A Lisp Editor for T-Deck

[johnsondavies]

This is an experimental Lisp Editor designed to make it easier to enter and edit Lisp programs on your T-Deck.

Unlike a screen editor like vi or pico, this editor understands the structure of a Lisp program, and works on Lisp subexpressions. In a screen editor you could inadvertently delete one of a pair of matching brackets – the program will then no longer make sense, and it may be very confusing getting it back to a valid program. In this Lisp Editor the edits will always leave the program correctly structured, making it easier to focus on the changes you want to make without having to worry about ending up with an invalid structure.

This version of the T-Deck Lisp Editor is written in Lisp. The advantage of this is that it’s easier to modify it to change its behaviour, or to add your own commands. In future it could be coded in C and provided as a uLisp Extension file.

Update

28th October 2023: Following initial feedback on this article I’ve added a How to use it section to make it clearer how you use the editor.

To add the T-Deck Lisp Editor to your copy of uLisp

• Download Release 5 of the T-Deck firmware from https://github.com/technoblogy/ulisp-tdeck.

  This includes a special version of the prettyprinter, used by the Lisp Editor, that can highlight a specified subexpression.

• Load this file: T-Deck Lisp Editor.

• Save this file as LispLibrary.h.

• Put this in the same Arduino project folder as the uLisp source file.

• Comment out the definition of LispLibrary[] at the start of the uLisp source, since we want to supply the LispLibrary[] definitions in a separate file:

  // Lisp Library
  // const char LispLibrary[] PROGMEM = "";
  

• Add a reference to the LispLibrary.h file in the main uLisp source file by uncommenting the line:

  #include "LispLibrary.h"
  

• Uncomment the #define lisplibrary:

  #define lisplibrary
  

  This will cause the LispLibrary[] definitions to be loaded automatically each time you restart uLisp.

• Compile and upload uLisp to your board.

The screen editor function definition edit will be added to your Lisp workspace, together with the auxiliary functions %edit and butlast, and the global variable cmds.

How to use it

All the editing commands are single lower-case letters. There are two types of commands: selection commands, to specify what you want to edit, and editing commands, to perform the edit.

After each command the Lisp Editor shows your program, restructured to reflect the changes you have made.

Selection commands

The main selection commands are d (cdr), a (car), and b (back).

d takes the cdr of the current expression, and displays a green block cursor to show your position.

a takes the car of the current expression, and highlights the resulting expression.

b backs up the tree, returning to the state before an a or d.

Editing commands

The main editing commands are r (replace) and i (insert).

r prompts for an expression at the bottom of the screen, and then replaces the current selection with what you type in.

i prompts for an expression at the bottom of the screen, and then inserts what you type in at the current position.

Undo

If you make a mistake at any time you can press z to undo the last operation(s) you performed.

Example

Here’s an example session demonstrating how you use the Lisp Editor.

For this example we will try and write a simple routine to find the least prime factor of a number. Here’s our first attempt; it uses mod to find the remainder when the number n is divided by a series of divisors d from 1 to n:

(defun factor (n)
  (let ((d 1))
    (loop
     (when (> d n) (return n))
     (when (zerop (mod n d)) (return d))
     (incf d))))

Let’s try the function with 2146654199, which is 46327 x 46337:

21326> (time (factor 2146654199))
1
Time: 0 ms

Unfortunately it finds the factor 1, which is not what we intended. We need to change the initial value of d to 2.

• Edit the function by typing:

  (edit 'factor)
  

  The T-deck Lisp Editor shows the function definition:

  

• Type d d to move the green block cursor to the start of the let expression:

  

• Type a to select the let block:

  

• Type d to move the cursor in front of the let assignment and a to select it:

  

• Type a d a to select the 1, and type r to replace the currently selected expression.

  You are prompted at the bottom of the screen to enter an expression to replace it:

  

• Enter 2 and press Return to enter it.

  The new value is shown in the definition:

  

We can make the factor function more efficient by only testing divisors up to the square root of n, because the smallest factor must be less than or equal to this. Let’s add this improvement:

• Back up to select the let assignment again by typing b b b :

  

• Position the cursor at the end of the let assignment by pressing d :

  

• Now press i to insert an expression at the cursor position.

  You are prompted at the bottom of the screen for the expression to insert:

  

• Enter (top (sqrt n)) and press Return.

  The expression is shown where you inserted it:

  

  Next, change the comparison to compare d with top :

• Type b b d a to select the loop block.

• Type f n to find the next occurrence of n .

• Press r top to replace it with top :

  

  Finally, type s to save the changes to the definition of function and return to the uLisp prompt.

Check that it works:

21302> (time (factor 2146654199))
46327
Time: 1.5 s

Command summary

Here is an alphabetical summary of the commands:

Command	Name	Description
a	car	Selects the car of the current subexpression.
b	back	Backs up the tree.
c	cons	Prompts for an expression, and conses in onto the front of the current subexpression.
d	cdr	Moves the cursor to the cdr of the current subexpression.
f	find	Prompts for an expression, and selects the next occurrence of it.
i	insert	A synonym for c. Inserts what you type in at the cursor position.
q	quit	Quits from the editor, discarding your edits.
r	replace	Prompts for an expression, and replaces the selected subexpression with it.
s	save	Saves your edits to the function you were editing.
x	delete	Deletes the car of the current form.
z	undo	Undoes the last command.

How it works

Here’s a brief summary of how the editor works:

The edit function reads the key presses you type, and rejects any illegal key presses. It builds up a list of the editing commands you have given in the global variable *cmds*. After each key press it calls %edit , which executes the list of editing commands on the function (or variable) you are editing, and prettyprints the result, using pprint.

This initial release of the Lisp Editor has the restriction that the function you’re editing must fit within the top 22 lines of the screen, above the two-line prompt area. I haven’t worked out the best way to extend the editor to cope with arbitrarily long programs, and would welcome any suggestions.

I can provide information about adding your own commands to the editor in a future article if it’s of interest.

[hasn0life]

This works very nicely! My main feature request is to change the get-key function to be non blocking, so it returns nil when there’s nothing and the character when there’s something. It would look something like

object *fn_getkey (object *args, object *env) {
  (void) env, (void) args;
  Wire1.requestFrom(0x55, 1);
  if (Wire1.available()){
    char temp = Wire1.read();
    if ((temp != 0) && (temp !=255)){
      if (temp == '@') temp = '~';
      if (temp == '_') temp = '\\';
      return character(temp);
    }
  }
  return nil;
}

Then one can turn it into a blocking function in ulisp

(defun wait-key () 
      (loop (let ((key (get-key)))
                 (when key (return key)))))

if necessary, but also have it not block for more control. I hope that’s not too weird of a request.

As far as feature ideas go, I think copy and paste functionality could be nice.

I have considered how to implement longer functions but haven’t figured anything particular out. You can try something like

//in the edit function

 (defvar processed (%edit fun))
 (defvar to-show (with-output-to-string (str) (pprint processed str)))

and then cut to-show to fit the screen, but I don’t know enough lisp to write this very well.

[johnsondavies]

My main feature request is to change the get-key function to be non blocking

Actually, I tried something similar but I couldn’t get it to work reliably. I think the problem is that the Lisp loop is likely to miss getting the key press.

[hasn0life]

That’s strange but I haven’t extensively tried this function myself so I can’t say that it works reliably. I will experiment with it and report back.

[hasn0life]

Here’s my take on the arbitrarily long programs issue. I trim the pretty printed string to the amount of lines that fit on screen and print that. Right now it can scroll but I didn’t figure out how to jump to cursor yet. I suspect it would involve figuring out which line the first #\STX appears and setting scroll-pos to that line. I figured I’d just publish what I got so far since it picks some low hanging fruit.

new commands:

• u scroll up by 1 line
• j scroll down by 1 line
• y scroll up by 5 lines
• h scroll down by 5 lines

// T-Deck Lisp Editor - Version 1 - 27th October 2023
// modified by hasn0life 2023-11-03

const char LispLibrary[] PROGMEM =

"(defvar *cmds* nil)"

// returns the position of the nth newline in a string
"(defun linepos (lines n)             "
"  (let ((count 0) (pos 0) (res 0))   "
"    (loop                            "
"      (setq res (search              "
"          (string (code-char 10))    "
"          (subseq lines pos)))       "
"      (if (and (not (= n count)) res)"
"        (setq pos (+ pos 1 res)      "
"          count (1+ count))          "
"        (return pos)))))             "

// trims the string to a certain amount of lines
// str: string to trim 
// starting-line: line to start trimming from 
// span: how many lines to show leave 
"(defun trim-by-lines (str starting-line span)      "
"   (let ((first-line-pos (linepos str starting-line)))"
"        (subseq str first-line-pos                    "
"		(linepos str (+ starting-line span)))))     "

//goes through the list of commands in cmds recursively and applys them to edit the function
"(defun %edit (fun)"
"  (cond"
"   ((null *cmds*) fun)"
"   ((eq (car *cmds*) #\\b) (pop *cmds*) fun)"
"   ((eq (car *cmds*) #\\e) (pop *cmds*) (%edit (list fun)))"
"   ((eq (car *cmds*) #\\h) (pop *cmds*) (%edit (cons 'highlight (list fun))))"
"   ((consp (car *cmds*))"
"    (let ((val (cdar *cmds*)))"
"      (case (caar *cmds*)"
"        (#\\r (pop *cmds*) (%edit val))"
"        ((#\\c #\\i) (pop *cmds*) (%edit (cons val fun)))"
"        (#\\f (cond"
"              ((null fun) nil)"
"              ((equal val fun) (pop *cmds*) (%edit fun))"
"              ((atom fun) fun)"
"              (t (cons (%edit (car fun)) (%edit (cdr fun)))))))))"
"   ((atom fun) (pop *cmds*) (%edit fun))"
"   ((eq (car *cmds*) #\\d) (pop *cmds*) (%edit (cons (car fun) (%edit (cdr fun)))))"
"   ((eq (car *cmds*) #\\a) (pop *cmds*) (%edit (cons (%edit (car fun)) (cdr fun))))"
"   ((eq (car *cmds*) #\\x) (pop *cmds*) (%edit (cdr fun)))"
"   (t fun)))"

"(defun butlast (lst) (subseq lst 0 (1- (length lst))))"

//handles the non editing commands, and puts the editing commands in the command buffer
"(defun edit (name)"
"  (let ((fun (eval name))"
"        cc"
"        processed  "
"        trimmed  "
"        (scroll-pos 0))"
"    (setq *cmds* nil)"
"    (loop"
"     (setq cc (append cc (list #\\h)))"
"     (setq *cmds* cc)"
"     (setq processed (with-output-to-string (str) (pprint (%edit fun) str)))" //turn the output into a string before printing to screen
"     (setq trimmed (trim-by-lines processed scroll-pos 20)) "  //trim the string to the screens length
"     (write-byte 12)"   //clear display      
"     (princ (concatenate 'string trimmed (string #\\ETX)))" //append ETX to stop the highlights
"     (setq cc (butlast cc))"
"     (let ((c (get-key)))"
"       (case c"
"         (#\\q (set name fun) (return name))"
"         (#\\s (setq *cmds* cc) (set name (%edit fun)) (return name))"
"         (#\\z (when cc (setq cc (butlast cc))))"
"         ((#\\r #\\c #\\i #\\f #\\e)"
"          (write-byte 11) (princ c) (princ #\\:)"
"          (setq cc (append cc (list (cons c (read))))))"
"         ((#\\d #\\a #\\x #\\b)"
"          (setq cc (append cc (list c))))"
"         (#\\u  (when (< 0 scroll-pos) (setq scroll-pos (1- scroll-pos )))) "
"         (#\\j  (setq scroll-pos (1+ scroll-pos ))) "
"         (#\\y  (if (< 5 scroll-pos) (setq scroll-pos (- scroll-pos 5 )) "
"                                     (setq scroll-pos 0))) "
"         (#\\h  (setq scroll-pos (+ scroll-pos 5))) "                                   
"         (t (write-byte 7)))))))"   //beep

;

PS: the nonblocking get-key works well for me so far, and using wait-key function replicates the blocking behavior without any weirdness on my end

[johnsondavies]

Nice! I’ll try it out.

[nanomonkey]

Now that functions have documentation strings, it may be useful to use C++ raw string literals for the LispLibrary file so that each line doesn’t need to be quoted individually with escape characters, and the documentation strings could be inline.

ie

const char LispLibrary[] PROGMEM = R"rawliteral(
(defun count (x lst)
  "Counts the number of items eq to x in lst."
  (if (null lst) 0
    (+ (if (eq x (car lst)) 1 0) (count x (cdr lst)))))

(defun count-if (tst lst)
 "Counts the number of items in lst for which tst is true."
  (if (null lst) 0
    (+ (if (funcall tst (car lst)) 1 0) (count-if tst (cdr lst)))))
)rawliteral";

The rawliteral portion is optional, and is just a token to indicate the start and end of your string in case your string has a sequence of characters that would cause problems. See here.

[johnsondavies]

For an improved version of the T-Deck Lisp Editor, which you can program with your own editing commands, see Extensible T-Deck Lisp Editor.