Simple Lisp program editor

#1

Here’s a simple program editor that you can add to uLisp. It lets you step through a function definition, editing it a bit at a time, using a set of simple single-key editing commands you type at the keyboard.

(defun e (l)
  (loop
   (let ((c (read)))
     (when (eq c 'b) (return l))
     (setq l
           (cond
            ((eq c 'r) (read))
            ((eq c '?) (princ l) (terpri) l)
            ((eq c 'c) (cons (read) l))
            ((atom l) (princ '*) l)
            ((eq c 'd) (cons (car l) (e (cdr l))))
            ((eq c 'a) (cons (e (car l)) (cdr l)))
            ((eq c 'x) (cdr l))
            (t (princ '!) (terpri) l))))))

(defun edit (l) (set l (e (eval l))))

Commands

Here is a summary of the commands:

  • Enter Prints the current form.
  • a (car) Takes the car of the current form.
  • d (cdr) Takes the cdr of the current form.
  • r (replace) Replaces the current form with what you type in.
  • c (cons) Conses what you type in onto the front of the current form.
  • x (delete) Deletes the car of the current form.
  • b (back) Backs up the tree.

You run the editor by typing:

(edit 'fun)

where fun is the name of the function, or variable, you want to edit.

To edit the function you type a series of a or d commands to step through the function to the part you want to edit, use r, c, or x to make changes, and type b repeatedly to go back until you exit from the editor.

At any stage you can press Enter to print the current part of the function you’ve reached.

If you type an invalid key the editor prints ?, and if you reach the end of the tree with a or d the editor prints !.

Example

As an example of using the program editor, enter the following program b that blinks the LED on pin 13 once a second:

(defun b (x) (pinmode 13 t) (digitalwrite 13 x) (delay 500) (b (not x)))

Suppose we now want to change the delay parameter to 250 to make it blink twice as quickly. First give the command:

(edit 'b)

The editor prints the current context:

(lambda (x) (pinmode 13 t) (digitalwrite 13 x) (delay 500) (b (not x)))

Get to the delay command by typing:

dddd

The editor prints:

((delay 500) (b (not x)))

Now get to the 500 by typing:

ada

The editor responds:

500

Replace this with 250 by typing:

r250

The editor responds:

250

We can confirm that we’ve changed the correct value by backing up with:

bb

The editor responds:

(delay 250)

Quit from the editor with:

bbbb

Finally run the program to confirm that the change has been made:

(b nil)

#2

Now, of course I found out about your editor WAY TOO LATE, even though you mention it very clearly here as well as here:

http://www.ulisp.com/show?1J1Z

Well, so I ended up writing my own. And I discovered a few bugs along the way (working with Lisp 1.9b on an Arduino 2560 r3):

  1. Not sure this is a bug, but: is there any chance to make “backspace” act as “backspace”, and not just add a further backspace to the presently edited form? - As I so no way of doing that (including Ctrl-H having that same unfortunate effect…), I wrote this editor in the first place.

  2. If you enter a gargantuan number such as 55555555555555555555… - the system actually accepts it, albeit wrongly. I wonder what one overwrites. I think one can live with this sort of bug, I am just telling you.

  3. Most interesting one: if you edit a function in my editor, e.g.:

(defun f (x y) (cond ((< x y) (quote x)) ((> x y) (quote y)) (t (quote z))))

and then try to trigger the else-clause:

(f 5 5)
evl

you will get an error that “T” is undefined. I circumvented it by adding a definition for a variable “TT”, which evaluates to “T”, and that works.

Anyway, next I am posting the entire program.

#3 
; PEQUOD'S
; - before you sink to the vortex of no-backspace-madness,
; you might try this editor. (The name is an allegation
; to the available editor commands.)

; By Nino Ivanov, version of 13th August 2018.
; This program is Free Software under the
; GNU Affero General Public License,
; Version 3, 19 November 2007, to be found under
; https://www.gnu.org/licenses/agpl.txt

; run as: (pequods)

; This is a little "editor" I wrote, trying to handle a system
; which had no "backspace" (i.e. you cannot correct any
; misspellings). What it does: it enters forms into the
; LISP system, sexp by sexp. Your editing consists in adding
; to the latest sexp further lists. As you extend the sexp,
; you can correct your mistakes (with udo and del, see below),
; and you can save your progress (with sav; retrieve with old).
; It was originally written for David Johnson-Davies' uLisp on
; an Arduino Mega 2560 r3 and takes just under 1000 cons cells.
; The idea is: if you mistype something, then just "Enter" it,
; but then undo it, or return to a previous version of your
; sexp, which circumvents the problem of having no backspace.

; Editor Commands:
; pak = merge new list into previous sub-list (often needed
; to continue adding further statements in a cond)
; evl = evaluate (e.g. add to the system a function definition)
; qit = quit (without modifying the system any further)
; udo = undo; works only for the one previous step
; old = retrieve a save (possibly older than undo)
; and set it as the present state
; del = delete previous input (but not the permanent one)
; sav = enter for saving, i.e. for retrieval with old
; symbol of any other kind = retry entry
; (old bk nx): old=saved form, bk = backed up (for undo),
; nx=next sexp (the one you work on)
; call with (edt nil nil nil)

; if you use SBCL:
; use (finish-output nil) after the first terpri,

; Circumvention of a bug, where the Lisp system did not
; recognise "T" as "true":
(defvar tt t)

(defun edt (old bk nx)
  (progn (print (list '(Pak Evl Qit Udo Old Del Sav)
                       nx))

         (terpri)

  (let ((rd (read)))
    (cond ((eq rd 'del) (edt old nx nil))
          ((eq rd 'udo) (edt old bk bk))
          ((eq rd 'old) (edt old bk old))
          ((eq rd 'sav) (edt nx bk nx))
          ((eq rd 'pak) (edt old nx
                             (reverse (cons
                               (append
                                 (cadr (reverse nx))
                                 (list (car (reverse nx))))
                               (cddr (reverse nx))))))
          ((eq rd 'evl) (progn 
                             (print nx)
                             (print (eval nx))
                             (edt old nil nil)))
          ((eq rd 'qit) nil)
          ((null (listp rd)) (edt old bk nx))
          (tt (edt old nx (append nx rd)))))))

; or, in another, more compact form, easier to paste into a terminal:

; (defun edt (old bk nx) (progn (print (list '(Pak Evl Qit Udo Old Del Sav) nx)) (terpri)
; (let ((rd (read))) (cond ((eq rd 'del) (edt old nx nil)) ((eq rd 'udo) (edt old bk bk))
; ((eq rd 'old) (edt old bk old)) ((eq rd 'sav) (edt nx bk nx)) ((eq rd 'pak) (edt old nx
; (reverse (cons (append (cadr (reverse nx)) (list (car (reverse nx)))) (cddr (reverse nx))))))
; ((eq rd 'evl) (progn (print nx) (print (eval nx)) (edt old nil nil))) ((eq rd 'qit) nil)
; ((null (listp rd)) (edt old bk nx)) (t (edt old nx (append nx rd)))))))

(defun pequods () (edt nil nil nil))

; HERE IS A SAMPLE RUN:

; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; (defun g (x y) (cond ((< x y) (quote x)) ((> x y) (quote y))))
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y)))) 
; sav ; Let's say we start with a definition of a function that shall compare two numbers
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y)))) 
; ((TT '=)) ; now add a form for showing an equal-sign in the "other" case, where no number is larger
; 
; ((PAK EVL QIT UDO OLD DEL SAV)
;  (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y)) (TT '=))) 
; pak ; i.e. "pack" the last form under the previous sub-list (that is, under the COND).
; 
; ((PAK EVL QIT UDO OLD DEL SAV)
;  (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y) (TT '=)))) 
; evl ; "evaluate" - that is, the function definition we just obtained.
;
; STYLE-WARNING: redefining G in DEFUN
; (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y) (TT '=))) 
; G 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) ; after you evaluate, the system resets to nil - i.e. new form
; old ; but you may still demand an old form, e.g. for re-editing it.
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y)))) 
; del ; delete it, start anew
; 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; (g 4 5) ; now let's use that function we just defined!
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 4 5)) 
; evl ; Evaluate it. This is like a "delayed" REPL, which is useful if messed up the function call.
; 
; (G 4 5) 
; X 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL)
; (g 5 4)
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 5 4)) 
; evl
; 
; (G 5 4) 
; Y 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; (G 3 3)
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 3 3)) 
; evl
; 
; (G 3 3) 
; = 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; old
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (DEFUN G (X Y) (COND ((< X Y) 'X) ((> X Y) 'Y)))) 
; del ; we still can re-call and delete that saved form from before.
; 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; (G 5 6 7) ; ooops, function of two arguments called with three... Undo!
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 5 6 7)) 
; udi ; ... but here, we mistyped the command itself. Just enter it, it will do nothing.
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 5 6 7)) 
; udo ; now you get a new chance to "undo" (and you could have mistyped on purpose to "cancel").
; 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL) 
; (G 6 7)
; 
; ((PAK EVL QIT UDO OLD DEL SAV) (G 6 7)) 
; evl
; 
; (G 6 7) 
; X 
; ((PAK EVL QIT UDO OLD DEL SAV) NIL)
; qit ; quit PEQUOD'S
; NIL
#4

Nice alternative approach to a Lisp sexp editor.

In reply to your points:

Not sure this is a bug, but: is there any chance to make “backspace” act as “backspace”, and not just add a further backspace to the presently edited form? - As I so no way of doing that (including Ctrl-H having that same unfortunate effect…), I wrote this editor in the first place.

The reason that backspace doesn’t delete is that the reader doesn’t have an input buffer; it parses each element as you type it in. This isn’t usually a problem when you’re entering a uLisp program into the Arduino Serial Monitor, because that provides an input buffer, but I assume you’re using a terminal to interface with uLisp.

If you enter a gargantuan number such as 55555555555555555555… - the system actually accepts it, albeit wrongly. I wonder what one overwrites.

It just gets truncated to a 16-bit integer, but it won’t cause anything (a buffer) to overflow.

Most interesting one: if you edit a function in my editor, you will get an error that “T” is undefined.

The true value, t, should be in lower case. In your examples the characters are getting converted to upper case, and I’m not sure why. Does your terminal do that?

#5

Thank you for your elucidation. And yes, I got a Brother EP-44 and I am using it as a serial terminal! I am sort of “building a laptop”… ;)

Now I hit a new issue: Arduino prints “too fast” for the typewriter. The effect is that everything gets stuck and only a powercycle helps. It happens when you try to print slightly above one and a half lines of text, i.e., “pretty soon”. I have the following options:

  1. Adjust my editor to not print the result of editing so far, or to print only the last sub-form; you anyway need only the “tail” in order to see should you “pack” anything. Advantage: small & fast. Disadvantage: not solving the issue permanently.

  2. Pepper your print object function in the source code with delays. Maybe I will do that. Advantage: not bleeding cons-cells like crazy. Disadvantage: harder to set up (need to calibrate the delays, which needs multiple recompiles). That is the way forward for anything below DUE.

  3. Write a princ-like function in Lisp. Advantage: adaptability, portability. Disadvantage: I am going “bankrupt” on cons cells. Well, I did one this morning, not yet tested on Arduino (curly braces due to the Lisp-reader on my ECL-installation on my phone getting confused); I am using tt here as “t” (thank you for explaining the error anyway):

(defvar tim 80)

(defun slowprinc (x)
  (cond ((null x) (progn (princ "} ") nil))
        ((not (listp (car x)))
          (progn (princ (car x))
                 (cond ((not (null (cdr x)))
                   (princ " ")))
                 (delay tim)
                 (slowprinc (cdr x))))
        (tt (progn
              (sprinc (car x))
              (slowprinc (cdr x))))))

;
        
(defun sprinc (x)
  (cond ((not (listp x)) 
          (progn (princ x)
                 (delay tim)))
        (tt (progn (princ "{") (slowprinc x)))))

Edit: you then print whatever you like with (sprinc something).

#6

A few more thoughts about input buffers.

I could include a keyboard buffer in uLisp, so characters are buffered as you type them in, and they get parsed when you enter return. This would allow you to press backspace to correct mistakes when you’re interacting with uLisp via a terminal.

This is the approach I took with my Tiny Lisp Computer; it includes a buffer of 165 characters, large enough for a screenful of characters:

Tiny Lisp Computer 2

However, the downside is that the buffer reduces the RAM available to uLisp, and it limits the length of lines you can type in. Currently you can paste a program into uLisp and there’s no restriction on the length of lines between return characters.

A better solution would to provide the buffering and line editing in the terminal. This is something I’ve been thinking about…

#7

Hmmm… how about a “sliding buffer”? That is, you have a SMALL sliding window, say, some 7 or 15 characters, anything further gets “parsed”. And within this mini-window you can correct mistakes. - The point is: the most common types of “typing errors” are recognised within a minimial distance. All you need to do to greatly increase “comfort” in a terminal is to allow for minimal corrections along the lines of “aah, missed a letter” or “that was a parenthesis in the wrong place”. If anyone messes up anything further away - tough luck, but that will be what, 2-5% of all cases?

And yes, I found your Tiny Lisp Computer very inspirational, and also that SMS-sending one, great work!

#8

You could write a buffer that recognises backspace in Lisp, something like this:

(defun repl ()
  (let ((buf "") c l)
    (loop
     (setq c (code-char (read-byte)))
     (setq l (length buf))
     (cond
      ((eq c #\backspace) (when (> l 0) (setq buf (subseq buf 0 (1- l)))))
      ((eq c #\return) (print (eval (read-from-string buf))) (setq buf ""))
      (t (setq buf (concatenate 'string buf (string c))))))))

This reads characters into the string buf until you enter a return, allowing you to delete with backspace. It then reads from the string, evaluates it, and prints the result just like the normal REPL.

It needs tidying up a bit, but I hope it’s useful as a starting point.

Using backspace key in serial monitor
#9

Ah, David, that is SO AWFULLY KIND OF YOU! It was EXACTLY this sort of thing I did not even suspect was possible!

Well, I admit, because I wanted to understand this somewhat better, I did two variations of my own; I am including them for future reference here (and for the Windows users, it be remarked, “gcl.exe” (GNU Common Lisp), the “primitive” version, works just fine with them, whereas my version of SBCL for Windows chokes on them - so use GCL). I have not tried them on Arduino yet, but I assume I only need to change “(read-byte standard-input nil)” to “(read-byte)”. One version is recursive, and you need to leave with ! [Enter]; the other version is “one-shot”, i.e., you finish one definition and you are out. Enter is NOT treated specially (because on my typewriter, I may have to simply “enter” into a new line), you terminate a definition instead with $ [Enter]. They are used as (rv1 nil) and (rv2 nil) respectively:

; one-shot version:
(defun rv1 (col)
  (let ((nc (code-char (read-byte *standard-input* nil))))
    (cond ((eq nc #\$)
            (print (eval (read-from-string
              (concatenate 'string (reverse col))))))
          ((eq nc #\backspace) (rv1 (cdr col)))
          (t (rv1 (cons nc col))))))
;
; recursive version:
(defun rv2 (col)
  (let ((nc (code-char (read-byte *standard-input* nil))))
    (cond ((eq nc #\$)
            (progn
              (print (eval (read-from-string
                (concatenate 'string (reverse col)))))
              (terpri)
              (rv2 nil)))
          ((eq nc #\!) nil)
          ((eq nc #\backspace) (rv2 (cdr col)))
          (t (rv2 (cons nc col))))))

Ah, thank you so much for showing me the way forward here!

… And… not to get too lofty ideas, but you surely notice, if one leaves out the “eval” element, both in your editor and in mine one could implement such a function that allows “reading a form with backspace-corrections”. ;)

#10

OK, so here comes a little “how I FINALLY did it”; I got some difficulty in that “$” was difficult to access and moreover, my terminal has no backslash! - Which meant I needed a workaround for inputting characters, and I generally used (char “mycharacter” 0) to do this.

I created a function named “mstr” for “make string” that turns into a string a given list:

(defun mstr (x)
  (eval (cons 'concatenate (cons (quote 'string)
    (mapcar princ-to-string x)))))

This works e.g. like this: (mstr '(4 5 6)) --> “456”

… and again I worked with:

(defvar tt t)

… although, of course, I could have adjusted just the C-code as you suggested, David.

Then I created my auxiliary function rvr, to (reverse-)read input:

(defun rvr (col)
  (let ((nc (code-char (read-byte))))
    (cond ((eq nc (char "!" 0))
            (read-from-string (mstr (reverse col))))

And thus, (rvr nil) made it possible to read in a list like this:
(this is a test)!

Then I defined edt and pequods as before, but in edt I specified rd not as (read) but as follows:

(let ((rd (rvr nil))) ...

My editor then works as before, NOW allowing me to use backspace, and I terminate input with “!” like so: (+ 2 3)! or evl!

#11

It’s hard to append elements to a long list with (edit …), you need to press 'd + enter as much times as the length of the list, so I added a 'p command to (append old new) to help on this case:

diff --git a/LispBadgeLE.ino b/LispBadgeLE.ino
index c74abcd..e4fa690 100644
--- a/LispBadgeLE.ino
+++ b/LispBadgeLE.ino
@@ -2024,6 +2024,13 @@ void supersub (object *form, int lm, int super, pfun_t pfun) {
   pfun(')'); return;
 }
 
+object *append (object *arg1, object *arg2) {
+  if (!listp(arg1)) error(notalist, arg1);
+  if (!listp(arg2)) error(notalist, arg2);
+  object *args = cons(arg1, arg2);
+  return fn_append(args, NULL);
+}
+
 object *edit (object *fun) {
   while (1) {
     if (tstflag(EXITEDITOR)) return fun;
@@ -2033,6 +2040,7 @@ object *edit (object *fun) {
     else if (c == 'r') fun = read(gserial);
     else if (c == '\n') { pfl(pserial); superprint(fun, 0, pserial); pln(pserial); }
     else if (c == 'c') fun = cons(read(gserial), fun);
+    else if (c == 'p') fun = append(fun, read(gserial));
     else if (atom(fun)) pserial('!');
     else if (c == 'd') fun = cons(car(fun), edit(cdr(fun)));
     else if (c == 'a') fun = cons(edit(car(fun)), cdr(fun));

Now you can just use 'p to append elements when you’re on a list, the input also must be a list. (same as (append list*))

It’s also hard to go back to upper level from a long list, I don’t know how to implement a quick command to do so.:)

#12

Hi David

I’m trying to add a new 't command to go to up level list When I’m in the middle of a long list. The following patch is not the final version, my question is why uLisp can’t load save image any more after I applied this patch, it print out many nil nil nil…? the main change would be the new added global var ‘edit_last_list’ plus the code size change.

Please ignore the correctness of the implementation, I guess we need a stack to save the list pointers so that we can go up level one by one.

$ git diff
diff --git a/LispBadgeLE.ino b/LispBadgeLE.ino
index e4fa690..73e7eb4 100644
--- a/LispBadgeLE.ino
+++ b/LispBadgeLE.ino
@@ -2031,19 +2031,25 @@ object *append (object *arg1, object *arg2) {
   return fn_append(args, NULL);
 }
 
+object *edit_last_list = NULL;
 object *edit (object *fun) {
   while (1) {
     if (tstflag(EXITEDITOR)) return fun;
     char c = gserial();
     if (c == 'q') setflag(EXITEDITOR);
     else if (c == 'b') return fun;
+    else if (c == 't') return (edit_last_list != NULL? edit_last_list : fun);
     else if (c == 'r') fun = read(gserial);
     else if (c == '\n') { pfl(pserial); superprint(fun, 0, pserial); pln(pserial); }
     else if (c == 'c') fun = cons(read(gserial), fun);
     else if (c == 'p') fun = append(fun, read(gserial));
     else if (atom(fun)) pserial('!');
     else if (c == 'd') fun = cons(car(fun), edit(cdr(fun)));
-    else if (c == 'a') fun = cons(edit(car(fun)), cdr(fun));
+    else if (c == 'a') {
+      object *arg = car(fun);
+      if (listp(arg)) edit_last_list = arg;
+      fun = cons(edit(arg), cdr(fun));
+    }
     else if (c == 'x') fun = cdr(fun);
     else pserial('?');
   }
@@ -3795,8 +3801,10 @@ object *fn_edit (object *args, object *env) {
   object *fun = first(args);
   object *pair = findvalue(fun, env);
   clrflag(EXITEDITOR);
-  object *arg = edit(eval(fun, env));
-  cdr(pair) = arg;
+  object *arg = eval(fun, env);
+  edit_last_list = arg;
+  cdr(pair) = edit(arg);
+  edit_last_list = NULL;
   return arg;
 }

BR.

#13

Hi @Walter,

Just to be clear, are you working on an editor written in Lisp, and loaded into uLisp, or the equivalent built-in version of this written in C, and invoked with the (edit) function?

Also, are you aware of the improved version of this editor I wrote for the T-Deck?

A Lisp Editor for T-Deck

and the extensible version of it I wrote:

Extensible T-Deck Lisp Editor

It should be possible to port these to the Lisp Badge LE, although the smaller number of screen lines might be a problem.

Finally in reply to your specific questions:

I’m trying to add a new 't command to go to up level list

Doesn’t the b command already do this?

I guess we need a stack to save the list pointers so that we can go up level one by one.

This shouldn’t be necessary. The editor calls itself recursively, so the context is saved automatically. To “go up a level” all you should need to do is return from the current invocation of the (e) function. This is what the b command does.

#14

Hi David

I’m working on the built-in (edit) function.

Yes, I’m aware of the editor for T-Deck, I can try it on Lisp Badge LE later. But I tend to use the built-in (editor) to save the flash space for other data.

Given that I have a function hpb as the following, If I want add new tone '(4 2) at the end of the list, I need to press d d a d …d d d… many many times to reach the end of the tones list, then use r to replay the last nil to add the new tone, after that how can I go back to the up level of list at '(500 (5 1) …? press b b b … many many times.

In the previous patch, I utilized the append() function to add a new tone '(4 2) at the end of the list without traveling to that position first.

Any way, my major question is why my saved data can’t be loaded after I added a new global var in the code.

(defun hpb nil
  (sing
    '(500
       (5 1)
       (5 1)
       (6 2)
       (5 2)
       (11 2)
       (7 4)
       (5 1)
       (5 1)
       (6 2)
       (5 2)
       (12 2)
       (11 4)
       (5 1)
       (5 1)
       (15 2)
       (13 2)
       (11 2)
       (7 2)
       (6 2)
       (14 1)
       (14 1)
       (13 2)
       (11 2)
       (12 2)
       (11 5))))
#15

Any way, my major question is why my saved data can’t be loaded after I added a new global var in the code.

Ah, sorry, I didn’t understand that question before.

Adding a global variable to the C program shifts the position of the other uLisp addresses in memory, so all the address pointers change, and the image restored by (load-image) is no longer valid. There is a hint about this in the Reference:

Note that saved Lisp images may not be compatible across different releases of uLisp.

There are obvious ways to work around this.

#16

Given that I have a function hpb as the following, If I want add new tone '(4 2) at the end of the list

One solution, if you’re usually adding tones to the end of the list, is to store the list in reverse, add new notes to the front of the list, and use (reverse) when you’re playing the tune!