Packages and persistent storage

#1

Now that the T-deck has sd-card support I’ve been using save-image and edit to create new functionality that is persistent. Unfortunately when I update the firmware these functions are no longer loadable. So I started exporting using the pprintall function to save a text version of the functions that I had written to a separate file:

(load-image)

(with-sd-card (str "image.txt" 2)
  (pprintall str))

When writing a function to load the stored functions from the text file, I realized that I was starting to write a package loader of sorts:

(defun load-package (filename)
  (with-sd-card (str filename)
    (loop
     (let (ln (eval (read str)))
       (unless ln (return nothing)))))))

Which lead me to wanting to write a function that given a package file name, and a list of symbols, would write these out to a file to be loaded later:

(defun save-package (filename lst)
  (with-sd-card (str filename 2)
   (dolist (f lst)
      (pprint f str))))

(defun add-to-package (filename list)
   (with-sd-card (str filename 1)
     (dolist (f lst)
         (pprint f str))))

Sadly pprint either prints only the name if they are quoted, or the lambda of the symbol if given a list of symbols

(save-package "packages.lsp" '(save-package add-to-package))

At which point I realized I would need to write out a describe function that returns the whole definition of a function, and I might as well have it do all symbols, which lead me to coming here and seeing if anyone has put any thought into this. Could this be build into edit. Do we need to include shadowing, gensym and other package management components, or should a package just load into the global space.

Thoughts?

#2

A package loader sounds an interesting idea.

Sadly pprint either prints only the name if they are quoted, or the lambda of the symbol if given a list of symbols

I think if you look at the definition of pprintall it may do some of what you want. It constructs the definition of a function by adding defun and the function name, and also handles defvars.

#3

I got some time to look over the pprintall function, and came up with the following additions, which I’ve added to my extensions file:

object *fn_sym_def (object *args, object *env) {
  (void) env;
  object *obj = first(args);
  pfun_t pfun = pstreamfun(cdr(args));
  #if defined(gfxsupport)
  if (pfun == gfxwrite) ppwidth = GFXPPWIDTH;
  #endif
  object *pair = findvalue(obj, env);
  object *var = car(pair);
  object *val = cdr(pair);
  pln(pfun);
  if (consp(val) && symbolp(car(val)) && builtin(car(val)->name) == LAMBDA) {
    superprint(cons(bsymbol(DEFUN), cons(var, cdr(val))), 0, false, pfun);
  } else {
    superprint(cons(bsymbol(DEFVAR), cons(var, cons(quote(val), NULL))), 0, false, pfun);
  }
  pln(pfun);
  ppwidth = PPWIDTH;
  return bsymbol(NOTHING);
}

const char string_sym_def[] PROGMEM = "symbol-def";

const char doc_sym_def[] PROGMEM = "(symbol-def symbol [str])\n"
"Prints the definition of a symbol (variable or function) defined in ulisp using the pretty printer."
"If str is specified it prints to the specified stream. It returns no value.";

{ string_sym_def, fn_sym_def, 0212, doc_sym_def },

This creates a ulisp function symbol-def which pretty prints the definition of a symbol that is currently saved within globals. With this I can now use the following ulisp functions to do some basic “package” management:

(defun load-package (filename)
  (with-sd-card (str filename)
    (loop
     (let (ln (eval (read str)))
       (unless ln (return nothing)))))))

(defun save-package (filename lst)
  (with-sd-card (str filename 2)
   (dolist (f lst)
      (symbol-def f str))))

(defun add-to-package (filename list)
   (with-sd-card (str filename 1)
     (dolist (f lst)
         (symbol-def f str))))

Note these loaded “packages” will overwrite any existing symbols, and does no attempt at creating a namespace, shadowing variable or anything that you’d normally expect from a Common Lisp package.

#4

Hello, I believe this code no longer works. Or I am having some issue with getting it to work!

I have added the not-lisp to the extensions file I made

load-package seems to complete but do nothing
save-package and add-to-package do not even complete or I provide the wrong arguments.

#5

Are you getting any error messages?

I assume that you’ve enabled the sdcard reader, and have tested that you can read and write to it.

You may want to read the section on adding extensions:
http://www.ulisp.com/show?19Q4

The C function definition, doc strings, and symbol names need to be added for the uLisp code will work. As will an entry in the symbol lookup table, which is probably what is missing from your code if you’ve only added what I placed above. You’ll need to add something like the following to the symbol lookup table:

{ string_sym_def, fn_sym_def, 0212, doc_sym_def },

#6

I have enabled sd card reader
I have closely followed the extensions sample
I have added the C function definition, doc strings, and symbol names

I do not get errors but load-package doesn’t seem to work
It fails to load silently

Which might be a problem of me not using save-package properly

Forgive me because of my lack of lisp knowledge but does your decls of save-package and add-to-package have typos related to list and lst

#7

Sorry for the double post but here is my code

https://git.ablecorp.us/able/tdeck-ulisp

#8

Yes, I believe the add-to-package uLisp code above has a typo, it should be lst throughout.

Note: filename is the file you want to write to, and lst is a list of uLisp functions to write (or append in the case of add-to-package) to the file.

(add-to-package "my_functions.lsp" '(new_function another_function))

#9

Wonderful! I believe saving works.

However when I use (load-package "hello.lsp") the hello fun is not loaded.

999853> (directory)
("funcs.lsp" "ULISP.IMG" "start.lisp" "hello.lsp" "ext.lisp" ".Trash-1000")

999853> (with-sd-card (str "hello.lsp")   (read str))
(defun hello nil (print "hello") )

999853> (load-package "hello.lsp")

999853> (pprintall)

(defun start nil (print "hinr"))

(defun add-to-package (filename list)
  (with-sd-card (str filename 1) (dolist (f list) (symbol-def f str))))

(defun save-package (filename list)
  (with-sd-card (str filename 2) (dolist (f list) (symbol-def f str))))

(defun load-package (filename)
  (with-sd-card (str filename) (loop
      (let (ln (eval (read str)))
        (unless ln (return nothing))))))

999853>
#10

It looks like there is another typo, there should be two parenthesis around the let binding:

(defun load-package (filename)
  (with-sd-card (str filename) 
    (loop
      (let ((ln (eval (read str))))
        (unless ln (return nothing))))))

I don’t have the ability to test this right now…I’ll check my notes when I get home if this doesn’t work for you.

If you want to load just one function from a file use load:

(defun load (filename)
  (with-sd-card (s filename)
    (eval (read s))))
#11

The updated load-package function works on code I write to the sd card from another computer

save-package leaves a strange unprintable character in the dumped file which leaves us with

999852> (load-package "hello.func")
Error: undefined: 
999836>