What would you like to see in uLisp in 2024?

#36

dragoncoder047 has implemented a pretty full implementation of macros in uLisp

Has anyone gotten this to work in their own uLisp codebase? I tried to port over the changes they made and kept getting errors. There are quite a few code convention changes that may have hung me up, but I was under the assumption that there was still work to be done.

(defmacro triple (x) `(+ ,x ,x ,x))

(triple 4) 
=> Error: 'defun' too few arguments
(macroexpand-1 triple)
=>(rest (x) (defun (+ (backquote x) (backquote x) (backquote x))))

Any thoughts?!

I have to agree, a lisp without macros feels more like a shell scripting language rather than a true lisp. Macros would allow the ability to get rid of many of the #IFDEFs used for configuring a system, allow for writing cleaner/readable code that does destructuring, error correcting or creates special forms that aren’t currently implemented in C.

As far as interrupts go, could the interrupt handlers be simple lambdas that are evaluated with their own limited environments so as to not mess with the greater REPL environment? Or streams can be generalized (currently I believe there are only two?) so that the interrupt handler can output to a dedicated stream.

#37

Has anyone gotten this to work in their own uLisp codebase? I tried to port over the changes they made and kept getting errors. There are quite a few code convention changes that may have hung me up, but I was under the assumption that there was still work to be done.

Yes, I managed to get it working, but I had to try several times before it worked. I think it is important to get stuff in the right order in the enums and the list of functions, and there were also some things that was not mentioned in the blog, but was in the code in his repo. Also, some things needed to be updated to work with newer uLisp version.

Take a look at my fork, where I’ve successfully applied it to the latest ulisp-arm. I put everything related to the macros in a single commit, so it should be pretty easy to pick out. Let me know if it works for you.

#38

Interesting, my update was essentially the same as yours, only with a slightly different ordering of the function definitions. I placed them similar to yours, and got rid of the forward declaration for is_macro_call and now get the following response:

21047> (defmacro triple (x) `(+ ,x ,x ,x))
triple

21025> (triple 4)
Error: 'unquote' not valid outside backquote

which appears to be the function

object* bq_invalid (object* args, object* env) {
    (void)args, (void)env;
    error2(PSTR("not valid outside backquote"));
    // unreachable
    return NULL;
}

Which is surprising, even though I see that the symbols unquote and unsplicing are linked to that function

  { stringunquote, bq_invalid, 0311, docunquote },
  { stringuqsplicing, bq_invalid, 0311, docunquotesplicing },

I would assume that these symbols would be replaced by macroexpand, and not evaluated as symbols.

Did you test unquote(,) and unquotesplicing (,@)? What happens when you run said macro?

#39

Hi David

I’d like to have:

  1. Upload new uLisp binary into device without erase the saved image/datas

Since I’d like to save some persist data in the device, I’d like them persist all the life time, otherwise, I need to find a way to backup data every time I want to burn a new uLisp version into the device and restore them later.

#40

Whether the saved image becomes invalid depends on the upgrade, and how saveimage is implemented on the platform, but it would be difficult to solve this in all cases.

The simplest way to make a backup of your data is to do:

(pprintall)

in the REPL, and then copy and paste the listing into a text file. You can then evaluate it after the upgrade to reinstall your data.

#41

Agreed, it’s a natural way in Lisp.

I tested upload new uLisp image into Lisp Badge LE, the saved image(data) remains loaded after reboot.

#42

3 posts were split to a new topic: Would it be possible to add “make-symbol”?

#45

A post was split to a new topic: How to output text only to the REPL

#46

I am the same dragoncoder047 from GitHub.

My sincerest apologies for any confusion caused by me not testing my own tutorial.

Well, I’m here now, ask me questions.

#47

As far as l know, to make things thread safe, you just need to add locks around all global variable writes.

FreeRTOS (on the ESP32) already has a built-in std::mutex implementation that plays nicely with its tasks, so this may be all that is necessary.

For Teensy, I found this library that provides std::thread, std::mutex, and std::lock_guard:

I can’t think of any other platform that would make sense to implement threads on, so I stopped looking after that.

#48

Some more things I’d love to see in the next version:

  1. A way for programs to quickly test if a particular extension or feature is available by name, and not have to rely on kludges or tricks. For example, to test if the platform has floating point:

    (string= (princ-to-string (/ 1 2)) "0.5")

    But to test if WiFi support is included:

    (not (eq nothing (ignore-errors with-client t)))

    These work but it’s not entirely clear what they’re testing for.

    s7 Scheme has (provided?) to test if a feature is available. The equivalent of the above two tests would be:

    (provided? 'floating-point)
(provided? 'wifi)

    The equivalent name in lisp speak would probably be featurep.

    s7 also has a global variable *features* that holds all of the currently-loaded features, which provided? uses age the user program can modify. On my particular fork, *features* would probably be set to (core floating-point wifi catch-throw macros).

  2. Ability to have an arbitrary number of extensions loaded, subject only to available memory. If each extension has a name, then those names could be used to populate the *features* list.

  3. Function overloading by extensions. This would allow extensions to handle new cases of existing functions, and still defer to the core function. I could conceivably see this used to allow the bigint and floating point extensions to actually be extensions and be loaded in with (2), and, defer to the core arithmetic functions that can only do integers. The bigint functions could just overload the existing arithmetic and turn regular numbers into bigints.

    This could be implemented by the reader always picking the last extension’s overload of the function, and then if there is a number-of-parameters mismatch, the function longjmp’s out to a special point, or it returns a special sentinel low pointer, the evaluator would try again with the next overload, up to and including the last one (there core implementation), and then finally throw an error if none of them work.

  4. Better streams. Provide (open-X) for all the (with-X) forms as well as (close). Allow multiple SD files to be open for reading at once. (peek-char).

    I previously discussed some implementation details here.

  5. Proper **&key**word arguments.

    At the very least, :if-not-exist on all of the stream-opening functions. In some cases, you want to open the stream to test if the stream exists (for example, I2C scanner) and you would want :if-not-exist nil, but other times you expect the target of the stream to exist and so :if-not-exist :error will be appropriate.

I drafted some code that might work to allow this and some other things here.

#49

@dragoncoder047 Are you able to run the macros that I suggested earlier and not get an error? I’m curious where my codebase has gone wrong. I think I translated your function binary flag macros correctly, and I’m not missing any functions. Perhaps the ordering of definitions is off.

#50

I get the expected output:

8977> (defmacro triple (x) `(+ ,x ,x ,x))
triple

8953> (triple 4)
12

8953> (triple (print 'foo))

foo 
foo 
foo 
Error in +: argument is not a number: foo

8953> (macroexpand-1 '(triple (print 'foo))) ; how macroexpand-1 is supposed to be used
(+ (print (quote foo)) (print (quote foo)) (print (quote foo)))

8953> (macroexpand-1 triple) ; how nanomonkey used it
(macro (x) (backquote (+ (unquote x) (unquote x) (unquote x))))

I’m not sure what’s going wrong on your version. Start again with a fresh clean stock uLisp, and then apply my patches exactly as described. If you get any errors that’s on me, sorry… Please let me know what errors you do get because I want to be able to fix them.

Do note that if you’re viewing my page on a smartphone, the highlighted lines may be wrong. It is a known bug in the syntax highlighting library I am using (the code itself text-wraps but the highlighting does not, leading to misalignment). edit: I disabled code line wrapping on my website, the highlights will be correct but you’ll have to scroll.

#51

Thanks for your list of “Some more things I’d love to see in the next version”.

1. A way for programs to quickly test if a particular extension or feature is available by name

Great idea! The Common Lisp way of doing this is with the *features* variable. You would simply be able to do:

(member :floating-point *features*)

to find if your version of uLisp has a particular feature. Initially I suggest populating it with features from this table:

Lisp for microcontrollers - Versions

2. Ability to have an arbitrary number of extensions loaded

My first attempt at the Extensions feature did allow an arbitrary number of extensions files, but it made it a lot more complicated, and there was an impact on uLisp’s performance. I suppose if there’s enough interest in this I could revisit it.

3. Function overloading by extensions.

It’s a nice idea, but do you think it would really be useful, apart from the bigint example?

Also, can’t the equivalent be achieved by redefining the built-in functions? For example:

(defun * (a b) ($* a b))

4. Better streams

Yes, we’ve talked about this before. I had a look at it but it was non-trivial to use peek rather than the way I do it at the moment with LastChar. I can have another look at it.

5. Proper &key parameters

As you know, I’m keen to keep uLisp compact, rather then letting it gradually grow towards a full Common Lisp, and this seemed a good point at which to draw the line.

Because uLisp is an interpreter, supporting &key parameters would have a performance impact, even when they’re not used.

Also, once uLisp supports &key parameters there would be an case for many functions to be extended; for example, all the sequence functions could be expanded to support the full set of keyword parameters such as :start, :end, :test etc.

I feel that there are other ways of achieving what you’re suggesting without having to add full support for &key parameters; for example, like I’ve done with make-array.

#52
  1. Yes, except I was intending for *features* to be something that gets pre-populated upon boot. Currently, it isn’t.

 

  1. That would work for the bigints, but I was thinking more of extensions that add new types, rather than just more numbers, where more extensive overloading would be necessary. A trivial example would be this:

    (defmacro + (&rest strings)
  `(concatenate 'string ,@strings))

    Now you can use + to concatenate strings, but you can’t add numbers anymore. If this was designed to work with both strings and numbers, it would get extremely long.

    TBH you probably don’t need this unless you manage (2), because if there is only one extension, it can explicitly defer to the built-in function.

    The only thing in here that needs to change for this to be supported is the reader, since it always picks the first entry for the symbol in the tables, not the last, and so an extension’s version will never be called if it used the same name as an existing function.

  2. Another argument here is that the “extensions” that have to use streams (WiFi and GFX), can’t be packaged as proper extensions in separate files, because the streams have to be included in the main uLisp source. If there was a way for extensions to declare what streams they provide, that would enable people to clearly turn on and off what features they want and don’t want, while also making core uLisp shorter so it’s easier to tinker with in the Arduino IDE (which I’ve found sometimes struggles on extremely large files).

    Currently the stream lookup functions are a bit of a mess and due for an overhaul, since they use a synchronized enum and string array to store the stream names (this reminds me of the enormous builtin_t enum of uLisp 4.3), but there is no table for the gfuns/pfuns — gstreamfun(), a weird function that is basically a lookup table made out of conditional statements, is used instead.

  3. Okay, since keyword arguments are a little complicated I understand your concern. However, I think that some of the more powerful platforms (ESP32, Teensy) have enough horsepower to handle proper keyword arguments. I don’t see how keyword arguments can be packaged purely as an extension, so could they be put behind a #define switch?

From what I can see, a lot of these limitations would be removed if the extreme reliance on enums that are used as indexes into a table is eschewed — why not just use a pointer directly to the entry? Especially with the built in functions and special forms. Currently, getting the table entry involves many operations: get index -> multiply by element size -> get 0th entry’s address -> add -> dereference pointer. If the pointer is stored directly, it becomes: get pointer -> dereference pointer. With that in place, there may even be enough time saved to allow (2).

#53
  1. Yes, except I was intending for *features* to be something that gets pre-populated upon boot. Currently, it isn’t.

Sorry, I meant “Initially I suggest that it would be pre-populated with features from this table”.

#54

2. Ability to have an arbitrary number of extensions loaded

I would really like to see this. My use case (robotics) often involves using several external libraries, for e.g. servo/motor drivers, diverse lcd displays, bignums, or whatever, and manually merging the extensions to support these into a single extensions file is proving enough of a pain point that I usually end up just using Arduino C or Micro/CircuitPython for those projects (with a higher level processor running Common Lisp talking to the uC). Would love to get to the point where I was Lisp all the way down.

#55

Whether it makes it into core uLisp remains to be seen, but if you’re interested, I figured out a (probably unoptimized) solution for multiple extensions at once in my fork:

In particular, these commits are where I added the multiple extension code:

(I would, however, recommend that you reference the latest version when applying the patches, since there were several bugs created, found, and fixed in the process of adding these features.)

#56

Thanks! I’ll try to apply these patches to my version (ulisp-arm 4.4a on an RP2040 Feather). Hopefully the “may not be portable to other platforms” comment won’t apply to mine…or if it does, I’ll switch to an ESP32 Feather.

#57

The “may not be portable…” comment was mostly about low-memory microcontrollers such as an ATmega328 where including the malloc libraries would waste way too much RAM and flash space (ulisp 4.4a already fills every last byte of the flash).

RP2040s have gobs of flash and RAM (similar to ESP32s), so I don’t think you’ll have any problems.