What would you like to see in uLisp in 2024?

nanomonkey · 2024-01-26 21:48:57 UTC

Things I’d like to see in uLisp for 2024 (and beyond) include:

Interrupt handling - being able to fire off a function when an interrupt is triggered, or store the fact that an interrupt was triggered in a queue or register would be nice.

Macros - I know these are dangerous, but it feels necessary for being a lisp to at least have them for more performant chips (teensy 4.0 and esp32). Not to mention it makes the language much more flexible.

Compile to assembly - this is a pie in the sky wish, but I’d love to be able to compile to one/all of the assembly languages. Not sure if this could be accomplished on the microcontroller, but even compiled on a computer and then passed back over the repl would be useful.

johnsondavies · 2024-01-27 09:21:36 UTC

Partial solutions to your three wishes:

Interrupt handling

The current options are:

Use the register command to set the appropriate bit in the pin change mask register, so a transition on a pin is latched in the pin change interrupt flag register. Then periodically poll the pin change interrupt flag register to detect when the pin change has occurred.

or:

Write a uLisp Extension file that implements interrupts, with an interrupt service that detects the interrupts and writes them to a location that can be queried from the Lisp program.

The terminology I’ve used is for AVR processors, but I assume something equivalent could be done on the other processors.

For information about the register function see: Programming AVR registers or Programming ARM registers .

For information about writing an Extension File see: Adding your own functions.

Macros

dragoncoder047 has implemented a pretty full implementation of macros in uLisp:

uLisp Extensions Requiring Major Edits

I would disagree that macros are dangerous; just difficult to understand, though I am not convinced that they are needed for most applications of uLisp.

Compile to assembly

The first step in doing this, namely compiling of assembler to machine code, is already available on ARM, AVR, and ESP; for example:

ARM assembler overview

The next step would be to write a compiler in uLisp to compile a subset of Lisp into assembler, or an intermediate bytecode. To be done!

Aeneas · 2024-02-05 15:16:03 UTC

Regarding macros, the situation is actually super funny: depending on WHO you ask, they are either totally essential (according to some Lispers), or near-complete nonsense (other Lispers + likely almost all of Haskellers).

Ginkgo · 2024-02-15 16:02:32 UTC

Another small proposal: Maybe it’s not just me who needs serial port communication non-blocking - if so, I’d suggest to include something like the following simple changes which seem to work:

[before platform specific settings]
#define SERIAL_TIMEOUT 1 // timeout for serial receive in milliseconds

[lines 2149 to 2157]
#if defined(ULISP_SERIAL3)
inline int serial3read () { unsigned long mymil = millis(); while (!Serial3.available() && (millis() < (mymil + SERIAL_TIMEOUT))) testescape(); return Serial3.read(); }
#endif
#if defined(ULISP_SERIAL3) || defined(ULISP_SERIAL2)
inline int serial2read () { unsigned long mymil = millis(); while (!Serial2.available() && (millis() < (mymil + SERIAL_TIMEOUT))) testescape(); return Serial2.read(); }
#endif
#if defined(ULISP_SERIAL3) || defined(ULISP_SERIAL2) || defined(ULISP_SERIAL1)
inline int serial1read () { unsigned long mymil = millis(); while (!Serial1.available() && (millis() < (mymil + SERIAL_TIMEOUT))) testescape(); return Serial1.read(); }
#endif

Could be done with micros() as well, of course - could also be implemented with another optional parameter for “with-serial”.

johnsondavies · 2024-03-21 08:04:26 UTC

2 posts were split to a new topic: Would it be possible to provide “return-from” in uLisp?

Ginkgo · 2024-02-17 08:51:59 UTC

Thanks for the advice! I figured cond might help here but wasn’t quite sure. I probably have to use it in connection with an error flag to prevent condition clauses further down within cond from being evaluated fully (i.e. always check the flag first in each clause and check the rest only if it’s not set) - to avoid unneccessary “evaluation cost”. Thanks again!

lrustand · 2024-02-20 20:20:11 UTC

I’d like to add a +1 for interrupts and macros. The interrupts need to be real interrupts though, not just the polling variant you suggested.

Also, a with-littlefs special form much like with-sd-card would be nice to have. We already have working LittleFS for save-image, now we only need to have a general function for reading and writing to arbitrary files as well.

Multithreading or multicore support would be a very nice to have, but this is lowest on my priority list right now.

johnsondavies · 2024-02-21 07:25:48 UTC

I’d like to add a +1 for … macros.

dragoncoder047 has implemented a pretty full implementation of macros in uLisp if you need them:

uLisp Extensions Requiring Major Edits

I’m not currently planning to add it to the standard version of uLisp.

and interrupts … The interrupts need to be real interrupts though, not just the polling variant you suggested.

uLisp is not thread safe, and it would be a major rewrite to change this, so I can’t see how this can be done.

Also, a with-littlefs special form much like with-sd-card would be nice to have.

Good suggestion, and it would be relatively easy. I’ll look at it.

Multithreading or multicore support would be a very nice to have

Sorry - see above!

lrustand · 2024-02-21 09:42:26 UTC

dragoncoder047 has implemented a pretty full implementation of macros in uLisp if you need them:

Yes, I know about this and already use them, but it would be good to add it to the core language. Macros are a very important part of Lisp, and a Lisp implementation without macros is simply not a real Lisp in my opinion. Leaving this out of the language and making everyone apply the patches themselves gives a lot of unnecessary friction to be able to use uLisp.

It also makes any code that is written using macros not portable. I want to be able to use macros in my code and to share this code with others without having to distribute the code alongside a patched version of uLisp.

uLisp is not thread safe, and it would be a major rewrite to change this, so I can’t see how this can be done.

Could it be done in a non-threadsafe way and let the programmer worry about the safety of his own threads? Or even an implementation where the interrupts/multithread/separate core is running in its own namespace or completely separate process with no access to the main workspace would still be a valuable addon.

Also, a with-littlefs special form much like with-sd-card would be nice to have.

Good suggestion, and it would be relatively easy. I’ll look at it.

Nice, thanks!

johnsondavies · 2024-02-21 10:45:24 UTC

Macros are a very important part of Lisp

I agree that they are important in a full Common Lisp, but I’m not convinced they are necessary for the applications that uLisp is targeted at. Can you give some examples of applications that you need macros for?

Could it be done in a non-threadsafe way and let the programmer worry about the safety of his own threads?

I don’t see how this would work. I’m assuming from what you said (“not just the polling variant you suggested”) you want an interrupt to be able to run Lisp code, or even a Lisp function. This implies that eval has to be reentrant, and I’m not sure how that would be possible.

lrustand · 2024-02-21 11:59:40 UTC

I agree that they are important in a full Common Lisp, but I’m not convinced they are necessary for the applications that uLisp is targeted at. Can you give some examples of applications that you need macros for?

I believe macros are essential to be able to clearly and concisely formulate good programs, no matter what the application is. I don’t think the size or type of application makes any difference to this. Macros allow you to create your own “syntactical sugar” and create things like custom with-* blocks, or implement doarray similar to dolist and all sorts of things you just can’t do with plain functions. Sure, you can implement the functionality of doarray as a function, but you can’t make the resulting function take the arguments in the same way as dolist without it being a macro, so the code would be uglier and less readable without macros, and it could not be implemented in a consistent way.

It would also improve interop of Common Lisp code in uLisp. Since uLisp is mostly a compatible subset of Common Lisp a lot of code can already be reused directly from CL. Adding macros would increase the amount of code reuse/interop that is possible. Not only directly from supporting defmacro and allowing to reuse existing CL code that uses macros, but it would also make it possible to use macros to quickly implement CL language features that are missing from uLisp in order to use CL code that depends on some missing language features.

I think that you are either underestimating what people are using uLisp for, or not really understanding the power of macros. Any program that is not simply a blinking LED “hello world” example is sufficiently advanced that it could potentially benefit from macros, so I do not agree at all that macros are not necessary for the type of applications uLisp is targeted at. Also, this type of reasoning becomes self-fulfilling, since the type of applications people are using uLisp for might be limited by missing features such as macros and interrupts, not the other way around.

I think that macros (and quasiquote and unquote) are possibly the biggest bang-for-the-buck addition to uLisp currently. It requires very little effort from your part to add one of the existing implementations to the language, but it gives a very big improvement to the power and expressiveness of the language. I don’t see any downsides or reasons why it shouldn’t be included.

I don’t see how this would work. I’m assuming from what you said (“not just the polling variant you suggested”) you want an interrupt to be able to run Lisp code, or even a Lisp function. This implies that eval has to be reentrant, and I’m not sure how that would be possible.

I don’t know too much about threads and thread safety, and I’m not familiar with how things are implemented in uLisp so I might be missing something obvious here. Could you explain what you mean by “eval has to be reentrant”?

johnsondavies · 2024-02-21 12:30:07 UTC

Could you explain what you mean by “eval has to be reentrant”?

An interrupt can happen at any time, such as during the evaluation of a Lisp statement. If you want the interrupt to be able to evaluate a Lisp function (the interrupt service routine), it needs to call eval. Therefore, the eval that’s executing needs to be re-entered to evaluate the interrupt service routine.

lrustand · 2024-02-21 21:58:52 UTC

Okay, that makes sense. In that case I think some kind of compromise between your suggested polling variant and a real interrupt could be good enough, such that it is uLisp itself that does the polling, not the Lisp program.

I don’t know exactly how the eval loop is implemented, but I assume it runs one function/expression at a time. Maybe the eval loop could poll for interrupt events each time it has processed one expression and call the triggered interrupt functions if any? Or would this cause a lot of overhead?

Alternatively, if I remember correctly there is a special register (on AVR chips at least) that enables and disables interrupt handling. Maybe this could be enabled and disabled when entering/leaving eval? (I’m not entirely sure if this register will queue up events that happened while the register was disabled or not though)

nanomonkey · 2024-02-23 18:05:44 UTC

dragoncoder047 has implemented a pretty full implementation of macros in uLisp

Has anyone gotten this to work in their own uLisp codebase? I tried to port over the changes they made and kept getting errors. There are quite a few code convention changes that may have hung me up, but I was under the assumption that there was still work to be done.

(defmacro triple (x) `(+ ,x ,x ,x))

(triple 4) 
=> Error: 'defun' too few arguments
(macroexpand-1 triple)
=>(rest (x) (defun (+ (backquote x) (backquote x) (backquote x))))

Any thoughts?!

I have to agree, a lisp without macros feels more like a shell scripting language rather than a true lisp. Macros would allow the ability to get rid of many of the #IFDEFs used for configuring a system, allow for writing cleaner/readable code that does destructuring, error correcting or creates special forms that aren’t currently implemented in C.

As far as interrupts go, could the interrupt handlers be simple lambdas that are evaluated with their own limited environments so as to not mess with the greater REPL environment? Or streams can be generalized (currently I believe there are only two?) so that the interrupt handler can output to a dedicated stream.

lrustand · 2024-02-25 17:53:46 UTC

Has anyone gotten this to work in their own uLisp codebase? I tried to port over the changes they made and kept getting errors. There are quite a few code convention changes that may have hung me up, but I was under the assumption that there was still work to be done.

Yes, I managed to get it working, but I had to try several times before it worked. I think it is important to get stuff in the right order in the enums and the list of functions, and there were also some things that was not mentioned in the blog, but was in the code in his repo. Also, some things needed to be updated to work with newer uLisp version.

Take a look at my fork, where I’ve successfully applied it to the latest ulisp-arm. I put everything related to the macros in a single commit, so it should be pretty easy to pick out. Let me know if it works for you.

nanomonkey · 2024-02-26 03:54:11 UTC

Interesting, my update was essentially the same as yours, only with a slightly different ordering of the function definitions. I placed them similar to yours, and got rid of the forward declaration for is_macro_call and now get the following response:

21047> (defmacro triple (x) `(+ ,x ,x ,x))
triple

21025> (triple 4)
Error: 'unquote' not valid outside backquote

which appears to be the function

object* bq_invalid (object* args, object* env) {
    (void)args, (void)env;
    error2(PSTR("not valid outside backquote"));
    // unreachable
    return NULL;
}

Which is surprising, even though I see that the symbols unquote and unsplicing are linked to that function

  { stringunquote, bq_invalid, 0311, docunquote },
  { stringuqsplicing, bq_invalid, 0311, docunquotesplicing },

I would assume that these symbols would be replaced by macroexpand, and not evaluated as symbols.

Did you test unquote(,) and unquotesplicing (,@)? What happens when you run said macro?

Walter · 2024-02-27 03:23:32 UTC

Hi David

I’d like to have:

Upload new uLisp binary into device without erase the saved image/datas

Since I’d like to save some persist data in the device, I’d like them persist all the life time, otherwise, I need to find a way to backup data every time I want to burn a new uLisp version into the device and restore them later.

johnsondavies · 2024-02-28 17:55:27 UTC

Whether the saved image becomes invalid depends on the upgrade, and how saveimage is implemented on the platform, but it would be difficult to solve this in all cases.

The simplest way to make a backup of your data is to do:

(pprintall)

in the REPL, and then copy and paste the listing into a text file. You can then evaluate it after the upgrade to reinstall your data.

Walter · 2024-02-29 00:22:48 UTC

Agreed, it’s a natural way in Lisp.

I tested upload new uLisp image into Lisp Badge LE, the saved image(data) remains loaded after reboot.

johnsondavies · 2024-03-18 17:59:00 UTC

3 posts were split to a new topic: Would it be possible to add “make-symbol”?