What would you like to see in uLisp in 2024?

#45

A post was split to a new topic: How to output text only to the REPL

#46

I am the same dragoncoder047 from GitHub.

My sincerest apologies for any confusion caused by me not testing my own tutorial.

Well, I’m here now, ask me questions.

#47

As far as l know, to make things thread safe, you just need to add locks around all global variable writes.

FreeRTOS (on the ESP32) already has a built-in std::mutex implementation that plays nicely with its tasks, so this may be all that is necessary.

For Teensy, I found this library that provides std::thread, std::mutex, and std::lock_guard:

I can’t think of any other platform that would make sense to implement threads on, so I stopped looking after that.

#48

Some more things I’d love to see in the next version:

  1. A way for programs to quickly test if a particular extension or feature is available by name, and not have to rely on kludges or tricks. For example, to test if the platform has floating point:

    (string= (princ-to-string (/ 1 2)) "0.5")

    But to test if WiFi support is included:

    (not (eq nothing (ignore-errors with-client t)))

    These work but it’s not entirely clear what they’re testing for.

    s7 Scheme has (provided?) to test if a feature is available. The equivalent of the above two tests would be:

    (provided? 'floating-point)
(provided? 'wifi)

    The equivalent name in lisp speak would probably be featurep.

    s7 also has a global variable *features* that holds all of the currently-loaded features, which provided? uses age the user program can modify. On my particular fork, *features* would probably be set to (core floating-point wifi catch-throw macros).

  2. Ability to have an arbitrary number of extensions loaded, subject only to available memory. If each extension has a name, then those names could be used to populate the *features* list.

  3. Function overloading by extensions. This would allow extensions to handle new cases of existing functions, and still defer to the core function. I could conceivably see this used to allow the bigint and floating point extensions to actually be extensions and be loaded in with (2), and, defer to the core arithmetic functions that can only do integers. The bigint functions could just overload the existing arithmetic and turn regular numbers into bigints.

    This could be implemented by the reader always picking the last extension’s overload of the function, and then if there is a number-of-parameters mismatch, the function longjmp’s out to a special point, or it returns a special sentinel low pointer, the evaluator would try again with the next overload, up to and including the last one (there core implementation), and then finally throw an error if none of them work.

  4. Better streams. Provide (open-X) for all the (with-X) forms as well as (close). Allow multiple SD files to be open for reading at once. (peek-char).

    I previously discussed some implementation details here.

  5. Proper **&key**word arguments.

    At the very least, :if-not-exist on all of the stream-opening functions. In some cases, you want to open the stream to test if the stream exists (for example, I2C scanner) and you would want :if-not-exist nil, but other times you expect the target of the stream to exist and so :if-not-exist :error will be appropriate.

I drafted some code that might work to allow this and some other things here.

#49

@dragoncoder047 Are you able to run the macros that I suggested earlier and not get an error? I’m curious where my codebase has gone wrong. I think I translated your function binary flag macros correctly, and I’m not missing any functions. Perhaps the ordering of definitions is off.

#50

I get the expected output:

8977> (defmacro triple (x) `(+ ,x ,x ,x))
triple

8953> (triple 4)
12

8953> (triple (print 'foo))

foo 
foo 
foo 
Error in +: argument is not a number: foo

8953> (macroexpand-1 '(triple (print 'foo))) ; how macroexpand-1 is supposed to be used
(+ (print (quote foo)) (print (quote foo)) (print (quote foo)))

8953> (macroexpand-1 triple) ; how nanomonkey used it
(macro (x) (backquote (+ (unquote x) (unquote x) (unquote x))))

I’m not sure what’s going wrong on your version. Start again with a fresh clean stock uLisp, and then apply my patches exactly as described. If you get any errors that’s on me, sorry… Please let me know what errors you do get because I want to be able to fix them.

Do note that if you’re viewing my page on a smartphone, the highlighted lines may be wrong. It is a known bug in the syntax highlighting library I am using (the code itself text-wraps but the highlighting does not, leading to misalignment). edit: I disabled code line wrapping on my website, the highlights will be correct but you’ll have to scroll.

#51

Thanks for your list of “Some more things I’d love to see in the next version”.

1. A way for programs to quickly test if a particular extension or feature is available by name

Great idea! The Common Lisp way of doing this is with the *features* variable. You would simply be able to do:

(member :floating-point *features*)

to find if your version of uLisp has a particular feature. Initially I suggest populating it with features from this table:

Lisp for microcontrollers - Versions

2. Ability to have an arbitrary number of extensions loaded

My first attempt at the Extensions feature did allow an arbitrary number of extensions files, but it made it a lot more complicated, and there was an impact on uLisp’s performance. I suppose if there’s enough interest in this I could revisit it.

3. Function overloading by extensions.

It’s a nice idea, but do you think it would really be useful, apart from the bigint example?

Also, can’t the equivalent be achieved by redefining the built-in functions? For example:

(defun * (a b) ($* a b))

4. Better streams

Yes, we’ve talked about this before. I had a look at it but it was non-trivial to use peek rather than the way I do it at the moment with LastChar. I can have another look at it.

5. Proper &key parameters

As you know, I’m keen to keep uLisp compact, rather then letting it gradually grow towards a full Common Lisp, and this seemed a good point at which to draw the line.

Because uLisp is an interpreter, supporting &key parameters would have a performance impact, even when they’re not used.

Also, once uLisp supports &key parameters there would be an case for many functions to be extended; for example, all the sequence functions could be expanded to support the full set of keyword parameters such as :start, :end, :test etc.

I feel that there are other ways of achieving what you’re suggesting without having to add full support for &key parameters; for example, like I’ve done with make-array.

#52
  1. Yes, except I was intending for *features* to be something that gets pre-populated upon boot. Currently, it isn’t.

 

  1. That would work for the bigints, but I was thinking more of extensions that add new types, rather than just more numbers, where more extensive overloading would be necessary. A trivial example would be this:

    (defmacro + (&rest strings)
  `(concatenate 'string ,@strings))

    Now you can use + to concatenate strings, but you can’t add numbers anymore. If this was designed to work with both strings and numbers, it would get extremely long.

    TBH you probably don’t need this unless you manage (2), because if there is only one extension, it can explicitly defer to the built-in function.

    The only thing in here that needs to change for this to be supported is the reader, since it always picks the first entry for the symbol in the tables, not the last, and so an extension’s version will never be called if it used the same name as an existing function.

  2. Another argument here is that the “extensions” that have to use streams (WiFi and GFX), can’t be packaged as proper extensions in separate files, because the streams have to be included in the main uLisp source. If there was a way for extensions to declare what streams they provide, that would enable people to clearly turn on and off what features they want and don’t want, while also making core uLisp shorter so it’s easier to tinker with in the Arduino IDE (which I’ve found sometimes struggles on extremely large files).

    Currently the stream lookup functions are a bit of a mess and due for an overhaul, since they use a synchronized enum and string array to store the stream names (this reminds me of the enormous builtin_t enum of uLisp 4.3), but there is no table for the gfuns/pfuns — gstreamfun(), a weird function that is basically a lookup table made out of conditional statements, is used instead.

  3. Okay, since keyword arguments are a little complicated I understand your concern. However, I think that some of the more powerful platforms (ESP32, Teensy) have enough horsepower to handle proper keyword arguments. I don’t see how keyword arguments can be packaged purely as an extension, so could they be put behind a #define switch?

From what I can see, a lot of these limitations would be removed if the extreme reliance on enums that are used as indexes into a table is eschewed — why not just use a pointer directly to the entry? Especially with the built in functions and special forms. Currently, getting the table entry involves many operations: get index -> multiply by element size -> get 0th entry’s address -> add -> dereference pointer. If the pointer is stored directly, it becomes: get pointer -> dereference pointer. With that in place, there may even be enough time saved to allow (2).

#53
  1. Yes, except I was intending for *features* to be something that gets pre-populated upon boot. Currently, it isn’t.

Sorry, I meant “Initially I suggest that it would be pre-populated with features from this table”.

#54

2. Ability to have an arbitrary number of extensions loaded

I would really like to see this. My use case (robotics) often involves using several external libraries, for e.g. servo/motor drivers, diverse lcd displays, bignums, or whatever, and manually merging the extensions to support these into a single extensions file is proving enough of a pain point that I usually end up just using Arduino C or Micro/CircuitPython for those projects (with a higher level processor running Common Lisp talking to the uC). Would love to get to the point where I was Lisp all the way down.

#55

Whether it makes it into core uLisp remains to be seen, but if you’re interested, I figured out a (probably unoptimized) solution for multiple extensions at once in my fork:

In particular, these commits are where I added the multiple extension code:

(I would, however, recommend that you reference the latest version when applying the patches, since there were several bugs created, found, and fixed in the process of adding these features.)

#56

Thanks! I’ll try to apply these patches to my version (ulisp-arm 4.4a on an RP2040 Feather). Hopefully the “may not be portable to other platforms” comment won’t apply to mine…or if it does, I’ll switch to an ESP32 Feather.

#57

The “may not be portable…” comment was mostly about low-memory microcontrollers such as an ATmega328 where including the malloc libraries would waste way too much RAM and flash space (ulisp 4.4a already fills every last byte of the flash).

RP2040s have gobs of flash and RAM (similar to ESP32s), so I don’t think you’ll have any problems.