The revamped streams thread


One of the biggest limitations of uLisp, as far as I’m concerned, is the lack of a sensible API for streams.

For example, I had the idea yesterday of trying to add a “tee” stream (the same sort of functionality as the Unix tee command), to work with streams between multiple threads. A set of tee streams could be created to wrap another output stream, and whenever output is received from the source stream, it is put into separate buffers that the tee’d streams read from, and one stream can be miles behind or ahead of another without affecting the parent stream. So, for example, you could tee a WiFi client stream and have one thread immediately dump the requests to serial, while the other thread actually parses the request and handles it.

There’s no way currently to make a custom stream like this, either in C++ (as an extension) or in pure uLisp.

One other example is that the ESP32 has Bluetooth capabilities, and it would stand to reason that these could be added as an extension. Since the ESP32 can support (in theory) an arbitrary number of connections, it would be sensible for the uLisp extension to also support an arbitrary number of Bluetooth-streams. But right now, there’s no easy way to support creating an arbitrary number of the same kind of stream, or even one of a new kind of stream, for that matter, without massively editing the main uLisp sources.

So here’s my proposal for fixing this:

Instead of a stream object having a low sentinel number in the cdr that indicates which of the builtin streams it is and everything else being done with global variables, the cdr of a stream object points to some cons pairs that hold what general “class” of stream it is, and some data that “individualizes” the stream. For example, it could hold the “file open for writing” type and the data could be a pointer to the Arduino File object. Or it could hold the “serial port” type and a pointer to the actual port object (Serial, Serial1, Serial2, SoftwareSerial, etc).

The point of this thread is to work out two things:

  1. The in-memory layout of a stream object.
  2. How extensions as well as user code can create new custom stream types.

Much feedback would be appreciated.

Using ulisp to upload to an FTP server