Working around serial buffer overflow with Emacs

#1

I’m setting up Emacs on Linux for use with my Cardputer over serial via inferior-lisp and lisp-mode as explained here. Aside from some spurious ^M characters in Emacs this setup can work well for me.

However, I haven’t figured out how to evaluate from a Lisp mode buffer a long expression that triggers a serial buffer overflow on uLisp. Even when the expression is preceeded by a line starting with a semicolon, C-x C-e (eval-last-sexp) and M-x eval-region both trigger overflows as Emacs likely skips the comment prior to sending the code to uLisp.

Do you have any tips for sending long expressions from Emacs to uLisp? How do you do it?

#2

Does this Swank implementation for SLIME handle sending expressions longer than the serial buffer?

#3

Only tangentially related: if you want to have some fun, I made a haiku OS app for interacting with the uLisp REPL: https://github.com/narvalotech/super-tyson

Feel free to steal some code from it :)

#4

Thanks, I actually already tried Super-Tyson on Linux as I seems to understand all the dependencies are available there. It builds fine on Mint but aborts with a core dump when I launch the program:

SuperTyson: /home/paolo/apps/super-tyson/src/main.cpp:10: int main(int, char**): Assertion `argc == 2' failed.
Aborted (core dumped)

Do I need to pass an argument? The README doesn’t mention any but perhaps I need to pass the serial device.

As for reusing code, I’m not fluent enough in C.

#5

Despite some other attempts Emacs still displays extra ^Ms in the inferior Lisp and sending expressions longer than the serial buffer hangs the session.

Emacs is not much usable with uLisp as is. Aside from the Serial Monitor, terminal emulators don’t support line editing. In addition, sending code to uLisp requires copypasting which introduces friction and potential mistakes. This limited environment doesn’t scale to typical medium-large Lisp code beyond short expressions one or two lines long.

The Cardputer keyboard is not an option for anything longer than a line. I have no practical way of interacting with uLisp over serial and sending code longer than the serial buffer. I’m stumped.

#6

I don’t know if you’re aware, but the serial buffer overflow problem is probably not anything to do with Emacs. The ESP32-S3 has native USB and it does the serial port via software, and currently it has a particularly nasty bug where if the receive buffer fills up too fast the ESP32-S3 will crash and disconnect. This also isn’t a problem with uLisp either — it’s with the Arduino core.

#7

sending code longer than the serial buffer

What serial buffer are you referring to?

To enter programs into uLisp I do:

  • Select a whole Lisp listing of several pages, such as the Mini text adventure game.

  • Do Select All, Copy, then Paste it into the Serial Monitor input field, and press Return.

This works reliably for me.

What microcontroller board are you using? There are problems with the serial interface software on the Arduino Core for some boards.

#8

@dragoncoder047 Yes I’m aware.

@johnsondavies I’m referring to the serial buffer related to the USB issue @dragoncoder047 explained. I assumed the Serial Monitor was somehow as limited as other terminal emulators but I’ll check it out in more depth with my M5Stock Cardputer.

Thanks both!

#9

Note that the Cardputer is based on the ESP32-S3 which I don’t recommend for use with uLisp because of an issue with the serial interface implementation:

As discussed on the page ESP32-S2, ESP-S3, and ESP-C3 boards, there is an outstanding issue with the ESP32-S3 built-in USB support that makes it hang up if you try to enter a program of more than a few lines from the Serial Monitor on a Mac computer.

Do you have another board you can try, such as the Raspberry Pi Pico or Pico 2?

#10

I read about that ESP32-S3 USB issue but thought it applied only to Macs whereas I use a Linux PC. The only board I have is a Cardputer.

#11

Here is the Serial Monitor connected to the Cardputer after copy-pasting the full source of the mini text adventure game and evaluating (adventure). I later evaluated two more expressions, 'abc and 123. Nothing happens, the Serial Monitor just echoes to the output pane any additional input.

serial-monitor-cardputer
serial-monitor-cardputer.png966x598 16.8 KB

The same occurs with input shorter than the adventure game source but longer than the serial buffer. It looks like the ESP32-S3 USB issue affects Linux PCs too, not just Macs.

#12

To test whether the issue is there you could try my MRE at the start of this post:

Native USB CDC hangs up on ESP32-S2 after about 320 characters

It affects the ESP32-S3 too.

#13

By the way, I believe that this issue only affects the use of the built-in USB support (CDC) in the ESP32-S2, ESP32-C3, or ESP32-S3.

If you used one of the widely available USB to Serial cables connected to the appropriate Rx and Tx ports on the StampS3 module, I believe the problem won’t occur (although I haven’t tried it).

For example, I had no problems using the ESP32-S3-DevKitM-1 with the UART connection; see:

ESP32-S2, ESP-S3, and ESP-C3 boards - ESP32-S3-DevKitM-1

#14

To test for the issue I need to flash the sketch in this post, right? At this point there are little doubts though.

I’m connecting a cable to the built-in USB-C port of the Cardputer, i.e. the one of the S3 on the right side of the Cardputer. As a software guy I’m not much familiar with hardware or assembling electronic components, so I’m not sure where else the UART connection is and what cable it requires.

#15

To test for the issue I need to flash the sketch in this post, right?

Yes, and then perform the test described there from the Serial Monitor.

I’m not sure where else the UART connection is and what cable it requires.

The cable would be something like this:

FTDI Serial TTL-232 USB Cable

I assume it would connect to Rx, Tx, and GND on the StampS3 module; perhaps someone else can give more help about this.

#16

I uploaded the sketch and performed the test. As expected the Cardputer is affected by the issue as the output of the test is:

64
128
192
256
320

Bummer. I’ll probably wait for Arduino to fix the issue but, since it has apparently been known for a couple of years, I’m not holding my breath.

Thanks anyway.