Ulisp-2.4-esp with 4 MByte PSRAM support

#5

Currently pprintall doesn’t accept a stream, but pprint does, so you could use the function described in this thread to pprint all your definitions to a stream:

#6

@Kaef can confirm this, but as I understand it there’s no corruption; it’s just that the standard ESP version of uLisp is very slow doing a (save-image) when using all the memory in the ESP32-WROVER module.

#7

Thank you very much, @johnsondavies !

I discovered this version has some issue with the garbage collection (or at least the way this is printed) - note the diminishing number of cells:

Screenshot_20181107-125402
Screenshot_20181107-125402.png1080x1920 122 KB

#8

Yes, that’s not right. I haven’t tested your version with PSRAM support. Could you try some additional tests:

  • The standard 2.4 version with the code to address PSRAM, but without your speedup improvements.

  • Your version on a board without PSRAM.

Thanks, David

#9

corruption of save-image

I’m not sure in this point.
I think I had corrupted saved images, but I wasn’t able to reproduce this behaviour.
I didn’t look at this because I want to make it possible to save the images on sd-card. I’m not sure if this solves this problem, but I think it may have something to do with the eeprom-functions for esp32 (which uses the flash).

My version of save-image (better: compactimage () and subfunctions) should be a lot faster than the original version if not much lispcells are in use. On a heavy used system compactimage() takes a lot of time (several minutes) (I added the output of ‘.’ so you can see that the system is working).

#10

If you recompile uLisp after making changes, and then try to load in a saved image, it’s not guaranteed to work because the saved image isn’t position independent, and the address of the workspace may have changed. Could this explain what you observed?

#11

diminishing number of cells

This is because of my patch to newsymbol () and repl (). If you call (gc) in between all cells are marked as free (this was done in repl (), but the system-reponse-time increases a lot). The (gc) is automatically called if less than 20 lispcells are free, so don’t worry ;-).

If you didn’t like this behaviour you can comment out the beginning of newsymbol () (in my version lines 361 to 363 and line 369) or you can comment out the definition LARGE_WORKSPACE (line 200), which will also removes my patch to repl (calling gc) (of course you can remove my patch to repl () without changing newsymbol () or vice versa).
The downside of doing this is that the interface gets slower without that patches.

I’m think this will be very hard to solve completely (ie. the system is still fast if many lisp-cells used) – one way could be to use small ‘chunks’ for the workspace and add a new chunk if there’s too less memory free. But if you would use many lisp-cells (large programs) ulisp would get slower the larger the workspace is, so I think this solution is not good enough (and it would be complicated to realize).
Therefore I think we have to gc the chunks separately and only if anything in it changed since last call to gc, but (again) to realize this is rather complex (and it has very low priority on my personal TODO-list…)

I think the best solution is to live with it and (if you want a predictive system-runtime) call (gc) when you have time for housekeeping in your program.

Kindly regards,
Kaef

#12

Thank you very much! Indeed, “(gc)” brings me to a high level again (524016 - higher than at bootup, 523997).

#13

Fine.
That the number of free lisp-cells increases may have to do with the LispLibrary (I activated it to test this functionality).
After starting ulisp you can see the defined symbols using (globals).
You can disable this by comment out the line ‘#define lisplibrary’ in ulisp-esp.ino.
Of cause you can put your own functions into the LispLibrary (second TAB File LispLibrary.h) if you like. There’s an excellent description of it at the ulisp-page (ulisp.com/ -> Getting started -> Lisp Library): http://www.ulisp.com/show?27OV (I think David did a very good job in writing the documentation of ulisp – this is one reason why I love to play with it – thank you David! ;-) ).

The functions I put in the LispLibrary.h are mostly copied from the ulisp homepage to do some tests.

Maybe I’ll use the LispLibrary feature it to define some basic functionality (some kind of ‘mini-lisp-bootloader’) to read more lisp-files from a sd-card. In my mind there could be an enhanced function of loadProgram (which is able to load programs which are written in more than one line), an enhanced (emacs-like?) editor and many more (sensor function for example) (that would be a small lisp-machine one day…).
Using a ulisp-image is of cause also possible, either variants have some pros and cons.

But to realize this I have to learn a lot about lisp first… ;-) At the moment I’m going to support more esp32 specific functions like deepsleep and storing ulisp-images on the sd-card.
Later I’d like to build a ‘Tiny-Lisp-Computer’ like the ones David has already built but with esp32 and a larger (> 5’’) display…
I find it also very interesting to show what a powerful language lisp is and that it’s sometimes much easier to solve problems in lisp than in other languages.

So, there’re a lot of ideas, and for me it is a lot of fun’ playing’ with it.

Regards,
Kaef

#14

Dear @Kaef ,

I have a Wemos ESP32-WROVER. Now, how do I use its SD-card? I am getting some sort of “SD card error”.

I see, you have defined in your code some MISO, MOSI etc.

Now, the SD-card area is marked as follows (on the back of the board):

SD
D2 - 12
D3 - 13
CMD - 15
CLK - 14
D0 - 2
D1 - 4

What do I need to change in your code to make SD card support possible on this board?

Kindly thank you in advance for your very obliging help.

#15

Hi Aeneas,

if your board is the one David describes at http://www.ulisp.com/show?2AJI – there is one solution shown on that page (patch the arduino file 'pins_arduino.h), but this will only work with my version of ulisp if you comment out the SDCARD_xx_yy defines at line 240.
I recommend to use the defines around line 240; they will be used in the SPI.begin() call (line 1657) :

#define SDCARD_SS_PIN    13 /* Kaef sdcard: 5 */
#define SDCARD_CLK_IO    14 /* Arduino standard: 18 */
#define SDCARD_MISO_IO    2 /* Arduino standard: 19 */
#define SDCARD_MOSI_IO   15 /* Arduino standard: 23 */

I inserted the values which should work for your board (of cause I didn’t test it because I don’t have the wemos board) (you have to enter ESP32-GPIO-numbers here, not PIN numbers!)

The sd-card must be formatted with fat-16 or fat-32 filesystem. On some boards you can only use “small” cards with 2GB or 4GB; I don’t know which sizes are supported by the wemos board.

You may copy some files to the card and enable the define SD_CARD_DEBUG (line 87). When starting ulisp the files from the sd-card should be listed (or you get an error-message).
If this works be sure that the filename given to (with-sd-card) starts with a leading “/”; if omitting the “/” you get the “SD card error” when try to read or write a file. (ie. (with-sd-card (str “/Greeting.txt” 2), not (with-sd-card (str “Greeting.txt” 2) )

HTH,
kindly regards,
Kaef

Here’s a log from my esp32:

uLisp 2.4a -- forked and extended by Kaef (https://github.com/kaef)
(c) by David Johnson-Davies - www.technoblogy.com
[...]

System information:
  reset reason: 1  wakeup cause: 0
  compiled: Nov 23 2018 09:21:43
  Features: resetautorun printfreespace serialmonitor sdcardsupport SD_CARD_DEBUG LARGE_WORKSPACE_SETUP
  initworkspace PSRAM 524281(512k - 7) cons allocated. 
SDCARD_SS_PIN = 5
SD Card Size: 1886MB
Listing directory: /
  FILE: /lisp.txt  SIZE: 48
  FILE: /foo.txt  SIZE: 13
  FILE: /hello.txt  SIZE: 11
[...]
done
  Free Heap: 233728
#16

Dear @Kaef ,

Thank you very much. Unfortunately, it still does not work. I enabled your debug-definition, and all I am getting is some sort of error telling me I did not mount the SD-card right. I then tried all possible values for the SS pin, i.e. 13, 12, 4 and 2, but still, error mounting the card upon boot-up. It is a known good card, successfully used in other Arduinos, too, and taken from another system just to be sure.

So… do I have any chance for SD card storage? :)

#17

Did you change the SDCARD_xx definitions (clk, miso, mosi)? It’s not enough changing the SS pin definition to get this work.

#18

Sorry, I didn’t read your post completely.
So you have change the definitions and it doesn’t work, rigght?
Please try an arduino example sketch and test if the card works with this.
You may also comment out my clk, mosi, miso definitions and try the way David suggested.
First, get the SD-card example working.

#19

Dear @Kaef ,

This is super annoying… Apparently I have a GENERAL issue with this board, the SD sketch does NOT work. Yes, I got that I need to change all pins, not just SS, and I found my pins_arduino.h for esp32, but no dice.

Well, I shall then order another board. Would it be possible to tell me which board EXACTLY you have? I want to get the same one as you, so debugging will be easier in the future. Is it this here, the one David recommended:

#20

Dear @Aeneas ,

I can confirm that your SD-card configuration doesn’t work!
I found a bug in my ulisp version (sometimes SD.begin() was called without parameters) – sorry for that.
I fixed (and tested) it with my module and I think it will work now, I uploaded the version to my dev-deepsleep branch (I will merge it to master soon).

Thank you very much for finding this bug!
Please let me know if it is working for you now.

I use a ESP32 WROVER module I soldered on a adapter board from Eckstein-shop ( https://eckstein-shop.de/HIMALAYA-Basic-ESP32-Adapter-Breakout-Board-for-ESP-WROOM-32-Partial-Kit). I don’t remember where I bought the ESP32 WROVER module, but there are several distributors. I connect the SD-Card with some wires, you can look at the photo on my github page. I also tested the connections which the WEMOS board without problems.
I also got a DSTIKE WiFi Packet Monitor board, but I didn’t test it with ulisp – I’ll do this ASAP.

Kindly regards,
Kaef

#21

Dear @Kaef ,

Thank you very much! - It still fails mounting, but I simply think my particular board might be faulty.

By the way, also for @johnsondavies , this might interest you both as it is quite cheap:

I guess that board should work, too. It does not look like it has SD-card support at all, but then again… it’s 5 EUR.

#22

Yes, that board looks like a bargain - thanks!

#23

Dear @Aeneas,

sorry for the delay.

I flashed my current ulisp-version 2.5a to the DSTIKE WIFI Packet MonitorV3 Board and I can confirm that the sd-card is working (see below).

I used the following settings for the sd-card io:

#define SDCARD_SS_PIN    13 /* Kaef            :  5, WEMOS ESP32 WOVER: 13 */
#define SDCARD_CLK_IO    14 /* Arduino standard: 18, WEMOS ESP32 WOVER: 14 */
#define SDCARD_MISO_IO    2 /* Arduino standard: 19, WEMOS ESP32 WOVER:  2 */
#define SDCARD_MOSI_IO   15 /* Arduino standard: 23, WEMOS ESP32 WOVER: 15 */

Here is a log from the serial monitor (I also defined SD_CARD_DEBUG):

uLisp 2.5a -- forked and extended by Kaef (https://github.com/kaef)
(c) by David Johnson-Davies - www.technoblogy.com
Licensed under the MIT license: https://opensource.org/licenses/MIT

System information:
  reset reason: 1  wakeup cause: 0
  compiled: Dec 19 2018 16:28:44
  Features: resetautorun printfreespace serialmonitor sdcardsupport SD_CARD_DEBUG lisplibrary LARGE_WORKSPACE_SETUP
  initworkspace PSRAM 524281(512k - 7) Workspace cells allocated. 
SDCARD_SS_PIN = 13
SD Card Size: 1886MB
Listing directory: /
  FILE: /lisp.txt  SIZE: 48
  FILE: /foo.txt  SIZE: 13
  [...]    done
  Free Heap: 233684

Press Btn0 to suppress restautorun...

loading image...
Error: No saved image
523767> (cat "/lisp.txt")

(defun sq (x) (* x x)) 
(defun tst nil (print 42)) 
nil

523694>

Kindly regards,
Kaef

#24

@Kaef The ESP32 Version of uLisp now fully supports PSRAM: see The latest ESP32 Version of uLisp now supports PSRAM.