Introducing the uLisp Builder


#1

Introduction

The uLisp Builder is a set of programs written in Common Lisp to allow you to build a version of uLisp for a particular platform from a common repository of source files.

I currently run it on LispWorks on a MacBook Pro, but it should run on any version of Common Lisp, such as SBCL.

Note that you don’t need to run the uLisp Builder to use uLisp. I wrote it for my own use in maintaining uLisp, and I’m making it available now in case it’s of interest to anyone else.

Get it from GitHub here: https://github.com/technoblogy/ulisp-builder

Why?

The aim of the Builder was to make it easier to maintain uLisp across multiple platforms. Where the C function for a particular uLisp feature is identical on all platforms there is just a single occurrence of that source in the Builder repository. The Builder also takes care of constructing the lookup table of function names and entry points used by uLisp for the built-in functions.

Why is the uLisp source for each platform a single file?

C and C++ projects are usually divided into a multiple file structure, with each logical set of functions in a separate .ino or .c file, and a separate .h header file for each.

I can see the benefits of this where a project has multiple people maintaining it, but I tried working with this sort of structure and found it much less efficient to work with. It’s also easier for users to install uLisp from a single file; there’s no setup involved, aside from double-clicking the .ino file.

Why not use the C preprocessor?

It’s been suggested that uLisp could be provided as a single set of multi-platform source files, with C preprocessor statements to select the sections of code specific to each platform.

However, this would involve a much larger source file, or set of source files, and the presence of all the preprocessor statements would make it very hard to follow.

Also, I believe that some functions performed by the Builder, such as constructing the table of function names and entry points, would be beyond the capabilities of the C preprocessor.

Current state of the Builder

The version of the Builder I’ve put on GitHub currently builds the latest uLisp Version 3.4 for the ARM, and ESP8266/32 platforms and 3.4a for the AVR platform. It currently doesn’t build consistent versions for the other platforms; I hope to remedy this over the next few days.

Disclaimer

The Builder was written for my own use, and has grown as I needed to add new features to uLisp. It isn’t designed to be easy to use or extend, and isn’t written in elegant Lisp. Feel free to ask me questions about it on the uLisp Forum.

Using the Builder

Here’s a step by step set of instructions for using the Builder:

  • Open the file Load Builder.lisp.

  • Change the first line in the file to the name of the platform you wnat to build for. For example:

    (push :arm *features*)
    
  • Compile the file. This will compile the files specified by the system builder .

  • In the Listener give the command:

    (build :arm)
    

where the parameter should specify the same platform you specified in the first step.

You will be prompted to enter a name for the file:

  • Enter a name and click Save.

The source file for the specified platform will be saved.

What the Builder does

The Builder’s main function is (build) which performs the following actions to build the uLisp source for a particular platform:

Sources

The Builder merges the sections of source for the chosen platform and outputs them to the source file.

Sources are represented in the Builder source files as strings, using the #" … "# sharp-double-quote reader macro syntax to avoid the need to escape double-quote marks within each string. I am grateful to Doug Hoyte’s excellent book Let Over Lambda for providing this reader macro.

Comments

I am in the process of adding comments to the uLisp sources, and these will be extended in future versions. The Builder strips out the comments to keep the sources for each platform as compact as possible.

Table of functions

The Builder processes the uLisp function definitions in the file functions.lisp, and uses these to generate the following entries in the uLisp source for each function:

A keyword in the enum function { }; list. For example:

ATOM

The source of the function. For example:

object *fn_atom (object *args, object *env) {
  (void) env;
  return atom(first(args)) ? tee : nil;
}

The name of the function as a string in program memory; for example:

const char string49[] PROGMEM = "atom";

An entry in lookup_table[] giving pointers to the name of the function and the function’s entry point, and an uint8_t giving the minimum and maximum number of parameters; for example:

{ string49, fn_atom, 0x11 },

Keywords

The Builder incorporated definitions for keywords from the list:

*keywords-avr*

or equivalent on other platforms in the platform-specific source file.

Utilities

The following files contain utilities used by the Builder:

  • extras.lisp - general Lisp utilities used by the Builder
  • build.lisp - the main build function and routines it uses

Common sources

The sources common to all platforms are given in the files:

  • preface.lisp - the C macros and constant definitions
  • utilities.lisp - the C typedefs, global variables, and utility functions
  • saveload.lisp - the definitions for save-image, load-image, and autorun
  • assembler.lisp - the assembler for the ARM and RISC-V platforms
  • prettyprint.lisp - the prettyprinter
  • postscript.lisp - the definitions for the function table lookup, eval, read, print, and the REPL

Platform-specific features

Sources specific to a particular platform are identified using two alternative mechanisms:

Platform-specific sources

The files avr.lisp, arm.lisp, esp.lisp, and riscv.lisp contain definitions for the following uLisp platform-specific features:

  • The source file preamble
  • The platform-specific settings: WORKSPACESIZE, etc.
  • The stream definitions: I2C, SPI, serial, etc.
  • The analogread and analogwrite I/O pin definitions
  • Definitions for note, sleep, and keywords

Platform-specific includes

The main source files also include read-time conditional statements to select platform-specific versions of code, as a more economical way of incorporating platform-specific differences. For example:

    #+esp
    #"
void pfstring (PGM_P s, pfun_t pfun) {
  char str[strlen_P(s)+1];
  strcpy_P(str, s);
  pstring(str, pfun);
}"#

incorporates a version of pfstring on the esp platform.


#2

This looks promising, I’ll try to convert my port for RedBear Duo to use this builder and mayhaps report on how easy to use it is for a third-party. Thank you for μλ!


#3

Thanks for releasing this; I always wondered how the uLisp Badge source code was generated. One question, should you add a license to the project somewhere?


#4

Good suggestion - thanks.


#5

Done (on GitHub).


#6

Here’s a version I’ve been working on:

I’ve reworked quite a bit of the builder, which I first saw considerably before this release. I believe that version is easier to work on, since it pulls the C source code away from the Lisp code - but portions of the C are still generated by Common Lisp. At present, it only works in the same way - by building a single .ino file for the Arduino environment, but my aim is to make it possible to work with it as a more conventional C/C++ project.

One of the objectives here is to divorce the base of the code from the Arduino environment, which should make it easier to build a version for the Raspberry Pi - I know that’s of interest to some here. I am also interested in seeing if it would be possible to make a version of uLisp that runs in WebAssembly - which oddly creates restrictions more similar to the AVR than any other architecture I remember dealing with.

The master branch there is mostly the same as uLisp 3.5, but in addition to that there is also an immediates branch, which at the moment contains code for fixnums on AVR and ARM. ‘fixnums’ means that integers which fit in 29 (ARM) or 13 (AVR) bits don’t need to be allocated in the Lisp workspace. That’s particularly useful when combined with arrays on the ARM.

I’d be happy to help anyone who wants to get a grip on how to use this version - right at this moment it’s not the friendliest system imaginable.