MQTT client - first extension

#1

hi, i’m needing to use MQTT for some iot projects, and since there isn’t anything like that within uLisp as of now i’ve decided to try to embed the arduino mqttclient library.

however it’s my first extension, and a pretty complex at that…
right now, i’m trying to implement a (mqtt/subscribe topic fn) that’d register callbacks for certain topics which the mqtt receive loop will be able to use to call the appropriate handler.

i’m struggling to retrieve the lisp-side lambda and to register it somehow in the C side; i’ve done a first (mqtt/connect addr &port) method and i’m stuck with that subscribe one…

i’d appreciate any help in designing that kind of “complex” extension

#include <ArduinoMqttClient.h>
#include <WiFi.h>

extern WiFiClient client;
MqttClient mqtt(client);

#define URI_LENGTH 2048
#define TOPIC_LENGTH 1024

// (mqtt/init URI)
object *fn_MQTTInitClient(object *args, object *env) {
  (void) env;
  int port = 1883;
  if (listlength(args) >= 3 || listlength(args) < 1) error2(PSTR("wrong arity"));
  if (listlength(args) >= 2) {
    port = checkinteger(second(args));
  }

  object *arg = first(args);
  char uri[URI_LENGTH] = "";
  cstring(arg, uri, URI_LENGTH);

  if (!mqtt.connect(uri, port)) {
    formaterr(arg, "cannot connect to mqtt broker, error: ~{", mqtt.connectError());
  }

  return nil;
}

// (mqtt/listen topic fn)
object *fn_MQTTListen(object *args, object *env) {
  if (listlength(args) != 2) error2(PSTR("wrong arity"));
  object *topic_obj = first(args);
  char topic[TOPIC_LENGTH] = "";
  cstring(topic_obj, topic, TOPIC_LENGTH);

  object *fn = second(args);

  return nil;
}
#2

I haven’t looked into MQTT so far, thus I’m not sure what the requirements are and what exactly you’re trying to achieve.
If I get you right you’re trying to pass a kind of function pointer which contains… exactly what?
Sorry for not understanding it fully, but I think in general it would not make much sense to pass a pointer to a C helper function or even an exposed uLisp function internally to be able to call some function from within uLisp. If there’s no interrupt involved (which would not be able to call a uLisp function without crashing uLisp, as far as I know), you could just wrap/expose all necessary C functions to uLisp and then call them and pass and store them as necessary within uLisp. Which I guess you want to do since you probably want to control the communication process with uLisp and not just “behind the scenes” - right?
Did I say anything useful? I’m not sure. It would help if you could describe what you need to implement and why.

#3

Working on my own stuff I now think I understood what you’re trying to do: You want to call a uLisp function whenever the MQTT loop detects a message of some kind. IMHO it’s not an easy decision how to handle that.

First, I believe you need to wrap mqttClient.poll() into a uLisp function to be able to control that from within uLisp.
Second, in case you want to react to a type of message using a uLisp function defined by yourself beforehand, I think you maybe have to do this:
To register the function and its parameters pass them to another uLisp wrapper function that calls mqttClient.onMessage and passes a third C function written by you as the callback function. This third function then needs to have access to the list containing your uLisp handler and its parameters stored before, i.e. the “args” passed to the onMessage wrapper, as well as the “env” passed to your “poll” wrapper.
Depending on the type of the incoming message this internal handler then needs to decide which uLisp handler to use, retrieve it from a permanent global storage and then “eval” it passing the stored uLisp-Object and the env that came from the “poll” wrapper.
I’m not really sure this is correct or the best way to deal with that, but you could give it a try. I believe the main thing here is that you 1) need to take control of the MQTT polling process and 2) need to store a reference to your uLisp function somewhere. You cannot register a uLisp function with onMessage as a handler directly, you need an internal C function that inserts your uLisp function into the uLisp evaluation process (hopefully) without crashing it.

#4

it’s a pretty accurate of my original intent, provide a dumb lisp-binding interface to mqtt, and once that’s figured out maybe go for something more lispy.

but i struggle most with how to actually do that, never did any cross-language lib bindings and haven’t found good resources on how to usually design that kind of stuff…

for storing the fns, i was thinking about doing some sort of globally-stored linked-list (since the mqtt client being globally stored is in C impls the simplest approach for basic use cases anyways), where each node would store the topic name and the lisp handler fn, but even just that is already out of my depth and i haven’t managed to get something coherent running… and from what i’ve seen the object type doesn’t support fns…

#5

I think what’s hard to grasp at first is the difference between compiled C code and interpreted uLisp. A uLisp “function” really is just a linked list of symbols which are bound to other (lists of) symbols and so on until you hit the built-ins and “atoms”. That’s why a uLisp object does not store C functions. David offers extensive documentation about that on this website, have a look at everything down from “Implementation” as well.
That said it was a steep learning curve for me, too: Understanding how Lisp interpreters internally work their way through lists of symbols bound to something and atoms. And more often than not I need to ask further questions.

So I’d say let go of the thought that C functions need to be stored in uLisp objects (as you discovered, they can’t be). It’s rather: The C functions of your extensions need to have runtime access to uLisp objects created in the running uLisp interpreter and report something back uLisp can work with. They must never really fully bypass/leave the interpreter, that’s why “env”, the current context the interpreter lives and works in, sometimes needs to be handed down. “Converting between C and uLisp” provides insight, as does the commented version of uLisp’s core. If you take all control away from the interpreter and ignore its state you can’t get back in.
Once I got the underlying principles well enough to work with I was hooked - I believe extending uLisp is much easier than, say, extending MicroPython, at least in most cases, since there are very few interface conventions that need to be served. So: No worries, it can be done!