FFI Updates and Notes

Development Language Bridging
1

Due to some complexities that had to do with the build system, the Ren-C FFI extension ceased being buildable somewhere on Dec 8, 2018.

Despite that, it still got some maintenance. So when things were being searched and replaced in the codebase at large, I would tinker with the code in the FFI a bit to try and make it roughly reflect what had changed. But even with that, a year's worth of drift without running it means it's going to get out of date.

In the last two days I've been able to get the demos back in running condition...including @szeng's GTK demo:

GTK Demo Screenshot
GTK Demo Screenshot2103×1684

Cool, but the Build Problems Haven't Gone Away...

There hasn't been a lot of time to focus on making Rebmake better at cross-platform inclusion of libraries that aren't by default available on the platform. There's no libffi that ships in Visual Studio, so it becomes something of a manual configuration process. Also, there's a bit of configuring that has to be done to get it working in cross-compilation of 32-bit binaries on a 64-bit Linux host. Similar difficulties for OS X.

Since no one is currently actively using the FFI (in the Ren-C builds at least), it seems the path of least resistance is to just have it building into the 64-bit Linux builds. That's where it's easy for the Travis container to install the package with sudo apt-get libffi and link with a simple -lffi. This is enough to keep the code compiling and run a couple of little smoke tests to make sure things haven't gone too far south.

To be clear: if you have the libraries all set up, then a "sufficiently motivated individual" could get the OS X, 32-bit Linux, and Windows versions to build. There's nothing about the code that shouldn't work (since it works on Linux 64-bit, it's more or less proven.) But they'd have to massage Rebmake to get it to spit out the right command lines and find the right files on the right platforms, when they're not around by default. It shouldn't be too hard, as the package setup code was already on Travis...but Rebmake is a bit of a beast and needs some design work vs. just hacking it up more.

Speaking of Tests...

I've been wanting tests that are extension-specific to live in the directory with the extension. So I'm moving them. Right now we have:

  • %printf.r - Very simple demonstration of calling printf
  • %varargs.r - More complex examples of calling printf and sprintf
  • %qsort.r - Simple callback example of C's qsort() function, where the sorting comparison function used by the C code is actually a Rebol function.
  • %qsort_r.r - More complex example of C's qsort_r() function, that allows passing a value through to the internal function that is not a global variable. Also makes that value a "tunneled" REBVAL*, so you can actually pass a Rebol value to the C as a pointer and then get it back in the Rebol callback as a REBVAL again.
  • %gtk.r - Gtk demo with toggle button, push button, font picker, and color picker

There was one file hanging around called %varargs_old.r. In Atronix R3 there were no variadics, so the only way to do something like printf was with a block. This means you'd have to COMPOSE or REDUCE to get your values in:

printf compose [
    "hello %p%c"
    "ffi" [pointer]
    (to integer! newline) [int8]
]

With variadics in Ren-C there was the variadic option, so I changed it to try it. For instance:

(printf "hello %p%c" "ffi" [pointer] to integer! newline [int8])

You could also use an expression barrier:

printf "hello %p%c" "ffi" [pointer] to integer! newline [int8] |

I don't know what the "right" answer for calling C variadics is. Maybe the compose was better? But it's easy enough to go back and forth that I don't think we need to preserve two copies of the same file to different conventions. Hence, instead I'm making a note here.

And as I mentioned in chat...

The big crazy difference with this FFI from the Atronix R3 is that it's all an extension. If you look closely you'll see a bit of an oddity in that it says make custom! and not make struct! . :-/ There needs to be some work on that particular ecology point.

The mechanics of having an arbitrary number of datatypes is now in place. So the system is no longer limited to the number of types that fit in a byte...only the built-in types use that for optimization to get use out of all 3 platform pointers (besides the pointer-sized bits used for the header). But a REB_CUSTOM type means that it is sacrificing one of those pointers in the cell for the extension type.

So...there's now a bunch of policy and ecology to straighten out on top of that.

But again, the good news of all of this is that it means that we aren't stuck putting a bunch of FFI-specific code in the core; so the WASM build can be lighter. And similarly, we don't have a bunch of JavaScript support in the core. So things are factoring well for using the language in different configurations.

1 Like