Is the ability to load external libraries considered a fundamental part of the core functionality of the language?
On the face of it, I would suggest so—off the top of my head I could cite SQLite and ImageMagick as desirable components that, though they could be incorporated in other ways such as CALL, would more efficiently be utilized through the library interface.
The promise in Rebol 3 Alpha and Rebol 2 (specific implementation differing slightly) was thus:
This is nominally possible in Ren-C through the FFI extension (I say nominally as I have not been able to confirm this first hand and understand this is still a less than ideal approach).
Or are extensions the prescribed way to achieve integration?
Which would open up a whole bunch of other questions.
I know that it's hard to develop a one-size-fits-all solution to this as the architecture of external libraries can vary. This also introduces dependency requirements—the external libraries in question—that doesn't necessarily fit the write-once-run-anywhere ethos of Rebol.
If that's not working, raise a GitHub issue w/@szeng. I haven't done any FFI tests on a mac, but much more significant demos are working on Linux/Windows, e.g. %gtk.r
Early on I was anti-FFI just because it had a lot of hooks into the GC and memory model, and having yet-another-function-type (ROUTINE!) was introducing a lot of complexity. Over time, I managed to accomplish a couple of things in the "OneFunction" design which made having new function dispatchers introduce less complexity...and also factored out the FFI entities to use stock OBJECT!s and BINARY! for their data/etc.
So now it's not going anywhere. It's gone from being scary to being a reasonably sophisticated test case for a user-defined type. If you find FFI useful, then don't let me stop you...and if you find bugs or have problems, you can try reporting them.
But my own point of view is that it's a bit of a red herring to say that the way to address a lack of C programming expertise is to use a shaky implementation method. Because if you asked me to use FFI to do things like SQLite, I wouldn't have the tools in hand to do it correctly..and would have to plaster it with usage warnings.
If you take a simple thing like passing the memory backing a STRING! to a C function, you already had problems in the old code of the string maybe being one byte per character or maybe being two bytes per character, as an implementation detail. So there are places in the GTK demo where your widget labels would be messed up with a string that was of the two byte form (a circumstance that could arise even if all your characters were ascii, e.g. if there ever was a wide character and then removed). UTF-8 Everywhere might fix that one, but sometimes things will hold onto pointers you pass them...and any mutating string operation can invalidate the memory and reallocate it at a new pointer. (I'd argue that the GC might want to just clean up and compact things, so even if you don't change things it could move.)
Perhaps it will suffice for your own careful usage--but if you're packaging a library for others to use, odds are you're taking what was a relatively safe high-level language and making it brittle and crashy.
We might be able to enhance the interface language to build a smarter/better FFI, that lets you achieve the kind of level of control that breaking off into C and building Rebol abstractions would give you. I don't know if it's worth it in an era where protocols are using more web-servicey type languages to exchange data.
Thanks for this response, appreciate the further insight. I will raise the issue, though I worry that I've already contaminated my system sufficiently trying to get it to work that I may not be able to reduce my problems as if I were attempting a clean install.
It's not just the lack of C expertise—it's also having as much of the logic used to wrap said libraries written in Rebol, for the same reasons you'd want to push any system C code to user-mode.
Would it not be the case that black-box strings may better map to the BINARY! type?
I understand that a lot of these services are using protocols, however in many environments much depends on having the capability to have these services running which isn't always the case. Both examples I used—SQLite or ImageMagick (cURL is another common one that springs to mind) can be used via CALL, but can be more efficiently used through the respective library—indeed that is how they were primarily intended to be used.