What will Rebol on the Web Look Like?


#1

First: We had emscripten as a black box %r3.js, which you could give a string to, and it would evaluate it and give a string back.

Second: We had the variadic-based libRebol manifesting as JavaScript. It’s important for making more complex interoperability, with API handles that offer certainty and can hold references over time…instead of guessing by picking apart strings with no running concept of identity.

Third: We now have “JavaScript user natives”, a category of ACTION! whose spec is a block of Rebol code, and whose body is a string of JavaScript. These can use the aforementioned libRebol to dissect their arguments into JavaScript values, and then produce return results back as Rebol again.

With these pieces in hand, we seem prepared to have a web page where JavaScript calls into Rebol…and then Rebol turns around and brings about actions in the JavaScript world. As exciting as that sounds, there are some technical hurdles…and three different directions to consider in how to address them:

  1. Rebol as a Guest in JavaScript’s House - This would take the view that Rebol is targeting becoming a relatively small but powerful download, in an era where WebAssembly languages can be loaded into a web app as easily as any other library. Here the predominant way of thinking is that your application structure and libraries are in JavaScript, but some of the work is being done in Rebol. People might want to use PARSE or invent other DSLs easily vs. worrying over how to do the equivalent thing in JS.

  2. JavaScript + Libraries as Display Driver - When you start a project you pick the appropriate set of JavaScript libraries to give the UI experience you want. But the application’s logic starts from a Rebol scripted foundation; getting data off of remote servers is done with read http://..., the UI is built by parsing a Rebol spec and then in the GROUP! rules that trigger it makes JavaScript calls that create windows or widgets or what-have you. Likely the only libRebol calls you make are inside JavaScript service natives to get the arguments or give back results…none of the “app itself” is JavaScript.

  3. Emscripten Rebol as Web Tutorial / Game - It’s possible to ignore broader issues of how Rebol might be used on the web, and just build a teaching tool that demonstrates interesting aspects of the design. I feel like Ren-C’s advances are largely unpublished and inaccessible, despite a huge amount of deep work. There’s little I can do to help that in a society that doesn’t really read even the people with ostensible interest in the topic, cough.

#3 seems the easiest. And sometimes I feel like getting it out there for people to try and discuss might build enough interest to get web-savvy people to come get involved and help. At the same time, the more ambitious the tutorial gets, the more its needs line up with #2, where that has to be done anyway.

#1 and #2 might seem like they’d be approached the same way. But…not exactly. To understand why, you should read and absorb the points in “On Giving libRebol JS more powers than JavaScript”.

The difference is that to accomplish a world where Rebol can “do things its way”, it really has to have more powers than JavaScript. And giving it those powers would involve technologies and trickery that are early alpha in today’s browsers…and may change or be canceled entirely.

It’s always been a risky gamble to try and run ahead of what the world has demonstrated readiness for. People have been trying to do this for some time…remember Google Native Client? Anyone run any Adobe Flex applications lately?

In this case, the technical issue is that Rebol scripts want to have synchronous effects, whatever that may be. Whether it’s a PRINT or an INPUT or a PARSE READ HTTP://…they want to take one step after the other. Making it even more necessary is that closures haven’t really been figured out; so even if we wanted to get callback-happy, things aren’t quite ready for that.

These synchronous behaviors would need to use APIs in JavaScript…whether they be fetch() or callbacks from DOM events, or whatever. Many-if-not-most of these behaviors must be running code on the GUI thread, and can involve several callback steps to fulfill one “synchronous” request.

If while making a synchronous request, Rebol doesn’t want to lose its state–e.g. how far it has counted in a loop or any state of any local variable on the stack–there are two options:

  • The emterpreter…which is a somewhat kludgey concept. You compile your C codebase not into WebAssembly directly, but into a bytecode. This is then run by a function called emterpret(). Since you’re not running on “bare metal” of the browser, it is possible to suspend the bytecode interpreter (hence preserving all of your stack and variables and such as far as C was concerned when you made the “synchronous” request). Then aysnchronous things happen to fulfill the request, and when they’re finished the C code is awoken from “suspended animation” and given the information.

  • Compile your code directly to WebAssembly and run it on a separate thread, e.g. a web worker. For the heap memory of the emterpreter, use a SharedArrayBuffer so that both the GUI thread and the worker thread have access to it. When the worker wants to make a synchronous request of the GUI, it posts a message containing the request and then uses an Atomics.wait on an indicator of when the result has been written to a specific heap location. The GUI does the requested work, writes the heap location, and does an Atomics.notify.

The first demonstration of synchronous input and output in a running script was done with “emterpreter”, but running the emterpreter code on a separate thread. Being on a separate thread is nice because the UI can stay responsive and receive events (such as pushing a cancel button, or doing anything else the worker isn’t needed to accomplish). But it means that JavaScript user natives are fairly useless, because pretty much everything you’d want to do when calling JavaScript from Rebol is on the GUI thread.

But this would really only be suitable for #3. While I took pains to build a transmission method that required mixtures of C code and messaging to accomplish the most basic of I/O, no one else would use a reb-lib.js they had to compile each time they wanted to add a new JavaScript call. And it fundamentally prohibited the mixture of libRebol code with DOM manipulation code. If the emterpreter is used, it has to be moved to the main thread…and accept the various downsides (polling to keep the UI responsive from Rebol evaluator loop, for instance).

SharedArrayBuffer was not pursued in the first test because it had been disabled in all major browsers due to a Spectre-based security flaw. Now–months later–it’s back on in Desktop Chrome…although only with asm.js support by default (you have to do --enable-features='WebAssembly SharedArrayBuffer' to get it to work with WASM, and it might not work even then). A lot of people are invested in it, but when it’s going to come back in other browsers is anyone’s guess. So it is a gamble to try to build on, and not necessarily trivial to work with.


That’s a lot of technical talk for philosophy. But it just makes the point. Where the bets are placed here depend on what the point is. I think I’ve probably said much of what I have to say about option #1 of being used as a casual service library for JavaScript programmers; I don’t think it’ll be ruled out, but I’m not sure that’s worth focusing on too much.


#2

In terms of numbers, I can’t envision many people using #1. But I’d be curious to see if anyone elso has thoughts about this. It would be nice to be able to use Rebol to interact with/query a live DOM managed by JavaScript. I could use Rebol to test or scrape dynamic web front-ends in that scenario-- but I’m unlikely to put Rebol in the execution path by offloading processing to Rebol in a live production app or environment.


#3

I actually found what appears to be a workaround for getting things to work using the “emterpreter” under today’s browsers.

It’s not as efficient as what can be done with SharedArrayBuffer+threads+direct-to-WASM compilation. But the good news is that it seems that the API surface will be similar. The capabilities might be more limited, but not so limited as to not be usable. I believe the extra capabilities can be offered as “perks” if you happen to be willing to compile with more aggressive technology

As a result, the merging of #2 and #3 has already started, which can be seen here:

http://hostilefork.com/media/shared/replpad-js/