What will Rebol on the Web Look Like?

Domains Web
1

First: We had emscripten as a black box %r3.js, which you could give a string to, and it would evaluate it and give a string back.

Second: We had the variadic-based libRebol manifesting as JavaScript. It's important for making more complex interoperability, with API handles that offer certainty and can hold references over time...instead of guessing by picking apart strings with no running concept of identity.

Third: We now have "JavaScript user natives", a category of ACTION! whose spec is a block of Rebol code, and whose body is a string of JavaScript. These can use the aforementioned libRebol to dissect their arguments into JavaScript values, and then produce return results back as Rebol again.

With these pieces in hand, we seem prepared to have a web page where JavaScript calls into Rebol...and then Rebol turns around and brings about actions in the JavaScript world. As exciting as that sounds, there are some technical hurdles...and three different directions to consider in how to address them:

  1. Rebol as a Guest in JavaScript's House - This would take the view that Rebol is targeting becoming a relatively small but powerful download, in an era where WebAssembly languages can be loaded into a web app as easily as any other library. Here the predominant way of thinking is that your application structure and libraries are in JavaScript, but some of the work is being done in Rebol. People might want to use PARSE or invent other DSLs easily vs. worrying over how to do the equivalent thing in JS.

  2. JavaScript + Libraries as Display Driver - When you start a project you pick the appropriate set of JavaScript libraries to give the UI experience you want. But the application's logic starts from a Rebol scripted foundation; getting data off of remote servers is done with read http://..., the UI is built by parsing a Rebol spec and then in the GROUP! rules that trigger it makes JavaScript calls that create windows or widgets or what-have you. Likely the only libRebol calls you make are inside JavaScript service natives to get the arguments or give back results...none of the "app itself" is JavaScript.

  3. Emscripten Rebol as Web Tutorial / Game - It's possible to ignore broader issues of how Rebol might be used on the web, and just build a teaching tool that demonstrates interesting aspects of the design. I feel like Ren-C's advances are largely unpublished and inaccessible, despite a huge amount of deep work. There's little I can do to help that in a society that doesn't really read even the people with ostensible interest in the topic, cough.

#3 seems the easiest. And sometimes I feel like getting it out there for people to try and discuss might build enough interest to get web-savvy people to come get involved and help. At the same time, the more ambitious the tutorial gets, the more its needs line up with #2, where that has to be done anyway.

#1 and #2 might seem like they'd be approached the same way. But...not exactly. To understand why, you should read and absorb the points in "On Giving libRebol JS more powers than JavaScript".

The difference is that to accomplish a world where Rebol can "do things its way", it really has to have more powers than JavaScript. And giving it those powers would involve technologies and trickery that are early alpha in today's browsers...and may change or be canceled entirely.

It's always been a risky gamble to try and run ahead of what the world has demonstrated readiness for. People have been trying to do this for some time...remember Google Native Client? Anyone run any Adobe Flex applications lately?

In this case, the technical issue is that Rebol scripts want to have synchronous effects, whatever that may be. Whether it's a PRINT or an INPUT or a PARSE READ HTTP://...they want to take one step after the other. Making it even more necessary is that closures haven't really been figured out; so even if we wanted to get callback-happy, things aren't quite ready for that.

These synchronous behaviors would need to use APIs in JavaScript...whether they be fetch() or callbacks from DOM events, or whatever. Many-if-not-most of these behaviors must be running code on the GUI thread, and can involve several callback steps to fulfill one "synchronous" request.

If while making a synchronous request, Rebol doesn't want to lose its state--e.g. how far it has counted in a loop or any state of any local variable on the stack--there are two options:

  • The emterpreter...which is a somewhat kludgey concept. You compile your C codebase not into WebAssembly directly, but into a bytecode. This is then run by a function called emterpret(). Since you're not running on "bare metal" of the browser, it is possible to suspend the bytecode interpreter (hence preserving all of your stack and variables and such as far as C was concerned when you made the "synchronous" request). Then aysnchronous things happen to fulfill the request, and when they're finished the C code is awoken from "suspended animation" and given the information.

  • Compile your code directly to WebAssembly and run it on a separate thread, e.g. a web worker. For the heap memory of the emterpreter, use a SharedArrayBuffer so that both the GUI thread and the worker thread have access to it. When the worker wants to make a synchronous request of the GUI, it posts a message containing the request and then uses an Atomics.wait on an indicator of when the result has been written to a specific heap location. The GUI does the requested work, writes the heap location, and does an Atomics.notify.

The first demonstration of synchronous input and output in a running script was done with "emterpreter", but running the emterpreter code on a separate thread. Being on a separate thread is nice because the UI can stay responsive and receive events (such as pushing a cancel button, or doing anything else the worker isn't needed to accomplish). But it means that JavaScript user natives are fairly useless, because pretty much everything you'd want to do when calling JavaScript from Rebol is on the GUI thread.

But this would really only be suitable for #3. While I took pains to build a transmission method that required mixtures of C code and messaging to accomplish the most basic of I/O, no one else would use a reb-lib.js they had to compile each time they wanted to add a new JavaScript call. And it fundamentally prohibited the mixture of libRebol code with DOM manipulation code. If the emterpreter is used, it has to be moved to the main thread...and accept the various downsides (polling to keep the UI responsive from Rebol evaluator loop, for instance).

SharedArrayBuffer was not pursued in the first test because it had been disabled in all major browsers due to a Spectre-based security flaw. Now--months later--it's back on in Desktop Chrome...although only with asm.js support by default (you have to do --enable-features='WebAssembly SharedArrayBuffer' to get it to work with WASM, and it might not work even then). A lot of people are invested in it, but when it's going to come back in other browsers is anyone's guess. So it is a gamble to try to build on, and not necessarily trivial to work with.

That's a lot of technical talk for philosophy. But it just makes the point. Where the bets are placed here depend on what the point is. I think I've probably said much of what I have to say about option #1 of being used as a casual service library for JavaScript programmers; I don't think it'll be ruled out, but I'm not sure that's worth focusing on too much.

3 Likes
2

In terms of numbers, I can't envision many people using #1. But I'd be curious to see if anyone elso has thoughts about this. It would be nice to be able to use Rebol to interact with/query a live DOM managed by JavaScript. I could use Rebol to test or scrape dynamic web front-ends in that scenario-- but I'm unlikely to put Rebol in the execution path by offloading processing to Rebol in a live production app or environment.

1 Like
3

I actually found what appears to be a workaround for getting things to work using the "emterpreter" under today's browsers.

It's not as efficient as what can be done with SharedArrayBuffer+threads+direct-to-WASM compilation. But the good news is that it seems that the API surface will be similar. The capabilities might be more limited, but not so limited as to not be usable. I believe the extra capabilities can be offered as "perks" if you happen to be willing to compile with more aggressive technology

As a result, the merging of #2 and #3 has already started, which can be seen here:

http://hostilefork.com/media/shared/replpad-js/

2 Likes
4

I wanted to say that the #2 and #3 merge did indeed go very well, and we no longer have to think of these as separate scenarios.

To recap what that meant: #2 was the idea that people would be able to code applications from a "Rebol Worldview" in the browser. They can use no JavaScript at all if they like (or throw in little bits of "inline JavaScript" when it suits them...e.g. they find there isn't something already bridged in the Rebol routines to do what they want.)

Because we have the baseline of that working, you can write a traditional app with PRINT and ASK TEXT! (formerly INPUT)...also doing READs from URL!s off their own domain (wherever the app is hosted) or from remote domains (if the files support CORS).

That isn't just enough to write a console with, that's enough to run the actual common console extension used by the interpreter itself. Also, @BrianOtto made a game/tutorial demo, and I made a guided interactivity test.

The big question is now about how people will make such apps that make the ReplPad console disappear into the background. I kind of want to punt on that until we know more...so if people could tolerate the idea that their applications are launched from the console I'd like to go that route. We can find various ways of making the URL format that auto-launches your app more palatable.

So what about option 1?

Option 1 was the idea that someone's web page is mostly JavaScript, and Rebol sneaks in as an assistant...to do parsing, or other various things.

This isn't impossible, but it's very hard to guarantee that a Rebol script doesn't have any I/O. And if it does have I/O...and there's no ReplPad...what should it do? Write to the JavaScript console? Should any ASK for input be treated as an error?

Imagine you want to do something like:

 var url = "http://hostilefork.com"  ; pretend I enabled CORS :-/
 var title = reb.Spell(
      "parse as text! read", url, 
          "[thru <title> copy title to </title>]",
      "title"
 );

That seems pretty cool. The variadic API even saves you from having to concatenate strings with +, which would make you put awkward spacing in that could be easy to forget ("... read " + url + " [thru..."). It's more readable this way.

But the problem I mentioned is still there: JavaScript cannot synchronously handle a network request like this. It just can't. You have to write something more like:

 var url = "http://hostilefork.com"
 reb.Promise(
      "parse as text! read", url, 
          "[thru <title> copy title to </title>]",
      "title"
 ).then(function(title) {
     ... do stuff with title ...
 );

Or, if you've followed all the rules of your entire stack to the top of your JavaScript code with async functions, you can say:

 var url = "http://hostilefork.com"
 var title = reb.Spell(await reb.Promise(
      "parse as text! read", url, 
          "[thru <title> copy title to </title>]",
      "title"
 ));

We can probably make this slightly better as title = await reb.SpellPromise(...) (or PromiseSpell? SpellAsync?)

But less important than the naming convention is the fundamental question: how did the caller know whether or not the Rebol code they called was going to be able to complete without doing something that requires yield processing in the JavaScript callsite?

This is one splendid argument for why Option #2 is a way forward for authoring code on the web. Because when you're not putting arbitrary JavaScript code in the driver's seat, you just don't worry about it. The entire application runs inside of One Giant reb.Promise().

But Would JS Programmers See Value in #1?

I think it's kind of odious to imagine every single thing you'd want to do with Rebol to require asynchronous handling. But the design of the API is pretty clever IMO, and it means you don't necessarily have to do a bunch of nitpicky API calls to get the data you want...you can usually tie it all up with one single call where the Rebol code has pared out everything you need...and you get the JS result you want. If we could transform BLOCK!s of data items to a JavaScript array (reb.UnboxArrayPromise()) then this could be even more true.

I also pointed out in the post "On Giving libRebol.JS More Powers Than JavaScript", I don't think JS programmers will see this as any particularly bad thing. They just plug along.

So if every single entry point to calling the Rebol interpreter was a promise, they'd probably like it. More time to let the main loop run! In fact, they're getting a powerful WASM web worker experience for free...it's like they can do background processing and still paint and process input. Hence I may be overthinking this, and we should just tell people using #1 to always use the Promise APIs because the non-Promise ones might error if any asynchronous requirement creeps in.

FWIW: This Is Pretty Well Understood Now, and Going Great...we just need to have people understand why it's great.

2 Likes
5

I think you answered your own question, and rather nicely at that. I'd surge forward down that path. Note for docs/tutorials.

1 Like
6

Probably! Biggest mental jam I've overcome is that I'm not as afraid of having JS programmers treat the Rebol interpreter instance as some kind of "server", that they're always communicating with asynchronously. There's enough trickery that you can fold up a lot of work into one request vs. making a lot of little ones...that's part of the intent and benefit of the variadic design. (Again, compare, say libRed :-/)

As I say: the "have to dispatch to a worker and get called back" could be marketed as a benefit ("hey, we've done all the hard work of multithreading for you, just include our boot code!")

By that token... the example should not be reb.Spell(<code>) <reaction> because of all the things that could go wrong if you need asynchronous services. It should be reb.PromiseSpell(<code>).then(<reaction>) which won't fall down if the code wants to do something interesting.

There's some tricks where we might cleanly stylize the pattern for the APIs like this:

reb.Promise.Spell(...)
reb.Promise.Value(...)
reb.Promise.Value.Q(...)  // maybe?

We can save the non-promise APIs for experts...after explaining the various terms upon which they can be used. They can be quick and dirty if you think you're sure things aren't going to be async. But like I say, you don't quite know...what if you turn on tracing and you start printing some spew when an operation runs that you didn't think printed? (e.g. ADD suddenly triggers output to say YOU JUST RAN ADD!)

With great power comes great... asynchronicity.

1 Like
7

A few thoughts (merely that):

This was one situation where Ports/Schemes provide options. Though it wasn't published, Rebol 2 had a console scheme that functioned for both INPUT/OUTPUT ports while in Repl mode and switched to a pseudo file mode when invoked as a shell script. A JavaScript console scheme could be used as an OUTPUT port for non-Repl, or could switch it out for a scheme that stores output in Local Storage. I'm not quite sure how one would go about writing an INPUT compatible scheme—this was never implemented in Rebol 3. However it works, WRITE-STDOUT should die (at least as a user-facing construct).

The problem as I see it is that there is no way with Option #2 to avoid dipping into JavaScript as a means to interacting with the DOM/Events. That is the wrinkle that needs papered over.

Events seem to me to be key, but what form do they come in? Whether or not they from the DOM or from requesting a READ on a remote resource. Otherwise #1 is tempting insomuch as to make JavaScript a dumb dispatcher with some type of Rebolizing layer to make it more palatable.

I know this is an old thread and I'm likely not making any profound observations, it's really just first impressions from digging under the replpad hood and rereading (hopefully) better understanding the stakes.

8

Very nice. I created a kind of matrix effect.

>> pwd
== http://hostilefork.com/media/shared/replpad-js/

>> cd ..
== %..
>> pwd
== %..
>> ls
239            187            191            60             33             
68             79             67             84             89             
80             69             32             104            116            
109            108            62             10             60             
104            116            109            108            32             
108            97             110            103            61    
and so on...
1 Like
9

So far, I mostly use Rebol to call JS libraries to do things such as bring up a chess GUI, or docx templating. Or, just ignore JS and use it as an accessible Rebol console. Having persistent storage is a great move forwards.