Ren-C is Now on the Clouds ☁️ with WASI

Note: WASI => WebAssembly System Interface

We've been running in a browser for a while. And over time, as that host environment has gotten better...we've been getting better, too. (It can sometimes pay to be an early adopter and co-evolve with a platform.)

After several long days of nuanced work, I've been able to take another step...running in a WasmEdge VM!

Unlike the web version, this has the command-line processing built into it. So you can pass wasmedge the %r3.wasm file, and say something like --do of the code you want to run:

$ wasmedge r3.wasm --do \
      "print ['** uparse {Hello, Cloud} [thru space across to <end>] '**]"

Then you get your result:

** Cloud **

See for yourself, on the GitHub Workflow that builds it!!!!

"So... er... so what?" (someone might ask)

... WH-

... WH- WHAT YOU DO MEAN, "SO WHAT?" ... YOU, you... IGNORAMUS!

:angry:
...

Oh, all right, well. Fine. Let me try and break down what this is... and what it might mean for Ren-C's future in the cloud space (and elsewhere).


First Let's Talk About "The Dream That Isn't" (yet)

...and that dream is "SERVERLESS"-ness.

  • Picture if you will, the idea that we take %httpd.reb and shoot it into the sun... :rocket: :sun_with_face: ...vowing to never try and write another web server in a Redbol again.

  • INSTEAD we dial the clock back to how people responded to web requests in the 90's: we think of it that each request starts a whole new copy of %r3.exe, which does its work, and exits.

    • ...this really is how systems used to work, back when there were 5 people on the Internet. (Modems were so slow that you couldn't get more than one request a week, anyway.)

    • But in the name of speed, webmastering became a mystical black art of figuring out how to force your PHP or Python or whatever to stay resident inside the Apache server and not exit.

    • The seeping complexity started to require watchdog processes to check to make sure your long-running process wasn't accruing memory across requests, or restarting it if it crashed.

    • Security was also being compromised, because bugs in your code could allow session data to seep between different requests...since the process was long-lived and kept memory open.

  • THE UTOPIA WE ARE BEING SOLD is that IF we package our code as a blob-o-WebAssembly...

    • We will get all the benefits of running fresh every request...

    • ...but with a performance characteristic like if we'd rigged up a way to stay running the whole time.

  • THERE WILL STILL BE A WEBSERVER RUNNING (of course)

    • However: it will be The One True Debugged WebServer (compiled in something fully formal like Rust or Haskell, right?)...so it won't have the same security problems that your dirty Wasm blob would have if it tried to implement a long-running web server itself.

    • It will have lightning fast abilities to spin up and down the Wasm blobs, much faster than an OS shutdown and launch would ever be.

I've Now Made Us One Of Those Wasm Blobs!

Hence we can focus our efforts on the high level logic of how to answer a web request without worrying about the details of writing, running, or administrating a web server.

If you've ever looked at some Platform as a Service (PaaS) like Heroku or AWS and seen a short list of languages supported, that's going to be a thing of the past. Anyone who builds to WebAssembly can be hosted and just pay the price of their storage/CPU/network use.

  • Yesterday we were a black sheep of languages...with a shoddy, slow, and insecure webserver.

  • Tomorrow... we'll still be a black sheep of languages. But one that's on an even playing field for providing higher level logic for processing and serving data on the web.

"Ah! Now I See the What of the So What!"

:smiling_face_with_three_hearts:

It's a quite compelling vision statement, and a lot of people are evangelical (you can tell by how many times people say the term "serverless" at conferences these days).

I'm definitely the sort of person this is an easy sell to. You should choose your battles when it comes to optimization, this is one of my favorite quotes:

"90% of a program's execution time is spent in only 10% of its code. The standard inference from this rule is that programmers should find that 10% of the code and optimize it ... But a second inference is just as important: programmers can deoptimize the other 90% of the code (in order to make it easier to use, maintain, etc.) -- Richard Pattis

So focusing performance research on the spin-up and tear-down sounds 100% right. Over the long run, the factors I mention probably mitigate any "catastrophic slowdown". Your higher level program logic is starting from a clean slate every time. There's no memory fragmentation or leaks, and all those security benefits I mentioned.

Not Just the Cloud, but One Binary For All Desktops

As WebAssembly has proven its mettle on the Web, people have asked why it couldn't be run from a terminal prompt. All you'd need would be a runtime host--simpler than a browser--that provided basic file, I/O, and network services.

If you could do that, you'd be able to have one %r3.wasm file that you could download and use for Linux, Mac, Windows, on the cloud, and in your browser...

...in theory. And that theory is almost practice. (I've remarked on another thread about it so take discussions of the non-cloud applications there.)

Of Course, It's Not All-There-Yet

What about those Wasm blob server providers?

They don't exist yet.

So if you're going to use the serverless methodology in the here and now, you're going to still be in the webmastering game. Only now you're learning how to set up Docker instances and configure bleeding-edge Wasm runtimes.

What about that "One True Webserver?"

Y'know, the webserver you're going to throw out yours for. The one that's bulletproof and supersonic and Utopian--the kind Jesus would have written--the one that's going save us all?

It's NodeJS. :man_facepalming:
at least for WasmEdge on AWS Lambda, for now

So I hope you like setting up NodeJS in Docker.

What about the "Language Agnosticism" in hosting?

The company behind WasmEdge talks a big game, and they have a platform for Functions as a Service (FaaS). But for various reasons this only works with Rust.

Kind of annoyingly, if you read the docs for develop a WasmEdge app you will see it say:

For compiled languages (e.g., C and Rust), WasmEdge WebAssembly provides a safe, secure, isolated, and containerized runtime as opposed to Native Client (NaCl).

Then you will see it list examples for Rust, JavaScript, Go, Swift, AssemblyScript, Kotlin, Grain, Python

Noticeably missing is C. And that's the one we need.

Without C support for things like sockets, our Ren-C Wasm blob cannot make network requests. We can do filesystem operations, but networking is just not there yet.

(The missing C bits relative to the other languages isn't technically WasmEdge's fault...it's just that the libc implementation for Wasi doesn't have some things that the Rust standard library does. I just wish they'd explained the details of how they got the other languages to work built upon Rust...rather than leaving it as an exercise for the reader to delve into it all to figure out.)

But Stay Tuned...

This is a pretty major development, and things are moving fast. I want @gchiu to keep the eyes on the prize here, and we see what we can do to make sure we've got some demos rolled out in this space. As we find ways to bring more features online, I'll let you know.

4 Likes

Rebol has for many years been at the forefront of personal computing so it's good to see it head back that way even it is personal cloud computing.

As far as I understand it, we need to wait until the hooks for WASI are built into the C/C++ compilers used to build Rebol so that not only can we print "EHLO world" but we can also do networking.

2 Likes

We can:

  1. Wait for that to happen (will happen < 1 year, I would bet for certain)

  2. Build a bridge to Rust (where the runtime has WASI networking), and have our r3.wasm built as a Rust project that embeds Ren-C.

    • I like Rust, and want a Rust bridge, Shixin started one that I have only glanced at; I know a lot more Rust now than then.

    • The "weird" Rust toolchain isn't necessarily any weirder than a "weird" clang wired up to target WASI, so on the topic of dependencies it's actually a thought-out thing that has some various virtuosities.

  3. Have me contribute to Wasi-SDK or Emscripten and write it myself:

    • This is a bad way to burn my cycles on something that will happen anyway.

    • The Rust bridge would be a much more forward looking idea

So default is (1), but maybe (2).

2 Likes
  1. means that eventually Rebol users will get access to WASI, but 2. means that Rust users will also get access to Rebol. There's more to be gained from the second choice, especially if it's something you've always wanted to do.
3 Likes

I'm for it. But it's one of those big new project things, that one has to claw one's way through. A lot of unknown unknowns.

"...as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know." --Donald Rumsfeld

But I did crack open the WasmEdge and do that, so, it's not like I can't do new things. It's just that we're at a real limit of carrying capacity for platforms for one person.

2 Likes

Perhaps do some experiments to explore the unknowns