2018 Retrospective: Elevating the Art

It seems like only a year ago that I was in a hotel in San Antonio doing a write-up that summarized Ren-C from its beginnings in early 2015, up until January 2018.

That's because it was a year ago. Here we are again. Whether you're here frequently, or only poke in when you can find the time...Happy New Year to you!!!


I joked the other day to someone about why I worked on Rebol. Said I: "well, Forth was just TOO popular. It wasn't cool anymore once everyone was doing it. I wanted to go more indie/underground, away from all those crowds." :stuck_out_tongue:

But on the positive side of being such a small group, I hope you feel like you're getting to play an important part in defining something you can be proud of--that makes you happy and makes you think--that will hopefully bring that same feeling to others one day. And you are making a difference. You're making a difference to me (hopefully it's bigger than just that, but I can promise it's at least that!)


First thing's first: a Vision Statement and Motto Re-cap

In film school they emphasized the importance of finding a short essential summary of your project. When people wanted to make the movie "Alien", they were pitching it around Hollywood simply as "Jaws...IN SPACE". Having an almost cartoonishly short vision statement was a touchstone for everyone working on it to be on the same page; what defines success, how will we even know if we did succeed.

So I've dug my heels in on "The Minecraft of Programming". Because I want people to feel that sense of uniformity and texture. A visceral relationship between you and the evaluator that feels like a wide open world, with all the satisfaction of LEGO back when LEGO cared about design or a spirograph. (It was actually DocKimbel who sent me that LEGO ad, so there should be some level at which we are all on the same page about this kind of thing.)

So that's my version of a vision statement. I don't think this is at odds with Carl's "putting the personal back in personal computing" phraseology. It's complementary.

But I noticed also this year, that my catch-phrase from circa 2015..."Modify With Confidence", wasn't something I was saying anymore. But I don't say it for the best of reasons: we take it for granted that the interpreter can adapt and change without exploding.

So moving on...the new motto I've used in 2018 is "Elevating the Art".

The tail end of 2017 introduced a fledgling concept called "invisibles". I was egged on by @MarkEye to implement something I'd flirted with as a concept to make COMMENTs evaluator-neutral. Little did I know it would open doors to solutions to problems that seemed nothing to do with commenting. When you looked closer you realized the mechanism was generic and extremely powerful, it is a gift that keeps giving.

I bring up invisibles as a watershed event--but not just for their functionality. Also that it was profound and could happen overnight. A conversation in chat was followed not by more weeks of chat and then nothing. It went from thought to action immediately.

2018 followed suit, with features that moved at the speed of thought. I said that Ren-C's development was planned so it could speed up vs. slow down. Our limitations are the limits of our ability to fully articulate what we want...not the limits of a codebase that had been stretched too far beyond its means.

Things would "just happen":

I don't want to keep going with a blow-by-blow of every interesting thing that happened, just read the posts or the commit log. But there's a pattern here. And it's a pattern that is showing no signs of slowing: just last week, I came up with a scheme to implement "lit bit", the arbitrary level of escaping per value. (not quite finished, but it's remarkably close.)

Time and again, things that once perhaps we could talk about--but have no clue how it could be done--those things are being done what seems like overnight. I swear the changes are elevating the art of programming in Rebol. Once people start using these features, they will realize they won't want to go back.

To me, the most interesting things that happened were things I didn't see coming. So I wanted to talk about how well that's gone before hitting some of the bullet points from last year's post. But, those bullet points were written down, so let's go over them.

libRebol

Progress here has been very notable. I don't think there's any C-based API out there quite like it, letting you mix up values and strings so smoothly.

I've kind of outdone myself. It's actually giving me a bit (just a bit) of empathy for those who are skeptics of C++, because just about every positive feature of RenCpp has been rethought in plain old C...and done even better. The limitations forced it to a kind of simplicity and elegance that I think is going to please people.

Some big cloudy questions have been hammered out...and the biggest thing was engineering a solution to JavaScript promises in order to get the illusion of synchronous interaction with the browser DOM. We have "javascript natives" now...functions whose specs are BLOCK!s and whose bodies are TEXT! of JS code using the API, including the "AWAITER" for simulated synchronous-ness.

UPDATE: A WASM-based web console has been implemented, and is now considered the most critical build for the near future!

The weakest areas are in exception handling and throws. Part of what makes that hard is that it's an intrinsically hard problem in languages. But I want to see it get shored up so we're at least no worse than any other language about this.

UTF-8 Everywhere

While the branch has only been partially committed with some of the groundwork for it, I actually got rather far with this. There are some hitches regarding what it does to performance in PARSE, that I want to get straight.

Every time I have to merge this branch it's a pain. I'd like to get it committed, and maybe I can if we are willing to say that during a PARSE, the only way you can modify the input series is if you use parse instructions (CHANGE, INSERT, REMOVE) to do it, vs. code in a GROUP! changing it arbitrarily. We need to talk about that.

Point is: Yes, it's still happening. And the API has been strongly influenced by experience gained developing it so far.

Derived Binding

Unless someone tells me otherwise, it seems to work. Code that would be intractable in Rebol2/R3-Alpha/Red can be quite zippy. If you want to put your head around the following, be my guest:

o-big: make object! collect [
    repeat n 256 [
        ;
        ; var-1: 1
        ; var-2: 2
        ; ...
        ; var-256: 256
        ;
        keep compose/only [
            (to set-word! unspaced ["var-" n]) (n)
        ]
    ]
    repeat n 256 [
        ;
        ; fun-1: method [] [var-1]
        ; fun-2: method [] [var-1 + var-2]
        ; ...
        ; fun-256: method [] [var-1 + var-2 ... + var-256]
        ;
        keep compose/only [
            (to set-word! unspaced ["meth-" n]) method [] (collect [
                keep 'var-1
                repeat i n - 1 [
                    keep compose [
                        + (to word! unspaced ["var-" i + 1])
                    ]
                ]
            ])
        ]
    ]
]

repeat i 2048 [
    derived: make o-big [var-1: 100000 + i]
    if derived/meth-255 <> 132639 + i [
        fail "This should not happen."
    ]
]

print "Success!"

The techniques that make this tractable are very clever--and are another example of the "art being elevated". And I hope the mechanics open doors to solutions to other kinds of problems...like threading context through the stack for parse rules, etc.

Rebmu

Honestly, Rebmu didn't get the attention I'd wanted to give it. You'd think it would be the "fun" thing--so it would be like goofing off, and I'd do it before other more "serious" work. But in truth, every time I get going on a feature there it shifts my attention back to developing some feature on the interpreter.

Bits of it feed back a little at a time. For instance, there's some interesting functionality in DOES now that grabs the word after it and specializes that function:

x: 1
foo: does catch [
     if x = 1 [throw <one>]
     print "Foo!"
     throw <other>
]

>> foo
<one>

>> x: 2

>> foo
Foo!
<other>

That came directly from work on Rebmu. Maybe this seems like a frivolous way to avoid one pair of brackets (e.g. two less characters than does [catch [...]]). But as you see with it being used with CATCH here, it actually eases the burden of DOES not having its own RETURN. It makes that more palatable, and there's good reasons why it shouldn't have its own return.

So I haven't forgotten about Rebmu--these points feed back into the system design. I'll try and set aside more time for it in 2019--it's one of the things I want you to be able to boot up easily in the browser console.

Debugging

Sadly, stepwise debugging didn't come online this year. :bug:

It's critical that whatever methodology the debugger uses, it needs to be able to reuse 90% of it based on whether it's in the browser or in a native console. I don't want to write two debuggers. :-/ Pushing us a little further down the road was one important accomplishment: the console has been factored into an extension. That's good news, as one of the big debugger issues is how to reuse the console in a nested way when a breakpoint happens.

I want to see the debugger and console and such be in usermode as much as possible, so I'm trying to fit the pieces together so that can be the case. Rushing it and doing it wrong wouldn't be very prudent. But there's enough pieces of it there that are close to working that the concerns it brings up are always in mind.

TLS 1.2

This wasn't on the list, and I'd really hoped someone else would do it. But the payment processing association deprecated TLS 1.0 this year, threatening any website that still used 1.0 with a negative security rating score. That created an extremely strong incentive for people to update, and it was getting to where Rebol couldn't talk to very many sites anymore.

So I read the darn crypto spec and patched it up. Super special thanks to @BlackATTR for sponsoring the work, making me feel less grumpy about having had to do it. :slight_smile: @iArnold chipped in a few dollars as well.

... and what will 2019 be ... ? BETA/ONE

I don't make promises often (ever?) but here we go: Beta/One will release in 2019.

Clearly it won't be this month. And I actually could have a lot of non-Rebol things to do in Q1, so who knows exactly when.

But we've got to get a demo/tutorial online that can be selling itself 24 hours a day...worldwide. That's the deliverable, and this is the year for it. Anything that anyone can do to help get there would be greatly appreciated! And one of the biggest things you can do is help think through things right here, on this forum (or in chat).

Happy 2019! Let's do this!

6 Likes

W00t!! Great year, transformative progress! Bravissimo @hostilefork!

Let’s see what kind of next-level art, minekraftwerk we can make happen for Beta/One in 2019!!

3 Likes

Best wishes for all, and this will definitely be something good.
Personally I am getting up to understanding a few more things and there also will be progress alongside this.
I will keep on to continue my (moral) support!

1 Like

Awesome job as usual HF. Really looking forward to Beta/One.
Will try to be both more supportive and more productive in 2019.
(May not be your top priority, but I have a huge interest in using PARSE for lexing.)

3 Likes

TCC integration has been advancing and we are probably not far from a bootstrap using only that (a tcc-powered Rebol that is the only tool you'd need on disk to build Rebol--no GNU make--no nothing else)...if that interests you, as well...more hands would help.

2 Likes

Duper awesome, I am even MORE interested in tcc-only. Dependency minimalist, me. I hope to be able to contribute in this direction very much, thanks!

3 Likes