Joe Marshall on Rebol Binding

Circa 2008, this is what Joe Marshall wrote about Rebol Binding in a comment on a blog called "Arcane Sentiment".

The binding model I used for Rebol 1.0 was basically a standard implementation of lexical binding. A lexical environment was carried around by the interpreter and when values were needed they were looked up. The only twist was that Carl wanted symbol instances to have lexical scope. So suppose you had this rebol function:

func [a] [ return 'a ]

The symbol that is returned can be evaluated in any context and it will return the value that was passed in to func. The trick was to close over the symbol. I had to tweak some of the other symbol routines to deal with this idea of closed over symbols, but it worked. (I don't think it was a good idea, but it was easy to implement.)

He wanted it to be the case that if you forced the evaluation of a symbol or block that you'd get the lexically scoped value. So you could return a block as a list of tokens and then call `do' on it and run it as if it were a delayed value. That's an interesting idea, but extending it to symbols themselves is probably going too far.

Carl never seemed to `get' how lexical environments work. The shape of the environment is constant from invocation to invocation, and this is how you get lexical addressing, but the instance of the environment changes.

When Carl ditched my code he went back to his original plan of allocating a binding cell for each lexical mention of the symbol in the source text. Imagine your source is like a Christmas calendar with the chocolate goodies behind each date. Behind each binding location is a box where the value is stored. This model has bizarre semantics. The first (and most obvious) problem is that it isn't re-entrant. If you recursively call a function, you'll end up smashing the values that are there for the outer invocation. The first release of Rebol 2.0 had this bug.

Carl patched this up with the following hack. If a cell is unbound, then binding it causes it to be assigned a value. If the cell already has a value, that value is saved away on the stack while the function is called, and then restored to the cell when the function returns. That fixes the re-entrancy problem, but you still have other issues.

The mechanism of saving the old binding away on the stack is plain-old bog-simple shallow dynamic scoping. But because Carl has a value cell for each binding site, rather than a single global cell, you only see the dynamic binding effect within a single function at a time. This makes it less likely that you'll suffer from inadvertant variable capture, but it doesn't eliminate it completely.

Carl's hack of leaving the previous binding in place if the cell was unbound before gives you a strange effect that variables can be used for some time after they are bound. Unfortunately, if a call to the routine that bound them occurs, the value gets smashed. This is `indefinite extent' in the truest sense --- you simply can't tell how long the variable will retain its value.

So Carl's binding methodology is a weird cross behind static binding, where each formal parameter has its own value cell, and shallow dynamic binding where the value is saved on the stack when a function is re-entered.

Carl's implementation has one other weird feature. If you copy a block of code, you also end up copying the value cells that are associated with the bindings in the code. Some rather enterprising Rebolers have used this trick to implement a `poor-man's lexical binding' by unsharing the dynamically bound value cells before leaving a function and thus getting the value to persist.

I've given my heavy criticism of R3-Alpha's binding model, and it's hard to imagine that it used to be even worse. But it sounds like it was.

As demonstrated by my writeups like "Rebol and Scopes: Well, Why Not?"... I am looking under every rock for possible ways that a model of binding in Rebol could be chosen which might lack some elements of rigor but that can still be useful (or "interesting", for some definition of interesting). So I wanted to see if there might be anything to mine out of Joe's opinions.

I don't think there's anything particularly new to be covered in Joe's opinions that I haven't already summarized about the Lisp/Scheme point of view... that Rebol is sloppy and slow, and tries to let you do more weird eval operations when the whole culture has been trying to minimize the use of arbitrary eval at all:

Lisps, Kernel, Clojure: limits of "Code is Data and Data is Code"

But I do think there are interesting things to see if you're not going to be entrenched too deeply in your idea of what a homoiconic language needs to be. Of course I think most of the non-trivially interesting things in the medium are Ren-C-isms... but they owe their existence to having meditated on questions that arose from this weirder/looser direction of working with code.

A post was split to a new topic: What if Blocks Had Scopes (But Not Individual Words)?