A lot of influencing factors have changed over time. Before making another major change, I want to run through the history of reasoning and look for any gaps, given modern understandings...in case some turn was taken that we would not take again given what kinds of things we know now.
To that extent, I'm going to somewhat "sanitize" the history to make it easier to absorb. e.g. NULL was initially called void...but for simplicity let's pretend it was just always called null. And there was no modern sense of TRY, it was called TO-VALUE. I'll try not to interrupt the flow by calling out such points inline.
Rebol historically had two "unit types": NONE! and UNSET!. Instances of both were values that could be put into blocks.
Ren-C started out with two parallel types: a simple renaming of NONE! called BLANK! (to match its updated appearance as
_), and NULL (a purified form of UNSET! which could not be put in a block).
The original NULL was "ornery" like an UNSET!. It was neither true nor false (caused an error in conditionals)...and it was the same state used to poison words in the user context against misspellings and cause errors. With NULL being so mean, BLANK! was the preferred state for "soft failures"...it was still the outcome of a failed ANY or ALL, or a failed FIND.
But as the point was rigorous technical consistency, something like a failed SELECT would distinguish returning BLANK! from NULL.
>> select [a 10 b 20 c _] 'c == _ >> select [a 10 b 20] 'c ; null
This provided an important leg to stand on for the operations that needed it (crucial to those trying to write trustworthy mezzanines). While casual users might not have cared or been able to work around it, writing usermode code that could be reliable was quite difficult without routines that were able to make this distinction. Too many things had to be expressed as natives, otherwise the usermode forms to be "correct" would be circuitous and overworked.
Along with this, failed conditionals were made to return NULL too. This provided a well-reasoned form of the pleasing behavior that made it into my non-negotiables list:
>> compose [<a> (if false [<b>]) <c>] == [<a> <c>]
There was no ambiguity as there was in Rebol2 (or as continues to be today in Red). They don't do my "non-negotiable" behavior, and has no way to put an UNSET! value into a block with COMPOSE...even though it's a legitimate desire, and you will get one with REDUCE:
red>> data: compose/only [(none) (do ) 1020] == [none 1020] red>> data: reduce [(none) (do ) 1020] == [none unset 1020] red>> compose [<a> (if false [<b>]) <c>] == [<a> none <c>] ; as with Rebol2, R3-Alpha, etc.
NULL's Prickliness Runs Up Against a Growing Popularity
As mentioned: by not being able to be put in blocks, NULL became the clear "right" mechanical answer to things like a failed SELECT. But it caused some friction:
>> if x: select [a 10 b 20] 'c [print "found C"] ** Error: cannot assign x with null (use SET/ANY)
Even if you had been able to assign null variables with plain SET-WORD! in those days, it would not be conditionally true nor false. You'd have to write something like if value? x: select ... or if x: try select ... or if try x: select ...
The rise of the coolness of ELSE also made it tempting to use NULL in more and more places. Those sites where BLANK! had seemed "good enough" since they didn't technically need to distinguish "absence of value" were not working with ELSE: ANY, ALL, FIND. Attempts to reason about why or how ELSE could respond to BLANK! in these cases fell apart--and not for lack of trying. This gave way to the idea of a universal protocol of "soft failure" being the returning of NULL.
NULL was seeming less like the pure form of UNSET!, but more like the pure form of NONE!. Its role in the API as actually translating to C's NULL (pointer value 0) became a critical design point.
The writing seemed to be on the wall that this non-valued state had to become conditionally false. Otherwise it would break every piece of historical code like:
if any [...] [...] if find [...] ... [...] if all [...]
This would start developing tics like:
if try any [...] [...] if value? find [...] ... [...] if ? all [...] ; one proposed synonym for VALUE?, still a "tic"
NULL Becomes Falsey, VOID! Becomes the New Meanie
It felt like Ren-C was offering rigor for who wanted it. You now knew when something really didn't select an item out of a block. All functions had a way of returning something that really couldn't mean any value you'd want to append anywhere.
But with a popular NULL that required GET-WORD! access to read it, the illusion of greater safety was starting to slip. :my-mispeled-variable was NULL too.
A path of reasoning led to the argument of resurrecting an ANY-VALUE! type which was not NULL that would be prickly. That's been called VOID!. It's more or less like an UNSET!, but terminologically makes more sense. I put it thusly:
"I like VOID because it's the return value of functions that aren't supposed to have any usable result. But more accurately, think of a paper check that has VOID written on it. You can still hand it to people, and you can get in trouble if you try to cash it. But you can't cause a problem by trying to cash a NULL check because there's no paper and nothing written on it--it is the literal absence of a check."
VOID! became a preferred choice for when branches accidentally became null (voidification). It replaced the previous "blankification" which was harder to catch anything unexpected happened, because blanks were so innocuous:
>> if false [null] ; null >> if true [null] == #[void]
Where This Stands Today
If the latest proposed change is adopted, reading NULLs out of a plain WORD! does not cause an error. VOID! more or less makes what was considered error-worthy in Rebol2 the same. And we come to a point where the "soft failures" of NULL are nearly as quiet as the "soft failure" of NONE! was historically.
"All we get" for Ren-C's trouble is more rigor for those who want it.
But that's a pretty pessimistic way of looking at it, because the rigor was the original motivation: to be able to write short clear usermode routines where you could be confident they were as correct as you could make a native. And there's actually a lot of ways in which the nulls raise errors when passed as parameters to things that don't take NULL (but which have behaviors for BLANK!). Yet under the proposal all refinements are implicitly optional arguments, hence NULL says nevermind. I feel good if that's what you meant, but uneasy if it wasn't.
One thing I've been thinking is that if NULLs have been "defanged" so much that you can assign and use them, that we need some short way of throwing in errors. Today we have REALLY, but I don't care for the name.
>> append x really select [a 10 b 20] 'c ** Error: REALLY did not succeed, it got a NULL ** Near: really ** select [a 10 ... (blah blah
I've started a separate thread to talk about renaming REALLY. But getting a better name for this is kind of my best idea so far of how to address a world where soft failures are accepted by every refinement (and some arguments). If your failure is a logic failure--e.g. something semantically really wrong happened which was not obvious from the routine itself (like FIND not being able to find something is a known result), then don't use NULL. It's too nice now. Use VOID! or raise an error.
I'll probably run with the null change a while before pushing it on others, to see what other thoughts it provokes. But really--any comments help!