UPDATE 2022: This thread covers issues that are also summarized in Shades of Distinction of Non-Valued Intents. Ultimately these should be refactored into one sort of "user guide" covering the modern state, and then one that helps explain the history.
To try and make this 2019 thread semi-comprehensible in modern terminology, I've sanitized the history to make it easier to absorb. e.g. NULL was initially called "void"...but for simplicity let's pretend it was just always called null. And for a time TRASH used the name "void" and for a time reused "none", so I've retconned it as if it was always trash. I'll try not to interrupt the flow by calling out such points inline.
Rebol historically had two "unit types": NONE! and UNSET!. Instances of both were values that could be put into blocks.
Ren-C started out with two parallel types: a simple renaming of NONE! called BLANK! (to match its updated appearance as _), and NULL (which I initially conceived as being a purified form of UNSET! which could not be put in a block).
The original NULL was "ornery" like an UNSET!. It was neither true nor false (caused an error in conditionals)...and it was the same state used to poison words in the user context against misspellings and cause errors. With NULL being so mean, BLANK! was the preferred state for "soft failures"...it was still the outcome of a failed ANY or ALL, or a failed FIND.
But as the point was rigorous technical consistency, something like a failed SELECT would distinguish returning BLANK! from NULL.
>> select [a 10 b 20 c _] 'c
== _
>> select [a 10 b 20] 'c
; null
This provided an important leg to stand on for the operations that needed it (crucial to those trying to write trustworthy mezzanines). While casual users might not have cared or been able to work around it, writing usermode code that could be reliable was quite difficult without routines that were able to make this distinction. Too many things had to be expressed as natives, otherwise the usermode forms to be "correct" would be circuitous and overworked.
Along with this, failed conditionals were made to return NULL too. This provided a well-reasoned form of the pleasing behavior that made it into my non-negotiables list:
>> compose [<a> (if false [<b>]) <c>]
== [<a> <c>]
There was no ambiguity as there was in Rebol2 (or as continues to be today in Red). They don't do my "non-negotiable" behavior, and has no way to put an UNSET! value into a block with COMPOSE...even though it's a legitimate desire, and you will get one with REDUCE:
red>> data: compose/only [(none) (do []) 1020]
== [none 1020]
red>> data: reduce [(none) (do []) 1020]
== [none unset 1020]
red>> compose [<a> (if false [<b>]) <c>]
== [<a> none <c>] ; as with Rebol2, R3-Alpha, etc.
NULL's Prickliness Runs Up Against a Growing Popularity
As mentioned: by not being able to be put in blocks, NULL became the clear "right" mechanical answer to things like a failed SELECT. But it caused some friction:
>> if x: select [a 10 b 20] 'c [print "found C"]
** Error: cannot assign x with null (use SET/ANY)
Even if you had been able to assign null variables with plain SET-WORD! in those days, it would not be conditionally true nor false. You'd have to write something like if value? x: select ... or if x: try select ... or if try x: select ...
The rise of the coolness of ELSE also made it tempting to use NULL in more and more places. Those sites where BLANK! had seemed "good enough" since they didn't technically need to distinguish "absence of value" were not working with ELSE: ANY, ALL, FIND. Attempts to reason about why or how ELSE could respond to BLANK! in these cases fell apart--and not for lack of trying. This gave way to the idea of a universal protocol of "soft failure" being the returning of NULL.
NULL was seeming less like the pure form of UNSET!, but more like the pure form of NONE!. Its role in the API as actually translating to C's NULL (pointer value 0) became a critical design point.
The writing seemed to be on the wall that this non-valued state had to become conditionally false. Otherwise it would break every piece of historical code like:
if any [...] [...]
if find [...] ... [...]
if all [...]
This would start developing tics like:
if try any [...] [...]
if value? find [...] ... [...]
if ? all [...] ; one proposed synonym for VALUE?, still a "tic"
NULL Becomes Falsey, TRASH Becomes the New Meanie
It felt like Ren-C was offering rigor for who wanted it. You now knew when something really didn't select an item out of a block. All functions had a way of returning something that really couldn't mean any value you'd want to append anywhere.
But with a popular NULL that required GET-WORD! access to read it, the illusion of greater safety was starting to slip. :my-mispeled-variable was NULL too.
A path of reasoning led to the argument of resurrecting a state which was not NULL that would be prickly. That took on the name TRASH. It's more or less like an UNSET!, but terminologically makes more sense...because variables are tested for unsetness, not values themselves.
TRASH became a preferred choice for when branches accidentally became null. "trashification" replaced the previous "blankification" which was harder to catch anything unexpected happened, because blanks were so innocuous:
>> if false [null]
; null
>> if true [null]
; trash
Then NULL Stopped Causing Errors When Read via WORD!
This brings things to a point where the "soft failures" of NULL are nearly as quiet as the "soft failure" of NONE! was historically. The key difference is that NULL can't be put in blocks.
Hence it's a good choice for variables which you want to easily test if they contain a meaningful value or not, but avoid accidentally treating as if they are meaningful by getting appended to blocks.
(Shixin had particularly harrowing experiences in Rebmake when he used BLANK! for yet-to-be-assigned variables, because they were always accidentally winding up in collections. It was easier to filter the blanks out at arbitrary moments than it was to find all the places where they could be accidentally appended.)
It started back with feeling I must be able to say