Should append null fail, append BLANK! add nothing unless /ONLY?

The Rebol design concept that used to be discussed under the label "none propagation" has become a system of what I think are pretty good rules:

  • Blank input gives Null output...for non-mutating operations
  • Mutating operations do not accept BLANK! targets as input
  • Null input gives an error for operations that aren't specifically about null handling
  • TRY can be used to "defuse" nulls into blanks

The reason this has merit is because it gives you a "hot potato" in the form of NULL, without going so far as to error in the moment. You get pinpointed error locality where a problem might be, and have an easy mitigation--if you want it--to use TRY to turn the null into a BLANK! and pass through all other results. And there are a host of null-driven operations to let you pick up the slack, e.g. first block else [code].

(I've suggested that if you ever actually wind up having to put a null state in a variable, test for null later with NULL? and act on it, that you probably could have done it better...reacting to the null without ever using it as a stored state. But it's your choice, I guess.)

So this is being applied across the board. first _ is null...just like first [] is.

Consequences for APPEND/INSERT/CHANGE...

These rules affect a current feature of things like APPEND that I doubt many people have tried:

 >> append copy [a b c] null
 == [a b c]

>> append copy [a b c] if false ['d]
== [a b c]

When it was introduced, I thought it made sense. Since NULL is not a value you can put in a block, you don't get confused about what that means. Then since BLANK! is a legitimate value that can appear in a block, you could append it:

 >> append copy [a b c] _
 == [a b c _]

But quietly accepting NULL loses the hot-potato benefit. If you say append foo third block and there is no third element in the block, doesn't that seem like a good time to complain? Rebol code can pretty quickly get hard to read and debug, and wouldn't it be nice if the person who wrote that line could articulate that there may not be a third thing in the block by saying append foo try third block?

This suggests making the default behavior of BLANK! to be to add nothing, and you need an /ONLY to override that. It may sound like a missed opportunity for null to exploit its out-of-band status. But realistically speaking, I don't think purposefully adding blanks is all that common--certainly a lot less common than adding a block as-is, and that requires /ONLY!

No one probably cared (...yet...)

Like I say--most people probably were unaware of this, and so I don't know there's a lot of append block if condition [...] out there today. We might need APPEND/OPT to be null tolerant, though the only reason to use it would be with /ONLY since otherwise you could say append block try if condition [...].

1 Like

So the answer, after delving and thinking on this issue, has come up as NO.

NULLs will continue to append or compose without error.

>> data: copy [a b c]
>> append data null
== [a b c]

>> compose [a (if false [<omitted>]) b]
== [a b]

But if you have a BLANK! in your hand, that is a "normal" value that will be COMPOSE'd in, or APPEND'd, etc.:

>> data: copy [a b c]
>> append data _
== [a b c _]

>> compose [a (_) b]
== [a _ b]

It's your job to OPT something that might be blank, if you want blanks to be turned into NULLs and evaporate.

Sound a bit awkward, given how many routines return BLANK!s when they fail? Well, no worries...

NULLs are taking over a lot of former BLANK! roles

A failed ANY or ALL, a FIND that didn't find anything, a MATCH that didn't take... these will all be returning NULL, and not blank. This is the new return value protocol, where BLANK! is no longer used as a return value meaning "no match". And it will be tolerable because NULLs will be conditionally false.

This gives BLANK!s a renewed purpose, but a purpose for which they should be thought of as "things".

It's a good feeling to pin down this previously oft-speculated-upon question, since it really was inelegant to have to say compose [a (opt any [...]) b], etc. But we need wonder no will be able to omit the OPT because ANY will now return NULL!

1 Like

This now has a fairly strong philosophical defense in NULL, first-class values, and safety.

The way I've started to feel comfortable with this is saying that NULL is what you use when you're talking about a "what". You can use a null to opt-out of that. But you can't use a null to opt-out of a "where". Only BLANK! can do that.

This is a really tough one to think about.

As the rules have shaped up, I often feel this should probably change to requiring an APPEND/ONLY or COMPOSE/ONLY, otherwise blanks should act like nulls. The premise is this:

It's likely better to embrace blank's "nothingness-without-triggering-errors-on-access" as its primary facet, with its ability to be represented in a block as secondary.

It's hard to see a clear answer purely from a nuts-and-bolts point of view. But it just seems like it's more natural. Consider:

  foo: try case [...]
  compose [... (opt foo) ...]

There's arguably a nice aspect to the annotation that it's optional, and you might say the random blanks popping up to tell you that you left off an OPT is a feature and not a bug.

But in practice I'm not feeling that. I didn't invoke the TRY because I wanted to create a thing. I put the TRY there because I wanted to say that my tolerance of nothingness was intentional, and I expect this variable to be defined...but conditionally false.

If I were nervous and felt I needed some kind of additional signaling not to use it in an optional sense, I'd have left off the foo would have been unset. Why should I have to repeat myself by way of an OPT annotation when I use the variable in this casual circumstance? Doesn't it kind of undermine the convenience of having blank in the first place?

I've made the argument that wanting to append literal blanks is probably less common than wanting to append literal blocks, and that takes an /ONLY. You can append a literal blank as easily as append block [_] if you want to, so it's not like there aren't good options for the cases when you need it.

What's tough about it is that it is a slippery slope. This makes one think "oh, well then should refinement arguments go back to being blank when they're not supplied?" And some of these things that sound good then turn around and bite you because conflating values with nulls loses the rigor of null and compromises the features it enabled. :-/

It's definitely a serious consideration to change this. So please speak up with any observations...

1 Like

To put myself on the other side of this statement: both aspects are going to come into play, whether you like it or not. It's not going to be all one way or the other. You're going to sometimes be thinking of a blank as a "thing", and sometimes as a "nothing".

Let's look away from APPEND and COMPOSE for a moment, and think about maps. Today these are different:

my-map/key: _ // sets the key to a blank value
my-map/key: null // removes the key

Here you don't have the option of an /ONLY. So if blanks were treated as nothings, you'd lose the ability to put them into maps with this mechanism. But as long as they are distinct, you have the choice to say either:

my-map/key: thing-that-might-be-blank
my-map/key: opt thing-that-might-be-blank

Going back to APPEND and COMPOSE, the /ONLY is already controlling one aspect: splicing or not. But...

  • ...what if you want to splice if it's a block, and treat it like a literal blank if it isn't?
  • ...what if you want to not splice if it's a block, but treat it as nothing if it's a blank?

When in Doubt, Favor the Mechanically Consistent Behavior

Sometimes things seem annoying when you look at them as individual cases, but you realize their necessity if the big picture is taken into account.

I think needing OPT at the callsite is just one of these necessary evils--that's not even that evil.

For historical perspective: When I initially complained about NONE!s not "vaporizing" in things like COMPOSE, BrianH was very adamant that NONE!s were things. He believed a lot of Rebol's idiomatic behavior involved having empty placeholders in slots of regular record-like structures. So he was generally resistant to using them to opt-out when blocks were being produced.

So I guess this just kind of informs what the system's bias is: Blanks are values first and foremost, and nothings second.

I started this thread talking about the two competing opportunities with NULL and APPEND.

  • The opportunity to have an "escape" out of a value bearing slot to say "no, I actually don't want to append a thing".

    >> append copy [<a> <b> <c>] (if 1 = 1 '[<d> <e>])
    == [<a> <b> <c> <d> <e>]
    >> append copy [<a> <b> <c>] (if 1 = 2 '[<d> <e>])
    == [<a> <b> <c>]
  • The opportunity to catch an error in intent when NULL is an accident.

    >> data: [<a> <b> <c>]
    >> block: [<d> <e>]
    >> append data third block
    ** Error: APPEND doesn't accept NULL as its value argument
    >> append data try third block
    == [<a> <b> <c>]

As evidenced by the back and forth above, I see both sides.

Null Behavior With The ^ Operator Shifts the Balance

There is an interesting property in the new ^ mechanisms that shows a way we could make blanks disappear and still be taking advantage of NULL's unique status.

Here is the property:

 >> absent: null

 >> ^absent
 == null

 >> try absent
 == _

 >> append [a b c] _
 == [a b c]

  >> append [a b c] absent
 ** Error: APPEND doesn't accept NULL as its value argument

  >> append [a b c] try absent
  == [a b c]

  >> append [a b c] ^absent
** Error: APPEND doesn't accept NULL as its value argument

  >> append [a b c] try ^absent
  == [a b c]

Don't see what's special about that yet? Think about how it would work with BLANK! instead.

  >> present: _

  >> append [a b c] present
  == [a b c]

  >> append [a b c] try present
  == [a b c]

  >> append [a b c] ^present
  == [a b c _]

  >> append [a b c] try ^present
  == [a b c _]

The magic of TRY here is that you can use the TRY on the outside of the literalizing operator and not wind up putting in blanks that get quoted.

So...Is This The Answer?

It's now a strong argument to say that APPEND and friends act with /ONLY semantics on QUOTED!, splice BLOCK!, do nothing on BLANK!, error on NULL.

>> append [a b c] "text"
== [a b c "text"]

>> append [a b c] <tag>
== [a b c <tag>]

>> append [a b c] [d e f]
== [a b c d e f]

>> append [a b c] _
== [a b c]

>> append [a b c] ^ _
== [a b c _]

>> append [a b c] [_]
== [a b c _]

>> append [a b c] null
** Error: APPEND doesn't accept NULL as its value argument

With quoting in reach as easily as ^var instead of var, or ^ some expression instead of some expression... it spins my view toward where we'd be instead wasting blank if we didn't take advantage of its unique status as being a thing you can put in a block and opt out of an operation with.

Then it will error unless you use QUOTE or ^:

>> append [a b c] 'd
** Error: Use QUOTE or ^ to literalize evaluative types with APPEND

>> append/only [a b c] 'd
== [a b c d]

>> append [a b c] ^ 'd
== [a b c d]

>> append [a b c] ^ ''d
== [a b c d]

>> append [a b c] [d]
== [a b c d]

>> append [a b c] ['d]
== [a b c 'd]

>> var: the 'd
>> append [a b c] ^var
== [a b c 'd]

One way of looking at this is that evaluative types are the ones that have the most to lose by conflating their quoted forms with their non-quoted forms on accident.

Another way of looking at it is that inert types exist in other languages as being able to be added to arrays, so making that simple gives parity on the common feature...while the domain of code building costs you an ^. Moreover, I think changing COMPOSE so that it leverages null and uses (( )) for splicing gives a powerful alternative for building up blocks of code.

It's certainly the best proposal so far. My confidence is fairly high it's about as good as this is going to get.

1 Like

The answer is...

Use VOID to mean append nothing, and NULL gets an Error

That's the conclusion I came to with COMPOSE and DELIMIT, and I think it has turned out to be the right one. It has eliminated many casual errors there, and provided new features from responding to the NULL errors definitionally.

So why not make APPEND 100% compatible?

 >> var: null

>> append [a b c] var
** Error: APPEND doesn't take NULL (use MAYBE if intentional)

>> append [a b c] maybe null
== [a b c]

We haven't historically thought of VOID as something to pass a function as a parameter. Because in the olden days, you couldn't receive a void: the evaluator would keep going until it actually produced a value (or NULL).

But non-interstitial invisibility is a thing of the past now (too dangerous, too messy). Hence ^META parameters can detect pure voids. And the interesting property of voidness (not being able to be stored in variables) means you have to do a head-check about "did I really mean this?"

Using MAYBE Feels Better Than TRY

It's not really an error to be appending nothing, that you should have to use TRY to "defuse". This is a different animal from something like first null.

And having nothingness as the parameter isn't just a no-op... e.g. when using CHANGE.

>> data: [a b c d]

>> change/part data void 2
== [c d]

 >> data
 == [c d]

Conditionals return void when they don't match a branch, which makes this convenient:

 >> append [a b c] switch 1 + 1 [10 [<ten>] 20 [<twenty]]
 == [a b c]

It looks like it's all tying together pretty well. Implementing this in the bootstrap shim is left as an exercise for the author. :slight_smile:


So there's a bit of a snag on this, when the thing you're making void isn't a parameter...but a product of something like the body on a MAP-EACH. At first it looks fine:

map-each item [1 <one> 2 <two> 3 <three>] [
    maybe match tag! item  ; leaving it as NULL would be an error
== [<one> <two> <three>]

But what if you had something else in the loop body?

map-each item [1 <one> 2 <two> 3 <three>] [
    append log spaced ["Logging:" item]
    maybe match tag! item  ; remember, void vanishes
== ["Logging: 1" "Logging: <one>" "Logging..." ...]

Now you've got this stray product of an append being the result of the MAP-EACH :-/

I don't know if this is a good enough reason to disallow a MAP-EACH body from evaluating to void and thus adding nothing. The same property applies everywhere, e.g. in COMPOSE slots--you can't just decide to evaluate the whole slot to void in the last expression. It isn't how invisibility works.

But it has some benefit, as it means you move your MAYBE to the outer level vs something tacked on down at the end:

compose [... (maybe any [...]) ...]

We might look at it this way with the MAP-EACH as well; if each iteration of the loop isn't going to be adding something, we say that at the top:

map-each item [1 <one> 2 <two> 3 <three>] [
    maybe (
        append log ["Logging:" item]
        match tag! item

Note that there is another tool here, which is CONTINUE:

map-each item [1 <one> 2 <two> 3 <three>] [
    append log spaced ["Logging:" item]
    match tag! item else [continue]
== [<one> <two> <three>]

In any case, this just shows that there's a bit of a twist to the use of void, making it something quite different from there's some risk of using it incorrectly.

1 Like

Historically, BrianH and others (including me) believed that mutating operations should not let you "opt out" of the target for the mutation. There was too much potential for confusion if you wrote:

 append block [1 2 3]

...and it silently didn't append, because block was a NONE!.

...Yet Another Historical Spectre Annihilated... :boom: :ghost: :boom:

The VOID in, NULL out solution handles this about as elegantly as we could hope.

If the variable were void, you'd get an error on accessing it:

>> unset 'block

>> append block [1 2 3]
** Error: BLOCK is unset, void, voided... whatever you want to call it...

And if you had a variable set to NULL, that wouldn't be glossed over either. The BLOCK reference would evaluate to null without an error, but it's not a valid parameter type and would fail in type checking:

>> block: _

>> append block [1 2 3]
** Error: APPEND doesn't accept null for its series argument (use MAYBE if intended)

But you can turn that null into a void with MAYBE

>> maybe block
; void

>> append maybe block [1 2 3]
; null

Here was some awkward code in UPARSE trying to use the blackhole when subpending was NULL, to skip the REMOVE-EACH:

remove-each item any [subpending #] [
    if group? item [eval item, true]

We see it instantly gets less confusing:

remove-each item (maybe subpending) [
    if group? item [eval item, true]

What's more, operations that have a natural "MAYBE" implicit in them--such as an IF or SWITCH or other branching construct, "just work"

>> block: [a b c]

>> flag: false

>> append if flag [block] spread [d e f]
; null

>> flag: true
>> append if flag [block] spread [d e f]
[a b c d e f]

^-- In regards to the "too many notes"... here's a good example to make "note" of, @rgchris ! NULL is a good thing, and an isotopic form of this non-value pays dividends--just as we've seen isotopic forms of regular values starting to show off, and only getting better...

1 Like

Something I didn't notice here is that we have another way of dealing with this, via an ordinary evaluated result... an empty splice !

map-each item [1 <one> 2 <two> 3 <three>] [
    append log spaced ["Logging:" item]
    match tag! item else [spread []]
== [<one> <two> <three>]

I don't know what we'd call this, let's just say it's nothing for now:

>> nothing
== ~[]~  ; isotope

Since VOID is reserved for the output of branching structures that never run, they could have this be what slips out of an empty branch:

>> if false []
; void

>> if true []
== ~[]~  ; isotope

We get the pure void distinction so that ELSE and THEN etc. know what to do... yet right off the bat we'd have several situations that would consider these equivalent!

>> append [a b c] if false []
== [a b c]

>> append [a b c] if true []
== [a b c]

There's no need for any special "decay" or anything, because the operation already has semantics for empty arrays, which matches that of void!

>> append [a b c] case [false [<skip>] true [print "Hi", nothing]] else ['d]
== [a b c]

>> append [a b c] case [false [<skip>] false [print "Hi", nothing]] else ['d]
== [a b c d]

>> unspaced ["a" if true [if true [print "MAGIC!", nothing]] "b"]
== "ab"