BLANK! (_) as SPACE in String-Oriented Dialects

(Note: I'm kicking off this discussion with a conclusion from an otherwise outdated thread, circa 2020.)

Having had a fair amount of time to reflect on the evolution of things, I think we need to make undecorated word fetches not produce an error with NULL. I've outlined the history of why they were erroring at the outset, and walking through it I think there's a coherent plan for when there is tolerance and when there is not (e.g. function arguments by default).

But today I realized something particularly pleasing about this. We had a concept that BLANK!...due to its non-erroring status, would be the way of "disarming" a null assignment to a variable. It's still going to be a way of disarming parameters for "blank in, null out", but not needed for a plain assignment.

That frees up blank for dialect uses distinct from NULL.

In particular, it recovers it for something that was tried for a time... being synonymous with space (#" "). The concept emerged when it was called into question whether PRINT should default to adding implicit spaces, and how you should avoid it doing so. print ["It" _ was _ "ugly"] if the common case didn't have implicit spaces, so ultimately we went with SPACED by default and you could write print unspaced [...] if that's what you wanted.

Yet _ was deemed to need to behave just as NULL did, as you couldn't put a NULL in a variable. So when you needed to, you would use blank, and it would be the non-erroring synonym. It was a sad loss for when you wanted a nice way to note spaces, but seemed to be the way it had to be.

Well, not any more!

With non-erroring NULLs, dialects are free to distinguish the behavior of blanks and nulls. NULL being the most obvious "nothing". So why not bring back BLANK! as being a synonym for space?

It still has the "no delimiters applied" status of a CHAR!. So spaced ["a" _ _ "b"] would still be just two spaces between the a and the b... not five.

Looking good!

Even better news here in 2022...'s not even used for that anymore!

With definitional errors as the underpinning, BLANK!-in-NULL-out has been replaced by NULL-in-TRY-out.

This is good. Because it had been uncomfortable dealing with the difference in meaning for a literal blank vs. a fetched blank:

>> maybe-unused-var: _

>> unspaced ["a" maybe-unused-var "b"]
== "ab"

>> unspaced ["a" _ "b"]
== "a b"

I tried various ways of rationalizing this (for instance, "a WORD! doesn't act like a fetched version of itself, why should BLANK! have to?") But I'll freely admit this was coming from a place of "I want blank for space in string dialects--so I'll keep justifying it until I find a good-enough sounding excuse, even if it's kind of broken."

Of course, I'm only hurting myself when I do that. This bit me when I was trying to factor the internals so that DELIMIT was based on the same code as REDUCE. Because if I thought of DELIMIT as being a kind of post-processing pass on what a reduce was doing, it wouldn't have enough information to accomplish the distinction above:

unspaced ["a" _ "b"]

Step 1: reduce ["a" _ "b"]
== ["a" _ "b"]
Step 2: ...
Step 3:
== "a b"


unspaced ["a" maybe-unused-var "b"]

Step 1: reduce ["a" maybe-unused-var "b"]
== ["a" _ "b"]
Step 2: ...
Step 3:
== "ab" 

:raised_hand: "What's step 2?"

Everything Changed...

Now any transitions from blanks to NULL-ness or VOID-ness or SPACE-ness (or anything else) will be conscious acts of a dialect. It can do that without being beholden to some idea that BLANK! has to be reserved for an implementation mechanic of something like TRY.

You would really make a variable NULL, and to have it disappear in something like a REDUCE or UNSPACED you'd literally use MAYBE to do that.

>> var: null

>> reduce ["a" var "b"]
** Error: var is NULL and REDUCE doesn't like that  ; paraphrased :-)

>> reduce ["a" maybe var "b"]  ; MAYBE turns NULL into VOID
== "ab"

With BLANK! now freed up, I think if the string conversions interpret it as space, that's great!

>> to text! [Hello _ New _ Blank _ World!]
== "Hello New Blank World!"

Armed with GET-BLOCK! as REDUCE, you have some great shorthand:

>> var1: "Hello"
>> var2: 'World!

>> to text! reduce [var1 _ 'New _ "Blank" _ var2]
== "Hello New Blank World!"

>> to text! :[var1 _ 'New _ "Blank" _ var2]
== "Hello New Blank World!"

Pleasing and solid. And the internals benefit--as I mentioned--by being able to do things like build DELIMIT on top of REDUCE.

:broom: :sparkles:

And it should go without saying at this point... but... Redbol and your own ideas could come in and do all this differently.


Things underwent another change, where we expect:

>> x: _

>> x
; null

This may seem like it makes the BLANK!-as-SPACE-in-DELIMIT a no-go.

But...with isotopes, I thought, wait... what if...

>> _
== ~_~  ; isotope

>> x: _
; null

There can be an intermediate state blank goes through on its way to decay to NULL. And if you're interested in that state you can decide if for your context it should be more null-like, or more space-like.

Rigging up DELIMIT on this, we can see... it works!

>> unspaced ["Hello" _ "World"]
== "Hello World"

Because DELIMIT isn't looking for blanks literally--only as evaluative products--it doesn't do nasty things like subvert enfix usages:

>> leftq: enfix lambda ['b [blank!]] ["BLANK!"]
== #[action! {leftq} ['b]]

>> unspaced ["Hello" _ leftq "World"]
== "HelloBLANK!World"

Another have-your-cake-and-eat-it-too feature... :cake: :stuck_out_tongue: