A Justification of Generalized Isotopes

Here is a train of thought to help people realize why isotopes are needed, and why unifying their behaviors and mechanisms under a common umbrella makes sense. It starts from the issue of solving /ONLY and then explains the generalization.

As time permits, I'll come back and try to improve this...


Years of fretting over the /ONLY debacle converged on a somewhat inescapable conclusion:

It's better to carry the intent of whether a value needs to be spliced on that value...as opposed to having subtle variants of core operations that modulate the splicing.

I'd worked up to a point where I was implementing the "mark of intent" by adding a quoting level to suppress splicing. Yet this faced likely accidents when someone had a quoted value in a variable...and really meant to use it somewhere as-is, with the quote--vs. thinking of the quote as a splice-suppression signal which the operation should remove.

Then @rgchris made this remark:

If Trying This In Historical Redbol, What Might One Do?

As a rough first cut, let's represent splices with a specially recognizable 2-element wrapper block. We'll signal it's a splice with a series in the first slot--checking for the unique identity of that series. Then put the block itself as the second element:

splice-cue: "!!!splice!!!"

spread: func [block [block!]] [
    return reduce [splice-cue block]
]

splice?: func [value] [
    if not block? :value [return false]
    return same? splice-cue first value
]

Then we can write our new versions of things like APPEND that are specifically aware of this construct.

append*: func [series [series!] value] [
    return either splice? :value [
        append series second value
    ][
        append/only series :value
    ]
 ]

It works more or less in your average Redbol, e.g. in Red:

red>> append* [a b c] spread [d e]
== [a b c d e]

red>> append* [a b c] [d e]
== [a b c [d e]]

red>> append* [a b c] 'd
== [a b c d]

red>> append* [a b c] first ['d]
== [a b c 'd]

In fact, this is essentially how the bootstrap executable for Ren-C simulates the SPREAD behavior.

But the weaknesses are immediately apparent!!! :pouting_cat:

Not A Distinct Type: Too Easy To Overlook Handling

There's no special type for the spliced block...it's just a BLOCK!. This means any routine that hasn't been written to handle it, will just let it leak through.

red>> reduce [spread [a b c] [a b c]]
== [["!!!splice!!!" [a b c]] [a b c]]  ; not [a b c [a b c]]

Changing to some other generic type that can contain a block...such as an OBJECT!...doesn't help matters. You are kind of in trouble any time an operation willfully lets you put these into an array.

The first instinct might be to introduce a new SPLICE! datatype, with a system-wide rule that splices can't be put into arrays. (Enforcing such a rule across all array-manipulating code is challenging...so let's sort of make a note of that fact, but continue.)

Because of the peculiar nature of not being able to be put in a block, there'd have to be a decision made about function arguments as to whether or not they took this type. Many functions designed to handle generic values would not be able to handle them, so there'd presumably need to be some typeset like ANY-NOTSPLICE! or ANY-NORMAL!.

How To Represent A Type That Can't Be Put In A Block?

Now we've got several things to ponder about our new type. For instance: what you should see here?

>> obj: make object! [foo: spread [d e]]
== make object! [
    foo: ???
]

We just said that a defining feature of SPLICE! is that you can't accidentally put them in blocks. But the argument to MAKE OBJECT!, namely [foo: ???], is a block. If ??? can't itself be a splice!, then what is it?

This brings up a possibly-related question: what if you want a way to put the intent of whether to splice or not into "suspended animation?"... in a way that you could collect it?

Here's a sort of contrived example of the puzzle:

generate: func [n [integer!]] [
   if even? n [return reduce [n n + 1]]
   return spread reduce [n n + 1]
]

wrap: func [
    return: [...]
    in [splice! block!]
][
    ...
]

unwrap: func [
    return: [splice! block!]
    wrapped [...]
][
    ...
]

n: 0
pending: collect [while [n < 4] [keep wrap generate n]]

data: copy []
for-each item pending [append data unwrap item]

How would you write WRAP and UNWRAP such that at the end of the code above, you'd get:

>> data
== [[0 1] 1 2 [2 3] 3 4]

If the system didn't provide some answer to this, you'd end up needing to re-invent something kind of equivalent to the primitive ["!!!splice!!!" [...]] mechanic as a means of persistence:

>> pending
== [[0 1] ["!!!splice!!!" [1 2]] [2 3] ["!!!splice!!!" [3 4]]]

Isotopes Were Designed For This!

Isotopes are a set of curated answers for these problems. Originally they were introduced to address issues like what an UNSET! was...which has some of the same class of problems as SPLICE! (such as not wanting to be put in BLOCK!s, and not accepted by default or by most routines).

Isotopes are:

  • general - all base value types (e.g. unquoted things that can be put into blocks, so not NULL or VOID or QUOTED!) have isotopic forms

  • efficient - isotopes do not require allocations, and merely are a different state of a byte in the value cell (the same byte that encodes quoting levels)

  • "meta-representable" - all isotopes have a corresponding single-value form known as a "quasi-form", which when evaluated (or run through a more narrow UNQUASI operation) will yield the isotopic form

I mentioned at the outset that it would be somewhat costly to bulletproof all of native code against the ability to do something like append a specific data type like "SPLICE!" to a block. But with isotopes this problem has been solved once for all the forms...so the same code that prevents a so-called "UNSET!" from winding up in arrays works for splices. That's because a splice is actually an isotopic block!, and an unset is actually an isotopic blank!

Above I asked:

What you should see here?

>> obj: make object! [foo: spread [d e]]
== make object! [
    foo: ???
]

Isotopes give us the answer, that it's foo: ~[d e]~. This is the previously mentioned "QUASI!" form of BLOCK!, which when evaluated produces an isotope.

But isotopes themselves have no canon representation. The console can print out a comment or show them in a different color, but to talk about them having a representation doesn't make much sense as you'll never see them in source.

>> ~[d e]~
== ~[d e]~  ; isotope

I also asked:

"How would you write WRAP and UNWRAP such that at the end of the code above, you'd get:"

>> data
== [[0 1] 1 2 [2 3] 3 4]

With isotopic blocks representing splices, you don't need to write WRAP and UNWRAP... because these operations are built in operations called META and UNMETA. And the pending array would look like:

>> pending
== ['[0 1] ~[1 2]~ '[2 3] ~[3 4]~]

When the QUOTED! blocks are UNMETA'd, they become regular blocks and then are appended as-is. When the QUASI! blocks are UNMETA'd they become isotopes and give the splice intent. This produces the desired "suspended animation" to preserve the intent.

That suspended animation is also used in the ^META parameter convention, which indicates a function argument can accept isotopes... and the add-quoting-or-quasi behavior brings those isotopic variables into a reified state so they can be safely handled.

The Proof Is In The Capabilities

I've explained about splices, and mentioned how it crosses needs with unset variable states.

But isotopes are also a cornerstone of how function and FRAME! specialization works.

The ERROR! isotope is used to have a sneaky out-of-band way to return definitional errors

As their applications expand, they are slated to allow us to have non-literal modes to ask something like FIND that you want to look for instances of a datatype by passing that type isotopically... vs. looking for the datatype's appearance concretely in the target. Concepts of making only isotopic ACTION!s run through WORD! references could make it completely safe to work with arguments passed as variables, saving you from needing GET-WORD!s to handle them.

It's natural for there to be some confusion with the new idea--especially given all its churn through the course of design. But the design is becoming clearer, and I think people are going to find this gives solidity to writing complicated but coherent code...vastly outpacing historical Redbol.

3 Likes

Great writeup and congrats on this foundational feature. It looks like a winner!
:clap: :clap: :clap:

1 Like

A small thought for now:

a: 'a b: 'b c: 'c
; for infinite reduction of 'a 'b and 'c

block: [a b c [a b c]]

reduce block
=> [a b c [a b c]]
; ok

find block spread pick block 4
=> [a b c [a b c]]
; ok

spread pick block 4
=> isotope value
; ok, or 'isotoped value' or 'isotopic value' or just 'isotope'?

reduce block
=> [a b c a b c]
; ??

Does SPREAD here apply the isotope to that value in place or does it create something new? I kind of suspect the latter, but is it reasonable to expect the above behaviour?

It creates something new.

It's reasonable to ask if it works that way. But it does not. :slight_smile:

The quoting level of a BLOCK! (or any other value) lives in its cell, not in the series pointed to that contains the contents. It's a property like whether a slot in a block has the NEW-LINE flag.

Remember that the same underlying array storage can be imaged multiple times:

>> block: [a b]

>> four: reduce [block, quote block, quote quote block, quasi block]
== [[a b] '[a b] ''[a b] ~[a b]~]

>> append block 'c

>> four
== [[a b c] '[a b c] ''[a b c] ~[a b c]~]

The quoting (and/or quasi) level lives resident in the bits of FOUR's array holding the cells. The cells point at the array storage for BLOCK.

The only way to change the quoting levels is to manipulate the actual contents of FOUR. You can't do it by modifying the BLOCK variable to change its quote level--that's only relevant to the cell stored in the context where block's value is looked up:

>> block: spread block
== ~[a b c]~  ; isotope

>> four
== [[a b c] '[a b c] ''[a b c] ~[a b c]~]  ; unchanged

Hence there's no means to subversively create the situation of an isotope inside a block remotely. You have to go through a function like CHANGE. They can spread the values, or error, or have some other reaction--but they'll never be allowed to put the isotope in the block.

2 Likes