Core Evaluator Mechanics: Complex Return Results

I mention in "The Cliffs of Complexity" that every nice-seeming feature brings along with it a ton of issues...where something formerly simple no longer has a simple answer.

For instance: invisible functions are fantastic. But in trying to implement invisible combinators in UPARSE, the parse engine has to invoke a function and know whether the result was invisible or not. How can you be sure?

One (bad) workaround would be to have some unique identity the function couldn't know about:

 unique-identity: []  ; say this BLOCK! can't be known to the function
 result: (unique-identity, some-func arg1 arg2)
 if same? result unique-identity [
     print "It must have been invisible..."
 ]

But what if you had a FRAME! for the function, and you used DO? Is it good for DO to be able to act invisibly?

>> f: make frame! :comment

>> f.discarded: "the commented thing"

>> 1 + 2 do f
; null

Today, the answer is no... DO of a frame for an invisible function just returns NULL.

I don't know if acting invisible is a good idea, or raising an error and making it so you have to use a special function for DO to be invisible, or what. The point is just that once you come up with a cool feature it has ripples all over the system.

My opinion is that invisibles are a good example of being worth their disruptiveness. Too many good ideas build on these features, so figuring out how to get the rough edges off of them is important.

Recent Thought: Every Function Is A MACRO?

When I think about "The Evaluator", I imagine it being a very mechanical and regular thing.

So it kind of felt dirty to add MACRO in a somewhat ad-hoc fashion. Yet it seemed necessary. Not having the ability to specify a function as what it "expands" into seemed weak.

Something that came to be on my mind was that with generic quoting, there could be a sort of generalized concept...where the "grind" of the mechanical evaluator would always receive back "what to splice into the evaluation feed". An ordinary function would simply always return a QUOTED! value of what it wanted to return...which would have one quote level dropped by the evaluator.

Then, an invisible would simply be an empty block of material to splice. A "true NULL" could be NULL, while a "valued NULL" could be simply a QUOTED! null.

e.g.:

 null => null
 ' => "null-2", e.g. although it's a NULL still triggers THEN
 [] => invisible
 '7 => just the number 7
 [7] => also just the number 7, but more costly representation
 [1 +] => more complex macro splice, partial expression

If this were the sort of "core" protocol for return values, then FUNC would give you a layer on top of it where you didn't have to worry about issues like invisibility. But if you wanted to write something that was sometimes invisible, you'd make a macro...so you could return an empty block as a way of saying to splice nothing into the stream.

We could imagine this level of protocol also having other non-quoted types like ERROR!...to signal the propagation of errors up the stack.

Neat Unification, Does It Buy Us Anything?

There are still tough questions. The average person calling DO on a FRAME! doesn't want to have to deal with the result being a BLOCK! representing a macro expansion. But some clients (like UPARSE) would like to have the full information on things like invisibility.

So maybe a DO* as a lower-level DO, that the higher level DO is built on.

 >> f: make frame! :add
 >> f.value1: 10
 >> f.value2: 20
 >> do* f
 == '30  ; asking for "core" result gets it quoted

 >> f: make frame! :comment
 >> f.discarded: "argument to comment"
 >> do* f
 == []  ; asking for "core" result reveals empty expansion, e.g. invisible

I guess we could create different types for ACTION! and MACRO!, and say you couldn't even try to call a macro with plain DO. But what if the expression is only sometimes partial or invisible? If the person making the frame thought of the particular parameterization as something they could use as if it were a function, why shouldn't that be permitted?

Multiple Return Values Are Going To Likely Just Have Limits

How I've been framing multiple return values is really just syntactic sugar on passing variables to refinements. But they're very limited...they look immediately to the right of the SET-BLOCK! for the function to search for multiple return results. Just throwing in a GROUP! breaks that:

[data header]: load %some-file.reb  ; this works

[data header]: (print "won't work", load %some-file.reb)

If you tried to dig into the group to interact with the first expression, you get something very incongruous with how the overall result usually comes from the last expression. And there's no a-priori way of knowing when you're interacting with the last expression.

I think this is...just okay. It's just a notation for convenience. If it doesn't help you, don't use it, and pass things by refinements like you would historically

data: (print "this works", load/header %some-file.reb 'header)

There's no current smart behavior for trying to use multiple returns with invisible function. I've challenged x: comment "hi" 10. But then, what about:

[x]: comment "hi" 10

My opinion is that [x]: ... and x: ... should act the same, or if they don't act the same then [x]: should error.

Anyway...the key point relevant to this post is that multiple returns are mostly separate from the main "return" protocol. That's probably for the best.

Mostly Just Wanted To Write Up The MACRO Unification

So I just wanted to kind of get down this concept of a core evaluation mechanic that could represent invisibility, NULL, and "NULL-2" in a way that was more clear...in particular that idea that an empty block could communicate a "splice of nothing", hence invisibility.

When I envision how the evaluator works, that feels more regular...that it sees expressions and keeps expanding them and collapsing them until it finishes a frame. I've talked about how the expansions mean that once you start evaluating a block, you leave the domain of block representation...and so your EVALUATE steps are actually on a frame...so that's important to contemplate as well: