Re-imagining EVAL/NEXT

Development Internals
1

NOTE: The nature of DO is to deal with whole scripts. We know do/next %foo.r doesn't make any sense, and in fact having DO take a BLOCK! of Rebol code should be a dialect about how to load and bind what you pass it... not expressions like do [1 + 2].

So in modern Ren-C, array evaluation is handled by a primitive called EVAL. Hence the /NEXT refinement has been removed from DO, and for a transitional period DO will not take BLOCK! at all... until all such references are gone. At which point it will take a dialected LOAD spec, probably aligning closely with what IMPORT takes.

Right Now, EVAL/NEXT Takes A Variable To Store The New Position

>> result: eval/next [1 + 2 10 + 20] $pos
== 3

>> pos
== [10 + 20]

This is exactly how Red and R3-Alpha handle DO/NEXT.

It was considered more convenient than how Rebol2 gave you a block of both the result and the new position...which you had to pick apart:

rebol2>> do/next [1 + 2 10 + 20]
== [3 [10 + 20]]

(That couldn't work at all in Ren-C, because evaluation can produce antiforms, and antiforms can't be put in blocks.)

One Twist: EVAL/NEXT of [] Returns A NULL Position

If you try to step with /NEXT over a BLOCK! like [1 + 2 10 + 20], then there are EXACTLY TWO steps with meaningful results of 3 and 30.

So if you're going to be doing the evaluations in a WHILE loop, you want the EVAL/NEXT position result to return success twice, and then have a third call that returns null to signal the looping is done.

This gives you the possibly surprising (or not?) result that EVAL/NEXT [] doesn't take a step and doesn't synthesize VOID, even though EVAL [] is VOID. It's a terminal condition. So if you're trying to take steps and generate an overall accumulated result, you have to seed your result with VOID... and then EVAL/NEXT [] will tell you there was nothing to do and you return your seeded result.

Make sense?

Rebol2, Red, and R3-Alpha all require you to check for the TAIL? of the block as your terminal condition. Because DO/NEXT on a tail position just produces an UNSET! and another tail position.

rebol2>> do/next [10 + 20]
== [30 []]

rebol2>> do/next []
== [unset []]

rebol2>> do/next []
 == [unset []]

That's quite a lot more awkward to handle for a terminal condition. In fact it forces you to check for TAIL? on the block you're evaluating before the first call to DO/NEXT (because seeing the tail afterward won't tell you if the previous step synthesized a valid UNSET!).

R3-Alpha and Red didn't change this, and still make you check for TAIL? before you take steps:

r3-alpha/red>> do/next [10 + 20] 'pos
== 30

r3-alpha/red>> pos
== []

r3-alpha/red>> do/next [] 'pos
; no console result here means unset

r3-alpha/red>> pos
== []

r3-alpha/red>> do/next [] 'pos
; no console result here means unset

r3-alpha/red>> pos
== []

Still very awkward, and unclear why they did this instead of making the POS be #[none].

But Ren-C Can Do Even Better: Multi-Returns!

What if EVAL/NEXT turned the return result into a parameter pack, where you get both the evaluation product and the new position?

(I'm going to make it so it switches the main return result to be the position, and the secondary result is the evaluation product. Then I'll explain why.)

>> block: [1 + 2 10 + 20]
== [1 + 2 10 + 20]

>> pos: eval/next block  ; don't have to heed both returns
== [10 + 20]

>> [pos /result]: eval/next pos  ; but you can heed both returns
== []

>> result
== 30

>> [pos /result]: eval/next pos
== ~null~  ; anti

>> result
== ~null~  ; anti <- not meaningful, because POS was null

The reason you (often) need the slash on /RESULT is that when EVAL/NEXT is done, it returns a pure null... not a multi-return. This makes it correctly reactive to THEN and ELSE, which consider nulls inside of parameter packs to be "something" instead of "nothing". But if you try to unpack a single null into two slots that is considered not enough.

Using a leading slash on a multi-return unpack is indication that you accept there may not be enough items in the pack to have one for that variable. We could choose to have the multi-return unpacker make it trash or null, but in general null is more useful.

(To distinguish from a null that was actually in a pack and unpacked, you'd have to use ^/result...which would meta the value so an unpacked null would be a quasiform ~null~, while the null resulting from too few values in the pack would be the antiform. Caring about this is rare, but good to have a way to tell the difference if you do care.)

Why Did I Make Position The Primary Return Result?

  1. It Makes It Easier to Loop Through an Evaluation - There are some situations where EVAL/NEXT doesn't care about the value synthesized, but pretty much no cases where you don't care about the new position. Being able to conditionally test if the returned position reached the end of a loop is super convenient.

    block: [1 + 2 10 + 20]

while [[block /result]: eval/next block] [
    print ["Step result was:" result]
]

    It's true that with Ren-C's super multi-return powers, you could ask for the second argument to be the overall main result. But that's uglier, why be ugly?

    block: [1 + 2 10 + 20]

while [[result @/block]: eval/next block] [
    print ["Step result was:" result]
]

  2. Avoids Ambiguity When EVAL Result Is Itself A Multi-Return - Imagine the following kind of confusion if we made the evaluation product the first result instead of the second:

    >> block: [1 + 2 comment "I don't care about this"]

>> result: eval/next block  ; I just want the first thing!
== 3  ; great, I didn't want that position anyway

>> block: [pack [<left> <right>] comment "I don't care about this"]

>> [left right]: eval/next block  ; just want to unpack that first thing
== <left>  ; great, just what I expected

>> right
== [comment "I don't care about this"]  ; whaaa? I wanted <right>!

    Encountering problems with this in the past has made me back off from using multi-returns in places they seemed like they would be perfect. But what I now realize is you simply don't want your primary return result of a multi-return to be something that can itself be a multi-return... unless you really know what you are doing.

    If you intend to do something with the evaluation product and want to be truly general, you of course have to be using ^META conventions:

    [pos ^result]: eval/next pos

    Whether you need to do that or not depends on what you are doing. Why are you stepping through arrays one step at a time, anyway? Usually intermediate results are discarded. What is it precisely you are looking for? (Again on my point of why making the position the primary result makes sense... usually you aren't looking at the result at all, you're a dialect and looking at what you advance to at the next position.)

LGTM :+1:

2 Likes