Stackless Generators and the API

Development Language Bridging
1

Imagine that you have some array you want to iterate over, and do some C processing on each element of that array. Pseudocode:

void ProcessBlock(REBVAL *block) {
    for-each item block [
        ProcessItem(item);
    ]
}

You've got a bit of a problem here, because you'd like to express that for-each as Rebol code...but you are in C. It seems like you can't use the FOR-EACH structure, you've got to write something like:

void ProcessBlock(REBVAL*block) {
    REBVAL *pos = rebValue(block);  // can't rebRelease(block) here, new handle

    while (rebNot("tail?", pos)) {
        REBVAL *item = rebValue("first", pos);
        ProcessItem(item);
        rebRelease(item);

        pos = rebValue("next", rebR(pos));  // release old pos, get new pos
    }
    rebRelease(pos);  // need to release the pos (it's a tail value, not null)
}

Ouch. There's certainly more ways to do it; we could get the item count and use a C integer as an index and PICK the items. But whatever we do, the point is that our iteration state is held in C variables.

Our ideal might look more like:

void ProcessBlock(REBVAL *block) {
    REBVAL *enumerator = rebEnumerator(block);

    REBVAL *item;
    while ((item = rebNext(enumerator)) {
        ProcessItem(item);
        rebRelease(item);
    }

    rebRelease(enumerator);
}

Yet the goal of the API is not to export every possible thing we would want as a rebXXX() function, rather to make it possible to use a few fixed extractors and value builders to push the burden onto a more flexible interpreter.

How about this, then?

void ProcessBlock(REBVAL *block) {
    REBVAL *generator = rebValue(
        "generator [for-each item", block, "[yield item]]"
    );

    REBVAL *item;
    while ((item = rebValue(generator)) {
        ProcessItem(item);
        rebRelease(item);
    }

    rebRelease(generator);
}

Ta da! It might not look like all that much of an advantage for this simple example, but the point is that it generalizes to much more complicated Rebol logic for the enumeration. Imagine if it were using PARSE and sometimes you want to handle the value with Rebol code (which you do inline) and sometimes you want to hand it back to C for processing (which you do with a yield).

I do want to get around to the C++ wrapper that would do the handle lifetime management implicitly, which if it mirrored RenCpp would look like:

void ProcessBlock(reb::Value &block) {
    auto generator = reb::Value {
        "generator [for-each item", block, "[yield item]]"
    };

    while (auto item = reb::Value { generator })
        ProcessItem(item);
}

But even the C version looks pretty darn nice for language interop!

1 Like