Should values given to C through API "handles" be mutable?

In libRed, the definition for a red_value is a void*. In the "lifetime of handles" post, I mentioned where these pointers point at...a small pool of values, that go bad quickly (last 50 API call results, at time of writing).

Due to the definition, the parameters don't give the appearance of pointers, they look like they're being passed-by-value, e.g. red_value redSelect(red_series series, red_value value); Hence it's not surprising that libRed doesn't have mutating operators on red_values themselves. This parallels how Rebol works in userspace; you don't fiddle with the bits inside values, you use operations that generate new values and overwrite old ones.

Consider their structure of a cell that holds a BLOCK!:

red-block!: alias struct! [
    header [integer!] ;-- cell header
    head   [integer!] ;-- block's head index (zero-based)
    node   [node!]    ;-- series node pointer
    extra  [integer!] ;-- (reserved for block-derivative types)
]

Imagine you wanted to visit each position in a BLOCK! from C. Since red_value is an opaque type, you can't twiddle the bits and increment the head index, you'd have to call redSkip. And that will produce a new cell for each position.

Their philosophy on keeping this from being "cost prohibitive" is to have a tiny pool of cells they reuse for these results. The results are temporaries, and you aren't supposed to get too attached to these cells. If you have some long-running operation and ask to tweak the index of a value you got some number of calls ago, you're going to run into trouble.

But RenCpp was designed differently. The cell behind a ren::AnyValue has the standard lifetime of the C++ object...it will exist so long as it is in scope. You can pass them by value (in which case they are copied and a new cell is created) or you can pass them by reference (which means mutations will be seen by other references to the same object). So rather than create 100000 distinct cells during iteration of 100000 positions in an array, when you call the .next() method on a series subclass, it just updates the bits of a single cell.

I'm not very eager to change this behavior of RenCpp. But this raises the question of how a C-friendly libRebol that parallels libRed--but with a different model of lifetimes of API handles--should operate. Should it offer functions that let you mutate the guts of a value you already have?

A less invasive way of thinking would be if every API function took as a parameter an existing cell to write to, perhaps with NULL meaning "create a new cell"...

REBVAL *gotten = rebAlloc();
for (i = 0; i < 100, i++) {
    rebGet(gotten, "a"); // fetch into already existing cell
    rebAppend(gotten, rebBlank());
}
rebFree(gotten);

The approaches aren't mutually exclusive. Every API could have an "Ex" version which has a slot to write to, and then specialize it:

#define rebGet(symbol) \
    rebGetEx(NULL, (symbol))

However one thing I do like about the idea of immutability and not thinking in terms of "cells the user can write to" would be the elimination of "nulled cells" from API awareness. Since you never ask to overwrite an existing cell address with an operation result, you never worry about what bit pattern that would need to hold the absence of a result...and you could just use NULL for that. That has very strong appeal for me.

(RenCpp has a C++-shaped tool for avoiding the need for expressing nulled cells...it uses std::optional.)

My leaning here is to say that RenCpp and libRebol just go down different paths with this. I guess I'd have to see more libRed example code to think about what users are interested in, and what the kind of code they'd write would look like...since my only real example case so far is Ren Garden, and libRed would be thoroughly unsuitable for it.

I feel like I've sort of answered my own question.

Whatever the C++ API does, the "C friendly" API should probably not have any mutators. If people have an iteration that they're particularly worried about the consequences of generating tons of superfluous handles, they should probably escape out from C to Rebol and do that iteration in Rebol code.

I think this is just going to have to be the performance balance. People writing repetitive procedures called into Rebol from C should probably register their C-part as a FUNCTION! and do their loop part as a string of Rebol code... OR... get comfortable with releasing handles manually... OR... not care and let the FRAME! clean it up... OR... use C++ and have it taken care of automatically... OR... use the rebRing() concept (rebTemp()?).

Not sure where this leaves reb_value vs REBVAL*. Windows programming involves a lot of HWND and HPROCESS and other H-prefixed stuff where you understand that you're responsible for it, while it's not expressed as a literal pointer. If we're going to be talking about lifetime management being in the client's consciousness, I'm not sure a name like reb_value that looks to be passed by value fits in. Also, one benefit of being forthcoming about it being a legitimate REBVAL* is that any code linked to the internal API would see the type as non-opaque and yet be able to use it with the same libRebol functions.

Anyway, guess I'm just saying there are enough options here to not sweat it, and indeed to say that NULL pointers mean what Ren-C calls a NULL. I guess we'll have to wait to see if libRed tries to introduce any ways to mutate cells in the ring, but under their current direction it doesn't seem likely that would happen.

To put it another way: If libRed ever allowed mutating operations, the by-value look of red_value being passed would seem a lie. When you pass a C function something that doesn't look like it's being passed by pointer, you semantically assume you got your own copy. The only way to make that work when your underlying implementation is a void pointer to a cell in a ring would be if copying meant nothing, e.g. immutability.