Input/Output/Error Ports

Development Ports
1

A fundamental Rebol 2 feature lacking from Rebol 3 (and by extension, Ren-C) is the concept of Input/Output ports (and missing from Rebol 2 was an Error port).

In Rebol 2, the Console scheme was used for both Input/Output when the interpreter was invoked without argument (thus entering the REPL loop), whereas when invoked with argument (at least on Mac/Linux), Input/Output used the File scheme on what I assume to be a virtual file.

That Rebol 3 uses the Console scheme for Input and no scheme (or port) for output reveal some limitiations, such as the bleeding of console logic and decoration in shell scripts, or redirecting output to a hard file, and the awkward workaround that is WRITE-STDOUT. The first case there is especially notable for me as I primarily develop and deploy scripts as shell scripts and the Console scheme should never be a consideration in that mode.

I understand that the whole Port model could use a fair degree of scrutiny—and I certainly can't speak to the rectitude of using FILE as the default I/O scheme (I don't have that systemic understanding), but it seems that the long arc of Rebol 3/Ren-C development has kind-of skirted this core concern.

2

I'd be amenable to replacing WRITE-STDOUT with WRITE STDOUT...and adding the capability for WRITE STDERR even if those "PORT!s" are done by hook or by crook for now. You'd be a good source of feedback for which messages in the system should be going where, given practical CSS scenarios.

I feel the concern has been heeded in the sense that things are being built out of parts in a clear way. For instance, the source for PRINT:

print: func* [
    {Textually output spaced line (evaluating elements if a block)}

    return: "NULL if blank input or effectively empty block, otherwise VOID!"
        [<opt> void!]
    line "Line of text or block, blank or [] has NO output, newline allowed"
        [<blank> char! text! block!]
][
    if char? line [
        if not equal? line newline [
            fail "PRINT only allows CHAR! of newline (see WRITE-STDOUT)"
        ]
        return write-stdout line
    ]

    (write-stdout try spaced line) then [write-stdout newline]
]

This pushes into plain view exactly where the decision of where PRINT happens is made. And it shows the logic of things in reusable parts. PRINT is clearly based on SPACED, which is based on DELIMIT, etc. But when things are packed into natives and circuitous code like R3-Alpha's I/O was done...especially in its "PORT!s", I'd definitely challenge anyone to make headway on a better design.

Just speaking up for the idea of whether this has been weighed as a serious consideration. It has. Of course you've still got questions like how PRINT would be parameterized to write to STDERR instead of STDOUT (if it can be at all). But the parts are there so anyone who can write Rebol can participate in that design.

One thing I've wanted to do is to get all of the cursor control and line editing / history logic into usermode. Right now we have a bunch of twisty POSIX C based on termios...which gives full control but hides everything (like the line history) in C code. Then there's a single monolithic call to ReadLine on Windows, which offers absolutely no control and prevents the implementation of even simple features like tab for file completion.

It may not sound like a huge priority... but... sorting this out for the Web REPL is a priority. And it would be nice if the same customizations would run on all platforms, because a console port was abstract enough to handle the features nicely across all three.

And in sorting that out, a graceful fallback to having fewer and fewer features for I/O is important to have. CGI is a good example of this.

3

This is where I start getting lost in the weeds a bit.

I'm not sure how the console scheme in Rebol 2 is formulated—seemingly system/ports/input and system/ports/output both use the console scheme when in REPL mode, but they are not the same port value so couldn't say what the relationship is or how they interact. I get the impression that the input port does most of the heavy lifting, output being a dumb 'print to terminal' affair.

A quick test at the Rebol 2 REPL shows that you can close and reopen system/ports/output without fuss, or switch it out for a different port! value. There are most certainly issues closing system/ports/input.

>> close system/ports/output
close system/ports/input 
Segmentation fault: 11

I similarly don't know if the file scheme maps to the way OS/Shell handles I/O, so don't know whether that was an appropriate choice or a hack.

The PRINT example is sound in itself, but is a level of abstraction above what I have in mind. It's more that WRITE-STDOUT is a black box stand-in where the elegance of the port concept should be. I don't necessarily see PRINT as relating to STDERR, rather just being shorthand for write system/ports/output "Stuff" with added compositional qualities—in Rebol 2, you could say: system/ports/output: open %some/file and that is where PRINT's output would end up. I suppose there could be print/error that would change the target to a proposed system/ports/error, but may be better handing off to a dump (or such) counterpart.

I do have a sense that solving this will lead to some better design outcomes.

4

in Rebol 2, you could say: system/ports/output: open %some/file and that is where PRINT's output would end up

That's the kind of thing I meant, when I said having things in usermode would certainly make that possible...trivially, even. We could go through the step where STDOUT is a port, and defaulted system/ports/output to that port...then the PRINT code could simply be changed to use WRITE SYSTEM/PORTS/OUTPUT. It would then work as you describe.

(Notably, if you said system/ports/output: %some/file and did not use open, then each PRINT statement would rewrite the file. This may or may not raise semantic issues of if WRITE should always assume it's appending to what's there, and you have to delete it beforehand otherwise. Or maybe there would need to be some kind of type trigger that would prevent modifications to system/ports/output that wasn't a PORT!...? Questions...)

My own wish would be to ergonomically flip it. It seems laborious for the average author to have to say write system/ports/output ">>" to get the indirection. I'd rather they say write stdout ">>" and get the appropriate indirection, shared with whatever PRINT sees.

...but I don't know how to do that with your assignment model, because saying stdout: xxx can't currently change the stdout that the PRINT in the lib context sees. You have to say at minimum sys/stdout: xxx to get them on the same page of changing the same context. It changes if you say stdout/set-target xxx which then gives you the ability to change a property of a common object.

If it would make you happy enough to code more for now, we can copy Rebol2's model...and it can be how things are done until something better comes along. I don't have a problem changing PRINT to WRITE to system/ports/output and make STDOUT a PORT! that's the default place it writes to. Then add STDERR that similarly defaults to system/ports/error.

We would make WRITE-STDOUT a specialization of WRITE to system/ports/output so it saves some typing. That could compromise between what I'd like to see and getting the indirection, and would be no worse than what's there today. Though that notably brings out the example of a specialization that does not want to capture a value... since you want it to see changes. You'd have to write it as:

write-stdout: adapt (specialize 'write [destination: <to-be-overwritten>]) [
    destination: system/ports/output
]

So the specialization removes the parameter and sets it to garbage, then the adaptation injects prelude code on each run to fetch another value. This "dynamic specialization" pattern is something I've been looking to see how to make easier. But...at least it is possible.

5

On reflection, the output port is somewhat trivial (your above qualms aside). I feel if you're the type of user that would change system/ports/output then you'd accept any quirks in setting it to something that doesn't make sense (such as a closed port). I am partial to having the canonical values within system/ports as system would seem the home for such things, but having STDIN/STDOUT/STDERR as synonyms would also make sense.

I see the console scheme-based system/ports/input as the broker for the console-isms and where the non-trivial event handling would happen.

6

If you're interested in hooking the standard out, the console's perspective on it can have a wide variety of what you want to hook. Do you want the console's prompt printing to the same place that your script code is outputting to? If you do this by simple assignment to system/standard/output, then what happens if you hit an error in mid-script...who puts it back? :bomb:

I did some work on an ECHO feature which tried to push on some of those questions, and I think was pretty neat. But it is mothballed for now, because there's just so much to do:

It seems a good start for a discussion of what's important. I'm pretty sure it's rare for features or compositions you don't plan for to just work "by accident"...

7

I think the better name for an output scheme when in REPL mode is 'terminal *.

I'm fine with system/ports/output being a monolith—I should say, at this point I'd advocate simply for the Rebol 2 model where the 'console scheme-based input port waits and processes input and I presume uses prin(t) as the conduit to the output port. I'm not certain what the merits are of sending >> or == to a different port than <result>. I would, of course, deviate from Rebol 2 in putting as much of that scheme in user-mode as is possible.

If you happen to redirect output to a 'file scheme, then it's on you to monitor when an error occurs—again, as per the Rebol 2 model.

It's also conceivable that system/ports/output and system/ports/error could point to the same port—I'm less concerned about how that works in the REPL environment (where you'd presumably wish to see errors and output appear in the same place) than being able to fine-grain your responses when invoked as a shell script (which goes hand-in-hand with being able to set your own exit status) or with a script argument.

*A 'terminal scheme might include the ability to detect console-like output and interject some terminal codes on select systems, as an example.

My goal here is to avoid the console and trappings altogether unless I'm explicitly using the REPL.

8

If you are using the Rebol 2 REPL and say close system/ports/output, the REPL still works, you just don't see any output. This seems fine and predictable to me, it's what you'd expect when you close the output port. It should maybe error on trying to write to a closed port, but how would you know**? :slight_smile:

I don't think the interpreter should protect you from making dumb choices like this, but I don't think anyone will accidentally do so. In Ren-C currently, the 'console scheme is a black box—bad things happen if you change the system/ports/input/actor value

**(print "Stuff" doesn't error, but insert system/ports/output "Stuff" does)

9

I am pretty much at 100% on understanding what is there, and I've not been shy about saying the emperor has no clothes.

But...if you say "I need X" I will do it. Seems you are getting interested again, and I will do as you wish. File issues, and you will see results.

10

I still don't fully appreciate this assertion—ports and schemes are how we interact with the outside world—why wouldn't that encompass the relationship with STDIN/OUT/ERR as opposed to black-box functions?