How Should Redirected Input Work For The Console?

When you run language interpreters without supplying a script to run on the command line, most will go into an interactive console mode.

But what if you didn't want interaction? It's possible to pipe input to a process, for instance:

$ echo "print(\"Hello\")" | python3
Hello

(Note: You have to backslash escape quotes to pass them in bash like that.)

The echo command produced some output, and python runs it as code. It doesn't do the same thing as if you'd typed the code--there's no console banner, and it doesn't give you a response about what the output is.

But Rebol, Red, and Ren-C all fail at this.

Rebol2 on Windows Ignores Piped Input

Rebol2 on Windows ignores the issue entirely. The following will just launch a GUI window, as if you hadn't said anything:

C:\Projects\rebol2> echo "print {Hello} 1 + 2" | rebol2-view
; Launches a GUI terminal window

C:\Projects\rebol2> echo "print {Hello} 1 + 2" | rebol2-core
; Also launches a GUI terminal widow

I don't have a 32-bit Linux handy to test rebol2 with, so we'll just look at R3-Alpha on Linux.

R3Alpha on Linux Runs The Input And Shows Half A Console

R3-Alpha does a little better, in terms of running the code you asked for. But it still runs the output side of the console, while not echoing the input side:

$ echo "print {Hello} 1 + 2" | ./r3-alpha
**************************************************************************
**                                                                      **
**  REBOL 3.0 [Alpha Test]                                              **
**                                                              
...(rest of the boot banner and help)...
Upgrade - check for newer releases
Changes - what's new about this version (web)

>> Hello
== 3

>>

If that would print the input code too so this looked coherent, you could imagine this being a interesting mode to offer. But while interesting, it's certainly not the default you'd want.

If we look into why R3-Alpha quits, it's because when it runs out of standard input to read a failure makes Fetch_Buf() return 0. You can see that it had at one time crashed under this condition, but that was commented out:

// If error, don't crash, just ignore it:
if (Std_IO_Req.error) return 0; //Host_Crash(stdio read);

When Fetch_Buf() returned 0, then Get_Line() would return 0....and that would stop the main loop and lead the console to exit.

Red On Windows Runs Out of Memory

If you don't use the --cli switch it will pop up a normal GUI Window and ignore the piped input like Rebol2 did.

But if you do use the --cli switch then it fails and exits.

C:\Projects\red> echo "print {Hello} 1 + 2" | red --cli
--== Red 0.6.4 ==--
Type HELP for starting information.

*** Internal Error: not enough memory
*** Where: append
*** Stack:

Ren-C Prints Console Banner/Prompt, Then Infinite Loops

Ren-C unified the way Windows worked somewhat with the way that the Linux build had worked, in terms of the relationship between the console and standard input. Hence the behavior (and failure modes) are similar.

Both of these get into an infinite loop, because the console uses ASK TEXT! in usermode and calls it in a loop. There's no special handling of pipe errors to break that loop, so it just keeps trying READ-LINE over and over again.

So Nothing Historically Works. What's The Plan?

We need to detect a piped situation and not print out console banners, prompts, or expression evaluation results.

I'm guessing this would imply not running the console at all. That seems saner than running the console in a mode where it still did the same REPL loop, but ASK had to worry about things like noticing when a pipe disconnected...and the printing out of expression evaluation results had to be overridden by hiding the output.

So that would imply that running from a pipe would be much more like running a script that was read from a file. But a pipe may not come up with its data all at once...it could come character by character, or line by line. What execution model would apply to pipes?

What Does Python Do?

Python3 seems to wait until all of the data is read before executing. If we pipe the output of the "cat" command (e.g. me typing lines of input) into Python3

$ cat | python3
print("Hello")  # I typed this line, hit enter
print("Goodbye")  # I typed this, hit enter, then Ctrl-D to end input
Hello 
Goodbye

Their logic for starting up does an interactivity detection, which can be driven by things like configuration flags vs. just guessing. If the various phases opt out of interactivity, the whole thing falls through to run using the same method as running a file... PyRun_AnyFileExFlags.

if (stdin_is_interactive(config)) {
    config->inspect = 0;
    Py_InspectFlag = 0; /* do exit on SystemExit */
    int exitcode;
    if (pymain_run_startup(config, cf, &exitcode))
        return exitcode;

    if (pymain_run_interactive_hook(&exitcode))
        return exitcode;
}

/* call pending calls like signal handlers (SIGINT) */
if (Py_MakePendingCalls() == -1)
    return pymain_exit_err_print();

if (PySys_Audit("cpython.run_stdin", NULL) < 0)
    return pymain_exit_err_print();

int run = PyRun_AnyFileExFlags(stdin, "<stdin>", 0, cf);
return (run != 0);

Conclusion

Long story short is that I think by default, input redirection should probably be such that:

$ command | r3

...acts basically like if you'd written:

$ command > output.reb
$ r3 output.reb

We'll want to offer more flexible behaviors for if you're processing data, e.g. line by line. But for source code it probably makes less sense to run it line by line.

Maybe there could be a console automation mode where it acts like you'd typed line-by-line and prints all the output the console would print--but that's a weird feature that doesn't seem terribly useful.

Of course, you should be able to write a script that reads from the stdin like a file...line by line or character by character, and process it. But that's a different question.

3 Likes