Dropping `<with>`, `<in>`, `<static>`

Development Functions
1

In R3-Alpha, there was the idea that FUNC was lower-level, and "FUNCTION" was built on top of it.

Its principal difference was automatically collecting SET-WORD!s as locals. But it added two refinements: /WITH and /EXTERN.

/EXTERN was a way of saying what shouldn't get collected as a local variable due to being a SET-WORD!:

global-var: 10

foo: function/extern [arg1 arg2] [
    local-var: "hi"
    global-var: 20  ; /EXTERN protected this from being collected local
    return arg1 + arg2
] [global-var]

/WITH was a way of making static variables:

accumulate: function/with [x] [
   return state: state + x
] [state: 0]

Early On, Ren-C Moved Everything Into The Spec

I didn't like seeing the refinement arguments at the end (of what could be a very long function definition).

It didn't occur to me to suggest that refinement arguments be moved to the head (which they probably should). But what did occur to me was that the function spec could incorporate these properties.

I actually thought <with> seemed better than extern, as a nicer word for "Use these existing variables". And <static> seemed like a well-known term for static variables:

foo: function [arg1 arg2 <with> global-var] [
    local-var: "hi"
    global-var: 20
    return arg1 + arg2
]

accumulate: function [x <static> state (0)] [
   return state: state + x
]

The subtlety of wanting to use an object instance was also added, as <in>:

obj: make object! [a: 10 b: 20]

bar: function [x <in> obj] [
   return a + b + x
]

To my eyes, that all seemed like improvement.

But it didn't come without cost: The spec had to be transformed into something the lower-level FUNC could understand.

This meant there was a layer of parsing and production of a new spec that was a tax on every function creation.

Time Passes, SET-WORD! Gathering Is Panned

It didn't take long for me to decide that SET-WORD! locals-gathering was bad... a gimmick that only made sense in very limited domains (perhaps code-golf)

This motivated having an answer for how to implement LET, as "virtual binding" became the new plan.

Once that transition went through, the effect of <with> was to become commentary. Since all it did was remove SET-WORD!s from the collection list, and there was no collection any longer.

Though virtual binding did open up a new possibility, that if your block had a different binding than the spec, then the WITH might import visibility of terms to that block:

global-variable: 10

block: /get-block-from-somewhere ...  ; doesn't know about GLOBAL-VARIABLE

/foo: function [x <with> global-variable] block

But this would be a binding operation, that is better generalized as:

/foo: function [x] (bind @global-variable block)

Pushing The Features To BIND Make The Most Sense

Not just <with>, but the <static> and <in> features seemed to be better as BIND operations as well.

The static syntax of not using SET-WORD!s was based on the idea that SET-WORD!s were reserved for local variables (and RETURN: syntax). So it was a WORD! followed by a GROUP! to initialize.

I think it's better done with just BIND to a FENCE!

accumulate: function [
    x
] bind {state: 0} [
   return state: state + x
]

<in> is similar.

obj: make object! [a: 10 b: 20]

bar: function [
   x
] bind obj [
   return a + b + x
]

Uglier? Maybe. More General? Yes. Faster? Definitely.

Feature-wise, there's a slight loss of the commentary capacity of <with>.

global-var: 10

foo: function [arg1 arg2 <with> global-var] [  ; no-op, but useful?
    global-var: 20
    return arg1 + arg2
]

But besides that, there's no loss of features to move everything to a BIND operation on the body.

Not having to PARSE the spec and generate a whole new one is a big performance win.

So I'm letting go of those features.

2 Likes
2

The Dying PARSE3-BASED FUNC Wrapper

Here's a snapshot of the code that was written to PARSE the spec, and produce a new one...creating any necessary objects for <static> etc.

As a shortcut, it first did a search with FIND to see if the spec contained any TAG!. If it did not, it would fall through to the normal func. (Though this did mean that any <local> tags would trigger the longer version, even though that was supplied by FUNC.)

It used PARSE3 because UPARSE was built on top of it, and because it would of course be way too slow at this time.

As I often say about these things... as grotesque as they may seem, they exercise the system asking if we can do certain things in usermode. And it shows a good bet for not having written this as tailored native code, because that would all be getting thrown out right now!

And it's not getting completely thrown out... because it's shifting to be part of the test code (and I'm making it use UPARSE, since the performance no longer matters).

/function: func [
    "Augment action with <static>, <in>, <with> features"

    return: [action?]
    spec "Help string (opt) followed by arg words (and opt type and string)"
        [block!]
    body "The body block of the function"
        [<const> block!]
    <local>
        new-spec var loc other
        new-body defaulters statics
][
    ; The lower-level FUNC is implemented as a native, and this wrapper
    ; does a fast shortcut to check to see if the spec has no tags...and if
    ; not, it quickly falls through to that fast implementation.
    ;
    all [
        not find spec tag?/
        return func spec body
    ]

    ; Rather than MAKE BLOCK! LENGTH OF SPEC here, we copy the spec and clear
    ; it.  This costs slightly more, but it means we inherit the file and line
    ; number of the original spec...so when we pass NEW-SPEC to FUNC or PROC
    ; it uses that to give the FILE OF and LINE OF the function itself.
    ;
    ; !!! General API control to set the file and line on blocks is another
    ; possibility, but since it's so new, we'd rather get experience first.
    ;
    new-spec: clear copy spec  ; also inherits binding

    new-body: null
    statics: null
    defaulters: null
    var: #dummy  ; enter PARSE with truthy state (gets overwritten)
    loc: null

    parse3 spec [opt some [
        :(if var '[  ; so long as we haven't reached any <local> or <with> etc.
            var: [
                &set-word? | &get-word? | &any-word? | &refinement?
                | quoted!
                | the-group!  ; new soft-literal format
            ] (
                append new-spec var
            )
            |
            other: block! (
                append new-spec other  ; data type blocks
            )
            |
            other: across some text! (
                append new-spec spaced other  ; spec notes
            )
        ] else [
            'bypass
        ])
    |
        other: group! (
            if not var [
                fail [
                    ; <where> spec
                    ; <near> other
                    "Default value not paired with argument:" (mold other)
                ]
            ]
            defaulters: default [inside body copy '[]]
            append defaulters spread compose [
                (var): default (meta eval inside spec other)
            ]
        )
    |
        (var: null)  ; everything below this line resets var
        bypass  ; failing here means rolling over to next rule
    |
        '<local> (append new-spec <local>)
        opt some [var: word! other: opt group! (
            append new-spec var
            if other [
                defaulters: default [inside body copy '[]]
                append defaulters spread compose [  ; always sets
                    (var): (meta eval inside spec other)
                ]
            ]
        )]
        (var: null)  ; don't consider further GROUP!s or variables
    |
        '<in> (
            new-body: default [
                copy:deep body
            ]
        )
        opt some [
            other: [object! | word! | tuple!] (
                if not object? other [
                    other: ensure [any-context?] get inside spec other
                ]
                new-body: bind other new-body
            )
        ]
    |
        '<with> opt some [
            other: [word! | path!]  ; !!! Check if bound?
        |
            text!  ; skip over as commentary
        ]
    |
        ; For static variables to see each other, the GROUP!s can't have an
        ; hardened context.  We ignore their binding here for now.
        ;
        ; https://forum.rebol.info/t/2132
        ;
        '<static> (
            statics: default [copy inside spec '[]]
            new-body: default [
                copy:deep body
            ]
        )
        opt some [
            var: word!, other: opt group! (
                append statics setify var
                append statics any [
                    bindable maybe other  ; !!! ignore binding on group
                    '~
                ]
            )
        ]
        (var: null)
    |
        <end> accept (~)
    |
        other: <here> (
            fail [
                ; <where> spec
                ; <near> other
                "Invalid spec item:" @(other.1)
                "in spec" @spec
            ]
        )
    ]]

    if statics [
        statics: make object! statics
        new-body: bind statics new-body
    ]

    ; The constness of the body parameter influences whether FUNC will allow
    ; mutations of the created function body or not.  It's disallowed by
    ; default, but TWEAK can be used to create variations e.g. a compatible
    ; implementation with Rebol2's FUNC.
    ;
    if const? body [new-body: const new-body]

    return func new-spec either defaulters [
        append defaulters as group! bindable any [new-body body]
    ][
        any [new-body body]
    ]
]
1 Like
3

So... deleting this information does seem lossy. It's helpful to see when something uses non-local declarations.

But the premise of the language is that you get these things as visible by default.

Some time ago I wondered about the idea that only functions would have this visibility, and any non-functions you would need <with> to see and modify.

Today that would mean you'd have to explicitly bind it:

global-var: 10

foo: function [
    arg1 arg2
]
bind @global-var [
    global-var: 20
    return arg1 + arg2
]

But this feels like an uphill battle, and you're probably inconveniencing as many or more situations than you are clarifying.

Perhaps it's better to just count one's small victories...e.g. that you have to pre-declare global-var, and typos like globl-var: won't work.

A lot of these cases are really situations where you should make an object with data members and methods, anyway... vs. have some sneaky relationship between global variables and certain functions.

I decided to tolerate <with> in the specs for now

What the native code actually does is a better-than-nothing enforcement, that it checks to make sure the thing you mention for with has a binding. This caught some stale <with>.

and... I Might Bring Back <static>

Since I've been able to easily implement the <local> initialization feature, as well as settle on why it uses a WORD!+GROUP! instead of a SET-WORD, I might be willing to add the feature back.

But as that thread explains, the notation of WORD!+GROUP! limits you in terms of what you can do with your assignments. The workaround for locals is to put the assignments in the body. The workaround for statics is to use BIND.

I'll weigh the pros and cons, but for right now BIND is getting exercised...and the PARSE wrapper that slowed down every function generation is gone.

1 Like