Pruning Complexity: Letting Go of "Derived Help"

Domains Help System
1

Almost exactly 3 years ago, the first Ren-C function derivation mechanism showed up: SPECIALIZE.

And with it came a question. What would the HELP for such functions say? The HELP string for PICK had historically been different from the help string for FIRST, and now FIRST was just a specialization of PICK.

Under the tentative name "REDESCRIBE" an idea was sketched out for updating just the parts of a function spec that you wanted to change. It took two arguments: a BLOCK! (that looked like a subset of an ordinary function spec), and a function to mutate the description of. Which could be a specialization, or not:

 first: redescribe [
      {Get the first item of a series if available, otherwise NULL}
 ](
      specialize 'pick [picker: 1]
 )

Concerns About Copying Parameter Descriptions

Early Ren-C development feared increasing the number of cells in code by too much. So if you had a function like APPEND which took 7 arguments and refinements, then making a "meta OBJECT!" which captured that as a mapping from parameter to object would be 7 cells of storage (rounded to 8 in a pool).

This object would be convenient and save people from having to parse the spec block to rediscover Rebol's interpretation of the information to implement HELP. And by making it a FRAME! it could use the function's own parameter description array as its "keylist" (a fundamental trick used by Ren-C), so it really would only be the 8 cells...not 16.

Then, if you run the code to create a function and then don't run it again, the original BLOCK from which the parameters were created would be GC'd. Assuming you had a description for every parameter (more or less) you'd pretty much break even.

How Many Copies in a Derivation?

If you derive functions like FIRST, SECOND, THIRD, all the way through TENTH off of PICK, then should each one have a duplicate of PICK's information? Or is it enough to have meta information that points and says "FIRST is a specialization of PICK", and then follow that link through to get the parameter descriptions from PICK?

Plus, having the meta information say "FIRST is a specialization of PICK" may provide useful knowledge.

Oh, what a tangled web...

This led to the existence of dig-action-meta-fields and other rather convoluted code. Every function that wants to do surgery on the HELP information gets burned by it. And if you ever do want to do surgery, you have to flatten it all anyway. Not to mention that it's only an object the size of the parameter cells involved...the actual description strings can appear multiple places without being copied. And if someone changes the help information on a base function, it will change the help information on derived functions--which you might want, but you also might not.

Because the routines were complex and early in the bootstrap, they have been good examples of generalized Rebol code that has had to weather core changes. It has been educational to have them. But the complexity represented gets in the way of the most basic things people want to do with manipulating HELP on derived functions...while not truly providing the intended value.

New Plan: Fundamental Derivations, Flat Help

Rather than investing deeper in this tangential design: help information will now be flat. The META information will be captured from the function being derived from at the time of function creation.

But to make things easier to understand, the derivations will be userspace code. Fundamental operations which do not create any HELP or META information at all will be available as SPECIALIZE*, ADAPT*, CHAIN*, ENCLOSE*, AUGMENT*. Performance-sensitive code (or functions that aren't meant to have their own HELP) can use these, and avoid the overhead.

As for being able to tell how a function is implemented, that seems to be a problem for SOURCE. We'll have to work on the rendering there to make that informative enough to show you that something is a CHAIN or a SPECIALIZE-ation of another function. That seems the right place to be conveying the information.

2

As an example of code that's being let go, here is DIG-ACTION-META-FIELDS. The problem isn't necessarily that the code itself is that complicated (and it could be improved). Rather it's the broader conceptual problem with burdening other parts of the system with the idea of having to go through such levels of abstraction to work with the information.

; Actions can be chained, adapted, and specialized--repeatedly.  The meta
; information from which HELP is determined can be inherited through links
; in that meta information.  Though in order to mutate the information for
; the purposes of distinguishing a derived action, it must be copied.
;
dig-action-meta-fields: func [value [action!]] [
    let meta: meta-of :value else [
        return make system/standard/action-meta [
            description: _
            return-type: _
            return-note: _
            parameter-types: make frame! :value
            parameter-notes: make frame! :value
        ]
    ]

    let underlying: try ensure [<opt> action!] any [
        select meta 'specializee
        select meta 'adaptee
        first try match block! select meta 'chainees
        select meta 'inner
    ]

    let fields: try all [:underlying | dig-action-meta-fields :underlying]

    let inherit-frame: func [parent [<blank> frame!]] [
        let child: make frame! :value
        for-each param words of child [  ; `for-each param child` locks child
            child/(param): maybe select parent param
        ]
        return child
    ]

    return make system/standard/action-meta [
        description: try ensure [<opt> text!] any [
            select meta 'description
            copy try select fields 'description
        ]
        return-type: try ensure [<opt> block!] any [
            select meta 'return-type
            copy try select fields 'return-type
        ]
        return-note: try ensure [<opt> text!] any [
            select meta 'return-note
            copy try select fields 'return-note
        ]
        parameter-types: try ensure [<opt> frame!] any [
            select meta 'parameter-types
            inherit-frame try select fields 'parameter-types
        ]
        parameter-notes: try ensure [<opt> frame!] any [
            select meta 'parameter-notes
            inherit-frame try select fields 'parameter-notes
        ]
    ]
]

Additionally, these were the prototypical META objects which were filled in as the meta information for derivations. The idea of having these defined as prototypical objects was that it would save on defining the keys for these objects for each ADAPT or CHAIN or SPECIALIZE. They'd reuse the keylist and only need to have an array long enough for the variables.

Each had a "description" field due to the belief that nearly every derivation would want a new description, and that it was rarer to want to change the return type or return description, or any of the parameter properties. So changing the description alone could be done without "flattening" the help information. But any other changes would flatten it (effectively leading to the situation now proposed as the default, avoiding this complexity).

(My hypothesis is that all this is getting in the way, and what people really likely want is a SOURCE command that gives insights into the derivation.)

; The common case is that derived actions will not need to be
; REDESCRIBE'd besides their title.  If they are, then they switch the
; meta archetype to `action-meta` and subset the parameters.  Otherwise
; HELP just follows the link (`specializee`, `adaptee`) and gets
; descriptions there.

specialized-meta: make object! [
    description:
    specializee:
    specializee-name:
        _
]

adapted-meta: make object! [
    description:
    adaptee:
    adaptee-name:
        _
]

enclosed-meta: make object! [
    description:
    inner:
    inner-name:
    outer:
    outer-name:
        _
]

chained-meta: make object! [
    description:
    chainees:
    chainee-names:
        _
]