Remember, I come from well outside the Rebol world. The reason I don’t understand is, to a large extent, because this way of reasoning about code is one I’ve never seen before in my life. This is the reason I’m asking so many questions, setting out my opinions, and having these discussions with you… precisely so that I can begin to understand why it is that way.
2 Likes
Yes, though also remember... 
I'm not advertising anything here...not fit for any purpose, or any use!
(I also didn't make this forum. That was done by @gchiu, and I initially protested it as something that would fall in my lap as yet-another-maintenance-headache. But I found Discourse could be a rather good notebook, with easy wiki-like reorganization features to mean the site wouldn't wind up stale and useless. So I ended up embracing it.)
I first met Carl Sassenrath at a party in Los Angeles. Rebol was not open source at the time. I'm a pretty big believer in Stallman's "Why Software Should Be Free"
(I also believe FSF should flip the acronym as "Foundation for Software Freedom", and that generally "Freedom" should be used where "Free" creates many confusions with "Gratis". But Stallman's idealism/autism makes him convinced that retaking the term "Free" to mean "Freedom" is so inherently logical that putting his thumb on the scale will change how everyone else hears the word.)
Carl had resisted open-sourcing Rebol. Over whiskey at the party's open bar, I lobbied him on the inevitability of developer tools as open source, and that closed-source couldn't be profitable. Much more profit would be opened up by being a steward of a successful open source project.
He said he didn't keep it closed-source to make money, rather that he "didn't want anybody messing with his art." Rebol was his "grand experiment" and he had very specific opinions about what was or wasn't an improvement of it.
Eventually it was open sourced. He came to face the fact he wasn't going to be able to deliver Rebol 3 himself, in the time he had, with the methods he'd been using. He wanted to move on with other things (went to work for Roku, on embedded-C products, closer to his EE background). So now it's in the hands of others.
While Carl and I may disagree about what exactly the artistic vision is, we do agree that it is an art... the medium is unique... and driven by a certain je ne sais quoi of what makes a good or bad decision. A big part of what we both believe is that dependency control is critical, and that a useful implementation can be made and understood without requiring a very advanced toolchain (and that the substrate not be intrinsically obfuscating, see recent xz hack for how important this all is).
The time I can put into the project is not infinite...and I haven't had much time in March (and will likely not have a whole lot in April, either). And I am a semi-retired person who is not paid to do this, so I heavily prioritize whether I'm having fun or not. It can be a real slog at times, and if a novel mechanism doesn't pop up every now and then to renew my interest...I can lose motivation.
I'm grateful that this year started off with cracking some of the rationale for how binding has to work, so thank you for your contributions to that! It gave a boost and a renewed faith that good-enough answers are possible. I've been letting that settle a bit, and sometimes these things take time to shake out vs. being able to finish the design all at one time. So I work on other things--sometimes just what interests me--while experience is gathered to realize where to point the big design questions.
But I don't expect anything to happen overnight. This has been a long haul project which needs a marathon mentality, and so patience is mandatory.
3 Likes
My apologies for taking so long to reply to this; I just moved countries (for research) and have been rather busy of late.
(Also, this is an important point, and I wanted to make sure I fully understood what’s been bothering me before writing anything.)
I think the key point here is this:
To state the obvious, je ne sais quoi literally means ‘I don’t know what’. (You learn these things, living in France.) And I feel this is really the underlying fact which makes me so uneasy when discussing Rebol design: it’s not necessarily obvious what justification to give for specific design decisions, beyond vaguely saying that ‘I feel it makes code generally simpler’.
The languages I’m used to are not at all like this. Haskell and family, for instance, are strongly informed by the mathematics of type theory. The core language is rigorously worked out in these terms, and everything follows from that. If someone has a bad suggestion, it’s generally obvious as such, with strong arguments along the lines of ‘it would make the language unprincipled in such-and-such a well-defined way’. This sort of thing helps to keep everyone on the same page, and gives the language development an overall direction to aim for.
A fairer comparison to Rebol might be Lisps, which don’t care nearly so much for mathematical rigour. But on the other hand, Lisps tend to have a very strong tendency towards minimalism, which helps to direct language development in a similar way. You can argue for ages about what ideas to base your Lisp on, but it’s generally possible to ground those arguments in the goals of minimalism and expressivity. And, once you’ve written out the (often quite short) specification, everything else just follows from that.
Now, returning to Rebol: what drives its language design? As far as I can tell, the answer seems to be: ‘whatever hostilefork feels is suitable’. Not that it’s unique to Ren-C by any means: Red feels largely driven by ‘whatever Nenad likes’, and similarly for Carl and historic Rebol. And then we have Oldes’s fork, which is yet another set of personal preferences.
To be quite clear, I don’t consider this to be a bad thing by any means. Making a language optimised for onesself, personally, is a lovely thing to do. (Probably I’ll try it myself at some point.) And as an artistic vision, it’s perfectly respectable. It just means that I’ll inevitably be of limited help with the design, insofar as my personal preferences diverge somewhat from yours. But as long as it’s clear that that’s the case, I’m OK with it.
(It’s not completely subjective, of course. I do think that Ren-C is an objective step forward compared to other Rebols, when it comes to ‘big picture’ stuff like isotopes and definitional returns. It’s on the little quibbles, like ‘which datatypes do we want’, that personal preference comes into play more than it does with Haskell or Lisp.)
1 Like
(... and definitional BREAK + CONTINUE ...)
The other Rebol derivatives quickly retreat to implementing dialects in C (or Red/System). They excuse it with "well, it would be too slow otherwise". But they literally don't know how to write and maintain non-trivial systems in the medium being discussed, without so many workarounds that it becomes more an exercise in constrained writing. Projects execute in spite of the language, not because of it... and that's what I'd call a Turing tar-pit.
I want to show cases where the language solves problems in an impressive and novel way... that doesn't fall apart like a house of cards the moment the problem statement changes by one word.
So calling Ren-C an "objective step forward" is accurate. 
The Rebol3 initiative put nearly every decision in Rebol2 under scrutiny, allowing for arbitrary incompatible changes...if they fixed fundamental issues. I found the language in the middle of that discussion. So I assumed "this all needs VERY serious rethinking" was the prevailing belief. But most who were questioning the design had been involved for a decade already, and were burnt out...so they wandered off. The few of us here are all that's left of Rebol3 (and Oldes, who is prolific in wiring up libraries, but not too concerned with language design 
)
The Red crowd had a completely different mission, of an open-source Rebol2 written in a C-like systems dialect. They considered Rebol2 to have been "good enough" to be copied more or less verbatim, and that deviating from it would cause delays and a requirement to write new documentation. If you show them a broken scenario they'll occasionally acknowledge it as bad. But the overwhelming tactic is to say "no language is perfect" (and Gregg can fill pages with saying this in about 100 different ways) vs. attempting to fix the problem.
Despite that, today's Red is incompatible with Rebol2 for seemingly trivial reasons, and is still missing many of its features (and backpedaling on open source). "Whatever Nenad likes" seems to be the edge cases of the GUI design. Because regardless of what they say publicly about the language or their goals, they envision RED.EXE winning the battle against the ever-bloating web browser. (They don't say it often, because their quirky insecure 32-bit design sounds like a poor bet against the sunk cost of the globally-vetted browser everyone has installed already.)
I'm a fan of consensus... when people are willing to work toward it... and you've been helpful so far!
I feel that the examples I look at provide a lot of food for thought. The Whitespace Dialect alone provides years of deep questions to work on. Emulating Rebol2 is a challenge. When I mix it all up with code that hasn't been written by me (e.g. @BlackATTR's Query and @gchiu's Midcentral) it gives me a good gauge of what is still left to do before I can be happy.
It would be good if you had some small project to add to the continuous integration roundup--something that I'd take responsibility for having an answer to keeping working as the system evolves. It would be code you were invested in, but not too invested in. 
Small scale, so no sequencing of genomes. This may help empathize with both what the joys of the medium are that I seek to distill and enhance, as well as the practicalities of what a slog this can be...
A new country to give a Rebol talk in! I've moved to the other side of the US (vertically) and have been busy as well...
Serious question: is Ren-C any better in this regard? For instance, we do have a UPARSE implementation in Ren-C, but in practise the key function is commented out in favour of a C implementation — precisely because it’s too slow.
This is interesting context, thanks!
This actually gives me another thought about the difference between Ren-C and the other language communities I mentioned: the others all have much larger communities.
For instance, in a Haskell language discussion, you can easily have ten or so people arguing about the merits of a proposal, and often more than that. That lets maintainers make an informed decision given the thoughts of many people. Even in Lisp, the discussions I’ve seen can easily get 3–5 people weighing in — and I’m not in any Lisp-specific fora online.
By contrast, most discussions I’ve had with you… well, they’ve mostly just been me and you. Two people. It’s possible to get a consensus with two people, but it’s not as easy, because there’s no good way to choose when they have differing views. Perhaps if Ren-C had a larger and more active community, language development would become easier.
You’re absolutely right here. I’ve been discussing a language I don’t use, and that’s not great.
Now that you mention it, I actually do have one project which would be well-suited to Rebol: a sound change applier. The concept comes from historical linguistics, but at it heart it boils down to something like a glorified regex-substitution program. I’ve developed a habit of writing one of these in every language I use seriously, with my latest and most successful being in Haskell.
[EDIT: apparently my webserver is down. Changed the link to GitHub instead.]
[EDIT 2: back up again! See https://bradrn.com/brassica.]
What’s particularly interesting about this is that it’s a perfect fit for dialecting. One of the biggest motivators for having a dedicated sound change applier is that it gives you a convenient syntax to use, so making it a dialect is an obvious choice. A sound-change applier embedded within a programming language could let you do a lot of very interesting stuff. Ren-C probably isn’t mature enough to make a really usable application — for that I’d want some form of interactivity, at least — but a simple implementation should certainly be doable.
Of course… like I said, I am busy. And even within the realm of SCAs, my primary focus is on improving my existing mature application. But it’s something for me to keep in the back of my head for when I get the chance.
COMBINATORIZE is in no way "the key function"! And it wasn't nativized in order to avoid something usermode could not accomplish. It was nativized with a few other things as an experiment... but the other things were all reverted because the design changed. COMBINATORIZE didn't change much (yet), so it was left using the native version for now. If the usermode code atrophied, that atrophy is hopefully minor.
Ultimately the goal is to nativize everything, but keep the scaffolding which wires everything together communicating in the abstractions of the language... and live-swappable with usermode implementations that do the same thing, just slower. (I've mentioned that I want to put the Rebol and C code for a given combinator in the same file, and have the Rebol code extracted and aggregated from comments in the C at build time. The same would apply to implementation guts like COMBINATORIZE.)
I've already demonstrated a primordial debugger for UPARSE. It depends on being able to programmatically dissect the stack frames of combinators, and that relies on those frames being bona fide FRAME! objects within the language. That's instead of some customized "parse stack" that only applies to parsing (e.g. Red's PARSE) and which would require a whole slew of specialized hooks to expose, and whose optimizations wouldn't speed anything else up.
What is nice is that by doing it my way, cross-cutting enhancements will help everything improve...not just UPARSE. The glue between combinators will still be in terms of usermode-exposable abstractions, so that you can mix and match native combinators and usermode ones. Using such methods does have cost over customized byte-level structures. But it's worth it for the reflection abilities--and of course allowing proof of the ability to express it within the confines of the language if you need to.
So yes: Ren-C is better. 
A large community of extremely patient people... maybe. But large communities of patient people who work for free on weird things are rare.
Large usually correlates with people being invested, and those who invest in things will resist changes that break their systems. To me, Red's thesis--that Rebol2 somehow is an end in and of itself--was disproven long ago. But I'm happy enough to let them continue chasing the three people in that audience.
I don't know what all the remaining massive breaking changes are, but there will be some. I do think "cut down stray bindings so you can assume bindings you encounter are intentional and need not be overridden" was a good example of one that will be rippling for a while.
Right now, I've decided that the third array type of FENCE! opens up a lot of design space that is interesting. That interestingness is scaled up as the viability of binding and dialecting ramps up. It's a tricky change affecting strings, and raising questions with how to have alternative loaders (e.g. if I want to continue pursuing Rebol2 compatibility layers).
So I'm still glad there aren't that many users to deal with disrupting...
Well, if I learn enough to understand Brassica in forming a Rebol version, maybe I can figure out how to contribute to the Haskell implementation as well, and it won't be a net loss of time.
I wasn’t thinking of COMBINATORIZE, but rather COMBINATOR itself, which has the following comment next to it:
; !!! COMBINATOR is used so heavily that having it be usermode makes UPARSE ; prohibitively slow. So it was made native. However the usermode version is ; kept as a proof of concept that a user *could* have made such a thing. It ; should be swapped in occasionally in the tests to overwrite the native ; version just to keep tabs on it.
One could quibble about how ‘key’ this really is, but I think my point stands.
I don’t know so much… there’s a lot of people interested in programming languages! At least some of them have found Rebol intriguing when I’ve mentioned it.
(The ‘constant disruption’ is certainly a barrier, but I think it’s something that early adopters are aware of.)
Thanks for the offer! Although personally, I think that most of the remaining challenges are on the design side of things — implementation is usually straightforward, once I’ve worked out the details.
It would if the comment weren't outdated...!
It's outdated, and COMBINATOR is running as usermode, not commented out.
But like I say: the eventual goal is to leverage native code as what the comment says. But keep the architecture be based around things that reflect and interchange with what could be entirely accomplished with usermode code, only slower.
That is a fundamental distinction from other approaches, that would not uncover issues like the use of CONTINUE in a GROUP! in PARSE when that could bind to code internal to the parse implementation. There are many examples like this you hit while trying to fully implement a system in usermode.
Ah, good to know! In that case I think the comment should be removed.