In 2019, I made a small debugging demo:
The "bad" (actually good) news is that those sticks and glue are all gone. Switching to stackless removed the hooks for writing code that intercepted the evaluator.
The "good" (no, actually good) news is that I've accomplished this using modern methods:
The methods that accomplish this are legitimate, and not likely to be overturned in the design.
However, they aren't part of a generalized notion of debugging. This debugger is tailored to PARSE, and parse only.
Not only that...it works only on a single parse call. You make a specific call to parse parameterized with a hook, e.g. PARSE-DEBUG. Only that call should be hooked. So if the debugger uses PARSE in its own implementation (which it does) it needs to use the non-DEBUG form to do so, otherwise it would break the in-progress debug session by interfering with its interface.
Could The Evaluator Use The Same Methodology?
PARSE-DEBUG is based on the idea of hooking a frame... using ENCLOSE. All parser combinators are ENCLOSE'd when they are built, with the ability to be hooked.
I've discussed the idea of the main evaluation process being driven by EVALUATORs, much like PARSE uses COMBINATORs. So you'd have a SET-WORD! evaluator function that took the SET-WORD! it was invoked with, and a variadic feed of code coming after it.
Under this idea, you could actually create something like REDBOL-DO, which had a different EVALUATOR for PATH! to do variable picking (while Ren-C's default evaluator would do variables with TUPLE!).
I think it could work. But it does mean that variadics are needed by the language, because the FRAME! built for evaluators can't see to the end of how much that evaluator is going to consume. (I'd been mulling over killing variadics as a "it's too much" feature now and again, but this kind of proves that's a bad idea...and they actually just need to be fixed up to be less of a mess.)
It would be a pretty gnarly performance penalty to turn every SET-WORD! or GET-TUPLE! into a function call. But I have an inkling on how to make it cheaper. As it happens, there's a stack frame already for the right hand side of the evaluation... and what this could do would be to morph that stack frame on-demand so it looked like a function call...but only when the debugger looked at it. It just might work.
Could PARSE-Debug and EVALUATOR-Debug Unify?
You run into some interesting questions in terms of whether you want to debug the combinators of a PARSE, or if you want to debug the evaluator implementing parse. Or if there's some way of switching modes.
Being able to switch modes on a whim would be something like being able to switch into an assembly view of an already-running C function in a C debugger.
Right now the PARSE debug demo is dependent on the hooked parse building up and tearing down its own stack level list in the combinator hook. That's already a problem, because it means that you can't trigger a breakpoint at an arbitrary point in the parse...because it only maintained this level list if it was tracing.
So the unified model would depend on some way to enumerate the stack API, and be able to make decisions about who to ask about rendering a stack level. Because a combinator is both an evaluator level and a parse level.
Prototyping EVAL Debugging in ReplPad is Probably Best
After having put together the PARSE stepping demo, I think that it makes the most sense to build debugger prototypes in the browser.
I continue to think this needs to be pushed on sooner rather than later. (I know it seems quite a bit "later" in the scheme of things, but there's always more later to come. :-P)