Updating TLS; bounty

So, I think this just basically means getting the elliptic curve encryption suites with SHA384 if we want to be up to date.

Anyway, we do need an indication of how much, if anything, people are willing to contribute to this bounty before I approach Cyphre again. I presume his estimate was in USD and not Koruna!

How would one actually go about sending money?

Count me in up to 50% assuming a $2k cap. I can adjust depending if there are other contributors, or if there are additional features we can get into the deliverables.

1 Like

I'm the treasurer for the Rebol Foundation so you can send the donation to me as PayPal, crypto, or whatever.

Thanks. Really appreciated.

I could donate $50 when I refresh my memory on how to use paypal.

1 Like

All good. Thanks.

Running total is now USD1050.

Count me in for $100

1 Like

Thanks .. running total now USD1150.

So we've got $1150 to work with so far. What does Cyphre want to get started? (I assume he's not expecting the $ up-front.)

He hasn't replied to me yet about that.

Ok, thanks. I just wanted to follow-up on this so it stays on the radar.

Basically he hasn't replied since he set the fee he was looking at. Maybe there's someone else amongst the old guard who are willing to do this.

Still no reply, eh? That's a disappointment.

@BlackATTR what sites do you have for testing that fail with the current ren-c builds?

I know that https://discourse.org fails

https://www.rci.com is an important one for me.

1 Like

Ok, @hostilefork is reworking the TLS scheme to make it more readable prior to updating it. But he's living out of his car out the moment so it might be paused while he's finding new digs.

And https://www.rci.com doesn't support TLS 1.0 which is why ren-c can't read it.

Though I don't like taking people's money, it wouldn't hurt to have some. :slight_smile: But bounties are difficult things...this article covers a lot of the points:


Financial incentives aside, I do appreciate that you have supported the development a lot as it is! So I had sifted around some about a week ago to understand the situation with TLS well enough to know what's involved.

That's from the perspective of "what the heck is with these different versions anyway", along with some amount of familiarity with what exactly it means to "support ECHDE" (e.g. there are a ton of different ways to calculate even thingsl like an 8-bit CRC, exactly how much code and variations of it would the binary need?)

So...what's up?

"Black Box" Implementations

One thing that feels like it may be a "bad" way to spend time would be to make an extension that does a simple, synchronous, TLS-aware READ and WRITE. I say it is bad, because that would be putting the TLS functionality into a black box...which makes it basically useless if you were writing some more interesting client or server program, holding a connection across several separate reads and writes.

But such black boxes exist...like libCurl (which really just gives you the services of curl without having to ship a separate EXE or exchange data through intermediate files). But on Windows, you don't even need that: the OS has the (IHttpRequest) built in. It's not rocket science to invoke...though it does use COM, which is a little ugly when called from C. Red's simple-io uses it, see %red/runtime/simple-io.reds

I'm not exactly clear on what blocks people from active experimentation. It's hard for me to understand if such a stopgap measure would really move the needle for anyone. Is calling out to curl, e.g. with @rgchris's curl wrapper, not sufficient to get started (assuming such things are brought up to date)?

Building on Cyphre's TLS

The TLS code was a bit of a nightmare for me, because if a core change ever broke HTTPS I was generally in for a world of pain. I had mostly no idea what was going on with it, and spent more time looking back at what behavior I might have caused to deviate vs. actually trying to understand the code.

Now looking over it and sort of getting the direction, I'd say it's not so bad. It was done largely in the spirit of how one might think a Rebol-based program should try to make a kind of "executable spec". So it has state transitions that look like what's described in the protocol, and makes a decent amount of sense. It's not a bad place to start from.

But as a small example of the aspect of it that was bad, look at this bit of code for "set the correct message lengths":

change at ctx/msg beg + 7 to-bin len: length? at ctx/msg beg + 10 3
change at ctx/msg beg + 4 to-bin len + 4 2
append clear ctx/handshake-messages copy at ctx/msg beg + 6
return ctx/msg

Does that look like "code of the future"? Not to me. :-/ What it's doing is that it has a packet structure, but some of the bytes are related to the size of sections in that packet. So it's easier to answer questions about what goes in those bytes after the whole header is formed. But it's using hardcoded numbers to do it.

The reason I picked this is because that's something I tweaked with a tiny dialect. I made it so that the binary spec could have SET-WORD! in it, and those SET-WORD!s recorded the position in the binary development, so you could go back and reference the values in based on ssl-record-length:, message-length: and ssl-record: in the block:

change ssl-record-length (to-bin (length of ssl-record) 2)
change message-length (to-bin (length of message) 3)
append ctx/handshake-messages ssl-record

Writing the dialect took only a minute. Figuring out that's what the original code was doing is what took the time. :-/

But like I said, I don't think the fundamental approach is bad. It's actually a short file, and I think we can keep making it shorter and better. It can be an opportunity to exercise Rebol and show something different...a TLS protocol running on an executable spec.

What does "TLS 1.2" support mean, anyway?

One thing you find pretty soon is that the spec is kind of big, and people make lists of the features or concerns they don't attend to.

Speaking of things that aren't attended to... people might not be aware that there's no "S" in Rebol's HTTPS. It extracts a certificate and then...takes its word. So saying that it has an "https implementation" is really saying "it will talk to anyone who hands it any certificate and encrypt the communication taking its word".

Doing certificate validation is...kind of...important? Anyway, that's an undeveloped area which would need attention.

Using a C TLS

If one is not going with a "black box", then one could imagine hooking up a Rebol-based HTTP or TCP server scheme on top of a C-level TLS implementation. It would appear native, the way that the TCP scheme does today.

It would be faster...but more importantly, it would be written and already support 1.0/1.1/1.2. Someone else would be responsible for maintaining it.

There are various options out there. People make OpenSSL out to be somewhat beastly, but it's documented and well-known. @gchiu has pointed out stuff like TLSe, which is a "Single C file TLS 1.2 (also 1.1 and 1.0) implementation, using libtomcrypt as crypto library"...very much in line with Rebol's stated mission.

If there were some kind of rush and people needed working TLS on a short-term timescale, I'd say this is the more reliable option. But the only thing to splice it into right-now-today would be the Rebol PORT! model, as a swap-in for a TCP port. And if the only usages people are imagining are HTTPS read and HTTPS posts, vs. writing some kind of interesting client/server with HTTPS needs... it doesn't make sense to do that rush instead of the black box!


TLS is just a protocol for two parties to agree on how to talk. When they do, they use a group of secure hashing and encryption functions. A client saying "hello" to a server offers it a list of what it can support, and the server looks through the list and picks the one it prefers.

One issue is when the server says it doesn't even like the client protocol itself. ("You said hello wrong".) So TLS 1.0 isn't being supported anymore. But the other is, that even when it is...or when you update the protocol, you still might not be able to speak a "cipher suite" that the server will accept.

Whether one sticks with the Rebol TLS or uses a C one, you still have to worry about having ciphers. Linking in new ones isn't exactly rocket science. You can get them various places...and people have brought up libSodium, which offers a "curve" which is compatible with ECHDE.

One edge C TLSes have is that they are often already geared up to use popular ciphers. So they are lined up in a "turnkey" way to just set a compiler option and use it, all the interface code for getting bytes back and forth from the cipher to the protocol has already been written by some poor soul. But for Rebol, each cipher implementation has to be wired up to read a BINARY! and and write a BINARY! out, etc.

Anyway it's not impossible, and it has been getting better rapidly due to libRebol. If you look at some of the code in %mod-crypt.c and compare it to the state of things in Saphir when it had to go through the RL_API, it's just night and day clearer--and more fault-tolerant. Still it's work that has to be done.


Previously I kept the %prot-tls.c around mostly because read https://example.com was one mean piece of test code, running a lot of stuff. If you asked me if it had a future, I'd have said probably not...and that it would likely be swapped out for some commodity C code at some point.

But I now think it probably does have a purpose and is relevant in telling the story of the language, and demonstrating its abilities. It does have a long way to go, though...as I mention, the certificate validation is a pretty big deal.

So it would be interesting if Cyphre wanted to come back and give it a revamp. However, there's a lot of friction with some other old-time Rebol people over the fact that I fix things that are broken (and yes, sometimes shuffle things around exploratorily while searching for those fixes, which is necessary). So, there's that. But as a bigger issue--I don't really know that a TLS of any kind, C or Rebol, has much purpose beyond a black box until the ports have more sorted out about them.

So on that note: If people showed me working programs they care about that were in Ren-C, that relied on calling out to curl.exe, and that there was a good reason why eliminating that curl dependency was important, I could consider that a reason to be compelled to do the black box versions. Getting https READ and POST to work on Windows via calling OS services would take a little time to write and debug, but not be terribly hard. libCurl is easy to compile with and link on linuxes for that functionality, also.

I can use (external) Curl for my short-term needs. And medium-term Curl could be embedded in Ren-C, as Red does. I do feel that Ren-C should handle common web protocols seamlessly, otherwise it will cease to be a "batteries included" network-ready language.
Ideally a solution would be part of Beta/One. Hopefully without stealing focus from more important features.
Happy to revise bounty terms/details and to venmo working solutions which are Ren-C worthy.