The most easily recognized form of encryption is symmetric, where the same password is used to encrypt something as to decrypt it.
But a weirder form is asymmetric encryption, where the key used to encrypt is different from the one used to decrypt.
RenC Supports RSA Public/Private Encryption
You generate your keys in a pair, which come back via multiple return results. You must specify a size for the public key (in bits, not bytes...because that's the convention)
We'll start using [raw] RSA encryption:
; use /INSECURE to override errors that tell you the key is too small
; (makes examples more readable)
>> [publickey privatekey]: rsageneratekeypair/padding/insecure 128 [raw]
== make object! [
padding: [raw]
n: #{C097238C34E7191561DD7D30BBB77C65}
e: #{010001}
]
>> length of publickey.n
== 16 ; 16 bytes is 128 bits
When you use raw mode encryption, the data you encrypt must be exactly the same size as the key you made. We'll show how to work around this later, but for now let's just deal with it.
>> data: #{0123456789ABCDEFFEDCBA9876543210}
>> length of data
== 16 ; same size as our key
>> encrypted: rsaencrypt data publickey
== #{3F7E8ACD2DEE61D09B7A6FC914E22295}
>> length of encrypted
== 16 ; also same length as key
Now, decryption is done with the private key:
>> rsadecrypt encrypted privatekey
== #{0123456789ABCDEFFEDCBA9876543210}
But [raw]
Makes The Same Message Every Time...
Raw RSA doesn't waste any space (e.g. 4096 bytes in means 4096 bytes out). But a given output always produces the same output with raw RSA:
>> rsaencrypt data publickey
== #{3F7E8ACD2DEE61D09B7A6FC914E22295}
>> rsaencrypt data publickey
== #{3F7E8ACD2DEE61D09B7A6FC914E22295}
This can make it easy (or at least, easier) for someone who doesn't have the private key to do some factoring with the public key, and see if they can generate input that produces that output...effectively defeating the encryption.
Also since it requires exactly the keysize of data coming in, you're likely going to have a lot of cases where you have to throw in some padding. But naive answers to padding also create security problems:

It would be bad to pad the data block with zeros (or whatever), because such predictable behavior generally makes it easier to guess what the encrypted information might be.

It would be bad to just put random data in the padding, because the person doing the decryption wouldn't know if the gibberish they got when decrypting was the same gibberish you put in.
 Attackers could leverage this random tolerance to forge data in the nonpadded portions, then forging the padding in a way that compensated.
RealWorld problems motivated the tricks that are used to resist attacks. You can read about them if you want...or just don't use [raw]
and you'll get a sensible default.
>> [publica privatea]: rsageneratekeypair/insecure 128
== make object! [
padding: [pkcs1v15]
n: #{A9EE9282744CC0FC6765824EB2B87539}
e: #{010001}
]
>> one: rsaencrypt #{DECAFBAD} publica ; Note: shorter input than key
== #{13C14811A3B1CD95100BA4F3273F0962}
>> two: rsaencrypt #{DECAFBAD} publica
== #{3245179A18C6B488FD39CDE7B4F5E3EC} ; different!
>> rsadecrypt one privatea
== #{DECAFBAD}
>> rsadecrypt two privatea
== #{DECAFBAD}
That Encrypts Smaller Than The Key...But What About Bigger?
A fundamental issue with RSA encryption/decryption is it is relatively SLOW. And if you want to futureproof a key to use with big file sizes, those keys would also be BIG!
But you can use a smaller asymmetric key as a stepping stone to providing a symmetric key.
It's easy! Let's say someone has published their 4096bit public key and you want to send them a 100 megabyte file that only they will know how to open. Follow these steps:

Generate a random string that's less than 4096 bits, and encrypt that with their public key

Encrypt the file using a faster symmetric algorithm, with that random string as the password

Send the recipient both the asymmetricallyencrypted random string and the symmetricallyencrypted 100mb file

Your recipient uses their private key to decrypt the oneoff password you randomly generated, and decrypts the symmetrically encrypted file with it.
It's All New, So Help Design the Interface...
The original motivation to research mbedTLS was so that @gchiu didn't have to call out to external utilities to generate RSA keys.
But the generation abilities weren't actually written until now. And honestly, I'm not sure the RSA stuff was even working at all. :/ Now there's at least a minimal amount of testing.
I think the multiple return values are a good start on making it clear, but it's certainly a good time for feedback if anyone has any.