RSA APIs in .NET - a crypto-trek rant

The stardate is 2017-06. In your .NET Enterprise mission to seek out biweekly paychecks you find yourself in need of RSA encryption. You check MSDN and do:

var rsa = RSA.Create();

Problem: the default RSA key size is 1024 (insecure).

You know about it because you've learned long ago that the only way to use .NET crypto APIs correctly is by reading implementation internals - you've seen 1024 hardcoded in the ctor. Defaults are forever, since "the needs of many outweigh the needs of the few" and all that. Fine, you'll set the key size explicitly. You double check MSDN again just to make sure that rsa.KeySize should do the job:

AssymmetricAlgorithm.KeySize: Gets or sets the size, in bits, of the key modulus used by the asymmetric algorithm.

Perfect, let's do it:

var rsa = RSA.Create();
rsa.KeySize = 3072;
// use rsa to encrypt secrets

However, you've been writing .NET crypto code long-enough to have zero trust in the API doing what the code seems to suggest it should be doing. Thus before you proceed to actually encrypt secrets, you do a quick check which you're careful not to commit because of how dumb it seems:

var rsa = RSA.Create();
rsa.KeySize = 3072;
Console.WriteLine(rsa.KeySize); // prints 1024

A junior developer would be surprised, but you're neither surprised nor deterred. You check the RSA modulus, just in case:

var rsa = RSA.Create();
rsa.KeySize = 3072;
Console.WriteLine(rsa.KeySize); // prints 1024
Console.WriteLine(rsa.ExportParameters(false).Modulus.Length * 8); // also prints 1024

Damn. At least you're glad you did not proceed to encrypt secrets with what you thought was a 3072bit RSA key, like a normal person would.

You've abandoned the .KeySize property (no time to care why it does not work), and after some googling you've found that another class, RSACryptoServiceProvider has an explicit keysize ctor. However, you've also seen a brand-new RSACng class added to .NET 4.6. CNG stands for "Cryptography Next Generation" - these must be the bestest crypto APIs to use, since luckily you're on .NET 4.7. Eager to stop blaming MS for the sins of .NET past, you decide to give it a go:

var rsa = RSACng.Create();
rsa.KeySize = 3072;
Console.WriteLine(rsa.KeySize); // prints 1024
Console.WriteLine(rsa.ExportParameters(false).Modulus.Length * 8); // also prints 1024

Nope, that does not work either. You also discover that you expected the API to return an instance of RSACng, but instead it gave you a default instance of RSACryptoServiceProvider. It turns out that .Create() is the same static method on the abstract RSA class. The old .NET API design keeps biting you, but you're not bitter and determined to win the battle:

var rsa = new RSACng();
Console.WriteLine(rsa.KeySize); // prints 2048, a better default
rsa.KeySize = 3072;
Console.WriteLine(rsa.KeySize); // prints 3072
Console.WriteLine(rsa.ExportParameters(false).Modulus.Length * 8); // also prints 3072

Aha! Finally! MS must've fixed at least some issues! Next Generation FTW. You boldly go where few devs have gone before, and try RSA-encrypt:

var data = new byte[640];
rsa.Encrypt(data, RSAEncryptionPadding.OaepSHA1);
// CryptographicException: The parameter is incorrect.

Sigh. Which parameter? Is it the padding? You try all other padding types, including the insecure Pkcs1, and they all generate the same helpful error message. Maybe it's the data? Let's half it in size:

var data = new byte[320];
rsa.Encrypt(data, RSAEncryptionPadding.OaepSHA1); // seems to work

Ok, good. Perhaps 320 bytes are below some upper limit. Fine. However, you've just recalled that SHA1*anything seems to spook the hell out of auditors, and your CSO has decreed that SHA256 is a new corporate standard to be used for everything including meeting room names and phone extensions. You make the change:

var data = new byte[320];
rsa.Encrypt(data, RSAEncryptionPadding.OaepSHA256);
// CryptographicException: The parameter is incorrect.

Hm. So the upper boundary on data size seems to be dependent on the RSA padding choices.. interesting (and thanks for nothing, MSDN). Experimentally, you find that the magic upper data size limit for OaepSHA256 padding seems to be 318 bytes. Is there a formula that can calculate the max data size given the padding type? Why, yes, yes, there is. What is it? I won't tell you (but MSDN should). Whatever - you can just use the too-convenient-to-be-real int GetMaxDataSizeForEncryption(RSAEncryptionPadding) method.

Live long and prosper, Earthling.
End of starlog.

sdrapkin/RSA APIs in .NET - a crypto-trek rant.md

RSA APIs in .NET - a crypto-trek rant