7

Can someone explain to me why this code throws javax.crypto.BadPaddingException: Decryption error on the final line when it's decrypting the key?

// Given an RSA key pair...
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(2048);
KeyPair keyPair = keyGen.genKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();

// ... and an AES key:
KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
keyGenerator.init(256);
SecretKey aesKey = keyGenerator.generateKey();

// When I encrypt the key with this Bouncy Castle cipher:
Cipher encryptionCipher = Cipher.getInstance("RSA/NONE/OAEPWithSHA256AndMGF1Padding", "BC");
encryptionCipher.init(Cipher.ENCRYPT_MODE, publicKey);
byte[] encryptedKey = encryptionCipher.doFinal(aesKey.getEncoded());

// Then trying to decrypt the key with this cipher...
Cipher decryptionCipher = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
decryptionCipher.init(Cipher.DECRYPT_MODE, privateKey);
// ... throws `javax.crypto.BadPaddingException: Decryption error` here:
decryptionCipher.doFinal(encryptedKey);

Is the following statement from https://stackoverflow.com/a/27886397/66722 also true for RSA with OAEP?

"RSA/ECB/PKCS1Padding" actually doesn't implement ECB mode encryption. It should have been called "RSA/None/PKCS1Padding" as it can only be used to encrypt a single block of plaintext (or, indeed a secret key). This is just a naming mistake of Sun/Oracle.

If so, I would expect these transformations to be equivalent and my test above to pass. The same padding has been specified in both, so why the BadPaddingException?

Either way, I would appreciate a layman's explanation of what the difference is.

Martin Dow
  • 4,813
  • 2
  • 27
  • 42

1 Answers1

21

For similar Stackoverflow questions with more information please see Maarten Bodewes answers to this and this.

The "mode" part of the transformation string has no effect. The problem is different defaults used by different providers. This is unfortunate and very definitely suboptimal. Should we blame Sun/Oracle? I have no opinion beyond being dissatisfied with the result.

OAEP is a fairly complicated construction with two different hash functions as parameters. The Cipher transform string lets you specify one of these, which you have specified as SHA-256. However, the MGF1 function also is parameterized by a hash function which you cannot specify in the cipher transformation string. The Oracle provider defaults to SHA1 whereas the BouncyCastle provider defaults to SHA-256. So, in effect, there is a hidden parameter that is critical for interoperability.

The solution is to specify more fully what these hidden parameters are by supplying an OAEPParameterSpec to the Cipher.init(...) method as in the following example:

Cipher encryptionCipher = Cipher.getInstance("RSA/NONE/OAEPWithSHA256AndMGF1Padding", "BC");
OAEPParameterSpec oaepParameterSpec = new OAEPParameterSpec("SHA-256", "MGF1",
                MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
encryptionCipher.init(Cipher.ENCRYPT_MODE, publicKey, oaepParameterSpec);
// ...
// ...
// ...
Cipher decryptionCipher = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
oaepParameterSpec = new OAEPParameterSpec("SHA-256", "MGF1",
                MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
decryptionCipher.init(Cipher.DECRYPT_MODE, privateKey, oaepParameterSpec);

The first one is effectively a no-op, because those are already the defaults for Bouncycastle.

President James K. Polk
  • 36,717
  • 16
  • 86
  • 116
  • 1
    Just what I was looking for - thanks. Are you able to point me to the evidence of this statement: "The Oracle provider defaults to SHA1 whereas the BouncyCastle provider defaults to SHA-256."? I'm curious to figure out how I could have discovered this myself - it's hard to follow the implementations hidden behind those magic transformation strings. – Martin Dow May 12 '18 at 06:00
  • 1
    @MartinDow: No, I can't find where it says that. Because I suspected it, I was able to step through the code with a debugger into the sun.* classes and see it. Oracle/Sun tend to be as standards-compliant as they can, and it is kind of implied I think in [PKCS#1, section A.2.1](https://tools.ietf.org/html/rfc8017#appendix-A.2.1) where is says `maskGenAlgorithm [1] MaskGenAlgorithm DEFAULT mgf1SHA1`. – President James K. Polk May 12 '18 at 14:17