java.security.UnrecoverableKeyException: Failed to obtain information about private key

Question

I have the following lines to get the private key from key store on Android

KeyStore keyStore = KeyStore.getInstance("AndroidKeyStore");
keyStore.load(null);

// generating key pair code omitted

KeyStore.PrivateKeyEntry privateKeyEntry = (KeyStore.PrivateKeyEntry) this.keyStore.getEntry("alias", null);

Everything works fine except that when the OS upgrades from Android 5.1.1 to Android 6.0.1, the 3rd line will throw java.security.UnrecoverableKeyException: Failed to obtain information about private key for very first execution. But it will work fine again afterward. Now my workaround is to execute the line for 2 times. At the same time, I am also wondering if there is better way to avoid the exception.

Update

The exception trace

W/System.err﹕ java.security.UnrecoverableKeyException: Failed to obtain information about private key
W/System.err﹕ at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStorePublicKeyFromKeystore(AndroidKeyStoreProvider.java:217)
W/System.err﹕ at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStoreKeyPairFromKeystore(AndroidKeyStoreProvider.java:253)
W/System.err﹕ at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStorePrivateKeyFromKeystore(AndroidKeyStoreProvider.java:263)
W/System.err﹕ at android.security.keystore.AndroidKeyStoreSpi.engineGetKey(AndroidKeyStoreSpi.java:93)
W/System.err﹕ at java.security.KeyStoreSpi.engineGetEntry(KeyStoreSpi.java:372)
W/System.err﹕ at java.security.KeyStore.getEntry(KeyStore.java:645)
W/System.err﹕ at com.example.keystoretest.MainActivity.onCreate(MainActivity.java:113)
W/System.err﹕ at android.app.Activity.performCreate(Activity.java:6251)
W/System.err﹕ at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107)
W/System.err﹕ at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2369)
W/System.err﹕ at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476)
W/System.err﹕ at android.app.ActivityThread.-wrap11(ActivityThread.java)
W/System.err﹕ at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344)
W/System.err﹕ at android.os.Handler.dispatchMessage(Handler.java:102)
W/System.err﹕ at android.os.Looper.loop(Looper.java:148)
W/System.err﹕ at android.app.ActivityThread.main(ActivityThread.java:5417)
W/System.err﹕ at java.lang.reflect.Method.invoke(Native Method)
W/System.err﹕ at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
W/System.err﹕ at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
W/System.err﹕ Caused by: android.security.KeyStoreException: Invalid key blob
W/System.err﹕ at android.security.KeyStore.getKeyStoreException(KeyStore.java:632)
W/System.err﹕ at android.security.keystore.AndroidKeyStoreProvider.loadAndroidKeyStorePublicKeyFromKeystore(AndroidKeyStoreProvider.java:218)
W/System.err﹕ ... 18 more

Most likely there isn't much you can do. Just in case though, are there any interesting lines in system log during the execution of KeyStore.getPrivateKey? Also, what's the full stack trace (incl. root causes) of the exception you're getting? — Alex Klyubin, Apr 18 '16 at 18:24
Sorry for the late the reply, the exception trace was added. I am testing the behavior with Nexus 5, which is one of the few devices that can upgrade from Android 4 to Android 6, according to the factory image list, https://developers.google.com/android/nexus/images — Dino Tw, Apr 20 '16 at 22:57
Thanks. Are there any interesting lines in the system log during the execution of KeyStore.getPrivateKey, other than the stack trace? — Alex Klyubin, Apr 21 '16 at 00:57
Not sure what system log you are referring to, but the exception trace is the print out seen in logcat by executing `exeption.printStackTrace()`. — Dino Tw, Apr 21 '16 at 03:33
logcat is the command that dumps/outputs the system log. Are there any interesting lines in the output of logcat during the execution of KeyStore.getPrivateKey, other than the stack trace? — Alex Klyubin, Apr 21 '16 at 18:35
No, I did not see other lines that would give me the hint of exception. — Dino Tw, Apr 22 '16 at 14:11

score 11 · Accepted Answer · edited May 23 '17 at 12:26

When this error happens and why?

Ans: When loading Android keys and storing public key from Keystore, this error may happen if the state is locked or uninitialized.

Error generating portion code is given below:

@NonNull
    public static AndroidKeyStorePublicKey loadAndroidKeyStorePublicKeyFromKeystore(
            @NonNull KeyStore keyStore, @NonNull String privateKeyAlias)
            throws UnrecoverableKeyException {
        KeyCharacteristics keyCharacteristics = new KeyCharacteristics();
        int errorCode = keyStore.getKeyCharacteristics(privateKeyAlias, null,
                null, keyCharacteristics);
        if (errorCode != KeyStore.NO_ERROR) {
            throw (UnrecoverableKeyException) new UnrecoverableKeyException(
                    "Failed to obtain information about private key")
                    .initCause(KeyStore.getKeyStoreException(errorCode)); // this exception is generated
        }
        ......
        ......
        ......
    }

KeyStore has 10 response code. They are

// ResponseCodes
NO_ERROR = 1;
LOCKED = 2;
UNINITIALIZED = 3;
SYSTEM_ERROR = 4;
PROTOCOL_ERROR = 5;
PERMISSION_DENIED = 6;
KEY_NOT_FOUND = 7;
VALUE_CORRUPTED = 8;
UNDEFINED_ACTION = 9;
WRONG_PASSWORD = 10;

KeyStore has 3 states. They are UNLOCKED, LOCKED, UNINITIALIZED

NO_ERROR is only happened when the state is UNLOCKED. For your upgrading case the state is LOCKED or UNINITIALIZED for first time, so the error is happened only once.

State Checking code is given below:

public State state() {
    execute('t');
    switch (mError) {
    case NO_ERROR:
        return State.UNLOCKED;
    case LOCKED:
        return State.LOCKED;
    case UNINITIALIZED:
        return State.UNINITIALIZED;
    default:
        throw new AssertionError(mError);
    }
}

Resource Link:

UPDATE:

From your error log, it is now clear that

W/System.err﹕ Caused by: android.security.KeyStoreException: Invalid key blob

this is the main issue which is caused when user tries to UNLOCK from LOCK/UNINITIALIZED. It is by default defined as 30 secs for timing. This problem is it's API related implementation issue.

/**
 * If the user has unlocked the device Within the last this number of seconds,
 * it can be considered as an authenticator.
 */
private static final int AUTHENTICATION_DURATION_SECONDS = 30;

For encryption/decryption some data with the generated key only works if the user has just authenticated via device credentials. The error occurs from

// Try encrypting something, it will only work if the user authenticated within
// the last AUTHENTICATION_DURATION_SECONDS seconds.
cipher.init(Cipher.ENCRYPT_MODE, secretKey); // error is generated from here.

Actual error is thrown from here. Your error is generated from InvalidKeyException.

Solution:

You have to remove the InvalidKeyException class from the catch argument. This will still allow you to check for InvalidKeyException. After checking you have to try for second time with code so that the problem is not shown in eye but doing 2 times checking it may solve your issue. I have not tested the code but should be like below:

try {
....
KeyStore.PrivateKeyEntry privateKeyEntry = (KeyStore.PrivateKeyEntry) this.keyStore.getEntry("alias", null);
....
} catch (final Exception e) {
    e.printStackTrace();
    if (e instanceof InvalidKeyException) { // bypass InvalidKeyException
        .......
        // You can again call the method and make a counter for deadlock situation or implement your own code according to your situation
        if (retry) {
            keyStore.deleteEntry(keyName);
            return getCypher(keyName, false);
        } else {
            throw e;
        }
    }
}

Resource Link:

Thank you for showing me the source code. The key store state is also something I guessed before, but I could not seem to find a method to UNLOCK the key store before getting the private key besides causing the exception purposely. — Dino Tw, Apr 20 '16 at 23:03
Actually the issue is for UNLOCK from LOCK/UNINITIALIZE. But you can be tricky on that case. you can make the call again by handling exception with counter. I have updated my answer with solution. Please have a try. — SkyWalker, Apr 24 '16 at 17:19
Yes, my workaround is similar to your answer. I try to get the privateKeyEntry in `onCreate()` for the first time, then get it again when I really need it and it's value is `null`. It seems working fine so far. But I will accept you answer. Thank you for the help. — Dino Tw, Apr 25 '16 at 05:29
I'm a little baffled by this answer. It isn't clear how trying getEntry() twice resolves the issue. What does it have to do with 30 seconds timeout? Also, why is this necessary: "You have to remove the InvalidKeyException class from the catch argument"..? Why not just catch the specific InvalidKeyException to begin with for second try? Are you intentionally swallowing all other exceptions and not re-throwing them? — yura, Dec 07 '17 at 00:39
@yura 30 sec timeout is [If the user has unlocked the device Within 30 seconds, it can be considered as an authenticator.] described in [here](https://github.com/googlesamples/android-ConfirmCredential/blob/master/Application/src/main/java/com/example/android/confirmcredential/MainActivity.java#L65). For **invalidKeyException**, I have clarified how to solve the issue in Solution part. Without throwing an exception, I can retry for few counters. If it fails for counter time, then I will throw the exception. Please check the solution again. — SkyWalker, Dec 07 '17 at 01:21
How is it an acceptable solution to delete the entry and then generate a new one? What if the user has encrypted information that needs to be restored using the existing private key in the key store? — haventchecked, Jan 03 '18 at 03:46
Android lets you specify whether user authentication is required to use a key, and the 30 second timeout in that example is passed to `setUserAuthenticationValidityDurationSeconds`. The default is not to require user authentication, though, and the 30-second timeout looks unrelated here. The `Invalid key blob` error is thrown from native code and maybe the root cause is closer to the secure hardware. If retrying to fetch the key fixes things, there's probably a race condition somewhere. — ide, May 22 '18 at 02:13
The code solved the problem on my side. What i dont understand is why the issue is not occuring everytime. I ran the same code on different emulators and only some of them had this issue. — shalama, Sep 08 '18 at 14:34

Ahmad Melegy · Answer 2 · 2020-08-08T16:39:03.690

UPDATE (August 2020):

Updating the security library to version 1.0.0-rc03 fixes the problem for me.

In the changle log they mention:

Tink update should gracefully handle AndroidKeyStore concurrency failures.

Old Answer:

There is an open issue on issuetracker for this

Here is a reply of one of Google Engineers

Some OEM implementations of the AndroidKeyStore are broken and do not work properly. Unfortunately, Jetpack Security relies on the AndroidKeyStore to securely store and generate keys. If this is not working, all you can do is trust the devices that have failures, less and not use encryption. Ideally a check would be nice in the library to find these issues so you can know about this without random crashes.

I wrote a test class you could use in the meantime to test the KeyStore. Basically, you have to do an end to end, encrypt/decrypt to know if the device KeyStore is fully working.

https://gist.github.com/jmarkoff/44f5a9cab1a881c8b0abc787791add08

/*
 * Copyright 2020 The Android Open Source Project
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

//packgage com.company.app

import android.content.Context;
import android.content.SharedPreferences;
import android.security.keystore.KeyGenParameterSpec;
import android.util.Log;

import androidx.annotation.NonNull;
import androidx.security.crypto.MasterKeys;
import androidx.security.crypto.EncryptedSharedPreferences;

import java.io.IOException;
import java.security.GeneralSecurityException;

/**
 * Convenient method to test the Android Keystore before using encryption/decryption. A small number
 * OEMs have devices with a bad keystore and KeyStore exceptions will occur.
 *
 * Requires Jetpack Security - https://developer.android.com/jetpack/androidx/releases/security
 *
 * Bugs:
 *
 * https://issuetracker.google.com/issues/147480931
 * https://issuetracker.google.com/issues/134417365
 * https://issuetracker.google.com/issues/150221071
 *
 */
public final class TestKeyStore {

     /**
     * Test the keystore, encryption and decryption on the device. This is useful to find devices
     * that have a bad keystore and encryption should not be used. It is up to the developer to
     * decide how to handle when a bad keystore is encountered. We recommend that the device be
     * trusted less by your app if possible.
     *
     * @param keyGenParameterSpec The key encryption scheme
     * @return true if the keystore can be relied on, false otherwise
     */
    public static boolean trustDeviceKeyStore(@NonNull KeyGenParameterSpec keyGenParameterSpec,
            @NonNull Context context) {
        try {
            String keyAlias = MasterKeys.getOrCreate(keyGenParameterSpec);
            SharedPreferences sharedPreferences =
                    EncryptedSharedPreferences.create("test_keystore", keyAlias,
                            context,
                            EncryptedSharedPreferences.PrefKeyEncryptionScheme.AES256_SIV,
                            EncryptedSharedPreferences.PrefValueEncryptionScheme.AES256_GCM);
            SharedPreferences.Editor editor = sharedPreferences.edit();
            editor.putString("TestKeyStore", "Testing");
            editor.commit();
            String value = sharedPreferences.getString("TestKeyStore", "Failed");
            if (value.equals("Testing")) {
                return true;
            }
        } catch (GeneralSecurityException ex) {
            Log.e(TestKeyStore.class.getSimpleName(),
                    "SecurityException: Could be a keystore issue, check the error for more "
                            + "details message: " + ex.getMessage() + ".\n Stacktrace:\n"
                            + ex.getStackTrace().toString());
        } catch (IOException ex) {
            Log.e(TestKeyStore.class.getSimpleName(),
                    "IOException: Check to make sure you have enough disk space and that the "
                            + "file doesn't exist." + ex.getMessage());
        }
        return false;
    }

}

For those who are eager to know more, this is a coressonding issue that was fixed in Tink library - https://github.com/google/tink/issues/339. Android X Security depends on this library, and this was released in 1.0.0-rc03. Now Android X Security does a self-test before actually using keystore. — Vadim Kotov, Sep 04 '20 at 08:30

score 2 · Answer 3 · answered Sep 06 '20 at 05:33

Actually Android keystore is not thread-safe, even we EncryptedSharedPreferences, it is also not thread-safe. Google has clearly told us in this document,

Note: The methods in both the EncryptedFile class and the EncryptedSharedPreferences class aren't thread-safe.

Once there are multiple threads trying to access the Android keystore, any exception could happened. Like above exception is happened when one thread is using key, but another thread tries to get the key again.

All Android keystore operations should be put into synchronized block.

For more details, please refer to https://medium.com/@weidianhuang/how-to-write-thread-safe-code-for-android-keystore-40a0fa17f416 — Weidian Huang, Sep 08 '20 at 12:48

java.security.UnrecoverableKeyException: Failed to obtain information about private key

Update

3 Answers3

When this error happens and why?

Error generating portion code is given below:

State Checking code is given below:

UPDATE:

Solution:

Resource Link:

Linked