How do some apps reach the contents of ".../Android/..." sub-folders on Android 11 without root?

Question

Background

There are various storage restrictions on Android 10 and 11, which also includes a new permission (MANAGE_EXTERNAL_STORAGE) to access all files (yet it doesn't allow access to really all files ) while the previous storage permission got reduced to grant access just to media files :

1. Apps can reach the "media" sub folder freely.
2. Apps can never reach "data" sub folder and especially the content.
3. For "obb" folder, if the app was allowed to install apps, it can reach it (to copy files to there). Otherwise it can't.
4. Using USB or root, you could still reach them, and as an end user you can reach them via the built-in file-manager app "Files".

The problem

I've noticed an app that somehow overcome this limitation (here) called "X-plore": Once you enter "Android/data" folder, it asks you to grant access to it (directly using SAF, somehow), and when you grant it, you can access everything in all folders of "Android" folder.

This means there might still be a way to reach it, but problem is that I couldn't make a sample that does the same, for some reason.

What I've found and tried

It seems this app targets API 29 (Android 10), and that it doesn't use the new permission yet, and that it has the flag requestLegacyExternalStorage. I don't know if the same trick they use will work when targeting API 30, but I can say that on my case, running on Pixel 4 with Android 11, it works fine.

So I tried to do the same:

1. I made a sample POC that targets Android API 29, has storage permissions (of all kinds) granted, including the legacy flag.

2. I tried to request access directly to "Android" folder (based on here), which sadly didn't work as it goes to some reason (kept going to DCIM folder, no idea why) :

val androidFolderDocumentFile = DocumentFile.fromFile(File(primaryVolume.directory!!, "Android"))
val intent = Intent(Intent.ACTION_OPEN_DOCUMENT_TREE)
        .setFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_ACTIVITY_RESET_TASK_IF_NEEDED) 
        .putExtra(DocumentsContract.EXTRA_INITIAL_URI, androidFolderDocumentFile.uri)
startActivityForResult(intent, 1)

I tried various flags combinations.

3. When launching the app, when I reach the "Android" folder myself manually as this didn't work well, and I granted the access to this folder just like on the other app.

4. When getting the result, I try to fetch the files and folders in the path, but it fails to get them:

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)
    Log.d("AppLog", "resultCode:$resultCode")
    val uri = data?.data ?: return
    if (!DocumentFile.isDocumentUri(this, uri))
        return
    grantUriPermission(packageName, uri, Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION)
    contentResolver.takePersistableUriPermission(uri, Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION)
    val fullPathFromTreeUri = FileUtilEx.getFullPathFromTreeUri(this, uri) // code for this here: https://stackoverflow.com/q/56657639/878126
    val documentFile = DocumentFile.fromTreeUri(this, uri)
    val listFiles: Array<DocumentFile> = documentFile!!.listFiles() // this returns just an array of a single folder ("media")
    val androidFolder = File(fullPathFromTreeUri)
    androidFolder.listFiles()?.forEach {
        Log.d("AppLog", "${it.absoluteFile} children:${it.listFiles()?.joinToString()}") //this does find the folders, but can't reach their contents
    }
    Log.d("AppLog", "granted uri:$uri $fullPathFromTreeUri")
}

So using DocumentFile.fromTreeUri I could still get just "media" folder which is useless, and using the File class I could only see there are also "data" and "obb" folders, but still couldn't reach their contents...

So this didn't work well at all.

Later I've found out another app that uses this trick, called "MiXplorer". On this app, it failed to request "Android" folder directly (maybe it didn't even try), but it does grant you full access to it and its sub-folders once you allow it. And, it targets API 30, so this means it's not working just because you target API 29.

I've noticed (someone wrote me) that with some changes to the code, I could request access to each of the sub-folders separately (meaning a request for "data" and a new request for "obb"), but this is not what I see here, that apps do.

Meaning, to get to "Android" folder, I get use this Uri as a parameter for Intent.EXTRA_INITIAL_URI :

val androidUri=Uri.Builder().scheme("content").authority("com.android.externalstorage.documents")
                    .appendEncodedPath("tree").appendPath("primary:").appendPath("document").appendPath("primary:Android").build()

However, once you get an access to it, you won't be able to get the list of files from it, not via File, and not via SAF.

But, as I wrote, the weird thing is that if you try something similar, of getting to "Android/data" instead, you will be able to get its content:

val androidDataUri=Uri.Builder().scheme("content").authority("com.android.externalstorage.documents")
                 .appendEncodedPath("tree").appendPath("primary:").appendPath("document").appendPath("primary:Android/data").build()

The questions

1. How can I request an Intent directly to "Android" folder that will actually let me access to it, and let me get the sub-folders and their contents?
2. Is there another alternative for this? Maybe using adb and/or root, I could grant SAF access to this specific folder ?

Answer 1

Here is how it works in X-plore:

When on Build.VERSION.SDK_INT>=30, [Internal storage]/Android/data is not accessible, java File.canRead() or File.canWrite() returns false, so we need to switch to alternative file system for files inside of this folder (and possibly also obb).

You already know how Storage access framework works, so I'll just give details about what needs to be done exactly.

You call ContentResolver.getPersistedUriPermissions() to find out if you already have saved permission for this folder. Initially you don't have it, so you ask user for permission:

To request access, use startActivityForResult with Intent(Intent.ACTION_OPEN_DOCUMENT_TREE).putExtra(DocumentsContract.EXTRA_INITIAL_URI, DocumentsContract.buildDocumentUri("com.android.externalstorage.documents", "primary:Android")) Here you set with EXTRA_INITIAL_URI that picker shall start directly on Android folder on primary storage, because we want access to Android folder. When your app will target API30, picker won't allow to choose root of storage, and also by getting permission to Android folder, you can work with both data and obb folders inside, with one permission request.

When user confirms by 2 clicks, in onActivityResult you'll get Uri in data which should be content://com.android.externalstorage.documents/tree/primary%3AAndroid. Make needed checks to verify that user confirmed correct folder. Then call contentResolver.takePersistableUriPermission(uri, Intent.FLAG_GRANT_WRITE_URI_PERMISSION | Intent.FLAG_GRANT_READ_URI_PERMISSION) to save permission, and you're ready.

So we're back to ContentResolver.getPersistedUriPermissions(), which contains list of granted permissions (there may be more of them), the one you've granted above looks like this: UriPermission {uri=content://com.android.externalstorage.documents/tree/primary%3AAndroid, modeFlags=3} (same Uri as you got in onActivityResult). Iterate the list from getPersistedUriPermissions to find uri of interest, if found work with it, otherwise ask user for grant.

Now you want to work with ContentResolver and DocumentsContract using this "tree" uri and your relative path to files inside of Android folder. Here is example to list data folder: data/ is path relative to granted "tree" uri. Build final uri using either DocumentsContract.buildChildDocumentsUriUsingTree() (to list files) or DocumentsContract.buildDocumentUriUsingTree() (for working with individual files), example: DocumentsContract.buildChildDocumentsUriUsingTree(treeUri, DocumentsContract.getTreeDocumentId(treeUri), DocumentsContract.getTreeDocumentId(treeUri)+"/data/"), you'll get uri=content://com.android.externalstorage.documents/tree/primary%3AAndroid/document/primary%3AAndroid%2Fdata%2F/children suitable for listing files in data folder. Now call ContentResolver.query(uri, ...) and process data in Cursor to get folder listing.

Similar way you work with other SAF functionality to read/write/rename/move/delete/create, which you probably already know, using ContentResolver or methods of DocumentsContract.

Some details:

• it doesn't need android.permission.MANAGE_EXTERNAL_STORAGE
• it works on target API 29 or 30
• it works only on primary storage, not for external SD cards
• for all files inside of data folder, you need to use SAF (java File won't work), just use file hierarchy relative to Android folder
• in future Google may patch this hole in their "security" intentions, and this may not work after some security update

---

EDIT: sample code, based on Cheticamp Github sample. The sample shows the content (and file-count) of each of the sub-folders of "Android" folder:

class MainActivity : AppCompatActivity() {
    private val handleIntentActivityResult =
        registerForActivityResult(ActivityResultContracts.StartActivityForResult()) {
            if (it.resultCode != Activity.RESULT_OK)
                return@registerForActivityResult
            val directoryUri = it.data?.data ?: return@registerForActivityResult
            contentResolver.takePersistableUriPermission(
                directoryUri, Intent.FLAG_GRANT_READ_URI_PERMISSION or Intent.FLAG_GRANT_WRITE_URI_PERMISSION
            )
            if (checkIfGotAccess())
                onGotAccess()
            else
                Log.d("AppLog", "you didn't grant permission to the correct folder")
        }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        setSupportActionBar(findViewById(R.id.toolbar))
        val openDirectoryButton = findViewById<FloatingActionButton>(R.id.fab_open_directory)
        openDirectoryButton.setOnClickListener {
            openDirectory()
        }
    }

    private fun checkIfGotAccess(): Boolean {
        return contentResolver.persistedUriPermissions.indexOfFirst { uriPermission ->
            uriPermission.uri.equals(androidTreeUri) && uriPermission.isReadPermission && uriPermission.isWritePermission
        } >= 0
    }

    private fun onGotAccess() {
        Log.d("AppLog", "got access to Android folder. showing content of each folder:")
        @Suppress("DEPRECATION")
        File(Environment.getExternalStorageDirectory(), "Android").listFiles()?.forEach { androidSubFolder ->
            val docId = "$ANDROID_DOCID/${androidSubFolder.name}"
            val childrenUri = DocumentsContract.buildChildDocumentsUriUsingTree(androidTreeUri, docId)
            val contentResolver = this.contentResolver
            Log.d("AppLog", "content of:${androidSubFolder.absolutePath} :")
            contentResolver.query(childrenUri, null, null, null)
                ?.use { cursor ->
                    val filesCount = cursor.count
                    Log.d("AppLog", "filesCount:$filesCount")
                    val nameIndex = cursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
                    val mimeIndex = cursor.getColumnIndex("mime_type")
                    while (cursor.moveToNext()) {
                        val displayName = cursor.getString(nameIndex)
                        val mimeType = cursor.getString(mimeIndex)
                        Log.d("AppLog", "  $displayName isFolder?${mimeType == DocumentsContract.Document.MIME_TYPE_DIR}")
                    }
                }
        }
    }

    private fun openDirectory() {
        if (checkIfGotAccess())
            onGotAccess()
        else {
            val primaryStorageVolume = (getSystemService(STORAGE_SERVICE) as StorageManager).primaryStorageVolume
            val intent =
                primaryStorageVolume.createOpenDocumentTreeIntent().putExtra(EXTRA_INITIAL_URI, androidUri)
            handleIntentActivityResult.launch(intent)
        }
    }

    companion object {
        private const val ANDROID_DOCID = "primary:Android"
        private const val EXTERNAL_STORAGE_PROVIDER_AUTHORITY = "com.android.externalstorage.documents"
        private val androidUri = DocumentsContract.buildDocumentUri(
            EXTERNAL_STORAGE_PROVIDER_AUTHORITY, ANDROID_DOCID
        )
        private val androidTreeUri = DocumentsContract.buildTreeDocumentUri(
            EXTERNAL_STORAGE_PROVIDER_AUTHORITY, ANDROID_DOCID
        )
    }
}

Answer 2

Well, I tried this code and it works on Android API 29, Samsung Galaxy 20FE:

@TargetApi(Build.VERSION_CODES.LOLLIPOP)
private void triggerStorageAccessFramework() {
    Intent intent = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
    startActivityForResult(intent, REQUEST_CODE_STORAGE_ACCESS);
}

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    if (requestCode == REQUEST_CODE_STORAGE_ACCESS) {
        Uri treeUri = null;
        // Get Uri from Storage Access Framework.
        treeUri = data.getData();

        // Persist URI in shared preference so that you can use it later.
        // Use your own framework here instead of PreferenceUtil.
        MySharedPreferences.getInstance(null).setFileURI(treeUri);

        // Persist access permissions.
        final int takeFlags = data.getFlags()
                & (Intent.FLAG_GRANT_READ_URI_PERMISSION | Intent.FLAG_GRANT_WRITE_URI_PERMISSION);
        getContentResolver().takePersistableUriPermission(treeUri, takeFlags);

        createDir(DIR_PATH);


        finish();
    }
}

private void createDir(String path) {
    Uri treeUri = MySharedPreferences.getInstance(null).getFileURI();

    if (treeUri == null) {
        return;
    }

    // start with root of SD card and then parse through document tree.
    DocumentFile document = DocumentFile.fromTreeUri(getApplicationContext(), treeUri);
    document.createDirectory(path);
}

I'm calling this from a button onClick:

Button btnLinkSd = findViewById(R.id.btnLinkSD);
    btnLinkSd.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            triggerStorageAccessFramework();
        }
    });

In the UI, I'm pressing "show internal storage", I navigate to Android directory and press allow. After that, in debugging, if I try to list all files under android I'm getting a list of all directories in Data. If that's what you are looking for.

And finally, results in debug: