How to extract text form a specific block of the image? [ML-Kit]

Question

I m desperated. I have 16 TextViews. The goal is to take a picture of a prescription. What I want is to extract the right text for every Textview I have. This is my last problem.

Here is the code of taking a photo, create a file, recognize the text and put it in a TextView.

private fun dispatchTakePictureIntent() {
    Intent(MediaStore.ACTION_IMAGE_CAPTURE).also { takePictureIntent ->
        // Ensure that there's a camera activity to handle the intent
        takePictureIntent.resolveActivity(packageManager)?.also {
            // Create the File where the photo should go

            val photoFile: File? = try {
                createImageFile()
            } catch (ex: IOException) {
                // Error occurred while creating the File
                null
            }
            // Continue only if the File was successfully created
            photoFile?.also {
                photoURI = FileProvider.getUriForFile(
                    this,
                    "com.example.android.fileprovider2",
                    it
                )
                takePictureIntent.putExtra(MediaStore.EXTRA_OUTPUT, photoURI)
                startActivityForResult(takePictureIntent, REQUEST_TAKE_PHOTO)
                /*if (REQUEST_TAKE_PHOTO == 1)
                {
                    return imageView.setImageURI(photoURI)
                }*/

            }
        }
    }
}



@Throws(IOException::class)
private fun createImageFile(): File {
    // Create an image file name
    val timeStamp: String = SimpleDateFormat("yyyyMMdd_HHmmss").format(Date())
    val storageDir: File? = getExternalFilesDir(Environment.DIRECTORY_PICTURES)
    return File.createTempFile(
        "JPEG_${timeStamp}_", /* prefix */
        ".jpg", /* suffix */
        storageDir /* directory */
    ).apply {
        // Save a file: path for use with ACTION_VIEW intents
        currentPhotoPath = absolutePath

    }
}











fun startRecognizing(v: View) {
    if (imageView.drawable != null) {
        editText.setText("")
        v.isEnabled = false
        val bitmap = (imageView.drawable as BitmapDrawable).bitmap
        val image = FirebaseVisionImage.fromBitmap(bitmap)
        val detector = FirebaseVision.getInstance().onDeviceTextRecognizer


        detector.processImage(image)
            .addOnSuccessListener { firebaseVisionText ->
                v.isEnabled = true
                processResultText(firebaseVisionText)
            }
            .addOnFailureListener {
                v.isEnabled = true
                editText.setText("Failed")
            }
    } else {
        Toast.makeText(this, "Select an Image First", Toast.LENGTH_LONG).show()
    }

}

private fun String.toEditable(): Editable =  Editable.Factory.getInstance().newEditable(this)


private fun processResultText(resultText: FirebaseVisionText) {
    if (resultText.textBlocks.size == 0) {
        editText.setText("No Text Found")
        return
    }
    for (blocks in resultText.textBlocks) {
        val blockText = blocks.text
        val stringText  = resultText.text
        val stringTextSplit = stringText.lines()
        krankenkasse.text = stringTextSplit[1].toEditable()
        et2.text = stringTextSplit[4].toEditable()
        et3.text = stringTextSplit[5].toEditable()
        et4.text = stringTextSplit[6].toEditable()
        et5.text = stringTextSplit[7].toEditable()
        et6.text = stringTextSplit[8].toEditable()
        et7.text = stringTextSplit[9].toEditable()
        et8.text = stringTextSplit[11].toEditable()
        editText.append(blockText + "\n")
    }
}

If you have questions. Please ask.

You having 16 TextViews is not relevant to the problem. Please describe the exact issue you are having, include snippets of code focused on the description what you've tried and why it didn't work. — Matt from VisionApp, Apr 17 '20 at 21:23
I did. I don't want to detect and extract the whole picture, I want to extract the text in specific areas of the picture. I'm working with an Intent. Maybe it's better to work with Camera2 or cameraX — Pyllic, Apr 19 '20 at 08:14

score 2 · Answer 1 · answered Apr 24 '20 at 17:33

If you want to detect some specific regions in the image, you could crop the image based on the regions, and send the cropped images into ML Kit.

Another option is to detect the whole image, but filter out texts which are not in the regions. Each detected text has its own bounding box coordinates.

https://firebase.google.com/docs/reference/android/com/google/firebase/ml/vision/text/FirebaseVisionText.TextBlock#getBoundingBox()

How to extract text form a specific block of the image? [ML-Kit]

1 Answers1