6

Context

I'm using the Android firebase-ml-vision to scan barcodes using SurfaceView with continuous ByteBuffer of pictures frames. I used the ML kit quickstart project as a starting point and it works great.

The objective of my project is to recognise the product associated with the barcode and add it to the list of scanned items.

Problem

Once the camera focuses, the barcode processor would detect the same barcode multiple times, so you would scan 20 rather than 1 barcode in a second.

This is javadoc from CamereSource.FrameProcessingRunnable.run

 * As long as the processing thread is active, this executes detection on frames continuously.
 * The next pending frame is either immediately available or hasn't been received yet. Once it
 * is available, we transfer the frame info to local variables and run detection on that frame.
 * It immediately loops back for the next frame without pausing.

I have tried to add a "paused" check into the FrameProcessingRunnable, but I was still getting the same barcode recognised at least twice, as the next frame/frames were already being fed in for detection:

private class FrameProcessingRunnable implements Runnable {
private volatile boolean paused = false;
.
.
.
public void pause() {
  synchronized (lock) {
    this.paused = true;
    lock.notifyAll();
  }
}

public void resume() {
  synchronized (lock) {
    this.paused = false;
    lock.notifyAll();
  }
}

public void run() {
.
.
.
synchronized (processorLock) {
        if (!paused) {
          Log.d(TAG, "Process an image");
          frameProcessor.process(...

Solution using stop & start

As I couldn't get it to pause, I opted for stop & start when a barcode is detected from the buffer:

private CameraSourcePreview preview;
public void pauseImageProcessing() {
  preview.stop();
  try {
      preview.start(cameraSource, graphicOverlay);
  } catch (IOException e) {
  }
}

This works, but there is about 1 second delay before it starts up again, camera starts focusing and the next barcode can be detected. No doubt this approach also consumes unnecessary resources. You may say this is fine, but in this video you'll see the difference between the camera scanner with Stop&Start and a Bluetooth scanner:

Better approach

I'm looking for a solution that would simply discard any frames immediately after the one with successful detection and start again, but so far I have failed. I'm using 20 frames per second.

VissionProcessorBase does have code for throttling

// Whether we should ignore process(). This is usually caused by feeding input data faster than
// the model can handle.
private final AtomicBoolean shouldThrottle = new AtomicBoolean(false);

But is doesn't go far enough for my need :(

Petr Kysela
  • 61
  • 3

1 Answers1

2

Once the camera focuses, the barcode processor would detect the same barcode multiple times, so you would scan 20 rather than 1 barcode in a second.

VisionProcessorBase.java

private void detectInVisionImage(
        FirebaseVisionImage image,
        final FrameMetadata metadata) {

    detectInImage(image)
            .addOnSuccessListener(
                    new OnSuccessListener<T>() {
                        @Override
                        public void onSuccess(final T results) {
                            new Handler().postDelayed(new Runnable() {
                                @Override
                                public void run() {
                                    shouldThrottle.set(false);
                                }
                            },1000);


                            VisionProcessorBase.this.onSuccess(results, metadata);

                        }
                    })
            .addOnFailureListener(
                    new OnFailureListener() {
                        @Override
                        public void onFailure(@NonNull Exception e) {
                            new Handler().postDelayed(new Runnable() {
                                @Override
                                public void run() {
                                    shouldThrottle.set(false);
                                }
                            },1000);
                            VisionProcessorBase.this.onFailure(e);
                        }
                    });
    // Begin throttling until this frame of input has been processed, either in onSuccess or
    // onFailure.
    shouldThrottle.set(true);



}
Subratsss
  • 647
  • 4
  • 11