How to incorporate tflite-yolov4 in Android App ?(Kotlin)

Question

Thanks for watch this post.I'm new to Android development. If any information is missing for verification, please let me know.

I'm trying to incorporate yolov4.tflite in my App.

For confirmation of detection results,I make an AnalyzeActivity Class. In the AnalyzeActivity Class, I wrote code to launch Viedeo in VieoView and display the detection result image below it.

When I ran the code, the following information was displayed and nothing was detected.

I/Choreographer: Skipped 30 frames! The application may be doing too much work on its main thread.

W/Parcel: Expecting binder but got null!

I have two questions here.

1. Am I dividing the threads wrong?How should I reduce work in main thread?
2. Do I have wrong way to use YoloV4 model?

The code and repository structure are below.

【code】

class AnalyzeActivity : AppCompatActivity() {
    companion object{
        const val TAG:String = "AnalyzeActivity"
    }

    private var pointsArray = mutableListOf<Point>()
    private var frameArray = mutableMapOf<Int,MutableList<Point>>()

    //Yolov4用パラメータ
    //viewModel作成
    private val viewModel by viewModels<YoloV4MainViewModel>{getViewModelFactory()}
    //binding作成
    private lateinit var binding: ActivityAnalyzeBinding

    @RequiresApi(Build.VERSION_CODES.P)
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        binding = ActivityAnalyzeBinding.inflate(layoutInflater)
        setContentView(binding.root)
        OpenCVLoader.initDebug()


        //Views
        val intent = getIntent()
        val videoName = intent.getStringExtra("videoName")
        val videoView = findViewById<VideoView>(R.id.videoView)
        val imgView = findViewById<ImageView>(R.id.imageView)
        val context = applicationContext

        //set Video to videoView
        val fileDir = context.getExternalFilesDir(null)
        val uri = Uri.parse(fileDir.toString() + "/$videoName")

        videoView.setVideoURI(uri)
        videoView.start();


        val retriever = MediaMetadataRetriever()
//        var bitmap = MediaMetadataRetriever.BitmapParams()

        retriever.setDataSource(context, uri)


        //Yolov4
            val sourceBitmap = assets.open("kite.jpg").use{
                inputStream->
                BitmapFactory.decodeStream(inputStream)
            }

             viewModel.setUpBitmaps(assets)
                imgView.setImageBitmap(sourceBitmap)

                try{
                    viewModel.setUpDetector(assets)
                    Log.d(TAG,"YoloV4 Detector Set Up Done")
                }catch(e: IOException){
                    Log.e(YoloV4MainActivity.TAG,"Exception initializing detector!")
                    Log.e(YoloV4MainActivity.TAG,e.stackTraceToString())

                    Toast.makeText(
                        baseContext,"Detector could not be initialized", Toast.LENGTH_LONG
                    ).show()

                    finish()
                }

             viewModel.setUpDetectionProcessor(
                 imgView,
                 findViewById(R.id.tracking_overlay),
                 resources.displayMetrics
             )
            
          thread{
              viewModel.processImage()
          }

    }

【repo structure】

Answer 1

If your main thread is doing too much task then you can use threads to perform the operations. Either use coroutines or Executor class. the implementation of executor class is below:

public static void getAllDataPagination(final Context context, final ImageDataControllerCallback imageDataControllerCallback) {
    final ExecutorService executor = Executors.newSingleThreadExecutor();
    final Handler handler = new Handler(Looper.getMainLooper());

    executor.execute(new Runnable() {
        @Override
        public void run() {
            final List<FolderDataDisplay> imageDataList = DatabaseClient.getInstance(context).getAppDatabase().imageDataDao().getAllDataPagination(from, to);

            handler.post(new Runnable() {
                @Override
                public void run() {
                    if (imageDataControllerCallback != null) {
                        imageDataControllerCallback.onImageDataPaginatedFetched(imageDataList);
                    }

                    /*executor.shutdown();
                    System.runFinalization();*/

                }
            });

        }
    });
}

The interface will look something like this:

public interface ImageDataControllerCallback {
    void onImageDataFetched(List<ImageData> data);
}