Moving from ImageJ to OpenCV

[rkrug]

I have a question concerning the OpenCV integration in R.

I am involved in the package bemovi (maintainer: @pennekampster) which is doing particle identification and measuring based on a video file, recorded e.g. from a microscope (for examples see http://bemovi.info/pages/example.html).

It uses ImageJ to do the processing and therefore Java. The package works, but we regularly have problems caused by Java versions and the whole processing is a bit cumbersome (creating ImageJ macros to do the processing based on a template).

So we would like to switch to another image processor engine and OpenCV seems to be the best option. Could somebody here give their opinion if you think that would be possible using rOpenCV?

It would be great, if we could give some ideas and suggestions in this regard, as one of us have used OpenCV before.

[jeroen]

@jwijffels @dmi3kno do you have any advice on this?

[jwijffels]

Is this the location of the ImageJ macros you are using on that site? https://github.com/pennekampster/bemovi/tree/master/inst/ImageJ_macros

[rkrug]

Yes - these are the macros. We have elicited the Video_overlay as we are creating the overlays (video overplayed with the id of the particle) now using ffmpeg and subtitles.

[rkrug]

The Video_to_morphology... are used to locate and measure the particles in the function https://github.com/pennekampster/bemovi/blob/master/R/Locate_and_measure_particles.R, and is doing most of the work.

[jwijffels]

morphology is available in the opencv R package
maybe you can explain a bit what the macro's are doing

[rkrug]

Thanks for your reply, @jwijffels For details, I would ask @pennekampster to chime in, as he has written the macros - I am only using them. But in principle, they are

1. Identifying the moving particles, and
2. tracks the particles, and
3. returns shape measurements of the particles.

I have done, based on my question at the OpenCV Forum some reading, and opened the issue pennekampster/bemovi#51 which is trying to summarise what I think would be necessary to do and also points to some C++ functions in OpenCV.

So far, I have identified, based on the example by OpenCV, the cv:BackgroundSubtractor to subtract the background, and https://learnopencv.com/object-tracking-using-opencv-cpp-python/ to track the particles to get info on direction and speed. Finally, something along the lines of https://docs.opencv.org/3.3.1/d3/dc0/group__imgproc__shape.html#ga107a78bf7cd25dec05fb4dfc5c9e765f to obtain the shape measurements.

This is very much in the early stages, which is why I am looking for ideas and suggestions.

Also, I am not a C++ programmer (rather R), wherefore using the OpenCV R package would be ideal.

[jwijffels]

Based on skimming pennekampster/bemovi#51, jeroen has did some example background substraction with cvmat_mog2, you can do image binarisation with R pkg image.binarization, morphology is available in the package, optical flow algorithms have currently not been mapped to R variants. So no sure how far you will get with the current functionalities inside the package but some modifications, I think it is possible you could replace your current ImageJ macros.

[rkrug]

That sounds good. But I have still a few questions:

1. background subtraction: I need the background file for this. But how can I extract the background file from the video? (probably @jeroen can help here?)
2. binarization: as mentioned, I don't know if it is necessary. I would prefer to keep the colours as long as possible, as they also contain information which might be useful at a later stage. Also, your package does the binarisation for images - I could split the video into individual dual frames, but this would cost time, and efficiency would be paramount as the analysis is quite time intensive. Am I correct?
3. morphology: Could you point me to the function(s) where this is implemented? (probably @jeroen can help here?)

[jwijffels]

Apologies, morphology was available in R package magick. Isn't mapped to R functions in R package opencv