spacer.png, 0 kB
  spacer.png, 0 kB
spacer.png, 0 kB
spacer.png, 0 kB
Optical Flow For Live Cell Imaging PDF Print
Article Index
Optical Flow For Live Cell Imaging
Optical Flow Methods
Ground Truth Image Series Generator
Motion Tracking Application
Related Publications

Optical Flow methods

Let the two consecutive frames of image sequence be given. Optical flow methods compute the displacement vector field which maps all voxels from the first frame into their new position in the second frame. We study and implement the three-dimensional variants of following methods:

Variational Optical Flow Methods

Variational optical flow methods determine the desired displacement as the minimizer of suitable energy functional. Particular energy functional consists of data term and smoothness term. Data term ensures that certain image properties (e.g. grey value, gradient magnitude) remain constant in time. Smoothness term regularises the non-unique solution by certain smoothness constraint.

We study common as well as the state-of-the-art variational optical flow methods published recently by Andrés Bruhn. We focus our interest on the multiscale warping based methods. These methods can handle motion larger than one pixel. This situation often occurs in live-cell image series. We implement and extend to three dimensions several variational optical flow methods, namely the following:

  • Classical Horn-Schunck variational optical flow (VOF) method
  • VOF method with isotropic image driven regularization (Charbonnier regularization)
  • VOF method with anisotropic image driven regularization (Nagel-Enkelmann regularization )
  • VOF method with isotropic flow driven regularization
  • VOF method with nonlinear data term and flow driven regularization
  • VOF method with nonlinear data term and anisotropic image driven regularization
  • Multiscale warping based variants of these methods (These can handle large motions).

 

We present some results and screenshots from our programs here. In the first example you can see two 2D frames of HL-60 cell nucleus. In the right picture is the example output of our optic flow demo program. The flow is visualised in color representation (top frame color codes direction, intensity length of the flow vector) and in vectors (bottom frame). Size of input frames is 400x400 pixels. The flow was computed with warping based method for large displacements.

a_1_tn.png a_2_tn.png Our Demonstration program. Results for two HL-60 cell frames
First frame of HL-60 sequence Second frame of HL-60 sequence Results visualised in 2D demo program

In the second example we demonstrate the capability of our software to handle 3D image sequences. The first two pictures show the input 3D frames (size 276 x 286 x 106). The first frame in red channel imposed over the second frame in green channel are shown in the third picture. In the fourth picture is the second frame backward registered using the computed flow. We again use the warping based method.

First 3D frame of HL-60 sequence
Second 3D frame of HL-60 sequence First frame in green, Second frame in red. Result. First Frame over second frame. First frame in red second in green
First Frame of 3D example. Second frame of 3D example.

 

First frame in red color over imposed on second frame (in green color) .

The 3D flow is applied on the second frame (Backward registration). First frame is over imposed over second frame. First frame in red color. Second frame in green frame.

 

The previous two examples are from a publication presented on VISAPP2007 conference [1] . You can read there which variational optical method was the best for local motion estimation in live-cell imaging.

We tried to use our software for registration of other biomedical images. We were successful in registration of 3D CT brain images as well as PET lung images using the variational optical flow. We cannot display our results here due to copyright.

Currently, these methods may be easily tested with our ofd, the GUI demo program for computation of optical flow. The program is part of our collection of libraries and tools related to the computation of optical flow . Once one is happy with the tunning of chosen optical flow method, she can easily start batch processing. The screen shot shows how to supply the ofcl tool with parameters used in the GUI program.

Phase-based and Energy-based Optical Flow Methods

We are currently working on these methods. In particular, we have already implemented the traditional Heeger's method and modified it to work better on biomedical images [4] . This optical flow computation method is based on processing energies obtained from spatio-temporal filtering.

Unfortunatelly, our implementation still suffers from a major limitation: the input image can't be scaled so that larger velocities can't be detected reliably. It also computes with subpixel velocities in multiples of 0.1.

The improvements were:

Anyway, we present some preliminary results on generated images with foreground motions and with global and additional local motions. In both cases, all velocities were up to 3px/frame.

 

the first frame the second frame both frames overimposed visualised ground-truth flow visualised computed flow flow visualization legend
the first frame, only local motions the second frame, only local motions both frames overlayed, the first is in red, the second is in green visualization of ground-truth flow field computed flow field flow field visualization: color and intensity gives direction and length of a vector, respectively
the first frame the second frame both frames overimposed visualised ground-truth flow visualised computed flow  
the first frame, global with additional local motions the second frame, global with additional local motions both frames overlayed, the first is in red, the second is in green visualization of ground-truth flow field computed flow field  

 

Note that the method, like any other method based on spatio-temporal filtering, requires more than just two frames. However, only two frames from a tested time-lapse image are presented above.

We still work on this method. The Fleet and Jepson method is planned too.

We implemented all these methods into the OpticalFlow library. This library is written in C++ and is multiplatform (tested Linux and Windows). Moreover, we use the sophisticated multigrid framework for solution of the optical flow problems. Therefore we achieve reasonable computation times even for 3D image sequences (seconds for 2D frames, minutes for 3D frames). The library is still under development but its source codes are available under the GNU license download the library. The authors are Jan Hubený and Vladimír Ulman.



Written by Vladimír Ulman   
Last Updated ( Wednesday, 19 September 2012 )
 
spacer.png, 0 kB
spacer.png, 0 kB
spacer.png, 0 kB