This is the original method of Heeger for the computation of 2D optical flow. More...

#include <ofHeeger.h>

Inheritance diagram for OFHeeger< VOXEL >:

Public Member Functions
	OFHeeger (i3d::Image3d< VOXEL > const &in)
virtual	~OFHeeger ()
virtual void	ClearWorkingData (void)
int	GetState (void) const
int	PrintState (void) const
int	SetParams (const int bankSize=4, const VOXEL s1=4.0, const VOXEL s2=3.0, const VOXEL s3=1.5, const VOXEL per=6.0)
virtual void	PrintSpecs (void) const
void	SaveWorkingImages (void) const
virtual int	SaveEnergyImages (const VOXEL Vx, const VOXEL Vy, const char *name) const
int	GetNumberOfTilts (void) const
double	GetTiltStepping (void) const
int	GetNumberOfFilters (void) const
double	GetBankStepping (void) const
VOXEL	GetxSigma (void) const
VOXEL	GetySigma (void) const
VOXEL	GettSigma (void) const
VOXEL	GetPeriod (void) const
virtual int	BankPreparation (void)
virtual int	BankConvolution (void)
virtual int	ComputeEnergy (void)
virtual int	ComputeParsevals (void)
virtual int	ComputeFlow (Image3d< VOXEL > &Fx, Image3d< VOXEL > &Fy)
virtual int	ComputeParsevalsAndFlow (Image3d< VOXEL > &Fx, Image3d< VOXEL > &Fy)
virtual int	FillResponsesCache (const int cx, const int cy)
virtual int	SaveResponsesCache (void)
Public Attributes
i3d::Image3d< VOXEL > const &	In
Protected Member Functions
unsigned int	FRIndex (const int bankTilt, const int bankFilter, const int vx, const int vy) const
unsigned int	FRCacheIndex (const int bankTilt, const int bankFilter, const int vx, const int vy) const
Protected Attributes
int	state
int	BankSize
int	BankTilts
VOXEL	xSigma
VOXEL	ySigma
VOXEL	tSigma
VOXEL	Period
struct GaborFilter3D< VOXEL > **	Filters
VOXEL *	FilterResponses
VOXEL *	FilterResponsesCache [2 NUMBEROFSPEEDS][2 NUMBEROFSPEEDS]
i3d::Image3d< VOXEL > ***	OutRe
i3d::Image3d< VOXEL > ***	OutIm
Static Protected Attributes
static const VOXEL	RAF [21]

Detailed Description

template<class VOXEL>
class OFHeeger< VOXEL >

This is the original method of Heeger for the computation of 2D optical flow.

The images the class works with must be then 3D, i.e. 2D+time.

The class splitted the method into several parts, which are roughly outlined here. The computation progress is reflected in the state property. There exists a method for each part of the method. These functions automatically inspect and toggle the state property. The process is started with method SetParams(). A wrapper that calls the respective methods in a correct sequence is included in the class as well.

Author:: Vladimír Ulman (xulman@fi.muni.cz)

Date:: 2009

Constructor & Destructor Documentation

template<class VOXEL >

OFHeeger< VOXEL >::OFHeeger ( i3d::Image3d< VOXEL > const & in )

Main constructor: Initiate the method with a read-only reference to the input 3D image.

The initialization may fail, when input image is smaller than 10x10x5 voxels. Therefore, inspect a state of the class after the initialization (state should be CREATED or INCORRECTINPUT).

Parameters:

[in] in an input 2D+t image on which an optical flow is going to be computed

template<class VOXEL >

OFHeeger< VOXEL >::~OFHeeger ( ) [virtual]

Destructor: Clears the memory before the class is destroyed. In fact, it only calls ClearWorkingData().

Member Function Documentation

template<class VOXEL >

void OFHeeger< VOXEL >::ClearWorkingData ( void ) [virtual]

Clears and frees all working data/arrays/images besides the input one.

The state is set to CREATED or PARAMSKNOWN depending if params were already known (i.e. state >= PARAMSKNOWN).

Reimplemented in OFHeegerANI< VOXEL >.

template<class VOXEL >

int OFHeeger< VOXEL >::GetState ( void ) const [inline]

Reads the state of the method.

template<class VOXEL >

int OFHeeger< VOXEL >::PrintState ( void ) const

Prints the human-readable name of the method's state, see the description of states, and returns it.

template<class VOXEL >

int OFHeeger< VOXEL >::SetParams	(	const int	bankSize = `4`,
		const VOXEL	s1 = `4.0`,
		const VOXEL	s2 = `3.0`,
		const VOXEL	s3 = `1.5`,
		const VOXEL	per = `6.0`
	)

Sets some parameters of the method.

It also resets the interal state to PARAMSKNOWN.

It does not allocate OFHeeger::Filters and working images OFHeeger::OutRe and OFHeeger::OutIm. However, it frees them from the memory prior reseting parameters of the method because this is the last place we know about their structure. After new parameters are set, there seems to be no way to safely free the old memory. The feeing is governed by the ClearWorkingData() function.

Parameters:

`[in]`	bankSize	number of filters in a bank to be used, minimum 2
`[in]`	s1	sigma along direction of movement, default 4.0
`[in]`	s2	sigma spatialy perpendicular to direction, default 3.0
`[in]`	s3	sigma temporarily perpendicular to direction, default 1.5
`[in]`	per	period of sine wave, default 6.0

Returns:

0 when requested params were OK
1 when negative or odd number of filters in a bank
2 when the number of filters in a bank was over MAX_FILTER_NUMBER, reached the limit specific only to MS Visual Studio
3 when the number of banks was over MAX_FILTER_NUMBER, reached the limit specific only to MS Visual Studio

template<class VOXEL >

void OFHeeger< VOXEL >::PrintSpecs ( void ) const [virtual]

Pretty print a content of parameters of the method.

Reimplemented in OFHeegerANI< VOXEL >.

template<class VOXEL >

void OFHeeger< VOXEL >::SaveWorkingImages ( void ) const

Saves complex working images, if possible.

Otherwise, it is an empty function.

The file names are heegerXXYYReZ.ics and heegerXXYYImZ.ics where XX is a number of tilt/bank, YY is a number of filter within a bank and Z stands for 'C' if the files are saved just after the convolution stage, for 'E' after energies are computed and for 'P' after Gaussian in the Parseval approximation stage.

If OFHeeger::state is inappropriate, the function does nothing.

template<class VOXEL >

int OFHeeger< VOXEL >::SaveEnergyImages	(	const VOXEL	Vx,
		const VOXEL	Vy,
		const char *	name
	)			const `[virtual]`

Saves Fourier images of filtering banks together with a Fourier image of a 2D motion plane.

The energy of filters is stored in red channel. The intersection of filter with a plane is in green channel. The 2D motion plane is in blue channel. Channels are outputs of functions GetGaborEnergyExtImage(), GetGaborEnergyExtImageWithMotion() and GetFourierEnergyOfMotion().

The output image is i3d::RGB16 image of dimensions (2*ONEPERIOD,2*ONEPERIOD,2*ONEPERIOD) with the DC frequency positioned at [ONEPERIOD,ONEPERIOD,ONEPERIOD]. The file names are heegerXXYYname.ics where XX is a number of tilt/bank, YY is a number of filter within a bank and name is supplied suffix. The suffix may be used to encode the motion plane (Vx,Vy).

If OFHeeger::state is not BANKPREPARED or higher, the function does nothing.

Parameters:

`[in]`	Vx	x-element of a movement
`[in]`	Vy	y-element of a movement
`[in]`	name	the suffix of the output images

Returns:

0 when images were saved OK
1 when OFHeeger::state is too small
2 when intermediate working images were created with different sizes! (internal error)
10-19 error states of GetGaborEnergyExtImage()
20-29 error states of GetGaborEnergyExtImageWithMotion()
30-39 error states of GetFourierEnergyOfMotion()

Reimplemented in OFHeegerANI< VOXEL >.

template<class VOXEL >

int OFHeeger< VOXEL >::GetNumberOfTilts ( void ) const [inline]

Returns the number of tilts used in the method or -1 if this number is not set yet.

template<class VOXEL >

double OFHeeger< VOXEL >::GetTiltStepping ( void ) const [inline]

Returns the tilt stepping in radians or -1 when the BankTilts is not set yet.

template<class VOXEL >

int OFHeeger< VOXEL >::GetNumberOfFilters ( void ) const [inline]

Returns the number of filters in each bank used in the method or -1 if this number is not set yet.

template<class VOXEL >

double OFHeeger< VOXEL >::GetBankStepping ( void ) const [inline]

Returns the degree in radians between adjacent filters in a bank or -1 when the BankSize is not set yet.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::GetxSigma ( void ) const [inline]

Returns the OFHeeger::xSigma or -1 when it hasn't been set yet.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::GetySigma ( void ) const [inline]

Returns the OFHeeger::ySigma or -1 when it hasn't been set yet.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::GettSigma ( void ) const [inline]

Returns the OFHeeger::tSigma or -1 when it hasn't been set yet.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::GetPeriod ( void ) const [inline]

Returns the OFHeeger::Period or -1 when it hasn't been set yet.

template<class VOXEL >

int OFHeeger< VOXEL >::BankPreparation ( void ) [virtual]

Prepares class for filtering.

The function allocates memory for filter templates, ideal filter responses, the 2nd-level cache for ideal filter responses and working images, which shall store convolution results. The (1st-level global) "normalized" ideal filter responses are computed. The 2nd-level local filter respones will be computed on demand during the flow computations (OFHeeger::ComputeFlow() and OFHeeger::ComputeParsevalsAndFlow()). More on this in docs of the OFHeeger::FilterResponsesCache. However, if a associated cache file exists and the internal state is PARAMSKNOWN then the 2nd-level cache is filled from this file.

The interal state is set to BANKPREPARED.

Returns:

0 when everything went just fine
1 when method parameters were not set already
2 on memory allocation error

Reimplemented in OFHeegerANI< VOXEL >, and OFHeegerANI_DC< VOXEL >.

template<class VOXEL >

int OFHeeger< VOXEL >::BankConvolution ( void ) [virtual]

A convolution with Gabor bank.

It removes DC from input image and then calls filtering routine ApplyIIRGaborInteger1D(). It removes DC from input image and then conducts filtering with routines ModulateImage(), ApplyIIRGaussInteger1DGB() and DemodulateImage(). The DC offset removing is ala Heeger, i.e. the local means are being subtracted.

The interal state is set to BANKCONVOLVED.

Returns:

0 filtering went okay
9 filtering is yet not defined/prepared
1-20 error value of the functions RemoveDCalaHeeger()
21-30 error value of the function ApplyIIRGaussInteger1DGB()
21-30 error value of the function ApplyIIRGaborInteger1D()

Reimplemented in OFHeegerANI< VOXEL >, and OFHeegerANI_DC< VOXEL >.

template<class VOXEL >

int OFHeeger< VOXEL >::ComputeEnergy ( void ) [virtual]

Computes Fourier energy from convolved images.

It works with the complex working images OFHeeger::OutRe and OFHeeger::OutIm directly, modifies them inplace. In fact, since the energy is real number, only the OFHeeger::OutRe's content changes to hold energies. The OFHeeger::OutIm is kept untouched (not released) because it will be used right in the follow-up function ComputeParsevals() in the method's workflow.

The internal state is set to ENERGYCOMPUTED.

Returns:

0 it went okay
1 filterings not conducted yet
2 filterings computed but not available, internal consistency problem

template<class VOXEL >

int OFHeeger< VOXEL >::ComputeParsevals ( void ) [virtual]

Approximates relation between local image energies and values available after Fourier transform by computing Gaussian on energy images.

It seems that most reasonable filters and their Fourier transforms can be covered under a Gaussian with sigma=0.8, whose Fourier transform covers approximately an area of frequencies -16 to 16. Hence, in this stage all energy images are filtered with isotropic Gaussian with sigma=0.8.

Since we use the ApplyIIRGauss3() in this function, which works with two separate input and output images, the smoothed energies are stored in OFHeeger::OutIm afterwards. The OFHeeger::OutRe is released from memory.

The internal state is set to PARSEVALSGAUSS.

Returns:

0 it went okay
1 energies not computed yet
2 energies computed but not available, internal consistency problem

template<class VOXEL >

int OFHeeger< VOXEL >::ComputeFlow	(	Image3d< VOXEL > &	Fx,
		Image3d< VOXEL > &	Fy
	)			`[virtual]`

Computes the optical flow sensed from the original (time-lapse) input image OFHeeger::In.

A 2D velocity is assigned to every voxel in the input image. The first elements of 2D velocities are stored voxel-wise in the output image Fx, the second elements in the Fy. The size of output images Fx and Fy is therefore the size of input image OFHeeger::In.

This function merely compares the "Parsevaled" energies measured in the input image to theoretical filter energies. The theoretical energies should correspond to filter responses on random 2D image moving with a certain velocity, the function IntegrateGaborQEnergy(). The comparison is quantitatively evaluated by computing an error rate: sum of squared differences. The goal is to find such 2D velocity that minimizes the error rate.

Note that we are inspecting results of OFHeeger::BankTilts times OFHeeger::BankSize filters. We are comparing this amount of energies computed from the input image to several the-same-sized groups of theoretical filter responses (each groups theoretical responses from 2D random image moving with certain given velocity). If we imagine a 3D plot with x-axis showing a filter tilt, y-axis showing a filter spatial tuning and z-axis showing response of a filter, each group as well as the sensed values samples some plane in this plot. What we do then to find a correct velocity is to find a theory-at-some-velocity-induced plane that matches the real-measured one the best.

We observed that zero-velocity can be judged directly from a shape of the real-measured plane. It forms a ridge in an otherwise flat plane. The ridge heads along the spatial y-axis and is positioned at "center" tilt (x-axis), i.e. filters tuned for zero-velocity with any spatial tuning had strongest responses. However, sometimes in the real data the ridge is not that strong and not that obvious. We implemented a RidgeRatio detection in this function which, in case it detects ridge, superseeds the plane-comparing evaluation and directly assigns velocity (0,0) to a voxel in question.

To compute the RidgeRatio, the function compares measured responses of a center tilt (a "no-velocity" tilt) with its two neighbors of the same spatial tuning and counts how many neighbors has strictly smaller response. If the number is strictly above 1.2 times the number of spatial tuning used in the method (the OFHeeger::BankTilts) the ridge is deemed present.

The (0,0) velocity is also assigned when measured responses are zero (or close to zero) for given voxel.

The internal state is set to FLOWCOMPUTED.

Parameters:

`[in]`	Fx	output image to be filled with x element of optical flow
`[in]`	Fy	output image to be filled with y element of optical flow

Returns:

0 it went okay
1 energies were not Gaussian filtered (Parseval's approximation)

Note:: The function also implements sort of normalization, according to the original Heeger's method. During the computation of normalization factor, we also zero (set to 0.0) voxels whose value is below 0.0001. We just zero values that are already nearly zero because we found it to improve accuracy a bit.; We strongly suggest to call the OFHeeger::SaveResponsesCache() right after this function.

template<class VOXEL >

int OFHeeger< VOXEL >::ComputeParsevalsAndFlow	(	Image3d< VOXEL > &	Fx,
		Image3d< VOXEL > &	Fy
	)			`[virtual]`

This is an optimized variant to computations of "Parseval" and optical flow stages.

Refer to original documentations of OFHeeger::ComputeParsevals() and OFHeeger::ComputeFlow() to learn some properties of this function.

The difference to the original OFHeeger::ComputeFlow() is that memory was rearranged to improve the memory accessing pattern and so the wall-time demand of the two stages. Originally, there are several images holding the computed energies from the input image OFHeeger::In. Assessing some velocity vector for voxel at given position required to read voxel values at this position from all images. When there is not enough free memory on the processor, the images had to swapped away resulting in shifting images back and forth between the main memory and swap for every single image position. The performance was completely lost.

The new arrangement sort of interlaces images such that voxel from all images at given position forms a continuous block in the memory. In fact all voxels from all images at given z-coordinate forms one such block. As a result, there are OFHeeger::In.GetSizeZ() such blocks in the memory (or many of them possibly swapped away). To assess velocity vector, however, only one block is required in the memory. The original working images OFHeeger::OutRe and OFHeeger::OutIm are released from memory to keep the overall memory usage at some reasonable pace.

Otherwise, nothing changes compared to the original functions (including the Note about zeroing nearly zero voxel values).

The internal state is set to FLOWCOMPUTED.

Parameters:

`[in]`	Fx	output image to be filled with x element of optical flow
`[in]`	Fy	output image to be filled with y element of optical flow

Returns:

0 it went okay
1 energies not computed yet
2 energies computed but not available, internal consistency problem
3 memory allocation error
4 error in computing 2nd-level cache responses (sub-pixel accuracy problem)

Note:: We strongly suggest to call the OFHeeger::SaveResponsesCache() right after this function.

template<class VOXEL >

int OFHeeger< VOXEL >::FillResponsesCache	(	const int	cx,
		const int	cy
	)			`[virtual]`

Fills given cell [cx,cy] of the 2nd-level cache OFHeeger::FilterResponsesCache.

If the cache cell is not empty, the function does nothing.

The building/filling of cache is time demanding.

If one is to measure the time required to compute flows on many input images, we suggest to first compute a flow on some single image (side effect will be the initiation of the cache) or just prefill the cache explicitly with this function before the measured computation shall happen. A subsequent flow computations should benefit from the cache and perform a lot faster.

The function can be used when the OFHeeger::state is BANKPREPARED (i.e. after executing OFHeeger::SetParams()) or "higher". The function does not modify the internal state OFHeeger::state.

Parameters:

`[in]`	cx	the first index to the array OFHeeger::FilterResponsesCache
`[in]`	cy	the second index to the array OFHeeger::FilterResponsesCache

Returns:

0 cache cell was filled successfuly
1 incorrect internal state
2 incorrect input parameters
3 could not allocate memory for a cache cell

Reimplemented in OFHeegerANI< VOXEL >.

template<class VOXEL >

int OFHeeger< VOXEL >::SaveResponsesCache ( void ) [virtual]

Saves the 2nd-level cache OFHeeger::FilterResponsesCache into a (persistent) file.

The file is read again automatically by the OFHeeger::BankPreparation(), when found. In fact, it is only read when the OFHeeger::state was PARAMSKNOWN which indicates that new/different filters are used. Otherwise, the state indicates that "still the same old" filters are used again and, thus, the cache remains valid.

The function is recomended to be used right after the OFHeeger::ComputeFlow(), OFHeeger::ComputeParsevalsAndFlow() or OFHeeger::FillResponsesCache() because only these functions build the 2nd-level cache. Hence, it is advisable to store the content of the cache before it is cleared (for instance, as a result of freeing all structures after the flow computation is done).

The building of cache is time demanding.

If one is to measure the time required to compute flows on many input images, we suggest to first compute a flow on some single image (side effect will be the initiation of the cache) or just prefill the cache explicitly with OFHeeger::FillResponsesCache() function before the measured computation shall happen. A subsequent flow computations should benefit from the cache and perform a lot faster.

The function does not modify the internal state OFHeeger::state.

Returns:

0 cache was stored to a file successfuly

Reimplemented in OFHeegerANI< VOXEL >.

template<class VOXEL >

unsigned int OFHeeger< VOXEL >::FRIndex	(	const int	bankTilt,
		const int	bankFilter,
		const int	vx,
		const int	vy
	)			const `[inline, protected]`

An inline indexer to the array OFHeeger::FilterResponses.

The parameters vx and vy must be integers within the bound of <-NUMBEROFSPEEDS,NUMBEROFSPEEDS>, refer to ofHeeger.h to see the actual number. Similarily, the parameter bankTilt must be between 0 and OFHeeger::BankTilts and the parameter bankFilter must be between 0 and OFHeeger::BankSize, all left-inclusively, right-exclusively.

Parameters:

`[in]`	bankTilt	number of filter bank
`[in]`	bankFilter	number of filter in a bank
`[in]`	vx	x element of a LUTed velocity
`[in]`	vy	y element of a LUTed velocity

Returns:

an index to OFHeeger::FilterResponses for given filter

Note:: Inputs are not checked against their bounds, for performance reasons.

template<class VOXEL >

unsigned int OFHeeger< VOXEL >::FRCacheIndex	(	const int	bankTilt,
		const int	bankFilter,
		const int	vx,
		const int	vy
	)			const `[inline, protected]`

An inline indexer to elements of the array OFHeeger::FilterResponsesCache.

The parameters vx and vy must be integers within the bound of <0,4>, representing decimals between 0.1 and 0.5, inclusively, in steps of 0.1. To read an ideal Gabor response for a filter tunned to, for instance, 2nd spatial direction and 3rd tilt and velocity (-2.4,1.8), one should read OFHeeger::FilterResponsesCache[-5+NUMBEROFSPEEDS][3+NUMBEROFSPEEDS][FRCacheIndex(3,2,1,3)], if it exists! (-2.4 and 1.8 are decomposed to -2.5 + 0.1 and 1.5 + 0.3 which points to cell [-2.5*2+NUMBEROFSPEEDS][1.5*2+NUMBEROFSPEEDS].)

Parameters:

`[in]`	bankTilt	number of filter bank
`[in]`	bankFilter	number of filter in a bank
`[in]`	vx	x decimal of a LUTed velocity
`[in]`	vy	y decimal of a LUTed velocity

Returns:

an index to OFHeeger::FilterResponsesCache for given filter

Note:: Inputs are not checked against their bounds, for performance reasons.

Member Data Documentation

template<class VOXEL >

i3d::Image3d<VOXEL> const& OFHeeger< VOXEL >::In

A read-only public reference to the input image.

template<class VOXEL >

int OFHeeger< VOXEL >::state [protected]

It defines the internal state of the computation, use the defines .

Note that some specialized versions of this method may compute more consequent steps at once, e.g. bank convolution and energy computations can be done simultaneously when certain method is used.

The computation is characterized by the following 7 steps:

CREATED when this class was instantiated
PARAMSKNOWN when all method parameters are established/defined
BANKPREPARED after the convolution recipes and mandatory memory allocations are done
BANKCONVOLVED after the convolutions are done including the DC removal procedure
ENERGYCOMPUTED after the energy is obtained from complex results of Gabor bank convolutions
PARSEVALSGAUSS after energy images are Gaussed, a neccessary step of the method
FLOWCOMPUTED after the motion parameters are established

Some more states may exist for different reasons:

INCORRECTINPUT when input image is not 3D or is smaller than 10x10x5 or when bad method's parameter was supplied
ERROR when some error occured during any of the above stages

template<class VOXEL >

int OFHeeger< VOXEL >::BankSize [protected]

The number of filters in a bank, must be positive even number not less than two.

template<class VOXEL >

int OFHeeger< VOXEL >::BankTilts [protected]

The number of banks (one tilt is one bank) used by the method is BankTilts.

It must be odd positive integer not less than one, because there must be always a "90deg-tilt" bank for stationary "movements". In fact, it minus one divided by 2 tells how many different velocities the method should be able to detect. This is so because two tilts are required to detect the same velocity in one and the opposite directions.

Currently, it is set to NUMBEROFSPEEDS+1 firmly. Note that NUMBEROFSPEEDS tells twice the number of maximum integer velocity, hence there are (NUMBEROFSPEEDS/2) *2+1 tilts.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::xSigma [protected]

The sigma of Gaussian envelope of template Gabor filter.

This one heads along the supposed direction of movement. This number must be strictly greater than OFHeeger::ySigma and OFHeeger::tSigma.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::ySigma [protected]

The sigma of Gaussian envelope of template Gabor filter.

This one heads perpendicular to the supposed direction of movement, perpendicular in the spatial plane.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::tSigma [protected]

The sigma of Gaussian envelope of template Gabor filter.

This one heads perpendicular to the supposed direction of movement, perpendicular in the temporal plane.

template<class VOXEL >

VOXEL OFHeeger< VOXEL >::Period [protected]

The period of sine wave of template Gabor filter.

The sine heads perpendicular to the supposed direction of movement, perpendicular in the temporal plane.

template<class VOXEL >

struct GaborFilter3D< VOXEL >** OFHeeger< VOXEL >::Filters [protected]

An array of pointers to sample filters each representing each filtering bank.

Filters in the associated bank differs only in the spatial directions which are, obviously, driven by the bank. The length of the array should be OFHeeger::BankTilts.

template<class VOXEL >

VOXEL* OFHeeger< VOXEL >::FilterResponses [protected]

An array of (some) precomputed ideal Gabor responses.

This is truly a look-up-table to avoid repetative computing of the same values. Note that for every pixel in the 2D+t image one must "compare" computed filter reponses to ideal responses over several velocities (vx,vy) to guess a real velocity present in the image. Since every pixel is first examined with the same set of velocities, the look-up-table comes as a welcomed time-saver. It is perhaps explained better in the OFHeeger::FilterResponsesCache.

The array is "linearized" multi-dimensional array originally of the format [vx][vy][FilterInBank][FilterBank]. That's where the term "filter cell", i.e. the number of velocities stored for each filter given by the constant FILTERCELLSIZE, come from. Now, the array "linearizes" the format [vx][vy][FilterInBank][FilterBank]. The number of all filters in this LUT is given as OFHeeger::BankTilts times OFHeeger::BankSize times the FILTERCELLSIZE.

Moral: One would have to allocate many intermediate arrays which may lie over the memory arbitrarily. We opted for this representation to ease allocation of the array and to make sure all values lie continuously in the memory for fast access (hoping to benefit from cpu caches).

To ease the navigation within this array, one may use the inline function FRIndex().

Note:: Comments on constants NUMBEROFSPEEDS and FILTERCELLSIZE can be found in the docs of the file ofHeeger.h .

template<class VOXEL >

VOXEL* OFHeeger< VOXEL >::FilterResponsesCache[2 *NUMBEROFSPEEDS][2 *NUMBEROFSPEEDS] [protected]

Cache for LUT of ideal Gabor responses on subpixel velocities (aka 2nd-level cache), similarily to the OFHeeger::FilterResponses array.

The OFHeeger::ComputeFlow() and OFHeeger::ComputeParsevalsAndFlow() functions search for a correct velocity using a global search first in the region of -0.5*NUMBEROFSPEEDS to +0.5*NUMBEROFSPEEDS in steps of 0.5 both for x and y. Subpixel search in the vicinity of a global minimum found in the first search is conducted then. Ideal Gabor responses to many different velocities must be available because they are (very often) used in both searches. Since the first search needs the same set of velocities for every voxel in the input image, a precomputed common LUT OFHeeger::FilterResponses becomes very handy. The second search may use different sets of velocities for different voxels. Hence an on-demand filled array, this array, is created during the second stage.

This array is filled on-demand, unlike the array for the first search. We recognize a velocity cell consisting of 25 * OFHeeger::BankTilts * OFHeeger::BankSize, i.e. there are 5 times 5 combinations of velocities each having to hold values of ideal Gabor responses for every tilt vs. spatial direction combination. For example, a cell [-3][2] (*) holds responses to velocities -1.4 to -1.0 for x and 1.1 to 1.5 for y -- right-adjacent inclusive intervals with stepping of 0.1. Note that the global search uses a step of 0.5 (BTW: that's why only 5 times 5 velocity combinations fit into each cell) and non-integer indices are not allowed in arrays, hence a multiple of two is used. Say, a global extreme was located at (-1,0) during the first search, the following four cells will be additionally examined: [-3+NUMBEROFSPEEDS][-1+NUMBEROFSPEEDS], [-3+NUMBEROFSPEEDS][0+NUMBEROFSPEEDS], [-2+NUMBEROFSPEEDS][-1+NUMBEROFSPEEDS] and [-2+NUMBEROFSPEEDS][0+NUMBEROFSPEEDS]. Everytime a cell's content is computed, it is saved until the end of flow computation, to be reused eventually.

(*) Actually, negative indices in arrays are not allowed in the C++. We use a shift by NUMBEROFSPEEDS that renders a "cell [0][0]" to an array position of [NUMBEROFSPEEDS][NUMBEROFSPEEDS]. The cell from the example is actually located at [-3+NUMBEROFSPEEDS][2+NUMBEROFSPEEDS].

The structure is initialized at creation time (in the constructors) and cleared with OFHeeger::ClearWorkingData(). Empty cell is indicated with NULL. Otherwise a pointer exists and points to an array of length 16 times OFHeeger::BankTilts times OFHeeger::BankSize. Every cell's content can be accessed via indexer FRCacheIndex().

Note:: For a single direction x or y, considering a real interval of -NUMBEROFSPEEDS/2 to +NUMBEROFSPEEDS/2 with steps of 0.5 to be used for correct flow velocity vectors, it gives 2*NUMBEROFSPEEDS+1 integer values. These tics/marks are examined in the first search. However, there is only 2*NUMBEROFSPEEDS cells. This is so because the second search deals with two adjacent intervals (remember, still talking about one direction) around a selected tic/mark and there is always one less intervals between tics/marks.

template<class VOXEL >

i3d::Image3d<VOXEL>*** OFHeeger< VOXEL >::OutRe [protected]

Real part of a complex working images.

These are stored in an array according to the tilt of the bank, the second index is a filter's index within a given bank, e.g. OutRe[tilt][direction] points to a certain image. Hence, after initialization, the array OutRe should be OFHeeger::BankTilts long with each element pointing at an array of length OFHeeger::BankSize.

template<class VOXEL >

i3d::Image3d<VOXEL>*** OFHeeger< VOXEL >::OutIm [protected]

Complex part of a complex working images.

These are stored in an array according to the tilt of the bank, the second index is a filter's index within a given bank, e.g. OutIm[tilt][direction] points to a certain image. Hence, after initialization, the array OutRe should be OFHeeger::BankTilts long with each element pointing at an array of length OFHeeger::BankSize.

template<class VOXEL >

const VOXEL OFHeeger< VOXEL >::RAF [static, protected]

Initial value:

{       0.0, 0.0, 0.000598874, 0.00296992, 0.00911867, 0.0214467, 0.0424803,
                                        0.0745317, 0.119364, 0.177901, 0.25, 0.334335, 0.428381, 0.528522,
                                        0.630265, 0.728553, 0.818135, 0.893976, 0.951655, 0.987726, 1.0 }

A helper array of constants that approximate filter Responses Adjusting Function (RAF).

The adjusting function is: y=(0.5*sin((y-0.5)*PI)+0.5)*(0.5*sin((y-0.5)*PI)+0.5) . The range and domain of y is the interval <0:1> with 0 and 1 included. Thus, the responses must be stretched to <0:1> beforehand.

The array splits the interval into 20 regular intervals (with 21 knots), each of length 0.05. Simple linear interpolation can be used to obtain values within any such interval.

The documentation for this class was generated from the following files:

libof/ofHeeger.h
libof/ofHeeger.cc

OFHeeger< VOXEL > Class Template Reference [Optical flow according to Heeger]

Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Static Protected Attributes

Detailed Description

template<class VOXEL> class OFHeeger< VOXEL >

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation

OFHeeger< VOXEL > Class Template Reference
[Optical flow according to Heeger]

template<class VOXEL>
class OFHeeger< VOXEL >