pandora.validation.interpolated_disparity

This module contains classes and functions associated to the interpolation of the disparity map for the validation step.

Module Contents

Classes

AbstractInterpolation

Abstract Interpolation class

McCnnInterpolation

McCnnInterpolation class allows to perform the interpolation of the disparity map

SgmInterpolation

SgmInterpolation class allows to perform the interpolation of the disparity map
class pandora.validation.interpolated_disparity.AbstractInterpolation[source]

Abstract Interpolation class

__metaclass__[source]
interpolation_methods_avail: Dict[source]
classmethod register_subclass(short_name: str)[source]

Allows to register the subclass with its short name

Parameters:

short_name (string) – the subclass to be registered

abstract desc() → None[source]

Describes the disparity interpolation method for the validation step :return: None

abstract interpolated_disparity(left: xarray.Dataset, img_left: xarray.Dataset = None, img_right: xarray.Dataset = None, cv: xarray.Dataset = None) → None[source]

Interpolation of the left disparity map to resolve occlusion and mismatch conflicts.

Parameters:

  • left (xarray.Dataset) –

    left Dataset with the variables :

    • disparity_map 2D xarray.DataArray (row, col)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

    • validity_mask 2D xarray.DataArray (row, col)

  • img_left (xarray.Dataset) –

    left Datset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • img_right (xarray.Dataset) –

    right Dataset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • cv (xarray.Dataset) –

    cost_volume Dataset with the variables:

    • cost_volume 3D xarray.DataArray (row, col, disp)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

Returns:

None

class pandora.validation.interpolated_disparity.McCnnInterpolation(**cfg: dict)[source]

Bases: AbstractInterpolation

McCnnInterpolation class allows to perform the interpolation of the disparity map

check_config(**cfg: dict) → None[source]

Check and update the configuration

Parameters:

cfg (dictionary) – optional configuration, {}

Returns:

None

desc() → None[source]

Describes the disparity interpolation method :return: None

interpolated_disparity(left: xarray.Dataset, img_left: xarray.Dataset = None, img_right: xarray.Dataset = None, cv: xarray.Dataset = None) → None[source]

Interpolation of the left disparity map to resolve occlusion and mismatch conflicts.

Parameters:

  • left (xarray.Dataset) –

    left Dataset with the variables :

    • disparity_map 2D xarray.DataArray (row, col)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

    • validity_mask 2D xarray.DataArray (row, col)

  • img_left (xarray.Dataset) –

    left Datset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • img_right (xarray.Dataset) –

    right Dataset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • cv (xarray.Dataset) –

    cost_volume Dataset with the variables:

    • cost_volume 3D xarray.DataArray (row, col, disp)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

Returns:

None

static interpolate_occlusion_mc_cnn(disp: numpy.ndarray, valid: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray][source]

Interpolation of the left disparity map to resolve occlusion conflicts. Interpolate occlusion by moving left until we find a position labeled correct.

Žbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. The journal of machine learning research, 17(1), 2287-2318.

Parameters:

  • disp (2D np.array (row, col)) – disparity map

  • valid (2D np.array (row, col)) – validity mask

Returns:

the interpolate left disparity map, with the validity mask update :

  • If out & MSK_PIXEL_FILLED_OCCLUSION != 0 : Invalid pixel : filled occlusion

Return type:

tuple(2D np.array (row, col), 2D np.array (row, col))

static interpolate_mismatch_mc_cnn(disp: numpy.ndarray, valid: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray][source]

Interpolation of the left disparity map to resolve mismatch conflicts. Interpolate mismatch by finding the nearest correct pixels in 16 different directions and use the median of their disparities.

Žbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. The journal of machine learning research, 17(1), 2287-2318.

Parameters:

  • disp (2D np.array (row, col)) – disparity map

  • valid (2D np.array (row, col)) – validity mask

Returns:

the interpolate left disparity map, with the validity mask update :

  • If out & MSK_PIXEL_FILLED_MISMATCH != 0 : Invalid pixel : filled mismatch

Return type:

tuple(2D np.array (row, col), 2D np.array (row, col))

class pandora.validation.interpolated_disparity.SgmInterpolation(**cfg: dict)[source]

Bases: AbstractInterpolation

SgmInterpolation class allows to perform the interpolation of the disparity map

check_config(**cfg: dict) → None[source]

Check and update the configuration

Parameters:

cfg (dictionary) – optional configuration, {}

Returns:

None

desc() → None[source]

Describes the disparity interpolation method :return: None

interpolated_disparity(left: xarray.Dataset, img_left: xarray.Dataset = None, img_right: xarray.Dataset = None, cv: xarray.Dataset = None) → None[source]

Interpolation of the left disparity map to resolve occlusion and mismatch conflicts.

Parameters:

  • left (xarray.Dataset) –

    left Dataset with the variables :

    • disparity_map 2D xarray.DataArray (row, col)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

    • validity_mask 2D xarray.DataArray (row, col)

  • img_left (xarray.Dataset) –

    left Datset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • img_right (xarray.Dataset) –

    right Dataset image containing :

    • im: 2D (row, col) or 3D (band_im, row, col) xarray.DataArray float32

    • disparity (optional): 3D (disp, row, col) xarray.DataArray float32

    • msk (optional): 2D (row, col) xarray.DataArray int16

    • classif (optional): 3D (band_classif, row, col) xarray.DataArray int16

    • segm (optional): 2D (row, col) xarray.DataArray int16

  • cv (xarray.Dataset) –

    cost_volume Dataset with the variables:

    • cost_volume 3D xarray.DataArray (row, col, disp)

    • confidence_measure 3D xarray.DataArray (row, col, indicator)

Returns:

None

static interpolate_occlusion_sgm(disp: numpy.ndarray, valid: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray][source]

Interpolation of the left disparity map to resolve occlusion conflicts. Interpolate occlusion by moving by selecting the right lowest value along paths from 8 directions.

HIRSCHMULLER, Heiko. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 2007, vol. 30, no 2, p. 328-341.

Parameters:

  • disp (2D np.array (row, col)) – disparity map

  • valid (2D np.array (row, col)) – validity mask

Returns:

the interpolate left disparity map, with the validity mask update :

  • If out & MSK_PIXEL_FILLED_OCCLUSION != 0 : Invalid pixel : filled occlusion

Return type:

: tuple(2D np.array (row, col), 2D np.array (row, col))

static interpolate_mismatch_sgm(disp: numpy.ndarray, valid: numpy.ndarray) → Tuple[numpy.ndarray, numpy.ndarray][source]

Interpolation of the left disparity map to resolve mismatch conflicts. Interpolate mismatch by finding the nearest correct pixels in 8 different directions and use the median of their disparities. Mismatched pixel areas that are direct neighbors of occluded pixels are treated as occlusions.

HIRSCHMULLER, Heiko. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 2007, vol. 30, no 2, p. 328-341.

Parameters:

  • disp (2D np.array (row, col)) – disparity map

  • valid (2D np.array (row, col)) – validity mask

Returns:

the interpolate left disparity map, with the validity mask update :

  • If out & MSK_PIXEL_FILLED_MISMATCH != 0 : Invalid pixel : filled mismatch

Return type:

tuple(2D np.array (row, col), 2D np.array (row, col))