# American Institute of Mathematical Sciences

April  2017, 11(2): 305-338. doi: 10.3934/ipi.2017015

## Optical flow on evolving sphere-like surfaces

 1 Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences, Altenberger Straße 69, 4040 Linz, Austria 2 Computational Science Center, University of Vienna, Oskar-Morgenstern-Platz 1,1090 Vienna, Austria 3 Johann Radon Institute for Computational and Applied Mathematics, Austrian Academy of Sciences, Altenberger Straße 69, 4040 Linz, Austria

* Corresponding author

Received  June 2015 Revised  April 2016 Published  March 2017

In this work we consider optical flow on evolving Riemannian 2-manifolds which can be parametrised from the 2-sphere. Our main motivation is to estimate cell motion in time-lapse volumetric microscopy images depicting fluorescently labelled cells of a live zebrafish embryo. We exploit the fact that the recorded cells float on the surface of the embryo and allow for the extraction of an image sequence together with a sphere-like surface. We solve the resulting variational problem by means of a Galerkin method based on vector spherical harmonics and present numerical results computed from the aforementioned microscopy data.

Citation: Lukas F. Lang, Otmar Scherzer. Optical flow on evolving sphere-like surfaces. Inverse Problems & Imaging, 2017, 11 (2) : 305-338. doi: 10.3934/ipi.2017015
##### References:

show all references

##### References:
Frames 140 (left) and 141 (right) of the volumetric zebrafish microscopy images recorded during early embryogenesis. The sequence contains a total number of 151 frames recorded at time intervals of $120 \, \mathrm{s}$. Fluorescence response is indicated by blue colour and is proportional to the observed intensity. The embryonic axis of the animal forms around the clearly visible dent. All dimensions are in micrometer ($\mu$m).
Depicted are frames no. 140 (left) and 141 (right) of the processed zebrafish microscopy sequence. Top and bottom row differ by a rotation of 180 degrees around the $x^{3}$-axis. All dimensions are in micrometer ($\mu$m).
From left to right, top to bottom: a) colour-coded surface velocity $\mathbf{\hat{V}}$, b) signed norm $\text{sign}(\partial_{t} \tilde{\rho}) \left\| {{\bf{\hat V}}} \right\|$, c) optical flow $\mathbf{\hat{v}}$ for $\alpha = 1$, and d) total motion $\mathbf{\hat{M}} = \mathbf{\hat{V}} + \mathbf{\hat{v}}$. Values are computed for the interval between frames 140 and 141. All surfaces are depicted in a top view.
Top view of the optical flow field computed on a static spherical geometry. The same values of α as in Fig. 10 were used.
Schematic illustration of a cut through the surfaces $\mathcal{S}^2$ and $\mathcal{M}_{t}$ intersecting the origin. In addition, we show a radial line along which the extension $\bar{f}(t, \cdot)$ is constant. The surface normals are shown in grey.
Commutative diagram relating spaces $\Omega$, $\mathcal{S}^{2}$, and $\mathcal{M}_{t}$, and tangent vector fields. We highlight that $\mathbf{y}_{p}$ is the coordinate representation, see Sec. 2.1, of a particular tangential vector spherical harmonic $\mathbf{\tilde{y}}_{p}$ and $\mathbf{\hat{y}}_{p}$ is its uniquely identified tangent vector field on $\mathcal{M}_{t}$.
Illustration of trajectories through the evolving surface. Their corresponding velocities are shown in grey.
Illustration of a triangular face (filled gray) intersecting the sphere $\mathcal{S}^{2}$ at the vertices (hollow circles). The six nodal points consist of the vertices of the triangle together the edge midpoints (filled black dots). The approximated sphere-like surface is shown by the hatched gray area. A radial line passing through the vertex $v_{i}$ is shown. The hollow circle indicates the intersection with $\mathcal{S}^{2}$ at which $\bar{f}(v_{i})$ in (49) is taken. $\bar{f}$ itself, as described in Sec. 5.2, is assigned by taking the maximum image intensity along the drawn radial line between the two cross marks.
Frames no. 140 (left) and 141 (right) of the processed image sequence in a top view. The embryo's body axis is oriented from bottom left to top right.
Tangent vector field minimising $\mathcal{E}_{\alpha}$. Depicted is the colour-coded optical flow field computed between frames 140 and 141 for different values of $\alpha$. The bottom row differs from the top view by a rotation of 180 degrees around the $x^{3}$-axis. From left to right: a) $\alpha = 10^{-2}$, b) $\alpha = 10^{-1}$, c) $\alpha = 1$, and d) $\alpha = 10$.
Function $\tilde{\rho}_{h}$ obtained by minimising $\mathcal{F}_{\beta}$ for frames 140 (left column) and 141 (right column). Colour corresponds to the radius ($\mu$m) of the fitted surface. The top row depicts $\mathcal{S}_{h}^{2}$ in a top view.
Top view of the optical flow field computed for different values of $\alpha$. From left to right, top to bottom: a) $\alpha = 10^{-2}$, b) $\alpha = 10^{-1}$, c) $\alpha = 1$, and d) $\alpha = 10$.
Summary of notation used throughout the paper.
 $\Omega$ coordinate domain $I$ time interval $\mathcal{S}^2$ 2-sphere $\mathcal{M}$ family of sphere-like surfaces $\mathcal{M}_{t}$ $T_{y}\mathcal{M}_{t}$ tangent plane at $y \in \mathcal{M}_{t}$ $\mathbf{\tilde{N}}, \mathbf{\hat{N}}$ outward unit normals to $\mathcal{S}^2$ and $\mathcal{M}$ $\bf{x}$ , $\bf{y}$ parametrisations of $\mathcal{S}^2$ and $\mathcal{M}$ $D\bf{x}$ , $D\bf{y}$ gradient matrix of $\bf{x}$ and $\bf{y}$ $\{ \partial_{1} \bf{x}, \partial_{2} \bf{x} \}$ basis for $T\mathcal{S}^2$ $\{ \partial_{1} \bf{y}, \partial_{2} \bf{y} \}$ basis for $T\mathcal{M}$ $\{ \mathbf{\hat{e}}_{1}, \mathbf{\hat{e}}_{2} \}$ orthonormal basis for $T\mathcal{M}_{t}$ $\mathbf{\hat{V}}$ surface velocity of $\mathcal{M}$ $\tilde{\phi}, D\tilde{\phi}$ smooth map from $\mathcal{S}^{2}$ to $\mathcal{M}$ and its differential $\tilde{f}$ , $\hat{f}$ , $f$ scalar function on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version $\nabla_{\mathcal{S}^2} \tilde{f}$ , $\nabla_{\mathcal{M}} \hat{f}$ surface gradient on $\mathcal{S}^2$ and $\mathcal{M}_{t}$ $\mathbf{\tilde{v}}$ , $\mathbf{\hat{v}}$ , $\mathbf{v}$ tangent vector fields on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version $\nabla_{\mathbf{\hat{u}}} \mathbf{\hat{v}}$ covariant derivative of $\mathbf{\hat{v}}$ along direction $\mathbf{\hat{u}}$ on $\mathcal{M}_{t}$ $\bar{f}$ , $\mathbf{\bar{v}}$ radially constant extensions of $\hat{f}$ and $\mathbf{\hat{v}}$ to $\mathbb{R}^{3} \setminus \{ 0 \}$ $\tilde{Y}_{n,j}$ scalar spherical harmonic of degree $n$ and order $j$ $\mathbf{\tilde{y}}_{n,j}^{(i)}$ vector spherical harmonic of degree $n$ , order $j$ , and type $i$ $\mathbf{\hat{y}}_{n,j}^{(i)}$ pushforward of $\mathbf{\tilde{y}}_{n,j}^{(i)}$ via the differential $D\tilde{\phi}$
 $\Omega$ coordinate domain $I$ time interval $\mathcal{S}^2$ 2-sphere $\mathcal{M}$ family of sphere-like surfaces $\mathcal{M}_{t}$ $T_{y}\mathcal{M}_{t}$ tangent plane at $y \in \mathcal{M}_{t}$ $\mathbf{\tilde{N}}, \mathbf{\hat{N}}$ outward unit normals to $\mathcal{S}^2$ and $\mathcal{M}$ $\bf{x}$ , $\bf{y}$ parametrisations of $\mathcal{S}^2$ and $\mathcal{M}$ $D\bf{x}$ , $D\bf{y}$ gradient matrix of $\bf{x}$ and $\bf{y}$ $\{ \partial_{1} \bf{x}, \partial_{2} \bf{x} \}$ basis for $T\mathcal{S}^2$ $\{ \partial_{1} \bf{y}, \partial_{2} \bf{y} \}$ basis for $T\mathcal{M}$ $\{ \mathbf{\hat{e}}_{1}, \mathbf{\hat{e}}_{2} \}$ orthonormal basis for $T\mathcal{M}_{t}$ $\mathbf{\hat{V}}$ surface velocity of $\mathcal{M}$ $\tilde{\phi}, D\tilde{\phi}$ smooth map from $\mathcal{S}^{2}$ to $\mathcal{M}$ and its differential $\tilde{f}$ , $\hat{f}$ , $f$ scalar function on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version $\nabla_{\mathcal{S}^2} \tilde{f}$ , $\nabla_{\mathcal{M}} \hat{f}$ surface gradient on $\mathcal{S}^2$ and $\mathcal{M}_{t}$ $\mathbf{\tilde{v}}$ , $\mathbf{\hat{v}}$ , $\mathbf{v}$ tangent vector fields on $\mathcal{S}^2$ , $\mathcal{M}$ , and their coordinate version $\nabla_{\mathbf{\hat{u}}} \mathbf{\hat{v}}$ covariant derivative of $\mathbf{\hat{v}}$ along direction $\mathbf{\hat{u}}$ on $\mathcal{M}_{t}$ $\bar{f}$ , $\mathbf{\bar{v}}$ radially constant extensions of $\hat{f}$ and $\mathbf{\hat{v}}$ to $\mathbb{R}^{3} \setminus \{ 0 \}$ $\tilde{Y}_{n,j}$ scalar spherical harmonic of degree $n$ and order $j$ $\mathbf{\tilde{y}}_{n,j}^{(i)}$ vector spherical harmonic of degree $n$ , order $j$ , and type $i$ $\mathbf{\hat{y}}_{n,j}^{(i)}$ pushforward of $\mathbf{\tilde{y}}_{n,j}^{(i)}$ via the differential $D\tilde{\phi}$
Radii $R$ of the colour disks used for colour-coded visualisation of tangent vector fields.
 Figures 9(a) 9(b) 9(c) 9(d) 10(a) 10(b) 10(c) 10(d) 11(a) 11(c) 11(d) 12(a) 12(b) 12(c) 12(d) $R$ $8.92$ $4.62$ $2.90$ $2.19$ $12.07$ $2.90$ $12.02$ $9.23$ $4.87$ $2.92$ $2.10$
 Figures 9(a) 9(b) 9(c) 9(d) 10(a) 10(b) 10(c) 10(d) 11(a) 11(c) 11(d) 12(a) 12(b) 12(c) 12(d) $R$ $8.92$ $4.62$ $2.90$ $2.19$ $12.07$ $2.90$ $12.02$ $9.23$ $4.87$ $2.92$ $2.10$
 [1] Thomas Schuster, Joachim Weickert. On the application of projection methods for computing optical flow fields. Inverse Problems & Imaging, 2007, 1 (4) : 673-690. doi: 10.3934/ipi.2007.1.673 [2] Jan Haskovec, Nader Masmoudi, Christian Schmeiser, Mohamed Lazhar Tayeb. The Spherical Harmonics Expansion model coupled to the Poisson equation. Kinetic & Related Models, 2011, 4 (4) : 1063-1079. doi: 10.3934/krm.2011.4.1063 [3] George Dassios, Michalis N. Tsampas. Vector ellipsoidal harmonics and neuronal current decomposition in the brain. Inverse Problems & Imaging, 2009, 3 (2) : 243-257. doi: 10.3934/ipi.2009.3.243 [4] Aniello Raffaele Patrone, Otmar Scherzer. On a spatial-temporal decomposition of optical flow. Inverse Problems & Imaging, 2017, 11 (4) : 761-781. doi: 10.3934/ipi.2017036 [5] Chjan C. Lim, Da Zhu. Variational analysis of energy-enstrophy theories on the sphere. Conference Publications, 2005, 2005 (Special) : 611-620. doi: 10.3934/proc.2005.2005.611 [6] Tom Goldstein, Xavier Bresson, Stan Osher. Global minimization of Markov random fields with applications to optical flow. Inverse Problems & Imaging, 2012, 6 (4) : 623-644. doi: 10.3934/ipi.2012.6.623 [7] Djano Kandaswamy, Thierry Blu, Dimitri Van De Ville. Analytic sensing for multi-layer spherical models with application to EEG source imaging. Inverse Problems & Imaging, 2013, 7 (4) : 1251-1270. doi: 10.3934/ipi.2013.7.1251 [8] Matthias Bergner, Lars Schäfer. Time-like surfaces of prescribed anisotropic mean curvature in Minkowski space. Conference Publications, 2011, 2011 (Special) : 155-162. doi: 10.3934/proc.2011.2011.155 [9] Franz W. Kamber and Peter W. Michor. The flow completion of a manifold with vector field. Electronic Research Announcements, 2000, 6: 95-97. [10] T. Varslo, C E Yarman, M. Cheney, B Yazıcı. A variational approach to waveform design for synthetic-aperture imaging. Inverse Problems & Imaging, 2007, 1 (3) : 577-592. doi: 10.3934/ipi.2007.1.577 [11] Chjan C. Lim, Junping Shi. The role of higher vorticity moments in a variational formulation of Barotropic flows on a rotating sphere. Discrete & Continuous Dynamical Systems - B, 2009, 11 (3) : 717-740. doi: 10.3934/dcdsb.2009.11.717 [12] Dieter Mayer, Fredrik Strömberg. Symbolic dynamics for the geodesic flow on Hecke surfaces. Journal of Modern Dynamics, 2008, 2 (4) : 581-627. doi: 10.3934/jmd.2008.2.581 [13] Francois Ledrappier and Omri Sarig. Invariant measures for the horocycle flow on periodic hyperbolic surfaces. Electronic Research Announcements, 2005, 11: 89-94. [14] Carlangelo Liverani. On the work and vision of Dmitry Dolgopyat. Journal of Modern Dynamics, 2010, 4 (2) : 211-225. doi: 10.3934/jmd.2010.4.211 [15] Marta García-Huidobro, Raul Manásevich, J. R. Ward. Vector p-Laplacian like operators, pseudo-eigenvalues, and bifurcation. Discrete & Continuous Dynamical Systems - A, 2007, 19 (2) : 299-321. doi: 10.3934/dcds.2007.19.299 [16] Huan Gao, Yu-Hong Dai, Xiao-Jiao Tong. Barzilai-Borwein-like methods for the extreme eigenvalue problem. Journal of Industrial & Management Optimization, 2015, 11 (3) : 999-1019. doi: 10.3934/jimo.2015.11.999 [17] S. J. Li, Z. M. Fang. On the stability of a dual weak vector variational inequality problem. Journal of Industrial & Management Optimization, 2008, 4 (1) : 155-165. doi: 10.3934/jimo.2008.4.155 [18] G. Mastroeni, L. Pellegrini. On the image space analysis for vector variational inequalities. Journal of Industrial & Management Optimization, 2005, 1 (1) : 123-132. doi: 10.3934/jimo.2005.1.123 [19] Bendong Lou. Spiral rotating waves of a geodesic curvature flow on the unit sphere. Discrete & Continuous Dynamical Systems - B, 2012, 17 (3) : 933-942. doi: 10.3934/dcdsb.2012.17.933 [20] Lori Badea. Multigrid methods for some quasi-variational inequalities. Discrete & Continuous Dynamical Systems - S, 2013, 6 (6) : 1457-1471. doi: 10.3934/dcdss.2013.6.1457

2018 Impact Factor: 1.469