# American Institute of Mathematical Sciences

December  2017, 11(6): 997-1025. doi: 10.3934/ipi.2017046

## Analysis of a variational model for motion compensated inpainting

 1 Istituto per le Applicazioni del Calcolo, CNR, Via dei Taurini 19,00185 Roma, Italy 2 Dipartimento di Matematica e Informatica, Universitá della Calabria, Via Pietro Bucci, Arcavacata di Rende, 87036 Cosenza, Italy

Received  October 2016 Revised  June 2017 Published  September 2017

We study a variational problem for simultaneous video inpainting and motion estimation. We consider a functional proposed by Lauze and Nielsen [25] and we study, by means of the relaxation method of the Calculus of Variations, a slightly modified version of this functional. The domain of the relaxed functional is constituted of functions of bounded variation and we compute a representation formula of the relaxed functional. The representation formula shows the role of discontinuities of the various functions involved in the variational model. The present study clarifies the variational properties of the functional proposed in [25] for motion compensated video inpainting.

Citation: Riccardo March, Giuseppe Riey. Analysis of a variational model for motion compensated inpainting. Inverse Problems & Imaging, 2017, 11 (6) : 997-1025. doi: 10.3934/ipi.2017046
##### References:
 [1] L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40. doi: 10.1007/BF00052492. Google Scholar [2] L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000. [3] L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321. doi: 10.1512/iumj.1993.42.42015. Google Scholar [4] G. Aubert, R. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182. doi: 10.1137/S0036139998340170. Google Scholar [5] G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308. doi: 10.1137/S003614109834123X. Google Scholar [6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006. [7] M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497. Google Scholar [8] K. Bredies, K. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526. doi: 10.1137/090769521. Google Scholar [9] K. Bredies, K. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454. doi: 10.1016/j.jmaa.2012.08.053. Google Scholar [10] T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3. Google Scholar [11] A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855. doi: 10.1093/imanum/drn038. Google Scholar [12] G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989. Google Scholar [13] J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530.Google Scholar [14] S. Conti, J. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487. doi: 10.1051/m2an/2015036. Google Scholar [15] A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193. doi: 10.1007/BF01759320. Google Scholar [16] F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190. Google Scholar [17] F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709. doi: 10.1512/iumj.1984.33.33036. Google Scholar [18] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156. Google Scholar [19] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172. Google Scholar [20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984. doi: 10.1007/978-1-4684-9486-0. [21] C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178. Google Scholar [22] B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. Google Scholar [23] S. H. Keller, F. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028. doi: 10.1109/TIP.2008.2003394. Google Scholar [24] S. H. Keller, F. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884. doi: 10.1109/TIP.2011.2106793. Google Scholar [25] F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787.Google Scholar [26] Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045. Google Scholar [27] S. Uras, F. Girosi, A. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87. Google Scholar [28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989. doi: 10.1007/978-1-4612-1015-3.

show all references

##### References:
 [1] L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40. doi: 10.1007/BF00052492. Google Scholar [2] L. Ambrosio, N. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000. [3] L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321. doi: 10.1512/iumj.1993.42.42015. Google Scholar [4] G. Aubert, R. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182. doi: 10.1137/S0036139998340170. Google Scholar [5] G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308. doi: 10.1137/S003614109834123X. Google Scholar [6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006. [7] M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497. Google Scholar [8] K. Bredies, K. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526. doi: 10.1137/090769521. Google Scholar [9] K. Bredies, K. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454. doi: 10.1016/j.jmaa.2012.08.053. Google Scholar [10] T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3. Google Scholar [11] A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855. doi: 10.1093/imanum/drn038. Google Scholar [12] G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989. Google Scholar [13] J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530.Google Scholar [14] S. Conti, J. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487. doi: 10.1051/m2an/2015036. Google Scholar [15] A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193. doi: 10.1007/BF01759320. Google Scholar [16] F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190. Google Scholar [17] F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709. doi: 10.1512/iumj.1984.33.33036. Google Scholar [18] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156. Google Scholar [19] M. Giaquinta, G. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172. Google Scholar [20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984. doi: 10.1007/978-1-4684-9486-0. [21] C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178. Google Scholar [22] B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. Google Scholar [23] S. H. Keller, F. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028. doi: 10.1109/TIP.2008.2003394. Google Scholar [24] S. H. Keller, F. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884. doi: 10.1109/TIP.2011.2106793. Google Scholar [25] F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787.Google Scholar [26] Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045. Google Scholar [27] S. Uras, F. Girosi, A. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87. Google Scholar [28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989. doi: 10.1007/978-1-4612-1015-3.
 [1] Bernard Dacorogna, Giovanni Pisante, Ana Margarida Ribeiro. On non quasiconvex problems of the calculus of variations. Discrete & Continuous Dynamical Systems - A, 2005, 13 (4) : 961-983. doi: 10.3934/dcds.2005.13.961 [2] Daniel Faraco, Jan Kristensen. Compactness versus regularity in the calculus of variations. Discrete & Continuous Dynamical Systems - B, 2012, 17 (2) : 473-485. doi: 10.3934/dcdsb.2012.17.473 [3] Felix Sadyrbaev. Nonlinear boundary value problems of the calculus of variations. Conference Publications, 2003, 2003 (Special) : 760-770. doi: 10.3934/proc.2003.2003.760 [4] Agnieszka B. Malinowska, Delfim F. M. Torres. Euler-Lagrange equations for composition functionals in calculus of variations on time scales. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 577-593. doi: 10.3934/dcds.2011.29.577 [5] Delfim F. M. Torres. Proper extensions of Noether's symmetry theorem for nonsmooth extremals of the calculus of variations. Communications on Pure & Applied Analysis, 2004, 3 (3) : 491-500. doi: 10.3934/cpaa.2004.3.491 [6] Nuno R. O. Bastos, Rui A. C. Ferreira, Delfim F. M. Torres. Necessary optimality conditions for fractional difference problems of the calculus of variations. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 417-437. doi: 10.3934/dcds.2011.29.417 [7] Kazuo Aoki, Ansgar Jüngel, Peter A. Markowich. Small velocity and finite temperature variations in kinetic relaxation models. Kinetic & Related Models, 2010, 3 (1) : 1-15. doi: 10.3934/krm.2010.3.1 [8] Nikos Katzourakis. Nonuniqueness in vector-valued calculus of variations in $L^\infty$ and some Linear elliptic systems. Communications on Pure & Applied Analysis, 2015, 14 (1) : 313-327. doi: 10.3934/cpaa.2015.14.313 [9] Gisella Croce, Nikos Katzourakis, Giovanni Pisante. $\mathcal{D}$-solutions to the system of vectorial Calculus of Variations in $L^∞$ via the singular value problem. Discrete & Continuous Dynamical Systems - A, 2017, 37 (12) : 6165-6181. doi: 10.3934/dcds.2017266 [10] Ioan Bucataru, Matias F. Dahl. Semi-basic 1-forms and Helmholtz conditions for the inverse problem of the calculus of variations. Journal of Geometric Mechanics, 2009, 1 (2) : 159-180. doi: 10.3934/jgm.2009.1.159 [11] Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161 [12] Ivar Ekeland. From Frank Ramsey to René Thom: A classical problem in the calculus of variations leading to an implicit differential equation. Discrete & Continuous Dynamical Systems - A, 2010, 28 (3) : 1101-1119. doi: 10.3934/dcds.2010.28.1101 [13] Nikos Katzourakis. Corrigendum to the paper: Nonuniqueness in Vector-Valued Calculus of Variations in $L^\infty$ and some Linear Elliptic Systems. Communications on Pure & Applied Analysis, 2019, 18 (4) : 2197-2198. doi: 10.3934/cpaa.2019098 [14] Pamela A. Marshall, Eden E. Tanzosh, Francisco J. Solis, Haiyan Wang. Response of yeast mutants to extracellular calcium variations. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 439-453. doi: 10.3934/dcdsb.2009.12.439 [15] Fabrizio Colombo, Graziano Gentili, Irene Sabadini and Daniele C. Struppa. A functional calculus in a noncommutative setting. Electronic Research Announcements, 2007, 14: 60-68. doi: 10.3934/era.2007.14.60 [16] Ken Abe. Some uniqueness result of the Stokes flow in a half space in a space of bounded functions. Discrete & Continuous Dynamical Systems - S, 2014, 7 (5) : 887-900. doi: 10.3934/dcdss.2014.7.887 [17] Vladimir V. Kisil. Mobius transformations and monogenic functional calculus. Electronic Research Announcements, 1996, 2: 26-33. [18] Hamed Azizollahi, Marion Darbas, Mohamadou M. Diallo, Abdellatif El Badia, Stephanie Lohrengel. EEG in neonates: Forward modeling and sensitivity analysis with respect to variations of the conductivity. Mathematical Biosciences & Engineering, 2018, 15 (4) : 905-932. doi: 10.3934/mbe.2018041 [19] Feng-Bin Wang, Sze-Bi Hsu, Wendi Wang. Dynamics of harmful algae with seasonal temperature variations in the cove-main lake. Discrete & Continuous Dynamical Systems - B, 2016, 21 (1) : 313-335. doi: 10.3934/dcdsb.2016.21.313 [20] Hassan Emamirad, Arnaud Rougirel. A functional calculus approach for the rational approximation with nonuniform partitions. Discrete & Continuous Dynamical Systems - A, 2008, 22 (4) : 955-972. doi: 10.3934/dcds.2008.22.955

2018 Impact Factor: 1.469