December 2017, 11(6): 997-1025. doi: 10.3934/ipi.2017046

Analysis of a variational model for motion compensated inpainting

1. 

Istituto per le Applicazioni del Calcolo, CNR, Via dei Taurini 19,00185 Roma, Italy

2. 

Dipartimento di Matematica e Informatica, Universitá della Calabria, Via Pietro Bucci, Arcavacata di Rende, 87036 Cosenza, Italy

Received  October 2016 Revised  June 2017 Published  September 2017

We study a variational problem for simultaneous video inpainting and motion estimation. We consider a functional proposed by Lauze and Nielsen [25] and we study, by means of the relaxation method of the Calculus of Variations, a slightly modified version of this functional. The domain of the relaxed functional is constituted of functions of bounded variation and we compute a representation formula of the relaxed functional. The representation formula shows the role of discontinuities of the various functions involved in the variational model. The present study clarifies the variational properties of the functional proposed in [25] for motion compensated video inpainting.

Citation: Riccardo March, Giuseppe Riey. Analysis of a variational model for motion compensated inpainting. Inverse Problems & Imaging, 2017, 11 (6) : 997-1025. doi: 10.3934/ipi.2017046
References:
[1]

L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40. doi: 10.1007/BF00052492.

[2] L. AmbrosioN. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000.
[3]

L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321. doi: 10.1512/iumj.1993.42.42015.

[4]

G. AubertR. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182. doi: 10.1137/S0036139998340170.

[5]

G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308. doi: 10.1137/S003614109834123X.

[6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006.
[7]

M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497.

[8]

K. BrediesK. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526. doi: 10.1137/090769521.

[9]

K. BrediesK. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454. doi: 10.1016/j.jmaa.2012.08.053.

[10]

T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3.

[11]

A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855. doi: 10.1093/imanum/drn038.

[12]

G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989.

[13]

J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530.

[14]

S. ContiJ. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487. doi: 10.1051/m2an/2015036.

[15]

A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193. doi: 10.1007/BF01759320.

[16]

F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190.

[17]

F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709. doi: 10.1512/iumj.1984.33.33036.

[18]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156.

[19]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172.

[20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984. doi: 10.1007/978-1-4684-9486-0.
[21]

C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178.

[22]

B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203.

[23]

S. H. KellerF. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028. doi: 10.1109/TIP.2008.2003394.

[24]

S. H. KellerF. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884. doi: 10.1109/TIP.2011.2106793.

[25]

F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787.

[26]

Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045.

[27]

S. UrasF. GirosiA. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87.

[28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989. doi: 10.1007/978-1-4612-1015-3.

show all references

References:
[1]

L. Ambrosio, Variational problems in SBV and image segmentation, Acta Appl. Math., 17 (1989), 1-40. doi: 10.1007/BF00052492.

[2] L. AmbrosioN. Fusco and D. Pallara, Functions of Bounded Variation and Free Discontinuity Problems, Oxford University Press, New York, 2000.
[3]

L. Ambrosio and D. Pallara, Integral representation of relaxed functionals on $BV(\mathbb{R}^n, \mathbb{R}^k)$ and polyhedral approximation, Indiana Univ. Math. J., 42 (1993), 295-321. doi: 10.1512/iumj.1993.42.42015.

[4]

G. AubertR. Deriche and P. Kornprobst, Computing optimal flow via variational techniques, SIAM J. App. Math., 60 (2000), 156-182. doi: 10.1137/S0036139998340170.

[5]

G. Aubert and P. Kornprobst, A mathematical study of the relaxed optical flow problem in the space $BV(Ω)$, SIAM J. Math. Anal., 30 (1999), 1282-1308. doi: 10.1137/S003614109834123X.

[6] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd Edition, Springer, New York, 2006.
[7]

M. Bertalmio, A. Bertozzi and G. Sapiro, Navier-Stokes, fluid-dynamics and image and video inpainting, in Proceedings of Computer Vision and Pattern Recognition, 2001,355-362. doi: 10.1109/CVPR.2001.990497.

[8]

K. BrediesK. Kunish and T. Pock, Total generalized variation, SIAM J. Imaging Sciences, 3 (2010), 492-526. doi: 10.1137/090769521.

[9]

K. BrediesK. Kunish and T. Valkonen, Properties of $L^1-TGV^2$: The one-dimensional case, J. Math. Anal. Appl., 398 (2013), 438-454. doi: 10.1016/j.jmaa.2012.08.053.

[10]

T. Brox, A. Bruhn, N. Papenberg and J. Weickert, High accuracy optical flow estimation based on a theory for warping, Proceedings of the 8th European Conference on Computer Vision (eds. T. Pajdla and J. Matas), Springer, 3024 (2004), 25-36. doi: 10.1007/978-3-540-24673-2_3.

[11]

A. Buffa and C. Ortner, Compact embeddings of broken Sobolev spaces and applications, IMA J. Numer. Anal., 29 (2009), 827-855. doi: 10.1093/imanum/drn038.

[12]

G. Buttazzo, Semicontinuity, Relaxation and Integral Representation in the Calculus of Variations, Pitman Research Notes in Mathematics Series, Vol. 207, Longman Scientific & Technical, UK, 1989.

[13]

J. P. Cocquerez, L. Chanas and J. Blanc-Talon, Simultaneous inpainting and motion estimation of highly degraded video-sequences, in Scandinavian Conference on Image Analysis, LNCS, Springer-Verlag, 2749 (2003), 523-530.

[14]

S. ContiJ. Ginster and M. Rumpf, A BV functional and its relaxation for joint motion estimation and image sequence recovery, ESAIM: Mathematical Modelling and Numerical Analysis, 49 (2015), 1463-1487. doi: 10.1051/m2an/2015036.

[15]

A. Corbo Esposito and R. De Arcangelis, Comparison results for some types of relaxation of variational integral functionals, Ann. Mat. Pura Appl., 164 (1993), 155-193. doi: 10.1007/BF01759320.

[16]

F. Demengel, Fonctions á hessien borné, Annales de l'Institut Fourier, 34 (1984), 155-190.

[17]

F. Demengel and R. Temam, Convex functions of a measure and applications, Indiana University Mathematics Journal, 33 (1984), 673-709. doi: 10.1512/iumj.1984.33.33036.

[18]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅰ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 143-156.

[19]

M. GiaquintaG. Modica and J. Soucek, Functionals with linear growth in the calculus of variations. Ⅱ, Commentationes Matematicae Universitatis Carolinae, 20 (1979), 157-172.

[20] E. Giusti, Minimal Surfaces and Functions of Bounded Variation, Birkhäuser, Boston, 1984. doi: 10.1007/978-1-4684-9486-0.
[21]

C. Goffman and J. Serrin, Sublinear functions of measures and variationals integrals, Duke Math. J., 31 (1964), 159-178.

[22]

B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203.

[23]

S. H. KellerF. Lauze and M. Nielsen, Deintarlacing using variational methods, IEEE Trans. Image Proc., 17 (2008), 2015-2028. doi: 10.1109/TIP.2008.2003394.

[24]

S. H. KellerF. Lauze and M. Nielsen, Video super-resolution using simultaneous motion and intensity calculations, IEEE Trans. Image Proc., 20 (2011), 1870-1884. doi: 10.1109/TIP.2011.2106793.

[25]

F. Lauze and M. Nielsen, A Variational algorithm for motion compensated inpainting, in British Machine Vision Conference (eds. S. Barman, A. Hoppe and T. Ellis editors), BMVA, 2 (2004), 777-787.

[26]

Y. G. Reshetnyak, Weak convergence of completely additive vector functions on a set, (Russian) Sibirski Mat. Zh., 9 (1968), 1386-1394; translation in Siberian Math. J., 9 (1968), 1039-1045.

[27]

S. UrasF. GirosiA. Verri and V. Torre, A computational approach to motion perception, Biol. Cybern., 60 (1988), 79-87.

[28] W. P. Ziemer, Weakly differentiable functions, Springer-Verlag, New York, 1989. doi: 10.1007/978-1-4612-1015-3.
[1]

Bernard Dacorogna, Giovanni Pisante, Ana Margarida Ribeiro. On non quasiconvex problems of the calculus of variations. Discrete & Continuous Dynamical Systems - A, 2005, 13 (4) : 961-983. doi: 10.3934/dcds.2005.13.961

[2]

Daniel Faraco, Jan Kristensen. Compactness versus regularity in the calculus of variations. Discrete & Continuous Dynamical Systems - B, 2012, 17 (2) : 473-485. doi: 10.3934/dcdsb.2012.17.473

[3]

Felix Sadyrbaev. Nonlinear boundary value problems of the calculus of variations. Conference Publications, 2003, 2003 (Special) : 760-770. doi: 10.3934/proc.2003.2003.760

[4]

Agnieszka B. Malinowska, Delfim F. M. Torres. Euler-Lagrange equations for composition functionals in calculus of variations on time scales. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 577-593. doi: 10.3934/dcds.2011.29.577

[5]

Delfim F. M. Torres. Proper extensions of Noether's symmetry theorem for nonsmooth extremals of the calculus of variations. Communications on Pure & Applied Analysis, 2004, 3 (3) : 491-500. doi: 10.3934/cpaa.2004.3.491

[6]

Nuno R. O. Bastos, Rui A. C. Ferreira, Delfim F. M. Torres. Necessary optimality conditions for fractional difference problems of the calculus of variations. Discrete & Continuous Dynamical Systems - A, 2011, 29 (2) : 417-437. doi: 10.3934/dcds.2011.29.417

[7]

Kazuo Aoki, Ansgar Jüngel, Peter A. Markowich. Small velocity and finite temperature variations in kinetic relaxation models. Kinetic & Related Models, 2010, 3 (1) : 1-15. doi: 10.3934/krm.2010.3.1

[8]

Nikos Katzourakis. Nonuniqueness in vector-valued calculus of variations in $L^\infty$ and some Linear elliptic systems. Communications on Pure & Applied Analysis, 2015, 14 (1) : 313-327. doi: 10.3934/cpaa.2015.14.313

[9]

Gisella Croce, Nikos Katzourakis, Giovanni Pisante. $\mathcal{D}$-solutions to the system of vectorial Calculus of Variations in $L^∞$ via the singular value problem. Discrete & Continuous Dynamical Systems - A, 2017, 37 (12) : 6165-6181. doi: 10.3934/dcds.2017266

[10]

Ioan Bucataru, Matias F. Dahl. Semi-basic 1-forms and Helmholtz conditions for the inverse problem of the calculus of variations. Journal of Geometric Mechanics, 2009, 1 (2) : 159-180. doi: 10.3934/jgm.2009.1.159

[11]

Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161

[12]

Ivar Ekeland. From Frank Ramsey to René Thom: A classical problem in the calculus of variations leading to an implicit differential equation. Discrete & Continuous Dynamical Systems - A, 2010, 28 (3) : 1101-1119. doi: 10.3934/dcds.2010.28.1101

[13]

Pamela A. Marshall, Eden E. Tanzosh, Francisco J. Solis, Haiyan Wang. Response of yeast mutants to extracellular calcium variations. Discrete & Continuous Dynamical Systems - B, 2009, 12 (2) : 439-453. doi: 10.3934/dcdsb.2009.12.439

[14]

Fabrizio Colombo, Graziano Gentili, Irene Sabadini and Daniele C. Struppa. A functional calculus in a noncommutative setting. Electronic Research Announcements, 2007, 14: 60-68. doi: 10.3934/era.2007.14.60

[15]

Ken Abe. Some uniqueness result of the Stokes flow in a half space in a space of bounded functions. Discrete & Continuous Dynamical Systems - S, 2014, 7 (5) : 887-900. doi: 10.3934/dcdss.2014.7.887

[16]

Vladimir V. Kisil. Mobius transformations and monogenic functional calculus. Electronic Research Announcements, 1996, 2: 26-33.

[17]

Hamed Azizollahi, Marion Darbas, Mohamadou M. Diallo, Abdellatif El Badia, Stephanie Lohrengel. EEG in neonates: Forward modeling and sensitivity analysis with respect to variations of the conductivity. Mathematical Biosciences & Engineering, 2018, 15 (4) : 905-932. doi: 10.3934/mbe.2018041

[18]

Feng-Bin Wang, Sze-Bi Hsu, Wendi Wang. Dynamics of harmful algae with seasonal temperature variations in the cove-main lake. Discrete & Continuous Dynamical Systems - B, 2016, 21 (1) : 313-335. doi: 10.3934/dcdsb.2016.21.313

[19]

Hassan Emamirad, Arnaud Rougirel. A functional calculus approach for the rational approximation with nonuniform partitions. Discrete & Continuous Dynamical Systems - A, 2008, 22 (4) : 955-972. doi: 10.3934/dcds.2008.22.955

[20]

Lukas F. Lang, Otmar Scherzer. Optical flow on evolving sphere-like surfaces. Inverse Problems & Imaging, 2017, 11 (2) : 305-338. doi: 10.3934/ipi.2017015

2017 Impact Factor: 1.465

Metrics

  • PDF downloads (13)
  • HTML views (96)
  • Cited by (0)

Other articles
by authors

[Back to Top]