
Previous Article
Variational source conditions and stability estimates for inverse electromagnetic medium scattering problems
 IPI Home
 This Issue

Next Article
Nonlinear Tikhonov regularization in Banach spaces for inverse scattering from anisotropic penetrable media
Foveated compressive imaging for low power vehicle fingerprinting and tracking in aerial imagery
HRL Laboratories LLC, 3011 Malibu Canyon Road, Malibu, CA 902654797, USA 
We describe a foveated compressive sensing approach for image analysis applications that utilizes knowledge of the task to be performed to reduce the number of required sensor measurements and sensor size, weight, and power (SWAP) compared to conventional Nyquist sampling and compressive sensingbased approaches. Our Compressive Optical Foveated Architecture (COFA) adapts the dictionary and compressive measurements to structure and sparsity in the signal, task, and scene by reducing measurement and dictionary mutual coherence and increasing sparsity using principles of actionable information and foveated compressive sensing. Actionable information is used to extract taskrelevant regions of interest (ROIs) from a lowresolution scene analysis by eliminating the effects of nuisances for occlusion and anomalous motion detection. From the extracted ROIs, preferential measurements are taken using foveation as part of the compressive sensing adaptation process. The taskspecific measurement matrix is optimized by using a novel saliencyweighted coherence minimization with respect to the learned signal dictionary. This incorporates the relative usage of the atoms in the dictionary. We utilize a patchbased method to learn the signal priors. A treestructured dictionary of image patches using KSVD is learned which can sparsely represent any given image patch with the tree structure. We have implemented COFA in an endtoend simulation of a vehicle fingerprinting task for aerial surveillance using foveated compressive measurements adapted to hierarchical ROIs consisting of background, roads, and vehicles. Our results show 113× reduction in measurements over conventional sensing and 28× reduction over compressive sensing using random measurements.
References:
[1] 
M. Aharon, M. Elad and A. Bruckstein, KSVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 43114322. Google Scholar 
[2] 
A. Ayvaci, M. Raptis and S. Soatto, Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010.Google Scholar 
[3] 
A. Bruckstein, D. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 3481. doi: 10.1137/060657704. Google Scholar 
[4] 
E. Candés and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 42034215. doi: 10.1109/TIT.2005.858979. Google Scholar 
[5] 
I. Ciocoiu, Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 2932. doi: 10.1109/ECCTD.2011.6043336. Google Scholar 
[6] 
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL).Google Scholar 
[7] 
J. P. Curzan, C. R. Baxter and M. A. Massie, Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318. doi: 10.1117/12.451183. Google Scholar 
[8] 
D. Donoho, A. Maleki and A. Montanari, Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 69206941. doi: 10.1109/TIT.2011.2165823. Google Scholar 
[9] 
J. DuarteCarvajalino and G. Sapiro, Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 13951408. doi: 10.1109/TIP.2009.2022459. Google Scholar 
[10] 
G. Georgiadis, A. Ayvaci and S. Soatto, Actionable Saliency Detection Proc. of CVPR, 2012.Google Scholar 
[11] 
Z. Harmany, A. Oh, R. Marcia and R. Willet, Motionadaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 15. doi: 10.1117/12.892726. Google Scholar 
[12] 
D. Heeger and A. Jepson, Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95117. Google Scholar 
[13] 
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwaveinfraredswircameras/.Google Scholar 
[14] 
R. Jenatton, J. Mairal, G. Obozinski and F. Bach, Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011.Google Scholar 
[15] 
R. Larcom and T. Coffman, Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145148. doi: 10.1109/SSIAI.2010.5483896. Google Scholar 
[16] 
T. Mundhenk, K. Ni, K. Kim and Y. Owechko, Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 114. doi: 10.1117/12.906491. Google Scholar 
[17] 
S. Soatto, Steps Towards a Theory of Visual Information Textbook Draft.Google Scholar 
[18] 
A. Soni and J. Haupt, Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 12501254. doi: 10.1109/ACSSC.2011.6190216. Google Scholar 
[19] 
P. D. Sturkie, Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego.Google Scholar 
[20] 
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPUaccelerated large displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the series Lecture Notes in Computer Science, (2010), 438451. doi: 10.1007/9783642155499_32. Google Scholar 
[21] 
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee, Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009. doi: 10.1109/AIPR.2009.5466304. Google Scholar 
[22] 
L. ZelnikManor, K. Rosenblum and Y. Eldar, Sensing matrix optimization for blocksparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 43004312. doi: 10.1109/TSP.2011.2159211. Google Scholar 
show all references
References:
[1] 
M. Aharon, M. Elad and A. Bruckstein, KSVD: An algorithm for designing overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing, 54 (2006), 43114322. Google Scholar 
[2] 
A. Ayvaci, M. Raptis and S. Soatto, Occlusion Detection and Motion Estimation with Convex Optimization Neural Information Processing Systems, 2010.Google Scholar 
[3] 
A. Bruckstein, D. Donoho and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Review, 51 (2009), 3481. doi: 10.1137/060657704. Google Scholar 
[4] 
E. Candés and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), 42034215. doi: 10.1109/TIT.2005.858979. Google Scholar 
[5] 
I. Ciocoiu, Foveated compressed sensing, Proc. of Europe. Conf. on Circuit Theory and Design, (2011), 2932. doi: 10.1109/ECCTD.2011.6043336. Google Scholar 
[6] 
Columbus surrogate unmanned aerial vehicle (CSUAV) dataset, United States Air Force Research Lab (AFRL).Google Scholar 
[7] 
J. P. Curzan, C. R. Baxter and M. A. Massie, Variable acuity imager with dynamically steerable, programmable superpixels, Infrared Technology and Applications, Proc. SPIE, 4820 (2003), p318. doi: 10.1117/12.451183. Google Scholar 
[8] 
D. Donoho, A. Maleki and A. Montanari, Noise sensitivity phase transition in compressed sensing, IEEE Transactions on Information Theory, 57 (2011), 69206941. doi: 10.1109/TIT.2011.2165823. Google Scholar 
[9] 
J. DuarteCarvajalino and G. Sapiro, Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization, IEEE Transactions on Image Processing, 18 (2009), 13951408. doi: 10.1109/TIP.2009.2022459. Google Scholar 
[10] 
G. Georgiadis, A. Ayvaci and S. Soatto, Actionable Saliency Detection Proc. of CVPR, 2012.Google Scholar 
[11] 
Z. Harmany, A. Oh, R. Marcia and R. Willet, Motionadaptive compressive coded apertures, Proc. of SPIE, 8165 (2011), 15. doi: 10.1117/12.892726. Google Scholar 
[12] 
D. Heeger and A. Jepson, Subspace methods for recovering rigid motion, Intl. J. of Comp. Vis., 7 (1992), 95117. Google Scholar 
[13] 
InView Shortwave Infrared (SWIR) Cameras, http://inviewcorp.com/products/shortwaveinfraredswircameras/.Google Scholar 
[14] 
R. Jenatton, J. Mairal, G. Obozinski and F. Bach, Proximal Methods for Sparse Hierarchical Dictionary Learning J. Machine Learning Research, 2011.Google Scholar 
[15] 
R. Larcom and T. Coffman, Foveated image formation through compressive sensing, Proc. of Southwest Symp. Image Anal. Interp., (2010), 145148. doi: 10.1109/SSIAI.2010.5483896. Google Scholar 
[16] 
T. Mundhenk, K. Ni, K. Kim and Y. Owechko, Detection of unknown targets from aerial camera and extraction of simple object fingerprints for the purpose of target reacquisition, Proc. of SPIE, 8301 (2012), 114. doi: 10.1117/12.906491. Google Scholar 
[17] 
S. Soatto, Steps Towards a Theory of Visual Information Textbook Draft.Google Scholar 
[18] 
A. Soni and J. Haupt, Efficient adaptive compressive sensing using sparse hierarchical learned dictionaries, Proc. of ASILOMAR, (2011), 12501254. doi: 10.1109/ACSSC.2011.6190216. Google Scholar 
[19] 
P. D. Sturkie, Sturkie's Avian Physiology 5th Edition, Academic Press, San Diego.Google Scholar 
[20] 
N. Sundaram, T. Brox and K. Keutzer, Dense point trajectories by GPUaccelerated large displacement optical flow, Chapter: Computer Vision C ECCV 2010, Volume 6311 of the series Lecture Notes in Computer Science, (2010), 438451. doi: 10.1007/9783642155499_32. Google Scholar 
[21] 
F. Tanner, B. Colder, C. Pullen, D. Heagy, M. Eppolito, V. Carlan, C. Oertel and P. Sallee, Overhead Imagery Research Data Set: An Annotated Data Library and Tools to aid in the Development of Computer Vision Algorithms Proc. of IEEE Applied Imagery Pattern Rec. Workshop, 2009. doi: 10.1109/AIPR.2009.5466304. Google Scholar 
[22] 
L. ZelnikManor, K. Rosenblum and Y. Eldar, Sensing matrix optimization for blocksparse decoding, IEEE Transactions on Signal Processing, 59 (2011), 43004312. doi: 10.1109/TSP.2011.2159211. Google Scholar 
Method  Measurement  Dictionary 
rand + flat  random Gaussian orthonormal measurements  (flat) ksvd dictionary 
rand + tree  random Gaussian orthonormal measurements  hierarchical (tree) dictionary 
mc + flat  minimum coherence measurements  (flat) ksvd dictionary 
mc + tree  minimum coherence measurements  hierarchical (tree) dictionary 
wmc + tree  weighted minimum coherence measurements  hierarchical (tree) dictionary 
Method  Measurement  Dictionary 
rand + flat  random Gaussian orthonormal measurements  (flat) ksvd dictionary 
rand + tree  random Gaussian orthonormal measurements  hierarchical (tree) dictionary 
mc + flat  minimum coherence measurements  (flat) ksvd dictionary 
mc + tree  minimum coherence measurements  hierarchical (tree) dictionary 
wmc + tree  weighted minimum coherence measurements  hierarchical (tree) dictionary 
[1] 
Yangyang Xu, Wotao Yin, Stanley Osher. Learning circulant sensing kernels. Inverse Problems & Imaging, 2014, 8 (3) : 901923. doi: 10.3934/ipi.2014.8.901 
[2] 
Vikram Krishnamurthy, William Hoiles. Information diffusion in social sensing. Numerical Algebra, Control & Optimization, 2016, 6 (3) : 365411. doi: 10.3934/naco.2016017 
[3] 
JianWu Xue, XiaoKun Xu, Feng Zhang. Big data dynamic compressive sensing system architecture and optimization algorithm for internet of things. Discrete & Continuous Dynamical Systems  S, 2015, 8 (6) : 14011414. doi: 10.3934/dcdss.2015.8.1401 
[4] 
Hong Jiang, Wei Deng, Zuowei Shen. Surveillance video processing using compressive sensing. Inverse Problems & Imaging, 2012, 6 (2) : 201214. doi: 10.3934/ipi.2012.6.201 
[5] 
Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems & Imaging, 2014, 8 (1) : 321337. doi: 10.3934/ipi.2014.8.321 
[6] 
Yonggui Zhu, Yuying Shi, Bin Zhang, Xinyan Yu. Weightedaverage alternating minimization method for magnetic resonance image reconstruction based on compressive sensing. Inverse Problems & Imaging, 2014, 8 (3) : 925937. doi: 10.3934/ipi.2014.8.925 
[7] 
Yingying Li, Stanley Osher. Coordinate descent optimization for l^{1} minimization with application to compressed sensing; a greedy algorithm. Inverse Problems & Imaging, 2009, 3 (3) : 487503. doi: 10.3934/ipi.2009.3.487 
[8] 
Jae Deok Kim, Ganguk Hwang. Crosslayer modeling and optimization of multichannel cognitive radio networks under imperfect channel sensing. Journal of Industrial & Management Optimization, 2015, 11 (3) : 807828. doi: 10.3934/jimo.2015.11.807 
[9] 
Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165191. doi: 10.3934/jcd.2015002 
[10] 
Ying Zhang, Ling Ma, ZhengHai Huang. On phaseless compressed sensing with partially known support. Journal of Industrial & Management Optimization, 2017, 13 (5) : 18. doi: 10.3934/jimo.2019014 
[11] 
Cesare Bracco, Annalisa Buffa, Carlotta Giannelli, Rafael Vázquez. Adaptive isogeometric methods with hierarchical splines: An overview. Discrete & Continuous Dynamical Systems  A, 2019, 39 (1) : 241261. doi: 10.3934/dcds.2019010 
[12] 
Miguel A. Dumett, Roberto Cominetti. On the stability of an adaptive learning dynamics in traffic games. Journal of Dynamics & Games, 2018, 5 (4) : 265282. doi: 10.3934/jdg.2018017 
[13] 
Shunfu Jin, Wuyi Yue, Shiying Ge. Equilibrium analysis of an opportunistic spectrum access mechanism with imperfect sensing results. Journal of Industrial & Management Optimization, 2017, 13 (3) : 12551271. doi: 10.3934/jimo.2016071 
[14] 
A Voutilainen, Jari P. Kaipio. Model reduction and pollution source identification from remote sensing data. Inverse Problems & Imaging, 2009, 3 (4) : 711730. doi: 10.3934/ipi.2009.3.711 
[15] 
Haruki Katayama, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Effect of spectrum sensing overhead on performance for cognitive radio networks with channel bonding. Journal of Industrial & Management Optimization, 2014, 10 (1) : 2140. doi: 10.3934/jimo.2014.10.21 
[16] 
Paolo Fergola, Marianna Cerasuolo, Edoardo Beretta. An allelopathic competition model with quorum sensing and delayed toxicant production. Mathematical Biosciences & Engineering, 2006, 3 (1) : 3750. doi: 10.3934/mbe.2006.3.37 
[17] 
Seungkook Park. Coherence of sensing matrices coming from algebraicgeometric codes. Advances in Mathematics of Communications, 2016, 10 (2) : 429436. doi: 10.3934/amc.2016016 
[18] 
Richard L Buckalew. Cell cycle clustering and quorum sensing in a response / signaling mediated feedback model. Discrete & Continuous Dynamical Systems  B, 2014, 19 (4) : 867881. doi: 10.3934/dcdsb.2014.19.867 
[19] 
Jan Haškovec, Dietmar Oelz. A free boundary problem for aggregation by short range sensing and differentiated diffusion. Discrete & Continuous Dynamical Systems  B, 2015, 20 (5) : 14611480. doi: 10.3934/dcdsb.2015.20.1461 
[20] 
MinFan He, LiNing Xing, Wen Li, Shang Xiang, Xu Tan. Double layer programming model to the scheduling of remote sensing data processing tasks. Discrete & Continuous Dynamical Systems  S, 2019, 12 (4&5) : 15151526. doi: 10.3934/dcdss.2019104 
2018 Impact Factor: 1.469
Tools
Metrics
Other articles
by authors
[Back to Top]