2014, 11(2): 331-342. doi: 10.3934/mbe.2014.11.331

Designing neural networks for modeling biological data: A statistical perspective

1. 

Department of Economics and Statistics - University of Salerno, Via Giovanni Paolo II, 132. 84084 Fisciano (SA), Italy, Italy

Received  October 2012 Revised  April 2013 Published  October 2013

In this paper, we propose a strategy for the selection of the hidden layer size in feedforward neural network models. The procedure herein presented is based on comparison of different models in terms of their out of sample predictive ability, for a specified loss function. To overcome the problem of data snooping, we extend the scheme based on the use of the reality check with modifications apt to compare nested models. Some applications of the proposed procedure to simulated and real data sets show that it allows to select parsimonious neural network models with the highest predictive accuracy.
Citation: Michele La Rocca, Cira Perna. Designing neural networks for modeling biological data: A statistical perspective. Mathematical Biosciences & Engineering, 2014, 11 (2) : 331-342. doi: 10.3934/mbe.2014.11.331
References:
[1]

D. K. Agrafiotis, W. Cedeño and V. S. Lobanov, On the use of neural network ensembles in QSAR and QSPR,, J. Chem. Inf. Comput. Sci., 42 (2002), 903. doi: 10.1021/ci0203702. Google Scholar

[2]

A. R. Barron, Universal approximation bounds for superposition of a sigmoidal function,, IEEE Trans. Inform. Theory, 39 (1993), 930. doi: 10.1109/18.256500. Google Scholar

[3]

J. K. Basu, D. Bhattacharya and T. Kim, Use of artificial neural network in pattern recognition,, International Journal of Software Engineering and its Applications, 4 (2010), 23. Google Scholar

[4]

H. M. Cartwright, Artificial neural networks in biology and chemistry- the evolution of a new analytical tool,, in Artificial Neural Networks: Methods and Applications (ed. D. J. Livingstone), (2009), 1. doi: 10.1007/978-1-60327-101-1_1. Google Scholar

[5]

X. Chen and H. White, Improved rates and asymptotic normality for nonparametric neural network estimators,, IEEE Trans. Inform. Theory, 45 (1999), 682. doi: 10.1109/18.749011. Google Scholar

[6]

W. Choe, O. K. Ersoy and M. Bina, Neural network schemes for detecting rare events in human genomic DNA,, Bioinformatics, 16 (2010), 1062. doi: 10.1093/bioinformatics/16.12.1062. Google Scholar

[7]

T. E. Clark and M. W. McCracken, Reality checks and comparison of nested predictive models,, J. Bus. Econom. Statist., 30 (2012), 53. doi: 10.1198/jbes.2011.10278. Google Scholar

[8]

T. E. Clark and M. W. McCracken, In-sample tests of predictive ability: A new approach,, J. Econometrics, 170 (2012), 1. doi: 10.1016/j.jeconom.2010.09.012. Google Scholar

[9]

V. Corradi and N. R. Swanson, Predictive density evaluation,, in Handbook of Economic Forecasting, (2006), 197. doi: 10.2139/ssrn.812104. Google Scholar

[10]

J. Devillers, Neural Networks in QSAR and Drug Design,, Academic Press, (1996). Google Scholar

[11]

R. De Veaux, J. Schumi, J. Schweinsberg and L. H. Ungar, Prediction intervals for neural networks via nonlinear regression,, Technometrics, 40 (1998), 273. doi: 10.2307/1270528. Google Scholar

[12]

G. Elliot and A. Timmermann, Optimal forecast combinations under general loss functions and forecast error distribution,, Journal Econometrics, 122 (2004), 47. doi: 10.1016/j.jeconom.2003.10.019. Google Scholar

[13]

J. H. Friedman, Multivariate adaptive regression splines,, Ann. Statist., 19 (1991), 1. doi: 10.1214/aos/1176347963. Google Scholar

[14]

T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning,, 2nd edition, (2008). Google Scholar

[15]

M. Jalali-Heravi, Neural network in analytical chemistry,, in Artificial Neural Networks: Methods and Applications (ed. D. J. Livingstone), (2009), 78. doi: 10.1007/978-1-60327-101-1_6. Google Scholar

[16]

K. Hornik, M. Stinchcombe and P. Auer, Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives,, Neural Computation, 6 (1994), 1262. doi: 10.1162/neco.1994.6.6.1262. Google Scholar

[17]

J. T. G. Hwang and A. A. Ding, Prediction intervals for artificial neural networks,, J. Amer. Statist. Assoc., 92 (1997), 748. doi: 10.1080/01621459.1997.10474027. Google Scholar

[18]

L. J. Lancashire, D. G. Powe, J. S. Reis-Filho, E. Rakha, C. Lemetre, B. Weigelt, T. M. Abdel-Fatah, A. R. Green, R. Mukta and R. Blamey, et al., A validated gene expression profile for detecting clinical outcome in breast cancer using artificial neural networks,, Breast Cancer Research and Treatment, 120 (2010), 83. doi: 10.1007/s10549-009-0378-1. Google Scholar

[19]

M. La Rocca and C. Perna, Variable selection in neural network regression models with dependent data: A subsampling approach,, Comput. Statist. Data Anal., 48 (2005), 415. doi: 10.1016/j.csda.2004.01.004. Google Scholar

[20]

M. La Rocca and C. Perna, Neural network modeling by subsampling,, in Computational Intelligence and Bioinspired Systems (eds. J. Cabestany, (3512), 200. doi: 10.1007/11494669_25. Google Scholar

[21]

M. La Rocca and C. Perna, Neural network modeling with applications to euro exchange rates,, in Computational Methods in Financial Engineering: Essays in Honour of Manfred Gili, (2008), 163. doi: 10.1007/978-3-540-77958-2_9. Google Scholar

[22]

C.-M. Kuan and T. Liu, Forecasting excange rates using feedforward networks and recurrent neural networks,, Journal of Applied Econometrics, 10 (1995), 347. doi: 10.1002/jae.3950100403. Google Scholar

[23]

Y. Makovoz, Random approximates and neural networks,, J. Approx. Theory, 85 (1996), 98. doi: 10.1006/jath.1996.0031. Google Scholar

[24]

H. Merdun and O. Cinar, Artificial neural network and regression techniques in modelling surface water quality,, Environment Protection Engineering, 36 (2010), 95. Google Scholar

[25]

A. Ossen and S. M. Rügen, An analysis of the metric structure of the weight space of feedforward networks and its application to time series modelling and prediction,, in Proceedings of the 4th European Symposium on Artificial Neural Networks (ESANN96), (1996), 24. Google Scholar

[26]

M. Qi and G. P. Zhang, An investigation of model selection criteria for neural network time series forecasting,, European Journal of Operational Research, 132 (2001), 666. doi: 10.1016/S0377-2217(00)00171-5. Google Scholar

[27]

J. P. Romano and M. Wolf, Stepwise multiple testing as formalized data snooping,, Econometrica, 73 (2005), 1237. doi: 10.1111/j.1468-0262.2005.00615.x. Google Scholar

[28]

G. E. Schwarz, Estimating the dimension of a model,, Ann. Statist., 6 (1978), 461. doi: 10.1214/aos/1176344136. Google Scholar

[29]

J. Shao and D. Tu, The Jackknife and the Bootstrap,, Springer Series in Statistics, (1995). doi: 10.1007/978-1-4612-0795-5. Google Scholar

[30]

T. Stamey, J. Kabalin, J. McNeal, I. Johnstone, F. Freiha, E. Redwine and N. Yang, Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate II radical prostactomy treated patients,, Journal of Urology, 16 (1989), 1076. Google Scholar

[31]

N. R. Swanson and H. White, A model selection approach to real time macroeconomic forecasting using linear models and artificial neural networks,, The Review of Economics and Statistics, 79 (1997), 540. doi: 10.1162/003465397557123. Google Scholar

[32]

I. V. Tetko, A. E. P. Villa and D. J. Livingstone, Neural network studies. 2. Variable selection,, J. Chem. Comput. Sci., 36 (1996), 794. doi: 10.1021/ci950204c. Google Scholar

[33]

R. Tibshirani, A comparison of some error estimates for neural network models,, Neural Computation, 8 (1996), 152. doi: 10.1162/neco.1996.8.1.152. Google Scholar

[34]

A. Tsanas, M. A. Little, P. E. McSharry and L. O. Ramig, Accurate telemonitoring of Parkinson's disease progression by non-invasive speech tests,, IEEE Transactions on Biomedical Engineering, 57 (2010), 884. Google Scholar

[35]

B. Turlach, Discussion of Least angle regression by Efron, Hastie, Jon- stone and Tibshirani,, Ann. Statist., 32 (2004), 494. Google Scholar

[36]

D. Urda, J. Subirats, L. Franco and J. M. Jerez, Constructive neural networks to predict breast cancer outcome by using gene expression profiles,, in Trends in Applied Intelligent Systems: 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, (2010), 1. doi: 10.1007/978-3-642-13022-9_32. Google Scholar

[37]

H. White, Learning in artificial neural networks: A statistical perspective,, Neural Computation, 1 (1989), 425. doi: 10.1162/neco.1989.1.4.425. Google Scholar

[38]

H. White, Connectionist nonparametric regression: Multi-layer feedforward networks can learn arbitrary mappings,, Neural Networks, 3 (1990), 535. Google Scholar

[39]

H. White, A reality check for data snooping,, Econometrica, 68 (2000), 1097. doi: 10.1111/1468-0262.00152. Google Scholar

[40]

A. Yasri and D. Hartsough, Toward an optimal procedure for variable selection and QSAR model building,, J. Chem. Inf. Comput. Sci., 41 (2001), 1218. doi: 10.1021/ci010291a. Google Scholar

show all references

References:
[1]

D. K. Agrafiotis, W. Cedeño and V. S. Lobanov, On the use of neural network ensembles in QSAR and QSPR,, J. Chem. Inf. Comput. Sci., 42 (2002), 903. doi: 10.1021/ci0203702. Google Scholar

[2]

A. R. Barron, Universal approximation bounds for superposition of a sigmoidal function,, IEEE Trans. Inform. Theory, 39 (1993), 930. doi: 10.1109/18.256500. Google Scholar

[3]

J. K. Basu, D. Bhattacharya and T. Kim, Use of artificial neural network in pattern recognition,, International Journal of Software Engineering and its Applications, 4 (2010), 23. Google Scholar

[4]

H. M. Cartwright, Artificial neural networks in biology and chemistry- the evolution of a new analytical tool,, in Artificial Neural Networks: Methods and Applications (ed. D. J. Livingstone), (2009), 1. doi: 10.1007/978-1-60327-101-1_1. Google Scholar

[5]

X. Chen and H. White, Improved rates and asymptotic normality for nonparametric neural network estimators,, IEEE Trans. Inform. Theory, 45 (1999), 682. doi: 10.1109/18.749011. Google Scholar

[6]

W. Choe, O. K. Ersoy and M. Bina, Neural network schemes for detecting rare events in human genomic DNA,, Bioinformatics, 16 (2010), 1062. doi: 10.1093/bioinformatics/16.12.1062. Google Scholar

[7]

T. E. Clark and M. W. McCracken, Reality checks and comparison of nested predictive models,, J. Bus. Econom. Statist., 30 (2012), 53. doi: 10.1198/jbes.2011.10278. Google Scholar

[8]

T. E. Clark and M. W. McCracken, In-sample tests of predictive ability: A new approach,, J. Econometrics, 170 (2012), 1. doi: 10.1016/j.jeconom.2010.09.012. Google Scholar

[9]

V. Corradi and N. R. Swanson, Predictive density evaluation,, in Handbook of Economic Forecasting, (2006), 197. doi: 10.2139/ssrn.812104. Google Scholar

[10]

J. Devillers, Neural Networks in QSAR and Drug Design,, Academic Press, (1996). Google Scholar

[11]

R. De Veaux, J. Schumi, J. Schweinsberg and L. H. Ungar, Prediction intervals for neural networks via nonlinear regression,, Technometrics, 40 (1998), 273. doi: 10.2307/1270528. Google Scholar

[12]

G. Elliot and A. Timmermann, Optimal forecast combinations under general loss functions and forecast error distribution,, Journal Econometrics, 122 (2004), 47. doi: 10.1016/j.jeconom.2003.10.019. Google Scholar

[13]

J. H. Friedman, Multivariate adaptive regression splines,, Ann. Statist., 19 (1991), 1. doi: 10.1214/aos/1176347963. Google Scholar

[14]

T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning,, 2nd edition, (2008). Google Scholar

[15]

M. Jalali-Heravi, Neural network in analytical chemistry,, in Artificial Neural Networks: Methods and Applications (ed. D. J. Livingstone), (2009), 78. doi: 10.1007/978-1-60327-101-1_6. Google Scholar

[16]

K. Hornik, M. Stinchcombe and P. Auer, Degree of approximation results for feedforward networks approximating unknown mappings and their derivatives,, Neural Computation, 6 (1994), 1262. doi: 10.1162/neco.1994.6.6.1262. Google Scholar

[17]

J. T. G. Hwang and A. A. Ding, Prediction intervals for artificial neural networks,, J. Amer. Statist. Assoc., 92 (1997), 748. doi: 10.1080/01621459.1997.10474027. Google Scholar

[18]

L. J. Lancashire, D. G. Powe, J. S. Reis-Filho, E. Rakha, C. Lemetre, B. Weigelt, T. M. Abdel-Fatah, A. R. Green, R. Mukta and R. Blamey, et al., A validated gene expression profile for detecting clinical outcome in breast cancer using artificial neural networks,, Breast Cancer Research and Treatment, 120 (2010), 83. doi: 10.1007/s10549-009-0378-1. Google Scholar

[19]

M. La Rocca and C. Perna, Variable selection in neural network regression models with dependent data: A subsampling approach,, Comput. Statist. Data Anal., 48 (2005), 415. doi: 10.1016/j.csda.2004.01.004. Google Scholar

[20]

M. La Rocca and C. Perna, Neural network modeling by subsampling,, in Computational Intelligence and Bioinspired Systems (eds. J. Cabestany, (3512), 200. doi: 10.1007/11494669_25. Google Scholar

[21]

M. La Rocca and C. Perna, Neural network modeling with applications to euro exchange rates,, in Computational Methods in Financial Engineering: Essays in Honour of Manfred Gili, (2008), 163. doi: 10.1007/978-3-540-77958-2_9. Google Scholar

[22]

C.-M. Kuan and T. Liu, Forecasting excange rates using feedforward networks and recurrent neural networks,, Journal of Applied Econometrics, 10 (1995), 347. doi: 10.1002/jae.3950100403. Google Scholar

[23]

Y. Makovoz, Random approximates and neural networks,, J. Approx. Theory, 85 (1996), 98. doi: 10.1006/jath.1996.0031. Google Scholar

[24]

H. Merdun and O. Cinar, Artificial neural network and regression techniques in modelling surface water quality,, Environment Protection Engineering, 36 (2010), 95. Google Scholar

[25]

A. Ossen and S. M. Rügen, An analysis of the metric structure of the weight space of feedforward networks and its application to time series modelling and prediction,, in Proceedings of the 4th European Symposium on Artificial Neural Networks (ESANN96), (1996), 24. Google Scholar

[26]

M. Qi and G. P. Zhang, An investigation of model selection criteria for neural network time series forecasting,, European Journal of Operational Research, 132 (2001), 666. doi: 10.1016/S0377-2217(00)00171-5. Google Scholar

[27]

J. P. Romano and M. Wolf, Stepwise multiple testing as formalized data snooping,, Econometrica, 73 (2005), 1237. doi: 10.1111/j.1468-0262.2005.00615.x. Google Scholar

[28]

G. E. Schwarz, Estimating the dimension of a model,, Ann. Statist., 6 (1978), 461. doi: 10.1214/aos/1176344136. Google Scholar

[29]

J. Shao and D. Tu, The Jackknife and the Bootstrap,, Springer Series in Statistics, (1995). doi: 10.1007/978-1-4612-0795-5. Google Scholar

[30]

T. Stamey, J. Kabalin, J. McNeal, I. Johnstone, F. Freiha, E. Redwine and N. Yang, Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate II radical prostactomy treated patients,, Journal of Urology, 16 (1989), 1076. Google Scholar

[31]

N. R. Swanson and H. White, A model selection approach to real time macroeconomic forecasting using linear models and artificial neural networks,, The Review of Economics and Statistics, 79 (1997), 540. doi: 10.1162/003465397557123. Google Scholar

[32]

I. V. Tetko, A. E. P. Villa and D. J. Livingstone, Neural network studies. 2. Variable selection,, J. Chem. Comput. Sci., 36 (1996), 794. doi: 10.1021/ci950204c. Google Scholar

[33]

R. Tibshirani, A comparison of some error estimates for neural network models,, Neural Computation, 8 (1996), 152. doi: 10.1162/neco.1996.8.1.152. Google Scholar

[34]

A. Tsanas, M. A. Little, P. E. McSharry and L. O. Ramig, Accurate telemonitoring of Parkinson's disease progression by non-invasive speech tests,, IEEE Transactions on Biomedical Engineering, 57 (2010), 884. Google Scholar

[35]

B. Turlach, Discussion of Least angle regression by Efron, Hastie, Jon- stone and Tibshirani,, Ann. Statist., 32 (2004), 494. Google Scholar

[36]

D. Urda, J. Subirats, L. Franco and J. M. Jerez, Constructive neural networks to predict breast cancer outcome by using gene expression profiles,, in Trends in Applied Intelligent Systems: 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, (2010), 1. doi: 10.1007/978-3-642-13022-9_32. Google Scholar

[37]

H. White, Learning in artificial neural networks: A statistical perspective,, Neural Computation, 1 (1989), 425. doi: 10.1162/neco.1989.1.4.425. Google Scholar

[38]

H. White, Connectionist nonparametric regression: Multi-layer feedforward networks can learn arbitrary mappings,, Neural Networks, 3 (1990), 535. Google Scholar

[39]

H. White, A reality check for data snooping,, Econometrica, 68 (2000), 1097. doi: 10.1111/1468-0262.00152. Google Scholar

[40]

A. Yasri and D. Hartsough, Toward an optimal procedure for variable selection and QSAR model building,, J. Chem. Inf. Comput. Sci., 41 (2001), 1218. doi: 10.1021/ci010291a. Google Scholar

[1]

Cheng-Hsiung Hsu, Suh-Yuh Yang. Traveling wave solutions in cellular neural networks with multiple time delays. Conference Publications, 2005, 2005 (Special) : 410-419. doi: 10.3934/proc.2005.2005.410

[2]

Zhigang Zeng, Tingwen Huang. New passivity analysis of continuous-time recurrent neural networks with multiple discrete delays. Journal of Industrial & Management Optimization, 2011, 7 (2) : 283-289. doi: 10.3934/jimo.2011.7.283

[3]

Ying Sue Huang. Resynchronization of delayed neural networks. Discrete & Continuous Dynamical Systems - A, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397

[4]

Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139

[5]

Ricai Luo, Honglei Xu, Wu-Sheng Wang, Jie Sun, Wei Xu. A weak condition for global stability of delayed neural networks. Journal of Industrial & Management Optimization, 2016, 12 (2) : 505-514. doi: 10.3934/jimo.2016.12.505

[6]

Benedetta Lisena. Average criteria for periodic neural networks with delay. Discrete & Continuous Dynamical Systems - B, 2014, 19 (3) : 761-773. doi: 10.3934/dcdsb.2014.19.761

[7]

Larry Turyn. Cellular neural networks: asymmetric templates and spatial chaos. Conference Publications, 2003, 2003 (Special) : 864-871. doi: 10.3934/proc.2003.2003.864

[8]

Zhuwei Qin, Fuxun Yu, Chenchen Liu, Xiang Chen. How convolutional neural networks see the world --- A survey of convolutional neural network visualization methods. Mathematical Foundations of Computing, 2018, 1 (2) : 149-180. doi: 10.3934/mfc.2018008

[9]

Karim El Laithy, Martin Bogdan. Synaptic energy drives the information processing mechanisms in spiking neural networks. Mathematical Biosciences & Engineering, 2014, 11 (2) : 233-256. doi: 10.3934/mbe.2014.11.233

[10]

Yong Zhao, Qishao Lu. Periodic oscillations in a class of fuzzy neural networks under impulsive control. Conference Publications, 2011, 2011 (Special) : 1457-1466. doi: 10.3934/proc.2011.2011.1457

[11]

Zbigniew Gomolka, Boguslaw Twarog, Jacek Bartman. Improvement of image processing by using homogeneous neural networks with fractional derivatives theorem. Conference Publications, 2011, 2011 (Special) : 505-514. doi: 10.3934/proc.2011.2011.505

[12]

Leong-Kwan Li, Sally Shao. Convergence analysis of the weighted state space search algorithm for recurrent neural networks. Numerical Algebra, Control & Optimization, 2014, 4 (3) : 193-207. doi: 10.3934/naco.2014.4.193

[13]

Jui-Pin Tseng. Global asymptotic dynamics of a class of nonlinearly coupled neural networks with delays. Discrete & Continuous Dynamical Systems - A, 2013, 33 (10) : 4693-4729. doi: 10.3934/dcds.2013.33.4693

[14]

Cheng-Hsiung Hsu, Suh-Yuh Yang. Structure of a class of traveling waves in delayed cellular neural networks. Discrete & Continuous Dynamical Systems - A, 2005, 13 (2) : 339-359. doi: 10.3934/dcds.2005.13.339

[15]

Benoît Perthame, Delphine Salort. On a voltage-conductance kinetic system for integrate & fire neural networks. Kinetic & Related Models, 2013, 6 (4) : 841-864. doi: 10.3934/krm.2013.6.841

[16]

Sylvia Novo, Rafael Obaya, Ana M. Sanz. Exponential stability in non-autonomous delayed equations with applications to neural networks. Discrete & Continuous Dynamical Systems - A, 2007, 18 (2&3) : 517-536. doi: 10.3934/dcds.2007.18.517

[17]

Tingting Su, Xinsong Yang. Finite-time synchronization of competitive neural networks with mixed delays. Discrete & Continuous Dynamical Systems - B, 2016, 21 (10) : 3655-3667. doi: 10.3934/dcdsb.2016115

[18]

Zuowei Cai, Jianhua Huang, Lihong Huang. Generalized Lyapunov-Razumikhin method for retarded differential inclusions: Applications to discontinuous neural networks. Discrete & Continuous Dynamical Systems - B, 2017, 22 (9) : 3591-3614. doi: 10.3934/dcdsb.2017181

[19]

Ruoxia Li, Huaiqin Wu, Xiaowei Zhang, Rong Yao. Adaptive projective synchronization of memristive neural networks with time-varying delays and stochastic perturbation. Mathematical Control & Related Fields, 2015, 5 (4) : 827-844. doi: 10.3934/mcrf.2015.5.827

[20]

Xilin Fu, Zhang Chen. New discrete analogue of neural networks with nonlinear amplification function and its periodic dynamic analysis. Conference Publications, 2007, 2007 (Special) : 391-398. doi: 10.3934/proc.2007.2007.391

2018 Impact Factor: 1.313

Metrics

  • PDF downloads (11)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]