2011, 1(1): 151-169. doi: 10.3934/naco.2011.1.151

Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins

1. 

Graduate School of Engineering, Osaka University, Yamada-Oka 1-2, Suita, Osaka 565-0871, Japan, Japan, Japan, Japan

Received  September 2010 Revised  December 2010 Published  February 2011

The all-together method is one of the support vector machine (SVM) for multiclass classification by using a piece-wise linear function. Recently, we proposed a new hard-margin all-together model maximizing geometric margins in the sense of multiobjective optimization for the high generalization ability, which is called the multiobjective multiclass SVM (MMSVM). Moreover, we derived its solving techniques which can find a Pareto optimal solution for the MMSVM, and verified that classifiers with larger geometric margins were obtained by the proposed techniques through numerical experiments. However, the experiments are not enough to evaluate the classification performance of the proposed model, and the MMSVM is a hard-margin model which can be applied to only piecewise linearly separable data. Therefore, in this paper, we extend the hard-margin model into soft-margin one by introducing penalty functions for the slack margin variables, and derive a single-objective second-order cone programming (SOCP) problem to solve it. Furthermore, through numerical experiments we verify the classification performance of the hard and soft-margin MMSVMs for benchmark problems.
Citation: Keiji Tatsumi, Masashi Akao, Ryo Kawachi, Tetsuzo Tanino. Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 151-169. doi: 10.3934/naco.2011.1.151
References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification,", Springer-Verlag, (2005).

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming,, Mathematical Programming, 95 (2003), 3.

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition,, in, (1994), 77.

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines,, Computational Optimization and Applications, 12 (1999), 53. doi: 10.1023/A:1008663629662.

[5]

M. Ehrgott, "Multicriteria Optimization,'', 2nd edition, (2005).

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs,, Neuro COLT2 Technical Report Series, (2000).

[7]

U. Kressel, Pairwise classification and support vector machines,, in, (1999), 255.

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines,, IEEE Trans. Neural Networks, 13 (2002), 181.

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers,, Mathematical Programming, 95 (2003), 407.

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms,, IEEE Trans. Neural Networks, 12 (2001), 181. doi: 10.1109/72.914517.

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines,, IEEE Trans. Neural Networks, 14 (2004), 45. doi: 10.1109/TNN.2003.820841.

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification,, in, 12 (2000), 547.

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition,, in, (2007), 1095. doi: 10.1109/SICE.2007.4421147.

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins,, Pacific Journal of Optimization, 6 (2000), 115.

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'', Cambridge University Press, (2004).

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().

[17]

J. Weston and C. Watkins, Multi-class support vector machines,, Technical report CSD-TR-98-04, (1998), 98.

[18]

V. N. Vapnik, "Statistical Learning Theory,'', A Wiley-Interscience Publication, (1998).

show all references

References:
[1]

S. Abe, "Support Vector Machines for Pattern Classification,", Springer-Verlag, (2005).

[2]

F. Alizadeh and D. Goldfarb, Second-order cone programming,, Mathematical Programming, 95 (2003), 3.

[3]

L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Sackinger, P. Simard and V. Vapnik, Comparison of classifier methods: A case study in handwriting digit recognition,, in, (1994), 77.

[4]

E. J. Bredensteiner and K. P. Bennett, Multicategory classification by support vector machines,, Computational Optimization and Applications, 12 (1999), 53. doi: 10.1023/A:1008663629662.

[5]

M. Ehrgott, "Multicriteria Optimization,'', 2nd edition, (2005).

[6]

Y. Guermeur, Combining discriminant models with new multiclass SVMs,, Neuro COLT2 Technical Report Series, (2000).

[7]

U. Kressel, Pairwise classification and support vector machines,, in, (1999), 255.

[8]

C. W. Hsh and C. J. Lin, A comparison of methods for multiclass support vector machines,, IEEE Trans. Neural Networks, 13 (2002), 181.

[9]

H. D. Mittelmann, An independent benchmarking of SDP and SOCP solvers,, Mathematical Programming, 95 (2003), 407.

[10]

K. R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Shölkopf, An introduction to kernel-based learning algorithms,, IEEE Trans. Neural Networks, 12 (2001), 181. doi: 10.1109/72.914517.

[11]

A. Passerini, M. Pontil and P. Frasconi, New results on error correcting output codes of kernel machines,, IEEE Trans. Neural Networks, 14 (2004), 45. doi: 10.1109/TNN.2003.820841.

[12]

J. C. Platt, N. Cristianini and J. Shawe-Taylor, Large margin DAG's for multiclass classification,, in, 12 (2000), 547.

[13]

K. Tatsumi, K. Hayashida, H. Higashi and T. Tanino, Multi-objective multiclass support vector machine for pattern recognition,, in, (2007), 1095. doi: 10.1109/SICE.2007.4421147.

[14]

K. Tatsumi, R. Kawachi, K. Hayashida and T. Tanino, Multiobjective multiclass support vector machines maximizing geometric margins,, Pacific Journal of Optimization, 6 (2000), 115.

[15]

J. S. Taylor and N. Cristianini, "Kernel Methods for Pattern Analysis,'', Cambridge University Press, (2004).

[16]

, UCI benchmark repository of artificial and real data sets, University of California Irvine,, Available from: , ().

[17]

J. Weston and C. Watkins, Multi-class support vector machines,, Technical report CSD-TR-98-04, (1998), 98.

[18]

V. N. Vapnik, "Statistical Learning Theory,'', A Wiley-Interscience Publication, (1998).

[1]

Yi Zhang, Yong Jiang, Liwei Zhang, Jiangzhong Zhang. A perturbation approach for an inverse linear second-order cone programming. Journal of Industrial & Management Optimization, 2013, 9 (1) : 171-189. doi: 10.3934/jimo.2013.9.171

[2]

Yubo Yuan, Weiguo Fan, Dongmei Pu. Spline function smooth support vector machine for classification. Journal of Industrial & Management Optimization, 2007, 3 (3) : 529-542. doi: 10.3934/jimo.2007.3.529

[3]

Xiaoni Chi, Zhongping Wan, Zijun Hao. Second order sufficient conditions for a class of bilevel programs with lower level second-order cone programming problem. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1111-1125. doi: 10.3934/jimo.2015.11.1111

[4]

Ye Tian, Shu-Cherng Fang, Zhibin Deng, Wenxun Xing. Computable representation of the cone of nonnegative quadratic forms over a general second-order cone and its application to completely positive programming. Journal of Industrial & Management Optimization, 2013, 9 (3) : 703-721. doi: 10.3934/jimo.2013.9.703

[5]

Xinmin Yang. On second order symmetric duality in nondifferentiable multiobjective programming. Journal of Industrial & Management Optimization, 2009, 5 (4) : 697-703. doi: 10.3934/jimo.2009.5.697

[6]

Liwei Zhang, Jihong Zhang, Yule Zhang. Second-order optimality conditions for cone constrained multi-objective optimization. Journal of Industrial & Management Optimization, 2017, 13 (4) : 1-14. doi: 10.3934/jimo.2017089

[7]

Qilin Wang, Shengji Li, Kok Lay Teo. Continuity of second-order adjacent derivatives for weak perturbation maps in vector optimization. Numerical Algebra, Control & Optimization, 2011, 1 (3) : 417-433. doi: 10.3934/naco.2011.1.417

[8]

Shiyun Wang, Yong-Jin Liu, Yong Jiang. A majorized penalty approach to inverse linear second order cone programming problems. Journal of Industrial & Management Optimization, 2014, 10 (3) : 965-976. doi: 10.3934/jimo.2014.10.965

[9]

Ning Lu, Ying Liu. Application of support vector machine model in wind power prediction based on particle swarm optimization. Discrete & Continuous Dynamical Systems - S, 2015, 8 (6) : 1267-1276. doi: 10.3934/dcdss.2015.8.1267

[10]

Yubo Yuan. Canonical duality solution for alternating support vector machine. Journal of Industrial & Management Optimization, 2012, 8 (3) : 611-621. doi: 10.3934/jimo.2012.8.611

[11]

Yanhong Yuan, Hongwei Zhang, Liwei Zhang. A smoothing Newton method for generalized Nash equilibrium problems with second-order cone constraints. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 1-18. doi: 10.3934/naco.2012.2.1

[12]

Xi-De Zhu, Li-Ping Pang, Gui-Hua Lin. Two approaches for solving mathematical programs with second-order cone complementarity constraints. Journal of Industrial & Management Optimization, 2015, 11 (3) : 951-968. doi: 10.3934/jimo.2015.11.951

[13]

Lijun Zhang, Chaudry Masood Khalique. Classification and bifurcation of a class of second-order ODEs and its application to nonlinear PDEs. Discrete & Continuous Dynamical Systems - S, 2018, 11 (4) : 759-772. doi: 10.3934/dcdss.2018048

[14]

Pooja Louhan, S. K. Suneja. On fractional vector optimization over cones with support functions. Journal of Industrial & Management Optimization, 2017, 13 (2) : 549-572. doi: 10.3934/jimo.2016031

[15]

Jian Luo, Shu-Cherng Fang, Yanqin Bai, Zhibin Deng. Fuzzy quadratic surface support vector machine based on fisher discriminant analysis. Journal of Industrial & Management Optimization, 2016, 12 (1) : 357-373. doi: 10.3934/jimo.2016.12.357

[16]

José F. Cariñena, Javier de Lucas Araujo. Superposition rules and second-order Riccati equations. Journal of Geometric Mechanics, 2011, 3 (1) : 1-22. doi: 10.3934/jgm.2011.3.1

[17]

Eugenii Shustin, Emilia Fridman, Leonid Fridman. Oscillations in a second-order discontinuous system with delay. Discrete & Continuous Dynamical Systems - A, 2003, 9 (2) : 339-358. doi: 10.3934/dcds.2003.9.339

[18]

Xinmin Yang, Xiaoqi Yang, Kok Lay Teo. Higher-order symmetric duality in multiobjective programming with invexity. Journal of Industrial & Management Optimization, 2008, 4 (2) : 385-391. doi: 10.3934/jimo.2008.4.385

[19]

Xiaoling Guo, Zhibin Deng, Shu-Cherng Fang, Wenxun Xing. Quadratic optimization over one first-order cone. Journal of Industrial & Management Optimization, 2014, 10 (3) : 945-963. doi: 10.3934/jimo.2014.10.945

[20]

Qilin Wang, Xiao-Bing Li, Guolin Yu. Second-order weak composed epiderivatives and applications to optimality conditions. Journal of Industrial & Management Optimization, 2013, 9 (2) : 455-470. doi: 10.3934/jimo.2013.9.455

 Impact Factor: 

Metrics

  • PDF downloads (1)
  • HTML views (0)
  • Cited by (1)

[Back to Top]