Departmental Papers (CIS)
Date of this Version
December 2002
Document Type
Conference Paper
Recommended Citation
Fei Sha, Lawrence K. Saul, and Daniel D. Lee, "Multiplicative updates for nonnegative quadratic programming in support vector machines", . December 2002.
Abstract
We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge monotonically to the solution of the maximum margin hyperplane. The updates optimize the traditionally proposed objective function for SVMs. They do not involve any heuristics such as choosing a learning rate or deciding which variables to update at each iteration. They can be used to adjust all the quadratic programming variables in parallel with a guarantee of improvement at each iteration. We analyze the asymptotic convergence of the updates and show that the coefficients of non-support vectors decay geometrically to zero at a rate that depends on their margins. In practice, the updates converge very rapidly to good classifiers.
Date Posted: 11 November 2004
This document has been peer reviewed.
Comments
Advances in Neural Information Processing Systems 15 (NIPS 2002), pages 1065-1072.
Publisher URL: http://books.nips.cc/nips15.html