Departmental Papers (CIS)

Date of this Version

December 2002

Document Type

Conference Paper


Advances in Neural Information Processing Systems 15 (NIPS 2002), pages 1065-1072.
Publisher URL:


We derive multiplicative updates for solving the nonnegative quadratic programming problem in support vector machines (SVMs). The updates have a simple closed form, and we prove that they converge monotonically to the solution of the maximum margin hyperplane. The updates optimize the traditionally proposed objective function for SVMs. They do not involve any heuristics such as choosing a learning rate or deciding which variables to update at each iteration. They can be used to adjust all the quadratic programming variables in parallel with a guarantee of improvement at each iteration. We analyze the asymptotic convergence of the updates and show that the coefficients of non-support vectors decay geometrically to zero at a rate that depends on their margins. In practice, the updates converge very rapidly to good classifiers.



Date Posted: 11 November 2004

This document has been peer reviewed.