Seminar

Incremental Optimization in Machine Learning

Nicolas Couellan (Université Paul Sabatier)

January 6, 2014, 15:00–16:00

Toulouse

Room MF 323

Decision Mathematics Seminar

Abstract

After a brief review of support vector machines (SVM) classification, we will discuss the underlying optimization issues and available methods to perform training and propose new first order constrained approaches. The methods exploit the structure of the SVM training problem and combine ideas of incremental gradient technique, gradient acceleration and successive simple calculations of Lagrange multipliers. Both primal and dual formulations will be presented and compared numerically. We will also discuss comparisons with an unconstrained large scale learning algorithm based on stochastic sub-gradient to emphasize that the proposed methods remain competitive for large scale learning due to the very special structure of the training problem.