Parallel Support Vector Machines Is A Supervised Machine Learning Alogrithom Used For Classification

1158 Words May 7th, 2016 5 Pages
Parallel Support Vector Machine
Junfeng Wu Junming Chen May 6, 2016
1 INTRODUCTION
Support vector machines is a supervised machine learning alogrithom used for classification. The problem could be written : minimize 1 |w |2 2 yi((w,xi)+b)−1≥0 where w is a linear combination of the training data: n w = αi k(xi ) i=1 this could be further written in a dual form[5]: min 1αTQα−eTα α2
0≤αi ≤C, yTα=0,
∀i ≤n where Q is the kernel matrix.
This dual form is a quadratic programming problem with linear constraints. A solver to this problem like SMO or IPM reuquires a time complexity of O(n3) and space complexity of O(n2), which makes SVM hard to scale. In our final project we implemented two parallel quadratic programming solvers for SVM and implemented a matrix factorization algorithm to improve the performance. Later we evaluated our implementation.
1
i =1,...,l,
2 SMO SOLVER IN PARALLEL
SMO is one of the most common ways to solve quadratic programming problem. SMO is a iterative alogrithm. In each iteration, the alogorithm optimize one pair of Lagrange multipliers (α1,α2) that could best accelerate the convergence util the lagrange multipliers meet the convergence condition.
2.1 SELECTING AND UPDATING α PAIR
We select alpha1 and alpha2 that make the largest progress towards the global maximum value on each side of the hyper plane according to the heuristic function. The heuristic function is as follow: n f (i) = aj yj k(xj ,xi )− yi j=1 The selection of α pair is as…
Open Document