Support vector machines (SVM) is a popular machine learning method
with variants for classification, regression and distribution
estimation that can learn a problem in a higher dimensional space
through the use of a kernel trick. Integrated with Orange is a popular
implementation by Chang and Lin, libsvm. Orange
currently embeds version 8.1 of the library, which supports:
- C support vector classification (C_SVC)
- NU support vector classification (Nu_SVC)
- ONE CLASS distribution estimation (OneClass)
- EPSILON support vector regression (Epsilon_SVR)
- NU support vector regression (Nu_SVR)
The support for the following kernel functions is provided by
libsvm library:
- linear: u'*v
- polynomial: (gamma*u'*v + coef0)^degree
- radial basis function: exp(-gamma*|u-v|^2)
- sigmoid: tanh(gamma*u'*v + coef0)
- custom kernel (any function that remotely resembles a distance measure between two examples that can be implemented in python)
See also LinearLearner for a fast linear SVM implementation.
SVMLearner
SVMLearner class constructs a SVMClassifier
Attributes
- svm_type
- Defines the type of SVM (can be SVMLearner.C_SVC (default), SVMLearner.Nu_SVC, SVMLearner.OneClass, SVMLearner.Epsilon_SVR, SVMLearner.Nu_SVR)
- kernel_type
- Defines the type of a kernel to use for learning (can be SVMLearner.RBF (default), SVMLearner.Linear, SVMLearner.Polynomial, SVMLearner.Sigmoid, SVMLearner.Custom)
- degree
- Kernel parameter (Polynomial) (default 3)
- gamma
- Kernel parameter (Polynomial/RBF/Sigmoid) (default 1.0/number_of_examples)
- coef0
- Kernel parameter (Polynomial/Sigmoid) (default 0)
- kernelFunc
- Function that will be called if
kernel_type
is SVMLearner.Custom. It must accept two orange.Example arguments and return a float.
- C
- C parameter for C_SVC, Epsilon_SVR, Nu_SVR
- nu
- Nu parameter for Nu_SVC, Nu_SVR and OneClass (default 0.5)
- p
- Epsilon in loss-function for Epsilon_SVR
- cache_size
- Cache memory size in MB (default 100)
- eps
- Tolerance of termination criterion (default 0.001)
- shrinking
- Determines whether to use shrinking heuristics (default True)
- probability
- Determines if a probability model should be build (default False)
SVMLearnerSparse
Same as above except that it learns from the examples mata attributes. Note that meta attributes dont need to be registerd with the dataset domain, or present in all the examples.
Use this if you are using large sparse datasets.
SVMClassifier
Classifier used for classification, regression or distribution estimation (OneClass). In the later case the return value of the __call__ function can be 1.0 (positive case) or -1.0(negative case).
For a multiclass classification problem with k classes there are k*(k-1)/2 1class vs. 1class internal binary classifiers being build. The multiclass classification is then performed by a majority vote.
Attributes
- examples
- Holds the examples used for training
- supportVectors
- Holds the support vectors. They are listed in the order of their classes (i.e. they are grouped by the order of classes as they apear in the domains
classVar.values
)
- nSV
- Number of support vectors for each class (the same order as above)
- rho
- Constants in decision functions in the order of 1v2, 1v3, ... 1vsN, 2vs3, 2vs4, ...
- coef
- Coefficients for support vectors in decision functions (coef[nClass-1][nSupportVectors]). If k is the total number of classes then, for each support vector there are k-1 coefficients y*alpha where alpha are dual solution of the following two class problems: 1 vs j, 2 vs j, ..., j-1 vs j, j vs j+1, j vs j+2, ..., j vs k; and y=1 in first j-1 coefficients, y=-1 in the remaining k-j coefficients
Methods
- getDecisionValues(example)
- Return the decision values of all nClass*(nClass-1)/2 internal binary classifiers in the order of 1v2, 1v3, ... 1vsN, 2vs3, 2vs4, ...
Examples
>>> import orange
>>> data=orange.ExampleTable("iris.tab")
>>> l=orange.SVMLearner()
>>> l.svm_type=orange.SVMLearner.Nu_SVC
>>> l.nu=0.3
>>> l.probability=True
>>> c=l(data)
>>> for e in data:
... print e[-1], c(e), c(e, c.GetProbabilities)
...
Iris-setosa Iris-setosa <0.971, 0.015, 0.014>
Iris-setosa Iris-setosa <0.964, 0.019, 0.016>
Iris-setosa Iris-setosa <0.968, 0.016, 0.016>
...
References
Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support
vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm