Making SVM run faster in python

Using the code below for svm in python:

    from sklearn import datasets
    from sklearn.multiclass import OneVsRestClassifier
    from sklearn.svm import SVC
    iris = datasets.load_iris()
    X, y = iris.data, iris.target
    clf = OneVsRestClassifier(SVC(kernel='linear', probability=True, class_weight='auto'))
    clf.fit(X, y)
    proba = clf.predict_proba(X)

But it is taking a huge amount of time.

Actual Data Dimensions :

    train-set (1422392,29)
    test-set (233081,29)

How can I speed it up(parallel or some other way)? Please help. I have already tried PCA and downsampling.

I have 6 classes. Edit: Found http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.SGDClassifier.html but I wish for probability estimates and it seems not to so for svm.

Edit:

    from sklearn import datasets
    from sklearn.multiclass import OneVsRestClassifier
    from sklearn.svm import SVC,LinearSVC
    from sklearn.linear_model import SGDClassifier
    import joblib
    import numpy as np
    from sklearn import grid_search
    import multiprocessing
    import numpy as np
    import math

    def new_func(a):                              #converts array(x) elements to (1/(1 + e(-x)))
        a=1/(1 + math.exp(-a))
        return a

    if __name__ == '__main__':
        iris = datasets.load_iris()
        cores=multiprocessing.cpu_count()-2
        X, y = iris.data, iris.target                       #loading dataset

        C_range = 10.0 ** np.arange(-4, 4);                  #c value range 
        param_grid = dict(estimator__C=C_range.tolist())              

        svr = OneVsRestClassifier(LinearSVC(class_weight='auto'),n_jobs=cores) ################LinearSVC Code faster        
        #svr = OneVsRestClassifier(SVC(kernel='linear', probability=True,  ##################SVC code slow
        #   class_weight='auto'),n_jobs=cores)

        clf = grid_search.GridSearchCV(svr, param_grid,n_jobs=cores,verbose=2)  #grid search
        clf.fit(X, y)                                                   #training svm model                                     

        decisions=clf.decision_function(X)                             #outputs decision functions
        #prob=clf.predict_proba(X)                                     #only for SVC outputs probablilites
        print decisions[:5,:]
        vecfunc = np.vectorize(new_func)
        prob=vecfunc(decisions)                                        #converts deicision to (1/(1 + e(-x)))
        print prob[:5,:]

Edit 2: The answer by user3914041 yields very poor probability estimates.

If you want to stick with SVC as much as possible and train on the full dataset, you can use ensembles of SVCs that are trained on subsets of the data to reduce the number of records per classifier (which apparently has quadratic influence on complexity). Scikit supports that with the BaggingClassifier wrapper. That should give you similar (if not better) accuracy compared to a single classifier, with much less training time. The training of the individual classifiers can also be set to run in parallel using the n_jobs parameter.

Alternatively, I would also consider using a Random Forest classifier - it supports multi-class classification natively, it is fast and gives pretty good probability estimates when min_samples_leaf is set appropriately.

I did a quick tests on the iris dataset blown up 100 times with an ensemble of 10 SVCs, each one trained on 10% of the data. It is more than 10 times faster than a single classifier. These are the numbers I got on my laptop:

Single SVC: 45s

Ensemble SVC: 3s

Random Forest Classifier: 0.5s

See below the code that I used to produce the numbers:

    import time
    import numpy as np
    from sklearn.ensemble import BaggingClassifier, RandomForestClassifier
    from sklearn import datasets
    from sklearn.multiclass import OneVsRestClassifier
    from sklearn.svm import SVC

    iris = datasets.load_iris()
    X, y = iris.data, iris.target

    X = np.repeat(X, 100, axis=0)
    y = np.repeat(y, 100, axis=0)
    start = time.time()
    clf = OneVsRestClassifier(SVC(kernel='linear', probability=True, class_weight='auto'))
    clf.fit(X, y)
    end = time.time()
    print "Single SVC", end - start, clf.score(X,y)
    proba = clf.predict_proba(X)

    n_estimators = 10
    start = time.time()
    clf = OneVsRestClassifier(BaggingClassifier(SVC(kernel='linear', probability=True, class_weight='auto'), max_samples=1.0 / n_estimators, n_estimators=n_estimators))
    clf.fit(X, y)
    end = time.time()
    print "Bagging SVC", end - start, clf.score(X,y)
    proba = clf.predict_proba(X)

    start = time.time()
    clf = RandomForestClassifier(min_samples_leaf=20)
    clf.fit(X, y)
    end = time.time()
    print "Random Forest", end - start, clf.score(X,y)
    proba = clf.predict_proba(X)

If you want to make sure that each record is used only once for training in the BaggingClassifier, you can set the bootstrap parameter to False.

From: stackoverflow.com/q/31681373