SVM regression with libsvm

1 min read

SVM is mostly commonly used for binary classifications. But one branch of SVM, SVM regression or SVR, is able to fit a continuous function to data. This is particularly useful when the predicted variable is continuous. Here I tried some very simple cases using libsvm matlab package:

1. Feature 1D, use 1st half to train, 2nd half to test. The fitting is pretty good.

linear 1D
linear 1D

2. Still 1D, but apparently the data is nonlinear. So I use nonlinear SVR (radial basis). The fitting is good.

nonlinear 1D
nonlinear 1D

3. What if we have a lot of dimensions? Here I tried feature space with up to 100 dimensions and calculated the correlation between predicted values and the actual values. For linear SVR (blue), the number of dimension doesn’t affect the correlation much. (red: nonlinear, blue:linear, same data for both cases)

curse of dimension
curse of dimension

One property of SVR I like is that, when two features are similar (i.e. highly correlated), their weights are similar. This is in contrast with “winner take all” property of general linear model (GLM). This property is desired in brain imaging analysis: neighbor voxels have highly correlated signals and you want them to have similar weights.

About performance: If different features have different scales, then normalization of data will improve the speed of libsvm. Also, the cost parameter c also affects the speed. The larger c is, the slower libsvm is. For the simulated data I used, the parameters don’t affect the accuracy.

MatLab code: test_svr.m

The normalization function (copy and save it into normalize.m):

function data = normalize(d)
% scale before svm
% the data is normalized so that max is 1, and min is 0
data = (d -repmat(min(d,[],1),size(d,1),1))*spdiags(1./(max(d,[],1)-min(d,[],1))’,0,size(d,2),size(d,2));

libsvm: http://www.csie.ntu.edu.tw/~cjlin/libsvm/#matlab

options:

-s svm_type : set type of SVM (default 0)

	0 -- C-SVC

	1 -- nu-SVC

	2 -- one-class SVM

	3 -- epsilon-SVR

	4 -- nu-SVR

-t kernel_type : set type of kernel function (default 2)

	0 -- linear: u'*v

	1 -- polynomial: (gamma*u'*v + coef0)^degree

	2 -- radial basis function: exp(-gamma*|u-v|^2)

	3 -- sigmoid: tanh(gamma*u'*v + coef0)

-d degree : set degree in kernel function (default 3)

-g gamma : set gamma in kernel function (default 1/num_features)

-r coef0 : set coef0 in kernel function (default 0)

-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)

-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)

-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)

-m cachesize : set cache memory size in MB (default 100)

-e epsilon : set tolerance of termination criterion (default 0.001)

-h shrinking: whether to use the shrinking heuristics, 0 or 1 (default 1)

-b probability_estimates: whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)

-wi weight: set the parameter C of class i to weight*C, for C-SVC (default 1)

The k in the -g option means the number of attributes in the input data.

>


Don't want to miss new papers in your field? Check out Stork we developed:

nirs2img, create an image file from NIRS data

I was asked where to get nirs2img script. Here it is. The download link is at the bottom of this article. nirs2img is to...
Xu Cui
51 sec read

Phrase frequency analysis on fNIRS hyperscanning with Stork Big…

In the previous blog, we have used Stork’s big analysis to analyze the literature of fNIRS. At that time, Stork only analyzed the frequency...
Xu Cui
1 min read

139 Replies to “SVM regression with libsvm”

  1. Your MatLab code goes wrong:
    ??? Error using ==> normalize at 43
    A must be an ATOM.

    Error in ==> test_svr at 13
    x = normalize(x);

  2. Looks like the normalize function is missing. Try:

    function data = normalize(d)
    % scale before svm
    % the data is normalized so that max is 1, and min is 0
    data = (d -repmat(min(d,[],1),size(d,1),1))*spdiags(1./(max(d,[],1)-min(d,[],1))’,0,size(d,2),size(d,2));

  3. Would you please publish the data set, the liblinear SVR input parameters, and the output accuracy (MSE, R2) for the non-linear case. I would like to test this and am having some trouble getting the octave interface compiled and also understanding the SVR output

    Thanks

  4. Thank you very much for svr_test.m example with libsvm.
    Excuse me,
    a question about svr_test.m mfile.
    Is it true that after the below commands zz and y(N/2+1:end)should be similar approximately? If it is true, why are zz and y(N/2+1:end)different?
    tic;model = svmtrain(y(1:N/2),x(1:N/2,:),[‘-s 4 -t 2 -n ‘ num2str(ii/2) ‘ -c ‘ num2str(1)]);toc
    tic;zz=svmpredict(y(N/2+1:end),x(N/2+1:end,:),model);toc

  5. @Amin
    No. zz is predicted value, and yy is original value. If the model is good, zz and yy is hopefully similar; otherwise they can be different.

  6. Hi,

    I have been working on epsilon SVR trying to figure out the best model, and I think I did based already. My question is, is it possible to obtain the regression coefficients/magnitudes for SV used in my model from the svmtrain code? If not, what is the best way to do it? I am mostly interested in finding something (SV magnitures/coef) like the regression coefficients from Multiple Linear regression … Is this possible?

    Thanks in advance!

    regards,

  7. Are w and b in test_svr.m what you are looking for?

    w = model.SVs’ * model.sv_coef
    b = -model.rho

  8. Thanks for the reply! Actually I am looking for the coef from training set I have. Can w and b be obtained from the svmtrain code on the original training set we have? My ultimate goal is to obtain the regression coef and use them to predict a different data set (not used in the original set). I hope it is more clear now.

  9. hi
    I want to know how we can apply error bound to our SVR to check the robustnees of my identified system?in epsilon regression

  10. Hi Xu Cui..
    I need your help about SVM Non Linear Reggression
    could you please reply me in my email ?

    Thanks

  11. Hi Xu,

    Great job about your tutorial regarding libsvm ! However, i don’t know how you get the plot above ? When i run your script I get only the red curve of the last plot. Could you help me to handle it ?

    Keep up the hard work !

    Thanks

  12. Thank you for your answer. By the way, do you think that we can extrapolate the function that interpolates the cloud to a wider range ?

  13. Hello
    could you give some examples about precomputed kernel in SVR ?

    how do we generate Xi and Xj ?

  14. @mary
    With the same technique used in regular SVM? Of course what to be maximized is the correlation of predicted vs actual, instead of accuracy.

  15. Hey Xu
    I am newbee to this field….just want to ask if we want to use 3 RBF functions how to do that in this case

  16. Hi Xu,

    as everyone else here, im working with SVR. Currently im working on the normalization data step and im using the normalization code you posted here. Yet i cant figure how to perform normalization for training and test data since training and testing data could have different min and max values. Should i use the min and max values of my training data to normalize the test data?
    Thx in advance

  17. Dear Xu Cui,

    I am very much interested in SVR. I am a beginner to SVM to guide me. I have a data set of mxn (100×40) with missing entries. I want to predict those missing entries through your code. But, right now the code uses 1D. can you please guide me on how should I alter ur code? Also, I want to know how to install the libsvm. I have learnt that SVM requires train set and test set. With my matrix how do i create a train and test set? Please guide me, i believe you can help me as soon as possible. Regards,manoraaju

  18. @Mano
    Mano,

    After downloading libsvm (the matlab version) you simply add its path to MatLab’s path. Then you can use it.

    Extending from 1D to 2D is very easy. My code above actually contains feature space from 1D to 100D.

    The training data would be the data without missing values; the test data would be the data w/ missing values.

  19. Dear Xu Cui,

    Thank you, I found a library for SVM which doesnt require MEX function at all. It process all the SVM functions. However Mr.Xu, Can i know at which place in your coding I should change the single dimension to two dimension. For my case mxn(100×40). So can you please post me a code that shall fit my data? I have clearly understood the meaning of test and train. However, I have only one set of data with missing values. The data(mxn 100×40) is actually, 100 patients answering 40 questions. Its a survey data. In this case, how shall I do it?

    I was thinking, I have the data 100×40. Using knn function in Matlab, I will impute the missing values. Next, with the matrix that is imputed, I will create a matrix with missing values. In this way, I can tell that my test data should approximate to training data. Am I right to say that?

    In the case of missing values, what number should I represent as? Like “NAN” or something?

  20. Dear Xu Cai,

    Can i know your email so that I can send the data, if your email is confidential please test email to I really appreciate for the help you are providing me. It is really helping me. Thank You.

  21. Hi Xu CAi,

    I am using LibSVM in regression for training Discrete Wavelet transform coefficients for use in image compression.Let a subband be of size 64×64 (ie: in each column there is a vector of size 64×1). I am training each 64×1 vector with svmtrain and encoding the weights thereby obtained. When I use Libsvm I get a model structure which I may need to pass to decompression routine also. But in realty this should not happen since only the weights should be transmitted and I want to recreate my wavelet coefficients back from these weights. Is there any workaround? This may require lot of code change right? With quadprog (qp solver in matlab) there is no issue.However the computation time is really an issue.

  22. @Bijoy Viswanath
    Bijoy,

    I am not familiar with image compression. From what you described, I am not sure if you are able to reconstruct the original coefficients from svr weights. The number of weights for a 2D SVM is 2, which I doubt is enough to reconstruct the original data.

    But I may not understand your problem enough.

  23. Dear Xu Cui,
    I am new to SVR. I have Matlab R2009b version. I am unable to instal LIBSVM package on it. Please help.
    Thanks
    S ZAIDI

  24. dear mr Xu Cui
    I am a beginner in svr and i would like to use svr for 3 input vectors and 1 output vector.I would like to use LIBSVM but i could not use it… i have matlab 7.10.0 R2010a and i put the extracted libsvm in my matlab path and i tried to open make.m and from readme file i tried to change my compiler to Visual C++ but i have not got this choice in my matlab…i have Visual C++ in my laptop but the matlab dose not know it as its compiler…
    what can i do for using libsvm and svr???
    could you possibly help me???
    I am waiting to hearing from you.it would be a kind of you.
    best regards
    a.b

  25. Type the following command on the matlab prompt.It will find the compiler for you and give you option to select.
    Hope it will help you.
    mex -setup

  26. hi,
    please tell me how to compile the libsvm 3.11
    i used the folllowing procedure
    1) mex -setup
    2) i have selected compiler Microsoft Visual C++ 2005 SP1
    3) make
    after this command it gives error like this
    >> make
    LINK : fatal error LNK1181: cannot open input file ‘user32.lib’

    C:\PROGRA~1\MATLAB\R2010A\BIN\MEX.PL: Error: Link of ‘libsvmread.mexw32’ failed.
    please respond me as early as possible….

    1. Hello

      I am an Algerian doctoral student and I work on modeling (artificial intelligence).
      If you can give me the MATLAB (Code) program for support vector machine (SVM).
      Thank you

  27. Hello there,
    I have a question about normalizing… do we have to normalize every column of our date separately respecting to their own min&max data?
    for example I have 4 column of input data and 1 column of output data, do I have to normalize every column separately, or normalize the whole data with one min & max?
    —————————-
    Thanks.

  28. Hi,
    Another question,
    when I’m using svmtrain, I get this error: “Group must be Vector.”
    My input data is not vector. I have 4 column for input data.
    and another problem is that when I want to use a vector data for input in another job I get this error: “??? Error using ==> sprintf
    Function is not defined for sparse inputs.

    Error in ==> num2str at 129
    t = sprintf(f,x(i,:));

    Error in ==> grp2idx>uniquep at 85
    b = cellstr(strjust(num2str(b), ‘left’));

    Error in ==> grp2idx at 23
    [gn,i,g] = uniquep(s); % b=unique group names

    Error in ==> svmtrain at 128
    [g,groupString] = grp2idx(groupnames);

    Error in ==> libsvmTest at 19
    model = svmtrain(Xtest, Xtrain, options);”

    I’m looking forward to hearing from you.
    Thanks for your time,

    Regards

  29. @Hamed
    Good question. I really don’t know why people usually don’t normalize y. If you find out the reason please let me know.

  30. i want to know how to install libsvm in matlab. Please give the step by step procedures as i am beginner.

  31. Hi, I have tested your SVM package for regression,
    using matlab 7.10 (R2010a)first there is an error message about normalize, I have used deleted x = normalize(x) and replaced x= mapstd(x), however there is always error message about : Undefined function or method ‘svmtrain’ for input arguments of type ‘double’
    why. is possible to give an example to follow it, for regression input/output purpose, of course
    thank you

  32. Hi,
    I have added path into matlab, like :addpath(‘c:/matlab/libsvm’), but
    there is always error message
    rror in ==> test_svr at 26
    tic;model = svmtrain(y(1:N/2),x(1:N/2,:),[‘-s 4 -t 2 -n ‘ num2str(ii/2) ‘ -c ‘ num2str(1)]);toc
    why.

  33. i am new to the support vector regression and using SVR for the prediction of the sales…can you provide me the algorithm of SVR in libsvm….

  34. Hi,
    i want to use libsvm on windows. i have downloaded it from the website, extracted it. now what to do next i am not getting please could you explore it step by step. though it is simple question but i am new to this.

  35. Hi,
    in svm regression,when we want to prove formula,we have alfa+ and alfa- ,
    in libsvm where can i find value of these alfa?

  36. Your Libsvm tutorial is very useful,Thanks for the same.

    I am using matlab.

    I have 4 columns & 55 rows data (real numbers)how to the scaling for this data & how to convert the data to libsvm format & save in libsvm format,also can we give the testing.txt file in comma separated or tab seperated file.Please let me know your suggestion to do the following
    scaling
    Coverting to libsvm data format.how to normalise or scale the data to[-1,+1] in matlab.

    I am using RBF kernel the accuracy is very low,I have used matlab.

    Looking forward for your reply

    Regards
    syeda

  37. Hi
    I want to use your code to find estimate output if training set consists of (x,y) x being the input and y being the output and the test set consists of (x’,y’). Given x’ we have to estimate y’. Kindly help me.

  38. Hi,

    I have data as below:
    GENDER Nationality Grade Age (Days) Service (Days)
    F Dutch CC.02 10679 789
    F South African CC.03 9313 1263
    M Brazilian FD.06 17150 1443
    F Chinese CC.02 8190 152
    M Trinidadian CC.02 9196 722
    F Filipino CC.03 10418 2010
    F Filipino CC.03 9628 1082
    F French CC.04 10556 1950

  39. Hi,

    I have data as below: with Service as output vector, Total sample size being 4500
    GENDER Nationality Grade Age(days)Servicedays)
    F Dutch CC.02 10679 789
    F South African CC.03 9313 1263
    M Brazilian FD.06 17150 1443
    F Chinese CC.02 8190 152
    M Trinidadian CC.02 9196 722
    F Filipino CC.03 10418 2010
    F Filipino CC.03 9628 1082
    F French CC.04 10556 1950

    Being a new being to SVM, please help me in
    –preparing the data, as input vector is not in numerical format
    –scaling data
    –choosing C,gamma
    –predicting
    is the code used above can be used for solving?

  40. @Xu Cui
    Thank you Xu,
    Yeah, I the samples are almost 5000 & there are two classes which have categorical attributes, will do as you adviced & will get back to you incase of any issue.

  41. @Xu Cui
    Hi Xu,
    How to do the normalization between (-1,1) using the below expression:

    data = (d -repmat(min(d,[],1),size(d,1),1))*spdiags(1./(max(d,[],1)-min(d,[],1))’,0,size(d,2),size(d,2));

    Regards,
    Suresh G

  42. @Xu Cui

    —I did understand that but when using the above expression, Normalising is happening in the range of (0,1), but not in the range of (-1,1).
    Also could you explain the looping conditions which I’m unable to understand: for M=m ,for ii =1 for JJ=1.
    I observe that (for ii=1 & jj=1) does run only once & for M=m runs for 10 times , could you plz explain?

  43. @Suresh Gorakala
    I see. Normalization to (0 1) is fine. There is no difference.

    ii and jj was supposed to try different parameter values, but not any more (just one time). M is to test for different dimensions of data.

  44. @Xu Cui

    Correct me if I’m wrong, for normalizing the test data I need to use the min/max values of the training data in the normalize equation.
    i.e
    data_test = (d_test -repmat(min(d_train,[],1),size(d_test,1),1))*spdiags(1./(max(d_train,[],1)-min(d_train,[],1))’,0,size(d_test,2),size(d_test,2));

  45. @Xu Cui

    Thank you Xu for the replies,
    Do we need to denormalize the predicted values (zz lables) we get after executing the [zz,acc] = svmpredict(zeros(size(x_test,1),1),x_test,model)?

    After we generate a model using training set, how to validate if our results are valid before predicting the data using test data?

  46. @Xu Cui
    Bear with my naive querries Xu,

    Please let me know how to calculate the accuracy rate using SVM, I have done validation using Gradient descent but not with SVM.

    – correct me if wrong: ‘-v’ option is used in svmtrain method while selecting ‘c,g’ values, once ‘c,g’ are selected we can use the svmtrain method without ‘v’ option.

  47. @Suresh Gorakala
    oops,I thought this is SVM.

    For SVR, instead of using “accuracy”, you may use correlation between the predicted and actual values.

    If you want to do SVR, you always want to use -v option (3 or 4). You don’t need to specify for spmpredict

  48. Correction to my earlier statement: I have validated trainig data using Gradient Descent technique not while using SVM using LIBSVM.Currently I’m currently working on SVM. 🙂

    how to find the correlation between the predicted & actual values?

  49. Xu Cui :
    w and b are from the training model and not from the test data.

    hi
    how are u?
    my name is hossein
    i am a msc student in water resource management
    i need a help. if you impossible,
    i want to a matlab code same az MatLab code: test_svr.m
    with 2 input and 1 out put with SVM regression with libsvm
    Thank you in advanced for your consideration and I am looking forward to
    hearing from you soon. Yours sincerely, H. Orouji (mail: orojih@ut.ac.ir)

  50. Xu Cui :
    w and b are from the training model and not from the test data.

    hi
    how are u?
    my name is hossein
    i am a msc student in water resource management
    i need a help. if you impossible,
    i want to a matlab code same az MatLab code: test_svr.m
    with 2 input and 1 out put with SVM regression with libsvm
    Thank you in advanced for your consideration and I am looking forward to
    hearing from you soon. Yours sincerely, H. Orouji

  51. @hossein
    hi
    thanks a lot for reply
    how do i create a train and test set with 2 input and 1 output?
    i need a code (sample MatLab code)
    Thank you in advanced for your consideration and I am looking forward to
    hearing from you soon. Yours sincerely, H. Orouji

  52. Dear Xu,

    One quick question, how to accomodate missing entries in the input data?
    for ex: in my data if few Age & Nationalities entries are missing then how to prepare the data for training?

  53. hi
    i need a help. if you impossible, how do i create a train and test set with 2 input and 1 output?
    i want to a matlab code same as : ex.m MatLab code.
    with 2 input and 1 out put with epsilon regression

    Thank you in advanced for your consideration and I am looking forward to
    hearing from you soon. Yours sincerely, H. Orouji

    code: ex code
    N = 30;
    u = linspace(0,1,N)’;
    y = 1 ./ (0.1 + u) + 0.3*randn(N,1);
    svm_type = 3;
    kernel_type = 2;
    gamma = 10;
    cost = 10;
    epsilon = 0.2;

    options = [‘-s ‘, num2str(svm_type),…
    fprintf(‘Starting LIBSVM\n’);
    tic;
    model = svmtrain(y, u, options);
    fprintf(‘Optimization finished in %3.2f sec\n’,toc);
    yp = svmpredict(y, u, model);
    figure
    plot(u,y,’*’)
    hold
    plot(u,yp,’r’,’linewidth’,2)
    xlabel(‘u’)
    ylabel(‘y’)
    legend(‘process’,’model’)

  54. @Suresh Gorakala
    Suresh, I myself don’t have experience in handling missing data. I googled and found there is a package called Weka but I have no experience with this package. If you find a solution please let me know.

  55. @Xu Cui
    I have done in the following way,
    Averaging all the feature (Column) values and replacing the missing values with the resultant average value. This worked for some extent but searching for better ways.

  56. @hossein
    hossein, sorry for not replying you earlier. I think to have 2 inputs is fairly easy, simple grouping the two input into one matrix with 2 columns – unless there is something in your question I don’t understand.

  57. Hi! I’m working with precomputed kernel matrix. I guess that svmtrain works well but I’m not sure because the svmpredict results are wrong. Could you give me some help with my code?

    sigma = 0.8;
    rbfKernel = @(X,Y) exp(- sigma .* DistanceMatrix(X,Y).^2);
    k1train = rbfKernel(trainz,trainz);
    c = 1;
    distance1 = @(X,Y) 1 – DistanceMatrix(X,Y).^2 ./ (DistanceMatrix(X,Y).^2 + c);
    k2train = distance1(trainc, trainc);
    alpha = 0.6;
    k = alpha.*k1train + (1-alpha).*k2train;
    ktrain = [(1:ntrain)’, k];
    model1 = svmtrain(trainy’, ktrain, ‘-s 3 -t 4 -c 1 -p 0.01′);

    k1test = rbfKernel(testz,testz);
    k2test = distance1(testc, testc);
    ktest = (alpha)*k1test + (1-alpha)*k2test;
    kt = [(1:neval)’, ktest];
    yfit3 = svmpredict(testy’, kt, model1)

  58. I am working with libsvm for regression model. I would like to know how support vectors can be identified in the final model structure. I know determining the number of support vectors in regression problem. However, I dont not know which patterns from training set are SV. Thank you

  59. Dear Xu Cui,
    I am trying to use LibSVM for travel time prediction. I have a feature vector matrix of form (latitude_StartPoint, longitude1_StartPoint, latitude2_EndPoint,longitude_EndPoint, distancebt_points,…….) and training_label_vector as time_taken_bt_points. My doubt is do I need to normalize both input and output vector

  60. @chanukya
    As long as your features are in similar range, it should be fine not normalize. I think you can try both to see if there is any real difference.

  61. Thanks Xu Cui for getting back to me. I have one more. I have 50 more instances that I use to train. But every instance has different (latitude1_StartPoint, longitude1_StartPoint, latitude2_EndPoint,longitude_EndPoint) even though I they belong to same route due to floating gps points. So do I need to standardize all the starting and ending points to fixed points(Latitude and Longitude) before I train them or can I proceed with out standardizing to fixed points?

  62. @chanukya
    The best way to find out the answer is, I think, to try both ways and see if there is big difference. I suspect there won’t be real difference here.

  63. Thanks Xu Cui,
    Did you work on R. Gunn toolbox for regression in matlab. Before I started using LibSVM I tried using that which gave me more accurate results with the same input vector epsilon svr. kernel = ‘linear’;C = 1000;loss = ‘quadratic’;e = 0.001; LibSVM linear kernel[-s 3 -t 0 -n 1:100 -c 1:10].Can you please give me some idea if I am inputing some wrong parameters.

  64. Dear Xu Cui,
    I am trying to use epsilon svr with radial basis function. I am trying to use [-s 3 -t 2 -g x -c y -p x]. How can I find the best values of [g c p]. I tried fixing c values and running two loops for g and p to find best values of correlation coefficient. But I am not sure how this works. Even I tried fixing p value and running for c and g. Do LibSVM internally perform cross validation or do I need to run it manually? Can you help me to get some clear idea.

  65. hi
    am new to work with SVM regression.
    actually am doing project in image compression using wavelets along with support vector machines . i want to know how to give the coefficients of the image which is obtained from wavelet decomposition as an input to svm regression to get support vectors?
    can you please help me

  66. hello sir,
    i want to know whether LIBSVM can useful for multi dimension regression (both linear and non linear).
    the meaning of 1D regression is to find only “W” and “b”. and meaning of multi dimension regression is there will be more than one “w”. am i correct ?.
    vishal mishra

  67. @satish
    hi Aj
    I installed one of the C compiler and created MEX files. Initially it wasnt happening in my PC i dont know why..i did it on my friends PC and created it and just copied MEX files in my folder and it works..

  68. a href=”#comment-1641″>@Xu Cui
    thanks sir ………
    LIBSVM for matlab interface is written in C/C++ format. is there any tool box or other LIBSVM which is completely written in MATLAB ?.

  69. Hi,
    I would like to know how to plot in matlab the hyperplan found for the case of the e-svr with a polymonial (for example ) kernel with 1D feature and not only the red test points (like in your figure above?
    thanks

  70. What if if we have only one feature? I mean we have the y label, and want to predict the future days. Which values will we use for x? Is ıt possible to use time (for i.e. days)?

  71. I’m not sure how to use SVR in either univariate and multivariate time series.

    Say we have stock prices for N days. For training inputs, y are the stock prices for N days, but what will we use for x ?
    1.Time series? For i.e. in one step ahead prediction 1,2,3…Z for Z days?
    2.(for one step ahead) sifting one day of y values?

    To explain more:
    matlab> model = svmtrain(training_label_vector,
    training_instance_matrix [, ‘libsvm_options’]);

    For univariate: I use the stock prices for N days in training_label_vector as a column vector and want to predict say next 30 days. I wonder which data I have to use in training_instance_matrix?

    For multivariate: say I have 22 more features (prices of other goodies), I use other features as column vectors in training_instance_matrix. But I’m not sure if I’m using the correct approach.

  72. @hakan
    You can use earlier stock price as X. For example, Y is the stock price at day 2, 3, …, N, X can be the stock price at day 1, 2, 3, …, N-1

  73. Hi,
    I have been using LIBSVM for along time. A regression problem is my interest. My problem is very important and i couldn’t solve it. I have two inputs and one output for SVR. Firstly i normalize my train data (both inputs and target) between 0 to 1 or -1 to 1 individually, and train the SVR with this normalized data, After training i use a test set which i similarly normalize test set by using min/max values of training stage. Finally i get the predicted values between 0 – 1 or -1 – 1. But when i denormalize this predicted values to actual values by using max/min values of target in training satage i get a prediction set which can not reach the max and min values of actual data. For axample in test stage actual outputs change between 0 – 100 but the predicted values change between 10 – 90. I tried lots of ways but couldn’t able to salve this. Please help me,
    Thanks

  74. Hello Xu Cui,
    I have question regarding selection of input features. Could you please look into the link where I posted my question http://stackoverflow.com/questions/18242784/training-libsvm-with-multivariate-data-in-matlab
    Could you please tell me which features should I select for training and testing? Please look into comments for the second response, I have given more description of my problem.
    For a time series data how can I input the time. I have my data collected on 6 different days at times starting from 8:00 to 11:00 am.So how do I input this data? Initially I get the UNIX time-stamp which I converted into day, time and entered into 2 different columns. For example now I have variable 1 as day with values 12,13,14,15,16,17 and variable 2 with times 8.0,8.25,8.5,8.75,9.0,9.25,…..11.0,8.0,8.25,8.5,8.75,9.0,9.25,…..11.0,8.0,8.25,8.5,8.75,9.0,9.25,…..11.0,8.0,8.25,8.5,8.75,9.0,9.25,…..11.0,…………….
    I have the data collected with 15minutes of frequency, so the experiment has all the stuff repeated for 6 days.
    So now I would be glad if you can suggest me how I could train LIBSVM with this input time series.

  75. Hello Sir,
    I have developed svm_model for regression, now i want to test it for a given input’x’. after reading the readme file i got that i should use
    Function: double svm_predict(const struct svm_model *model,
    const struct svm_node *x);
    but i do not know how to use? please help so that i can use it in matlab.
    thanks and regards
    Vishal mishra

  76. hi my friend
    i need your honestly help, i dont know what does svmtrain and quadprog in matlab mean?
    i dont know how should i solve this question?
    Suppose we have 5 points in 1D feature space as follows:
    x1=1, x2=2, x3=4, x4=5, x5=6, with 1, 2, 6 as class 1 and 4, 5 as class 2
    Use a polynomial kernel of degree 2 as K(x,z) = (xz+1)2 to find
    To do that, follow these steps:
    1) Find αi (i=1, …, 5) by
    2) Find support vectors
    3) Find the discriminant function f(z)
    4) Find the bias b
    5) Plot the discriminate function, 5 points and the regions for the classes
    Note that y=1 if x in w1 and y=-1 if x in w2

  77. Sir,
    My proj deals with emotion recognition from speech. Given that I have 40 samples of 6 emotions each, I have extracted the feature of mfcc from each of the sample. However, because the length of each speech sample is different, the order of the extracted mfcc’s are also different. does svm deal with unbalanced sizes of data? also, in the svmtrain and svmclassify programs available in the biolearning toolbox of matlab, do we need to add any parameters?

  78. Hi,

    Can you please send me the MATLAB code for Nonlinear function approximation.(Some thin like above parabolic function)

  79. Sir,
    While running LIBSVM – Multiclassification in MAT lab i’m getting the following error

    ??? Error using ==> svmtrain at 172
    Group must be a vector.

    Error in ==> ovrtrain at 8
    models{i} = svmtrain(double(y == labelSet(i)), x, cmd);

    Error in ==> get_cv_ac at 9
    model = ovrtrain(y(train_ind),x(train_ind,:),param);

    Error in ==> automaticParameterSelection at 67
    cv = get_cv_ac(trainLabel, trainData, cmd, Ncv);

    Error in ==> sample at 88
    [bestc, bestg, bestcv] = automaticParameterSelection(evalLabel, evalData, Ncv_param, optionCV);

    To make it work, make sure that the LibSVM library is part of your Matlab’s Search Path. One option would be to use the Matlab filebrowser (Current Folder) to go the LibSVM folder and use the menu Add to Path -> Selected Folders and Subfolders. _____________ I have tried this solution also.

    But i couldnt rectify my problem. Kindly help me to sort out this problem.

    Regards,
    Angeline

  80. could you please send the matlab code for the followin raw data and show hw to scale it between [0,1] and[+1,-1]

    function data = normalize(d)
    % scale before svm
    % the data is normalized so that max is 1, and min is 0
    data = (d -repmat(min(d,[],1),size(d,1),1))*spdiags(1./(max(d,[],1)-min(d,[],1))’,0,size(d,2),size(d,2));

    please could u tell how to accomodate(load) this data and perform scaling
    and call svmtrain and classify

    please send the matlab code for the same

    4 columns(feartures) and 55 samples(columns)

    7 classess
    4 features

    for ex:
    1 10 20 30 40
    1 values
    1 -do-
    1
    1
    1
    1
    2
    2
    2
    2
    2
    2
    2
    3
    3
    3
    3
    3
    3
    3
    3
    4
    4
    4
    4
    4
    4
    4
    4
    5
    5
    5
    5
    5
    5
    5
    5
    6
    6
    6
    6
    6
    6
    6
    6
    7
    7
    7
    7
    7
    7
    7
    7

    looking forward for ur reply..

  81. Hello Xu Cui,
    I am working in EEG signal processing field. I need nonlinear SVM for classifying my EEG signals.Can you please share the SVMtrain and svmclassify codes in matlab.

  82. Hello Xu Cui,
    I have done feature extraction of EEG signal(for epilepsy localization)using ICA. And got topoplot(in MATLAB) of independent components.Now i need to generate a matrix from this topoplot to give as an input for classification of signal(using SVM).How we can gennerate the matrix?

  83. I need your help to develop svm model using regression, how do i create a train and test set with 4 input column and 1 output column?

  84. Hi,

    what happens if the normalization has values inside [-5,5] or [-1,1]. [0,1] is loosing information ?

  85. @Andre
    Scaling to [0,1] is fastest and does Not lose you information. It could be that the SVM predicts slightly different prediction values depending on how you do your scaling. Remember to scale prediction date in the same way also.

  86. Hi,

    I’d like to use this library for outputs with two values, for example:

    X = [[2,5,3,8,1,1,1],[2,3,3,8,0,0,1],[3,5,1,2,1,1,1],[2,5,4,8,5,1,1]]
    y = [(2,3),(5,8),(5,7),(7,0)]

    I had an error when i tried this about the length of the output.
    Thanks

  87. Respected Sir,
    I am new to SVM. My feature set is of 100×24 (random number between 10:100) and my label is 100×1 (random numbers between 10:100). But let us say it has 5 class and I have to use svm-scale to normalize the data. How I have to do it?
    I WANT INFORMATION ABOUT THE .range4 FILE and svmguide4 IN THE SVM-SCALE (below syntax is defined).
    How it is created?, IF I want to create it how should I create it.
    PLEASE EXPLAIN IT WITH SMALL EXAMPLE

    ../svm-scale -l 0 -s range4 svmguide4 > svmguide4.scale

    for example (JUST randomly I have given)
    X 1 -1
    1 9 20
    2 56 98


    Thanking you sir

  88. I am using libsvm for regression problem How to check acuracy cross validation of C, gamma and epsilon parameters? Is it cross validation accuracy or Mean Square Error?

  89. Respected Sir

    There is any option for methodological change in SVM. Can we update the Kernal of SVM using any clustering technique.THen kindly explain how it be
    I would like to apply SVM and one more clustering technique.

    Thank u sir

  90. I have some question Sir,
    How much the number of support vector generated by SVR in a good case? is that normal if i got 90 SV from 100 data ?

    And is there any rule of minimum training data to build SVR model ?

  91. Hello sir
    I am using libsvm for regression. the Input is processed signal EMG and the output is measured torque. I want to use the epsilon-SVR to estimate knee torque by EMG test signal, but the estimated output is zero!
    number of samples (EMG & Torqe) for train is 801*1
    number of samples (EMG & Torqe) for test is 310*1
    and all of them is normalised between 0,1 (by samples/max(samples))

    and My code is:
    model = svmtrain(Torqetrain,EMGtrain,’-s 3 -c 40 -t 2 -p 0.1 -d 5′);
    [PredictedY,MSE] = svmpredict(Torqetest,EMGtest,model);

    my model after train is:
    model.Parameters [3;2;5;1;0]
    model.nr_class 2
    model.totalSV 281
    model.rho -2.521
    model.Label [ ]
    model.sv_indices 281×1 double
    model.ProbA [ ]
    model.ProbB [ ]
    model.nSV [ ]
    model.sv_coef 281×1 double
    model.SVs 281×1 sparse double

    can you help me please?

  92. Normalizing data range to [0,1] may not be enough. Train data divided by their standard deviation (std) may lead to better results.
    If so, you have to divide test data by the constant value given by std previously to be consistent.

  93. Hi,
    I am new to SVR, and would like to try your program, test_svr.m, before starting with my own data set. I can’t locate your data file.

    I also clicked your tab: “Download link sent”. I didn’t get the email. Checked spam version as well. Nothing there either. Could you please send me the link to the e-mail address shown above?
    Thanks.

  94. Hi again.
    I proceeded with your program using the random number generator. Now it is having problem with corrcoef function. Do I need to download some other program?

  95. Dear Mr Cui, I am new in matlab and SVM coding. Please I want to know how I can obtain the frequency, Scale, Phase etc parameters from Matching Pursuit and Gammatonne Frequency Cepstral Coefficient algorithms in order to feed them into the SVM classifier to detect and recognize environmental sound. Thank you very much as I await your response

  96. Dear Cui

    Please how can I get the mean and variance of each my GFCC frame coefficient, so that I can feed it to my SVM classifier.

    Thank you

  97. Again, please I do I show that a particular frame is an environmental sound frame or not. that is what are the criteria to know that a frame is an environmental sound frame. Thank you again

  98. down vote
    favorite
    i’m going to use the SVM with the iris data when i calculate the kernel function ,i have an error i’can’t fixed .some help please

    model1 = svmtrain( LS’,SSapp’,’-s 1 -t 1 -g 0.25 -c 0 -n 0.5 -r 0.35 -d 1 -b 1′); Error in svmtrain (line 57) errstring = consist(net, ‘svm’, X, Y);

    Error in irissataset (line 23) model1 = svmtrain( LS’,SSapp’,’-s 1 -t 1 -g 0.25 -c 0 -n 0.5 -r 0.35 -d 1 -b 1′);

  99. How can I use the probability values instead of labels in training data using svm regression?

Leave a Reply

Your email address will not be published. Required fields are marked *