本文参考机器学习周志华
基础内容如下
由上述原理,给出利用不使用核函数和软间隔的SVM处理用LAD降至2维的iris数据集的MATLAB源程序
main.m
tic clear;clc %导入数据 load matlab.mat global D D.X1 = D_new(1:48,:); D.Y1 = ones(size(D.X1,1),1); D.X2 = D_new(51:end-2,:); D.Y2 = -ones(size(D.X2,1),1); % 使用Matlab自带SVM对LDA处理后的iris数据二分类 % svmModel = fitcsvm([D.X1;D.X2],[D.Y1;D.Y2],'kernelFunction','rbf'); % Y_pred = predict(svmModel,[D_new(49:50,:);D_new(end-1:end,:)]); %% 使用自己求解SVM对LDA处理后的iris数据二分类 D.Label = [D.Y1;D.Y2]; D.Data = [D.X1;D.X2]; figure(1) hold on scatter(D.X1(:,1),D.X1(:,2),'r'); scatter(D.X2(:,1),D.X2(:,2),'g'); ITER = 100; objf = +inf; for Times = 1:ITER x0 = rand(size(D.X1,1)+size(D.X2,1),1); obj = 0; for i = 1:size(D.Data,1) for j = 1:size(D.Data,1) obj = obj+1/2*(x0(i)*x0(j)*D.Label(i)*D.Label(j)*D.Data(i)*D.Data(j)'); end obj = obj-x0(i); end if obj<objf objf = obj; bestx0 = x0; end end [alpha,val] = fmincon(@obj_fun,bestx0,[],[],[D.Label'],0,zeros(size(D.X1,1)+size(D.X2,1),1),[],[],[]); % h = []; % for i = 1:size(D.Data,1) % for j = 1:size(D.Data,1) % h(i,j) = D.Data(i,:)*D.Data(j,:)'*D.Label(i)*D.Label(j); % end % end % f = -1*ones(size(D.Data,1),1); % [alpha,val] = quadprog(h,f,[],[],D.Label',0,zeros(size(D.Data,1),1),[]); alpha(alpha<1e-3) = 0; w = zeros(1,size(D.Data,2)); for i = 1:size(D.Label,1) w = w+alpha(i).*D.Label(i).*D.Data(i,:); end [row,~] = find(alpha~=0); b = D.Label(row(1))-w*D.Data(row(1),:)'; plot([0,-b/w(1)],[-b/w(2),0]) toc
Obj_fun
function f = obj_fun(x) global D Data = [D.X1;D.X2]; Label = [D.Y1;D.Y2]; sum = 0; for i = 1:size(Data,1) for j = 1:size(Data,1) sum = sum+1/2*(x(i)*x(j)*Label(i)*Label(j)*Data(i)*Data(j)'); end sum = sum-x(i); end f = sum;
其中注释段代码为使用二次规划求解,直接运行是先由100次蒙特卡洛模拟生成初始解,再将初始解代入非线性规划求得的,二者在运行时间上差了5倍,但为非二次规划问题提供思路。
值得注意的是,如果将两类的标签置为0和1则不能得到正确的结果,只有将标签置为1和-1时才能得到正确的SVM二分类结果
积累:是否可以不定义全局变量global D,而使用其他手段定义目标函数Obj_fun