added solution to 5.5 and 5.6

exercises/ex05/solutions/mdr.m

+% In this script one solves the MDR (missing data recovery) problem
+% A data vector f represented by a sparse (few nonzeros xs) expansion
+% in a frame (here the discrete cosine transform DCT) Phi. f experiences
+% a loss of information, its last n-m components are lost in b. The
+% system Phi(1:m,:)*xr = b that would need to be solved to recover the
+% coefficients xr is underdetermined and thus has infinitely many
+% solutions. Since it is known that xs is sparse, one tries to
+% find a sparse xr vector by solving the optimization problem
+%
+%     min ||x||_0  subject to   Ax = b, ||.||_0 denotes no. of nonzeros
+%
+% This is a combinatorial problem and NP hard. One replaces the ||.||_0
+% norm by the ||.||_1 norm which encourages sparsity when minimized.
+%
+%     (MDR)     min ||x||_1  subject to   Ax = b  (x free)
+%
+% This problem can be rewritten as standard form LP and solved by
+% Mehrotra's interior point method. Sparsity should be exploited
+% Use m = 400, 350, 300
+%
+n = 500;			% data size
+m = 400;			% available data size; may be changed
+Phi = dctmtx(n);		% get the frame
+xs = zeros(n,1);		% initialize x*
+k = 10;				% number of nonzeros in x*
+rand('state',1011);		% for reproducibility
+p = randperm(60);		% indices of nonzeros in x*
+randn('state',11);		% for reproducibility
+xs(p(1:k)) = randn(k,1);	% random values for x*
+f = Phi*xs;			% complete data vector v*
+b = f(1:m);			% available data b
+A = Phi(1:m,:);			% matrix A
+subplot(4,1,1);			% put all graphs in one figure
+plot(1:n,f);			% plot complete data
+subplot(4,1,2);
+plot(1:n,[b;zeros(n-m,1)]);	% plot available data
+%
+% fill in here; set up the LP and solve with Matlab's linprog
+%
+nn = 2*n;
+xx = 1e0*ones(nn,1);
+c = ones(nn,1);
+AA = [A -A];
+[xx,ff] = meh(AA,c,b,xx,m,nn);
+xr = xx(1:n) - xx(n+1:nn);
+%
+%
+%
+subplot(4,1,3);
+plot(1:n,Phi*xr);		% plot recovered data
+subplot(4,1,4);
+plot(1:n,Phi*xr-f);		% plot error

exercises/ex05/solutions/sol05.txt

+|------------------|
+| Solution of ex05 |
+|------------------|
+
+---
+5.1
+---
+
+CPLEX.
+
+conv1.mod: CPLEX complains that there are diagonal QP coefficients of the wrong sign..
+
+conv2.mod: QP Hessian is not positive semidefinite.
+This only works with QP, while here the constraints are quadratic.
+
+conv3.mod: linearized constraint.
+Locally optimal solution of indefinite QP (using QP barrier iterations), solution (0,-2.5).
+
+conv4.mod: optimal integer solution (using branch-and-bound) (0,-2.5)
+
+
+
+---
+5.2
+---
+
+Gurobi
+
+conv5.mod: Gurobi complains that quadratic objective is not positive definite.
+
+
+
+---
+5.3
+---
+
+Xpress
+
+conv6.mod: Xpress complains that quadratic objective is not conves.
+
+
+
+---
+5.4
+---
+
+SCIP
+
+conv7.mod: optimal solution found (0,-2.5).
+
+Original problem with (convex) quadratic constraints.
+conv8.mod: optimal solution found, (0,1).
+There is another solution in (0,-1) (the problem is non-convex).
+
+Problem with non-convex feasible set.
+conv9.mod: optimal solution found (0,1.73).
+There is another solution in (0,-1.73) (the problem is non-convex).
+
+
+
+---
+5.5
+---
+
+See svm.mod.
+
+
+
+---
+5.6
+---
+
+See mdr.m.
+
+You need to double the number of variables, since now for each x_i there are x_i^+ (that is equal to x_i if x_i>0 and equal to 0 otherwise), and x_i^- (that is equalt to 0 if x_i>0 and equal to -x_i otherwise).
+
+The objective is 
+c^T x = e^T (x^+ + x^-) = e^T x^+ + e^T x^- = [e; e]^T [x^+; x^-]
+where e = [1; ... ; 1]
+
+The equality constraint is
+b = A x = A (x^+ - x^-) = A x^+ - A x^- = [A -A] [x^+ ; x^-]
+
+And there are the additional inequality constraints on the sign of x^+ and x^-

exercises/ex05/solutions/svm.com

+#option solver gurobi;
+#option solver cplex;
+option solver mosek;
+solve;
+display w;
+

exercises/ex05/solutions/svm.mod

+reset;
+
+param n integer;
+param m integer;
+
+param x{1..n,1..m+1};
+
+var w{1..m};
+var z{1..n} >= 0;
+var gamma;
+
+minimize obj:
+	0.5*sum{i in 1..m} w[i]^2 + sum{i in 1..n} z[i];
+
+s.t. c1{i in 1..n}: x[i,m+1]*(sum{j in 1..m} w[j]*x[i,j] + gamma) >= 1 - z[i];
+
+data svm.dat;
+
+#option solver gurobi;
+option solver cplex;
+#option solver cbc;
+
+solve;
+
+display w;