FMincon Introduction Advanced Econometrics Professor:Florian Pelgrin Assistant:Edward Shyu 1 Outline Fminunc Fmincon Single Variable Multiple Variables Maximize a function Equality constraints Lower and upper bounds nonlinear constraints Output options Gradient Examples: constraints 2 Purpose: Minimize a function Given a function f(x), find value x that results in the smallest f(x). Example: f(x) = (x-2)^2 Define function to minimize [x fval] = fminunc(@myfunUnc,x0) Define objective function (for example: myfunUnc) Save in a file of the same name: myfunUnc. m Guess a value for x [x fval] = fminunc(@myfunUnc,x0,options) Try x0 = 0

Call fminunc: get x, fval [x fval] = fminunc(@myfunUnc,x0,options) Results: x = 2. 0000 fval = 7. 5417e-018 ? 0 Inequality Constraints Find the value of x that minimizes a function of x f(x) = cos(x) Given some inequality constraints on x 0 ? x ? 2? Minimum expected: x = ? = 3. 14 f(x) = 0 Fmincon inputs: function f(x) [x fVal] = fmincon(@myfun,x0,A,b) @myfun: gives the ‘handle’ of function ‘myfun’, which contains the function to minimize. Save function in m-file (myfun. m) Fmincon inputs: constraints [x fVal] = fmincon(@myfun,x0,A,b) Constraints described by A*x ? b For 0 ? x ? 2? , rewrite as: -1*x ? 1*x ? 2? In Matrix notation: ? ? 1? ?0 ? ? 1 ? ? x = ? 2? ? ? ? ? ? { { A b In Matlab: A = [-1; 1] b = [0 ; 2*pi] Fmincon inputs: starting value [x fVal] = fmincon(@myfun,x0,A,b) x0: guess a starting value Try to guess a starting value for x that is close to the correct value that minimizes f(x). Try starting value = 1 In Matlab: x0 = 1 Fmincon input/output Output: x = 3. 1416 fVal = -1 Fmincon output: check x = 3. 1416 = ? fVal = -1 Example 2: Multiple x values Minimize f(x) = -x1x2x3 given the constraint 0 ? x1 + 2×2 + 3×3 ? 100 Now there is more than one variable to find: (x1, x2 and x3)

Fmincon inputs: function f(x) [x, fVal] = fmincon(@myfun2,x0,A,b) The x-variables are in vector x Define f(x) = -x1*x2*x3 ? x1 ? x = ? x2 ? ? ? ? x3 ? ? ? Fmincon inputs: constraints [x, fVal] = fmincon(@myfun2,x0,A,b) Constraint: 0 ? x1 + 2×2 + 3×3 ? 100 Rewrite as -1×1 – 2×2 – 3×3 ? 0 x1 + 2×2 + 3×3 ? 100 In matrix notation: In Matlab: A = [-1 -2 -3; 123] b = [0; 100] ?? 1 ? 2 ? 3? ? 0 ? ?1 ? ? x = ? 100? 2 3 ? ?2? 144 44 ? 2 3 13 A b Fmincon inputs: starting values [x, fVal] = fmincon(@myfun2,x0,A,b) Guess starting values ?10? x 0 = ? 10? ? ? ?10? ? ? In Matlab: x0 = [10 ; 10 ; 10] Input and output

Output: x= 33. 3333 16. 6666 11. 1111 fVal = -6. 1728e+003 Maximize a function To maximize a function, minimize the negative of that function. max f(x) = min –f(x) Equality Constraints [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq) Define Aeq and beq such that Aeq * x = beq Example: Equality constraint: x1 + x2 + x3 = 10 In Matlab: Aeq = [1 1 1] Beq = 10 ? x1 ? [1424 ]? ? x2 ? = [10] 1 1 1 3 ? ? { Aeq ? x3 ? beq ? ? Equality Constraints [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq) If there are no inequality constraints, set A=[] b=[] Call function: [x fVal] = fmincon(@myfun,x0,[],[],Aeq,beq) Lower bound, upper bound x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub) Define lower and upper bounds for x values Example: -10? x1? 10 -20? x2? 20 -5 ? x3? 50 In Matlab: lb = [ -10; -20;-5] ub =[ 10; 20; 50] Call function: x =fmincon(@myfun,x0,[],[],[],[],lb,ub) Remember, if an input value is not needed, fill in with [ ] Nonlinear constraints [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub,@mycon) Define mycon function in file mycon. m function [c,ceq] = mycon(x) c = … % C(x) ? 0 ceq = … % ceq(x) = 0 Call function: x = fmincon(@myfun,x0,[],[],[],[],[],[],@mycon) Matlab will minimize @myfun given the constraint that c(x) ? and ceq(x) = 0 This gives you more flexibility in defining your constraints, in case you cannot write them linearly as A*x? b or Aeq*x ? beq Nonlinear constraints [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub,@mycon) Example: define function mycon function [c,ceq] = mycon(x) c = x(1)^2 + x(2)^2 – 1 % x(1)^2 + x(2)^2 ? 1 ceq = x(2)^2 + x(3)^2 -1 % x(2)^2 + x(3)^2 = 1 Call function: x = fmincon(@myfun,x0,[],[],[],[],[],[],@mycon) Nonlinear constraints [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub,@mycon) If you don’t want to use one of these constraints, set them to null: [ ] Example: in function mycon, in file mycon. : function [c,ceq] = mycon(x) % no constraints: c = [ ] % C(x) ? 0 ceq = [ ] % ceq(x) = 0 Nonlinear constraints You can define inequality and equality constraints in mycon function Example: in function mycon, in file mycon. m: function [c,ceq] = mycon(x) c = A*x – b % A*x ? b ceq = Aeq*x – beq % Aeq*x = beq In main program, call function [x fVal] = fmincon(@myfun,x0,[],[],[],[],[],[],@mycon) Order of input arguments Remember that the order of arguments is: 1. Linear inequality constraints (A,b) 2. Linear equality constraints (Aeq,beq) 3. Lower bounds,Upper bounds (lb,ub) 4.

Nonlinear constraints (@mycon) Input arguments not used are filled in with a place holder [ ] [x,fval]=fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub,@mycon,options) [x,fval]=fmincon(@myfun,x0,[],[],[],[],lb,ub,@mycon,options) [x,fval]=fmincon(@myfun,x0,[],[],[],[],[],[],@mycon,options) Options [x fVal] = fmincon(@myfun,x0,A,b,Aeq,beq,lb,ub,@mcon,options) Diagnostics: Display diagnostic information about the function to be minimized. ‘on’ or ‘off’ Display: ‘off’ displays no output; ‘iter’ displays output at each iteration; ‘final’ (default) displays just the final output.

MaxFunEvals: Maximum number of function evaluations allowed MaxIter: Maximum number of iterations allowed TolFun:Termination tolerance on the function value. TolCon:Termination tolerance on the constraint violation. TolX:Termination tolerance on x. Optimset: Setting options Use Optimset to define the variable “options” options = optimset(‘param1′,value1,’param2’,value2,… ) Example: in main program: options = optimset(… ‘Diagnostics’,’on’,… ‘display’,’final’,… ‘MaxFunEvals’,10000,… ‘MaxIter’,1000,… ‘TolFun’,1e-10,… ‘TolCon’,1e-10,… ‘TolX’,1e-10) [x fVal] = fmincon(@myfun,x0,[],[],[],[],[],[],@mycon,options)

Diagnostics: sample output Diagnostic Information Number of variables: 3 Functions Objective: Gradient: Hessian: Constraints Nonlinear constraints: myfun2 finite-differencing finite-differencing (or Quasi-Newton) do not exist Number of linear inequality constraints: 2 Number of linear equality constraints: 0 Number of lower bound constraints: 0 Number of upper bound constraints: 0 MaxFunEval, MaxIter Matlab output will notify you if you need to increase the maximum number of evaluations or iterations. Sample output: Maximum number of function evaluations exceeded; increase OPTIONS. MaxFunEvals. To fix this, use ptimset to increase MaxFunEvals options = optimset(‘MaxFunEvals’,100000) [x fVal] = fmincon(@myfun,x0,[],[],[],[],[],[],@mycon,options) Gradient You can choose constraints on the gradient of the function, and/or on the constraints. 1a) Define the gradient in objective function and/or 1b) Define the constraints on the gradient in constraint function 2) Set options to turn on gradient object (required) Gradient 1a objective function Take the derivative of f(x) with respect to x1 and x2 Define the gradient, gradf in myfun function [f,gradf]=myfun(x) f=x(1)^4-x(1)^2+x(2)^2-2*x(1)+x(2); gradf=[4*x(1)^3 – 2*x(1) – 2; … 2*x(2)+1]; ( x ) = x1 ? x1 + x2 ? 2 x1 + x2 4 2 2 ?f 3 ( x ) = 4 x1 ? 2 x1 ? 2 ? x1 ? f ( x ) = 2 x2 + 1 ? x2 ? 4 x13 ? 2 x1 ? 2? gradf = ? ? 2 x2 + 1 ? ? Gradient 1b constraint Define the constraints on x and on f’(x) Save function mycon in file mycon. m The first column of gradc is the gradient-vector of the first constraint The second column of gradc is the gradient vector of the second constraint. function [c,ceq,gradc,gradceq]=mycon3(x) c =[ x(1)^2+x(2)^2-1;… ? x12 + x2 2 ? 1? ?0? c=? ?? ? 1-x(1)^2+x(2)^2] 2 2? ?1 ? x1 + x2 ? ?0? ceq=[]; gradc=[ 2*x(1),-2*x(1);… ? 2 x1 ? 2 x1 ? 2*x(2),2*x(2)]; gradc = ? ?? gradceq=[]; ? x2 2 x2 ? ?0 0? ?0 0? ? ? Gradient 2 Set the gradient option to ‘on’ This is required for the gradients that you provide (either the objective function gradient and/or the constraint gradient) options = optimset(‘GradObj’,’on’,’GradConstr’,’on’) Results x= -1. 0000 -0. 0000 10. 0000 fVal = 2. 0000 Examples: constraints Minimize f(x,y)=x14-x12+x22-2×1+x2 linear inequalities (a) (b) (c) (d) (e) (f) –x1+x2? 0 —- linear equalities -x1+x2=0 —– lower bounds x1? 0 —— upper bounds x2? 0 x1? 1, x2? 10 —– nonlinear constraints –x12+x22? 1 x12+x22=1 x12+x22=1, x12-x22? 1 x12+x22? 1, x12-x22? 1 Example (a)

Define the objective function in an m file function f=myfun(x) f=x(1)^4-x(1)^2+x(2)^2-2*x(1)+x(2); Constraints: x1? 0, x2? 0 ? 0 ? ? x1 ? ??? ?? ?? ? ? x ? ? ? 0 ? ? 2 ? ? 2? ? ? { 13 lb ub lb = [0;-Inf] ub = [Inf;0] [x,fval]=fmincon(@myfun,x0,[],[],[],[],lb,ub) Example (b) Constraints: x1+x2=0 Aeq = [1 1] beq = 0 ? x1 ? [1 1]? ? ? = 0 ? x2 ? x1? 1, x2? 10 lb = [ ] ub = [1 ; 10] ? x1 ? ? 1 ? ? x ? ? ? 10? ? 2? ? ? [x,fval]=fmincon(@myfun,x0,[],[],Aeq,beq,lb,ub) Example (c) Nonlinear constraints: define a function mycon, saved in file mycon. m Constraint: x12+x22? 1 x1 + x2 ? 1 ? 0 2 2 function [c ceq] = mycon(x) c=x(1)^2+x(2)^2-1; ceq=[];

Example (d) Nonlinear constraints: define a function mycon, saved in file mycon. m Constraint: x12+x22=1 x1 + x2 ? 1 = 0 2 2 function [c ceq] = mycon(x) c=[ ]; ceq=x(1)^2+x(2)^2-1; Example (e) Nonlinear constraints: define a function mycon, saved in file mycon. m x12+x22=1, x12-x22? 1 1 ? x1 + x2 ? 0 2 2 x1 + x2 ? 1 = 0 2 2 function [c ceq] = mycon(x) c=1-x(1)^2+x(2)^2; ceq=x(1)^2+x(2)^2-1; Example (f) Nonlinear constraints: define a function mycon, saved in file mycon. m x12+x22? 1, x12-x22? 1 ? x12 + x2 2 ? 1? ?0? ?? ? ? 2 2? ?1 ? x1 + x2 ? ?0? function [c ceq] = mycon(x) c1=x(1)^2+x(2)^2-1; c2=1-x(1)^2+x(2)^2; c=[c1;c2]; ceq=[];