MATLAB Can't get tutorial to work with new data

AI Thread Summary
The user is experiencing an error when running a backtracking tutorial in MATLAB, specifically at line 28 of their code, which calls the `maxSharpeRatioFcn`. The error message indicates that the portfolio object `p` is either empty or unbounded, leading to NaN values when executing `estimateBounds(p)`. The user has compared their data table with the tutorial's but is unable to identify the issue. They suspect that the problem lies in the constraints set for the portfolio, as the tutorial successfully returns valid bounds. Assistance is sought to resolve this issue with the portfolio constraints in MATLAB.
member 428835
Hi PF!

I'm going through a backtracking tutorial here. That code runs well for me, and is below:
Matlab: 
%% LOAD DATA

% Read a table of daily adjusted close prices for 2006 DJIA stocks.
T = readtable('dowPortfolio.xlsx');

% For readability, use only 15 of the 30 DJI component stocks.
assetSymbols = ["AA","CAT","DIS","GM","HPQ","JNJ","MCD","MMM","MO","MRK","MSFT","PFE","PG","T","XOM"];

% Prune the table to hold only the dates and selected stocks.
timeColumn = "Dates";
T = T(:,[timeColumn assetSymbols]);

% Convert to the table to a timetable.
pricesTT = table2timetable(T,'RowTimes','Dates');

% View the structure of the prices timetable.
head(pricesTT)

% View the size of the asset price data set.
numSample = size(pricesTT.Variables, 1);
numAssets = size(pricesTT.Variables, 2);
table(numSample, numAssets)

% USE FIRST 40 DAYS
% This example uses the first 40 days of the data set (about 2 months) to initialize the strategies
% The backtest is then run over the remaining data (about 10 months).
warmupPeriod = 40;

% No current weights (100% cash position).
current_weights = zeros(1,numAssets);

% Warm-up partition of data set timetable.
warmupTT = pricesTT(1:warmupPeriod,:);

%%
% Compute the initial portfolio weights for each strategy.
equalWeight_initial     = equalWeightFcn(current_weights,warmupTT);
maxSharpeRatio_initial  = maxSharpeRatioFcn(current_weights,warmupTT);
inverseVariance_initial = inverseVarianceFcn(current_weights,warmupTT);
markowitz_initial       = markowitzFcn(current_weights,warmupTT);
robustOptim_initial     = robustOptimFcn(current_weights,warmupTT);

%% LOCAL FUNCTIONS

function new_weights = equalWeightFcn(current_weights, pricesTT)
% Equal-weighted portfolio allocation

nAssets = size(pricesTT, 2);
new_weights = ones(1,nAssets);
new_weights = new_weights / sum(new_weights);

end

function new_weights = maxSharpeRatioFcn(current_weights, pricesTT)
% Mean-variance portfolio allocation

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT);  % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT
% Max 25% into a single asset (including cash)
p = Portfolio('NumAssets',nAssets,...
    'LowerBound',0,'UpperBound',0.1,...
    'LowerBudget',1,'UpperBudget',1); % Portfolio IS DICTIONARY WITH PREDEFINED KEYS
p = estimateAssetMoments(p, assetReturns{:,:}); % estimates mean & covariance of assetReturns from data for a Portfolio object
new_weights = estimateMaxSharpeRatio(p); % outputs sharp ratio

end

function new_weights = inverseVarianceFcn(current_weights, pricesTT) 
% Inverse-variance portfolio allocation
% Risk averse by weighting prop to inverse variance, or inverse uncertainty

assetReturns = tick2ret(pricesTT); % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT
assetCov = cov(assetReturns{:,:}); % cov calculates covariance, measures amount two quantities move together or inversed
new_weights = 1 ./ diag(assetCov);
new_weights = new_weights / sum(new_weights);

end

function new_weights = robustOptimFcn(current_weights, pricesTT) 
% Robust portfolio allocation
% maximizing return and minimizing risk with fixed risk-aversion
% coefficient (lambda)

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT); % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT

Q = cov(table2array(assetReturns));
SIGMAx = diag(diag(Q)); % make matrix of zeros with diag the cov_ii

% Robust aversion coefficient
k = 1.1;

% Robust aversion coefficient
lambda = 0.05;

rPortfolio = mean(table2array(assetReturns))'; % mean of assetReturns

% Create the optimization problem
pRobust = optimproblem('Description','Robust Portfolio');

% Define the variables
% xRobust - x  allocation vector
xRobust = optimvar('x',nAssets,1,'Type','continuous','LowerBound',0.0,'UpperBound',0.1);
zRobust = optimvar('z','LowerBound',0);

% Define the budget constraint
pRobust.Constraints.budget = sum(xRobust) == 1;

% Define the robust constraint
pRobust.Constraints.robust = xRobust'*SIGMAx*xRobust - zRobust*zRobust <=0;
pRobust.Objective = -rPortfolio'*xRobust + k*zRobust + lambda*xRobust'*Q*xRobust;
x0.x = zeros(nAssets,1);
x0.z = 0;
opt = optimoptions('fmincon','Display','off');
[solRobust,~,~] = solve(pRobust,x0,'Options',opt);
new_weights = solRobust.x;

end

function new_weights = markowitzFcn(current_weights, pricesTT) 
% Robust portfolio allocation

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT);

Q = cov(table2array(assetReturns));

% Risk aversion coefficient
lambda = 0.05;

rPortfolio = mean(table2array(assetReturns))';

% Create the optimization problem
pMrkwtz = optimproblem('Description','Markowitz Mean Variance Portfolio ');

% Define the variables
% xRobust - x  allocation vector
xMrkwtz = optimvar('x',nAssets,1,'Type','continuous','LowerBound',0.0,'UpperBound',0.1);

% Define the budget constraint
pMrkwtz.Constraints.budget = sum(xMrkwtz) == 1;

% Define the Markowitz objective
pMrkwtz.Objective = -rPortfolio'*xMrkwtz + lambda*xMrkwtz'*Q*xMrkwtz;
x0.x = zeros(nAssets,1);

opt = optimoptions('quadprog','Display','off');
[solMrkwtz,~,~] = solve(pMrkwtz,x0,'Options',opt);
new_weights = solMrkwtz.x;

end

function [buy, sell] = variableTransactionCosts(deltaPositions)
I've shortened it substantially, where I only go until we get initial portfolio weights. Now I try to run this code with different data, so the code slightly changes to this:
Matlab: 
load('T.csv')

% Convert to the table to a timetable.
pricesTT = table2timetable(T,'RowTimes','Dates');

% View the structure of the prices timetable.
head(pricesTT)

% View the size of the asset price data set.
numSample = size(pricesTT.Variables, 1);
numAssets = size(pricesTT.Variables, 2);
table(numSample, numAssets)

% USE FIRST 40 DAYS
% This example uses the first 40 days of the data set (about 2 months) to initialize the strategies
% The backtest is then run over the remaining data (about 10 months).
warmupPeriod = 40;

% No current weights (100% cash position).
current_weights = zeros(1,numAssets);

% Warm-up partition of data set timetable.
warmupTT = pricesTT(1:warmupPeriod,:);

%%
% Compute the initial portfolio weights for each strategy.
equalWeight_initial     = equalWeightFcn(current_weights,warmupTT);
maxSharpeRatio_initial  = maxSharpeRatioFcn(current_weights,warmupTT);
inverseVariance_initial = inverseVarianceFcn(current_weights,warmupTT);
markowitz_initial       = markowitzFcn(current_weights,warmupTT);
robustOptim_initial     = robustOptimFcn(current_weights,warmupTT);

%% LOCAL FUNCTIONS

function new_weights = equalWeightFcn(current_weights, pricesTT)
% Equal-weighted portfolio allocation

nAssets = size(pricesTT, 2);
new_weights = ones(1,nAssets);
new_weights = new_weights / sum(new_weights);

end

function new_weights = maxSharpeRatioFcn(current_weights, pricesTT)
% Mean-variance portfolio allocation

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT);  % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT
% Max 25% into a single asset (including cash)
p = Portfolio('NumAssets',nAssets,...
    'LowerBound',0,'UpperBound',0.1,...
    'LowerBudget',1,'UpperBudget',1); % Portfolio IS DICTIONARY WITH PREDEFINED KEYS
p = estimateAssetMoments(p, assetReturns{:,:}); % estimates mean & covariance of assetReturns from data for a Portfolio object
new_weights = estimateMaxSharpeRatio(p); % outputs sharp ratio

end

function new_weights = inverseVarianceFcn(current_weights, pricesTT) 
% Inverse-variance portfolio allocation
% Risk averse by weighting prop to inverse variance, or inverse uncertainty

assetReturns = tick2ret(pricesTT); % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT
assetCov = cov(assetReturns{:,:}); % cov calculates covariance, measures amount two quantities move together or inversed
new_weights = 1 ./ diag(assetCov);
new_weights = new_weights / sum(new_weights);

end

function new_weights = robustOptimFcn(current_weights, pricesTT) 
% Robust portfolio allocation
% maximizing return and minimizing risk with fixed risk-aversion
% coefficient (lambda)

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT); % tick2ret GIVES PERCENT RETURN AS DECIMAL FROM ONE DAY TO THE NEXT

Q = cov(table2array(assetReturns));
SIGMAx = diag(diag(Q)); % make matrix of zeros with diag the cov_ii

% Robust aversion coefficient
k = 1.1;

% Robust aversion coefficient
lambda = 0.05;

rPortfolio = mean(table2array(assetReturns))'; % mean of assetReturns

% Create the optimization problem
pRobust = optimproblem('Description','Robust Portfolio');

% Define the variables
% xRobust - x  allocation vector
xRobust = optimvar('x',nAssets,1,'Type','continuous','LowerBound',0.0,'UpperBound',0.1);
zRobust = optimvar('z','LowerBound',0);

% Define the budget constraint
pRobust.Constraints.budget = sum(xRobust) == 1;

% Define the robust constraint
pRobust.Constraints.robust = xRobust'*SIGMAx*xRobust - zRobust*zRobust <=0;
pRobust.Objective = -rPortfolio'*xRobust + k*zRobust + lambda*xRobust'*Q*xRobust;
x0.x = zeros(nAssets,1);
x0.z = 0;
opt = optimoptions('fmincon','Display','off');
[solRobust,~,~] = solve(pRobust,x0,'Options',opt);
new_weights = solRobust.x;

end

function new_weights = markowitzFcn(current_weights, pricesTT) 
% Robust portfolio allocation

nAssets = size(pricesTT, 2);
assetReturns = tick2ret(pricesTT);

Q = cov(table2array(assetReturns));

% Risk aversion coefficient
lambda = 0.05;

rPortfolio = mean(table2array(assetReturns))';

% Create the optimization problem
pMrkwtz = optimproblem('Description','Markowitz Mean Variance Portfolio ');

% Define the variables
% xRobust - x  allocation vector
xMrkwtz = optimvar('x',nAssets,1,'Type','continuous','LowerBound',0.0,'UpperBound',0.1);

% Define the budget constraint
pMrkwtz.Constraints.budget = sum(xMrkwtz) == 1;

% Define the Markowitz objective
pMrkwtz.Objective = -rPortfolio'*xMrkwtz + lambda*xMrkwtz'*Q*xMrkwtz;
x0.x = zeros(nAssets,1);

opt = optimoptions('quadprog','Display','off');
[solMrkwtz,~,~] = solve(pMrkwtz,x0,'Options',opt);
new_weights = solMrkwtz.x;

end

function [buy, sell] = variableTransactionCosts(deltaPositions)
but my code is choking at line 28. I've read the error message, compared my table T to the tutorial's, but can't seem to figure out the issue. Any help is greatly appreciated.
 

Attachments

  • T.csv
    T.csv
    15.3 KB · Views: 194
Physics news on Phys.org
joshmccraney said:
I've read the error message,
Which is ... ?
 
  • Like
Likes berkeman
Mark44 said:
Which is ... ?
Sorry, it says "
Error using mv_optim_transform
Portfolio set appears to be either empty or unbounded. Check constraints.

Error in Portfolio/estimateFrontierLimits>frontierLimitsContinuous (line 108)
[A, b, f0, f, H, g, d] = mv_optim_transform(obj);

Error in Portfolio/estimateFrontierLimits (line 75)
pwgt = frontierLimitsContinuous(obj, minsolution, maxsolution);

Error in Portfolio/estimateMaxSharpeRatio (line 71)
[pwgtAtMaxRet, pbuyAtMaxRet, psellAtMaxRet] = estimateFrontierLimits(obj, 'max');

Error in stocks>maxSharpeRatioFcn (line 180)
new_weights = estimateMaxSharpeRatio(p); % outputs sharp ratio

Error in stocks (line 81)
maxSharpeRatio_initial = maxSharpeRatioFcn(current_weights,warmupTT);"

For me, the MATLAB file is called stocks.m and line 81 refers to line 28 in post 1 (my code was a little bulky since I was importing and modifying data, where I just saved it for ease of reproducability above).

Line 28 calls the function maxSharpeRatioFcn, which is defined on line 44. Within that function call, the error is line 54, the estimateMaxSharpeRatio(p) function. When I googled the error it appears my portfolio p is not bounded, as when I execute estimateBounds(p) I get NAN for all seven values, where the tutorial gets 0. Any idea?
 
Caveat: I don't have MatLab and have never worked directly with it, at least not in the past 30 years or so..
joshmccraney said:
Line 28 calls the function maxSharpeRatioFcn, which is defined on line 44. Within that function call, the error is line 54, the estimateMaxSharpeRatio(p) function. When I googled the error it appears my portfolio p is not bounded, as when I execute estimateBounds(p) I get NAN for all seven values, where the tutorial gets 0. Any idea?
I don't see estimateBounds(p) anywhere in your program, so can't comment on why it is producing NAN for whatever values it's working on.

Your program differs from the tutorial quite a lot at the start. The tutorial calls readtable() to read in an Excel spreadsheet, while your modified version calls load() to load a CSV file. The tutorial also has an array of DJI stock symbols.

If you're not using a debugger, you should start doing so. Here's a link to the mathworks documentation on how to view the value of a variable - https://www.mathworks.com/help/matlab/matlab_prog/examine-values.html.
 
Mark44 said:
Caveat: I don't have MatLab and have never worked directly with it, at least not in the past 30 years or so..

I don't see estimateBounds(p) anywhere in your program, so can't comment on why it is producing NAN for whatever values it's working on.

Your program differs from the tutorial quite a lot at the start. The tutorial calls readtable() to read in an Excel spreadsheet, while your modified version calls load() to load a CSV file. The tutorial also has an array of DJI stock symbols.

If you're not using a debugger, you should start doing so. Here's a link to the mathworks documentation on how to view the value of a variable - https://www.mathworks.com/help/matlab/matlab_prog/examine-values.html.
Yea, the beginning is different, but that's because the data set I'm using is different (the tutorial's is built in). Once loaded, don't the two tables T have identical form and structure?

You don't see estimateBounds(p) because this was where google led me in order to debug. However, after receiving the NANs outputs, no further suggestions were given, so I'm kind of stuck.

I've used the debugger, but it doesn't look like there's a bug anywhere.
 
joshmccraney said:
Once loaded, don't the two tables T have identical form and structure?
No idea. Where I'd start with debugging is seeing what the table looks like after it's been loaded.
 
Mark44 said:
No idea. Where I'd start with debugging is seeing what the table looks like after it's been loaded.
My thoughts exactly. Looks identical, so I'm kinda of lost.
 
joshmccraney said:
My thoughts exactly. Looks identical, so I'm kinda of lost.
Single step through the tutorial program and write down what it does to the variables. Do the same with your program. There has to be a difference between the two programs, otherwise your version wouldn't be throwing an error message.
 
Mark44 said:
Single step through the tutorial program and write down what it does to the variables. Do the same with your program. There has to be a difference between the two programs, otherwise your version wouldn't be throwing an error message.
Well, this is very annoying, but evidently estimateMaxSharpeRatio only works with 10 or more assets (documentation doesn't say anything about this). I know this because when I trim the demo number from 15 to 7,8,9 I get an error but 10+ seems to be fine...very annoying.
 

Hot Threads

Recent Insights

Back
Top