Is diversification a sham? (financial advice)

[scientific601]

Is diversification a sham?

Yet this is the advice financial advisors are constantly giving us.
I can't stop thinking that it's all marketing hoax... giving people the false hope that there exists a real "strategy" and logic to anything.

To me it's equivalent to proclaiming that sticking to one brand of lottery is a poor method whereas purchasing multiple brands of lottery tickets increases your chances of winning significantly.

It just doesn't seem intuitive to me that splitting up a pot of money into multiple pots, each of which is now used to buy various 'diversified' lottery ticket brands, in any way increases your chance of winning. If it were the case then we'd be hearing abut the best ratio to use. Or, why not just ignore the 'poor' performing ones altogether.

Examples of multiple lottery brands are for example Powerball / Mega Millions / The Big Game or in Canada
Lotto 6/49, Lotto Max. If you only play one, would say those advisors, you're not being smart.

Can someone with a stronger statistical background explain how diversification is a valid technique for increasing ones average winning ratio? And why this technique is suspiciously absent as a tool in any other risky behavior. I said "curiously" because I have a strong suspicion that it's a bogus methodology.

It's like suggesting: "to reduce the risk of death in the sport of skydiving, you should also take up hang gliding. This will spread out the risk. as you're devoting less total time to just one thing."

This is not homework. Just a thought experiment... one that's been bothering me for years.

 

[andrewkirk]

Can someone with a stronger statistical background explain how diversification is a valid technique for increasing ones average winning ratio?

Diversification is not always about increasing the probability of winning. It can also be about reducing the size of potential losses. It does that by reducing variance.

Diversification only increases the probability of winning when that probability is already reasonably high. To be specific, when the expected payoff of a strategy is more than zero, the probability of a positive payoff can be improved by combining a number of such strategies. It does that by reducing the variance. It does not however increase the expected payoff.

This can be visualised by imagining a bell curve the height of which represents the probability of profit equal to the value on the x axis, and the peak of the bell curve - the expected profit - is to the right of the y axis, ie in positive territory. Say 40% of the area under the bell curve falls to the right of the y axis, ie representing losses. Then if we can narrow the spread of the bell curve without changing where its peak lies, less of the area will be to the left of the y axis. This is not getting something for nothing because, as well as reducing the probability of losses, it also reduces the probability of large gains. We have foregone potential large gains in order to reduce the likelihood of losses.

Diversification is the method by which we narrow that bell curve. It uses the simple fact that, if the variance of an investment in anyone of n stocks is s, then the variance of an investment divided equally between all n stocks is s/n.

Whether diversification can be used to 'increase one's winning ratio' depends on what game of chance you are talking about, and what you mean by a 'winning ratio'. In some situations and for some definitions of ratios, diversifying strategies can improve the ratio. In others they cannot.

 

[scientific601]

Thanks for that eloquent explanation. I'll have to digest that.

It appears that the counterexamples I gave are faulty because they exhibit an expected payoff that's less than zero. You're saying that only potentially positive payoff items may be skewed by narrowing its spread.

I'm still not entirely clear. If one forgoes potential large gains in order to reduce probability of losses, how is one further ahead in one or the other extreme of bell curve spreads?

Perhaps the answer is in how much one weighs a potential loss vs. an equivalent gain. If the loss, for example, is deemed much more 'important' than any equivalent gain. In other words, there is an additional, non-mathematical component - an emotional bias, that we're trying to accommodate. The raw numbers (gain vs. loss) can not be magically changed, but one's comfort level may be adjusted by twiddling the tone controls on the bell curve. Am I on the right track?

 

[lisab]

To me it's equivalent to proclaiming that sticking to one brand of lottery is a poor method whereas purchasing multiple brands of lottery tickets increases your chances of winning significantly.

So...you're equating investing with playing the lottery?

 

[andrewkirk]

Perhaps the answer is in how much one weighs a potential loss vs. an equivalent gain. If the loss, for example, is deemed much more 'important' than any equivalent gain. In other words, there is an additional, non-mathematical component - an emotional bias, that we're trying to accommodate. The raw numbers (gain vs. loss) can not be magically changed, but one's comfort level may be adjusted by twiddling the tone controls on the bell curve. Am I on the right track?

Yes.
The investor is only better off if they would regret a $50 loss more than they'd enjoy a $50 gain. Economists often utilise a mythical beast called a 'risk-neutral investor' who cares only about expected profits (in the formal statistical sense of expected value or mean) and not about the distribution of possible profits and losses. Option pricing theory is based around what a risk-neutral investor would do.

In practice, humans are always somewhat risk averse. The branch of economics that deals with this is called Ultility Theory. THe Utility function maps the wealth of an investor to their satisfaction. Utility curves are always upward sloping (more is better) but concave down ($1000 more is not ten times as good as $100 more). Given such a utility curve, an investor will be happy to remove the top 25% best possible investment returns in order to also remove the 25% worst possible investment returns.

 

[andrewkirk]

So...you're equating investing with playing the lottery?

The interesting thing (to me) about this comparison is that the process works in reverse for any game where the expected payoff is negative, which is the case with just about all forms of gambling.

Diversification increases the probability of receiving a payoff close to the mean. In investment, where the mean is positive, it increases the probability of a positive payoff. In gambling, where the mean is negative, diversification increases the probability of a negative payoff. It follows that, if one wishes to gamble, one is better off betting a lot of money on one number (switching the example to roulette) rather than smaller amounts on a bunch of numbers. But one will still expect to lose.

 

[Jeff Rosenbury]

Yes.
The investor is only better off if they would regret a $50 loss more than they'd enjoy a $50 gain. Economists often utilise a mythical beast called a 'risk-neutral investor' who cares only about expected profits (in the formal statistical sense of expected value or mean) and not about the distribution of possible profits and losses. Option pricing theory is based around what a risk-neutral investor would do.

In practice, humans are always somewhat risk averse. The branch of economics that deals with this is called Ultility Theory. THe Utility function maps the wealth of an investor to their satisfaction. Utility curves are always upward sloping (more is better) but concave down ($1000 more is not ten times as good as $100 more). Given such a utility curve, an investor will be happy to remove the top 25% best possible investment returns in order to also remove the 25% worst possible investment returns.

This is true for most investors. In large part this is because most people are investing for retirement. A modest retirement is much better than a 50/50 chance of being rich or starving.

But for those trying to build wealth, concentrating investments in an area where one has expertise is often a better solution. Concentration earns money. Diversification prevents loss.

 

[BWV]

The other issue - which ties to the positive expected value, is the economic arguments on what risks an investor is paid to bear. If you and I wager on a coin toss, neither one of us would offer better than even odds. However in financial markets, investors must be offered a return, i.e. a positive expectation, for purchasing equity or debt securities. The economic argument is that the only risks investors are compensated for assuming are those that are non-diversifiable. This means the only stock market risk, not individual company risk warrants a risk premium. If, for example, you buy a speculative biotech stock that is either worthless or worth 10x depending on whether its drug gets approved by the FDA, the fair pricing of that security involves a discount rate that only depends on how sensitive to market risk (beta in finance jargon) the stock is. Another example would be making risky loans - if you loan money at 10% to borrowers with a 5% default rate, you would be insane to only make a couple of loans.
You can earn the equity risk premium by purchasing a broadly diversified index fund, anything less than this just adds uncompensated risk to the returns.

 

[Hornbein]

Can someone with a stronger statistical background explain how diversification is a valid technique for increasing ones average winning ratio?
.

It isn't. The whole idea is to reduce variance, which in economics is called "risk."

The assumption is that values of the commodities over which one is diversifying are uncorrelated or, even better, negatively correlated.

 

[nsaspook]

The interesting thing (to me) about this comparison is that the process works in reverse for any game where the expected payoff is negative, which is the case with just about all forms of gambling.

Diversification increases the probability of receiving a payoff close to the mean. In investment, where the mean is positive, it increases the probability of a positive payoff. In gambling, where the mean is negative, diversification increases the probability of a negative payoff. It follows that, if one wishes to gamble, one is better off betting a lot of money on one number (switching the example to roulette) rather than smaller amounts on a bunch of numbers. But one will still expect to lose.

Losing slowly is usually a good 'investment' strategy while gambling. Lose fast and the house has little incentive to provide 'free' drinks, rooms and perks. Losing smaller amounts over a longer period of time means the house wants you to continue and will try to keep you there with those 'free' drinks, rooms and perks. One will still expect to lose but it's nice not being treated like a 'loser'.

 

[Nidum]

Bookies have a sophisticated system of laying off high risk bets placed with them by putting spread bets on other possible winners at other bookies .

They used to do it mainly intuitively but some of of the larger bookies now use computers .

 

[BWV]

Insurance companies do essentially the same thing

 

[scientific601]

Impressive amount of knowledge and expertise being demonstrated here! Thanks all so far! My inquiry has been better received and addressed than I had expected.

 

[Hornbein]

Impressive amount of knowledge and expertise being demonstrated here! Thanks all so far! My inquiry has been better received and addressed than I had expected.

The whole point of diversification is to avoid losing big.

 

[Czcibor]

I add one more thing - poorly done diversification can be indeed a sham. There is a non zero correlation between return from different investments. Let's say you invest only in one sector. Like you have in portfolio only some tech companies. Or a few banking companies and some bonds issued by banks. Or real estate fund and construction companies. And a crisis comes, (or a dot com bubble) then all such sector is in trouble.

On the good thing the correlation could also work in your favour - shares and gold. In case of some troubles the shares would plummet, but gold would go up.

 

[Jeff Rosenbury]

I add one more thing - poorly done diversification can be indeed a sham. There is a non zero correlation between return from different investments. Let's say you invest only in one sector. Like you have in portfolio only some tech companies. Or a few banking companies and some bonds issued by banks. Or real estate fund and construction companies. And a crisis comes, (or a dot com bubble) then all such sector is in trouble.

On the good thing the correlation could also work in your favour - shares and gold. In case of some troubles the shares would plummet, but gold would go up.

One decent measure of correlation is a stock's beta (β). Beta measures the correlation of the stock to the entire market. Stocks with a β of 1 move as the market moves. Negative β indicates stocks which move against market trends. Unfortunately there are few of these and they tend to trade at a premium.

A more sophisticated methodology is to correlate the posited portfolio against itself, but that takes some computing power.

 

[BWV]

The standard finance model is:

re_(portfolio)=α+β(re_market)+ε
re=return in excess of T-Bills or some other proxy for a risk-free rate of return
α = excess return (earned by stockpicking or trading skill, a zero sum game among investors)
β=cov(portfolio,market)/variance(market)
ε= idiosyncratic volatility

standard finance theory would state:
exp(re)>0
exp(α)=exp(ε)=0
so only β(market RE) is a source of expected return. Diversification reduces the variance of ε which is risk with no expected return

σ²_Portfolio=β²σ²_Market+σ²_ε
also, using geometric brownian motion to model stock returns (a reasonable assumption that keeps prices >0)
dp/dt=μdt+δdz
where μ is the drift or mean return and z is a normally distributed random variable
it can be shown using Ito's Lemma that

p_t=p₀Exp((μ-σ²/2)t+σz_t)

the -σ^2/2 corresponds to the difference between the arithmetic mean and geometric mean return (a year 1 return of +10% followed by a year 2 return of -10% is a 1% loss, not a zero return). This means that assuming uncompensated volatility by not diversifying diminishes the overall return (actually the expectation is the same, but higher volatility concentrates all the value in the right tail so the median goes to zero as σ→∞, holding r constant)

the standard deviations of individual stocks twice or more the overall market volatility, so the vol reduction benefits from diversification are substantial

 

[Tosh5457]

I add one more thing - poorly done diversification can be indeed a sham. There is a non zero correlation between return from different investments. Let's say you invest only in one sector. Like you have in portfolio only some tech companies. Or a few banking companies and some bonds issued by banks. Or real estate fund and construction companies. And a crisis comes, (or a dot com bubble) then all such sector is in trouble.

On the good thing the correlation could also work in your favour - shares and gold. In case of some troubles the shares would plummet, but gold would go up.

Agree. But even investing in stocks of different economic sectors is a poorly done diversification. Sure the correlation might be low, but that's a very poor measure to use when investing. When there is a bull market you'll be exposed to the different sectors rotation, but in the event of a serious market turn, they'll all go down, with few exceptions (and the long-term correlation won't change much).

 

[andrewkirk]

also, using geometric brownian motion to model stock returns (a reasonable assumption that keeps prices >0)
dp/dt=μdt+δdz
where μ is the drift or mean return and z is a normally distributed random variable

That equation is for arithmetic, not geometric Brownian Motion. The Geo BM equation is

$$dP=\mu P dt+ \sigma P dz$$

 

[andrewkirk]

One decent measure of correlation is a stock's beta (β).

Beta is more a measure of leverage than of correlation. Beta tells us nothing about how diversified a portfolio is - ie its level of unsystemic (aka idiosyncratic aka stock-specific) risk. It tells us how much systemic risk there is, and systemic risk is undiversifiable.

 

[Czcibor]

I did not only think about beta (this negative beta as such are generally not shares of companies per se, but more some investment funds with derivatives / short positions). I more thought about case where there is high, unindented correlation between assets:
-I mentioned tech companies before bubble burst - it looked like diversified portfolio, with promising new markets and uncorrelated risks related to market successes of different products.
-Creating a portfolio of assets (not necessary shares) spread all over sectors and countries (so seemingly perfect)... in oil rich countries. If the oil prices plummet then...

Interestingly Buffet tended to invest in barring companies with beta slightly lower than 1...

 

[FactChecker]

I once did a quick study based on historical data of the Dow and bond prices where I tried different allocation percentages, with periodic reallocation. Keeping all stocks was the best alternative. So I stayed in all stocks (US large cap index). But I got caught in the 1998 recession with no reserve to buy stocks at the bottom. Luckily, I kept my job and did not have to cash in at the bottom. It would have been a different story if I had lost my job or was retired. I am convinced now that even if all investment options are correlated, there will be some that are hurt less by a recession than others. So diversification will always allow some ability to rebalance and "buy low, sell (relatively) high". This assumes that all investments will eventually recover and that you can wait that long. It helps emotionally to know that there are some opportunities in a recession.

PS: I am talking about a balance of index investments (US large cap, US small cap, US bonds, Foreign stocks, Foreign bonds)

 

[FactChecker]

Two comments:
1) There is a big difference between diversification without rebalancing and diversification with periodic rebalancing. Rebalancing offers the oppertunity to "buy (relatively) low and sell (relatively) high". With more options, even if they are correlated, there will be a highest and a lowest compared with its target percentage.
2) A truly diversified portfolio should have a good mix of US large cap, US small cap, US bonds, Foreign stocks, Foreign bonds, and maybe more. The idea that investments in one sector can be "diversified" is wrong.

 

[Jeff Rosenbury]

I did not only think about beta (this negative beta as such are generally not shares of companies per se, but more some investment funds with derivatives / short positions). I more thought about case where there is high, unindented correlation between assets:
-I mentioned tech companies before bubble burst - it looked like diversified portfolio, with promising new markets and uncorrelated risks related to market successes of different products.
-Creating a portfolio of assets (not necessary shares) spread all over sectors and countries (so seemingly perfect)... in oil rich countries. If the oil prices plummet then...

Interestingly Buffet tended to invest in barring companies with beta slightly lower than 1...

Buffet's strategy is to build wealth, not protect it for retirement. He is not planning on selling stock -- ever. He buys significant shares of a company and hopes to influence board decisions. He looks for companies with good financials, a product, and an economic moat.

This is a vastly different strategy than most people saving for retirement.

 

[Jeff Rosenbury]

2) A truly diversified portfolio should have a good mix of US large cap, US small cap, US bonds, Foreign stocks, Foreign bonds, and maybe more. The idea that investments in one sector can be "diversified" is wrong.

It is possible-ish in some sectors. For example in buying automotive stocks one could buy car dealers matched by spare parts and mechanics stocks. In good times cars sell. In bad times people repair rather than buy new.

Still, that's more of a corporate investment strategy since there are few hedges like auto repair shops can afford to trade on the market.

Nor does that lead to full diversification since owning a car dealer and a repair shop in pre-Katrina New Orleans would still leave you open to obvious risks.

Finally remember that the ultimate value of all markets is zero. Make sure you are prepared for that with investments that actually pay money (often dividends). If it makes money it's an asset. If it doesn't, it's not -- accountants not withstanding.

 

[Czcibor]

Buffet's strategy is to build wealth, not protect it for retirement. He is not planning on selling stock -- ever. He buys significant shares of a company and hopes to influence board decisions. He looks for companies with good financials, a product, and an economic moat.

This is a vastly different strategy than most people saving for retirement.

Honestly speaking, except of shareholder activism I do not see in this case some ideas unsuitable for retirement saving, except maybe just before retirement. Long term horizon - yes. Stable companies from varied sectors - yes. Seems good enough also for retirement purpose.

 

[BWV]

Finally remember that the ultimate value of all markets is zero. Make sure you are prepared for that with investments that actually pay money (often dividends). If it makes money it's an asset. If it doesn't, it's not -- accountants not withstanding.

This is not correct. How can the ultimate value of all assets be zero? Also there is a theorem in finance that states a company's dividend policy does not effect its value:
http://pages.stern.nyu.edu/~adamodar/New_Home_Page/invfables/dividirrelevance.htm

 

[andrewkirk]

This is not correct. How can the ultimate value of all assets be zero?

Well one day the Earth will be engulfed by the Sun and become uninhabitable, and at that time the value of all stocks on Earthly stock exchanges will be zero. In practice, there will be some calamity that destroys the value of stockmarkets long before that, whether war, pandemic, runaway global warming, asteroid strike or Carrington event.

I think the point is that an investment has to pay out money in dividends or the like before some finite, knowable time, to be worth anything.

 

[BWV]

I think the point is that an investment has to pay out money in dividends or the like before some finite, knowable time, to be worth anything.

But that is not true. What about gold or art, for example? and what does 'finite and knowable' mean? Does that apply to say, the first round investors in Facebook?

 

[andrewkirk]

Even gold and art will have no monetary value when the asteroid hits.

A little less cryptically: all financial value of financial assets comes from discounted cash flow, either directly from the asset, or from using the ability to control that asset to benefit another financial asset. Those cash flows do not have to be in the next five years. But they must be planned to occur at some time.

The investors in Facebook were anticipating that at some stage, several years hence, the stock would start paying big dividends, or equivalent means of cash distribution, such as share buybacks.

My interest was pricked by this, so I looked up the dividend yields of major tech companies.

Newer tech giants (less than 20 years old) like Facebook, Amazon and Google pay no dividends yet. But more mature tech companies like Microsoft, Apple and Cisco pay steady, substantial dividends. The expectation of current investors in the former is that they will end up paying big divs like the latter.

 

[BWV]

Yes that is largely correct, but more generally investment is simply savings deployed to something that is expected to maintain or increase in value. We can safely ignore the eventual demise of earth, such is the power of compounding that the present value of a perpetual stream of dividends worth $100 today is reduced by less than a dollar if the company vanishes in 100 years

 

[Jeff Rosenbury]

Yes that is largely correct, but more generally investment is simply savings deployed to something that is expected to maintain or increase in value. We can safely ignore the eventual demise of earth, such is the power of compounding that the present value of a perpetual stream of dividends worth $100 today is reduced by less than a dollar if the company vanishes in 100 years

I may be wrong, but I seem to recall that the firm Data General never paid a dividend. It grew to be a tech giant in the 1980s. But it eventually went out of business, even giving up its domain name to Dollar General. The company grew, shrank, then disappeared without bringing much value to the owners/investors. This cycle did not take 100 years.

The management followed the Miller and Modigliani theory. How did that work out?

Well it worked out well for those who knew to sell at the top of the market. It worked out well for the management team. It didn't work so well for the common investor with a long term buy and hold strategy.

A stock produces two things. First, you get a nice annual report. Second, you get dividends. Unless a secondary market suddenly develops for old annual reports, the buy and hold strategy needs to rely on dividends.

Of course there are other reasons to buy stock. Influencing board decisions can help make money through synergy. Those who like to gamble can try day trading under the "bigger fool" theory of investment. Buying on margin can allow higher growth rates. (Mostly for the bank/broker if you are a common investor.) But for long term economic growth, there is no substitute for dividends.

So IMO, the long term value of a stock is equal to the present value of its expected dividend stream (remembering that this should grow as the company does) plus the recycle value of its annual reports.

 

[BWV]

Warren Buffett went 40 years without a dividend or buyback

 

[andrewkirk]

We can safely ignore the eventual demise of earth, such is the power of compounding that the present value of a perpetual stream of dividends worth $100 today is reduced by less than a dollar if the company vanishes in 100 years

That's the point I'm making. As long as dividends, or alternative cash distributions, are expected to be paid within the not-too-distant future (eg say the next 100 years), we can ignore the far distant future. Otherwise we can't.

 

[russ_watters]

Finally remember that the ultimate value of all markets is zero. Make sure you are prepared for that with investments that actually pay money (often dividends).

What does that even mean? I agree it sounds very wrong.