Improved Portfolio Optimizer, and Initial Results

I’ve been working hard on making the tools better, in particular on the optimizer, to make sure it produces great results. Keep in mind that studies that use historical returns can only be relied on so much. And we should never be using historical results to tweak the portfolio, because it leads to overfitting. If you are not aware, overfitting is a data science term that refers to making the model ‘fit’ the data. Like fitting for a suit of clothes. The problem is, that market performance won’t be duplicated, so the model might promptly fail on new ‘real’ results.

So with that said, I’ve improved the optimizer so even with smaller portfolios it produces better results. The optimizer is about reducing risk, and when it results in better returns and reduced risk, that’s a nice result. (again with the caveat I made about past results)

A couple weeks ago I made a blog post about optimizing some popular model portfolios from a Forbes article. At the time I thought it would be interesting to try optimizing these portfolios using RIAengine smart portfolio optimizer, and seeing what backtests looked like.

Now, with the improvements to the optimizer, I wanted to post updated results.

Buffet 90/10 Portfolio

This portfolio optimization had resulted in better risk and drawdowns, but returns were worse. Here are the new results:

So like with the original tests I ran, I optimized for 2003 and 2004, and then ran a backtest from 2005 through 2010. You don’t want to optimize and backtest over the same period since you’ll be overfitting. So that’s why the sequence there.

The original tests had resulted in worse performance, but improved risk and drawdowns. And the new test, with the improved optimizer, still resulted in better risk and drawdowns, but performance was higher too. This of course would be a best of both worlds scenario. Kind of a surprising result with only two assets to optimize for, but interesting too.

Merriman Ultimate Buy and Hold Strategy

The next model, the Merriman Ultimate Buy and Hold Strategy invests pretty much all in stocks, some large, some small, some value, some blend. Some are US and some international. Last, a little real estate gets added. As I said before, you need different asset classes to properly diversify. But let’s see how it goes.

The last time I ran these tests, the optimizer produced a worse backtest than the original. The drawdowns were worse, risk about the same, and returns worse. This time, returns still less but at least they are positive. Risk still the same and drawdowns worse.

As I said before, with an almost 100% stock portfolio, and only real estate to diversify things, it’s hard to figure out an asset mix that will perform better than any other. Ideally you’d reduce the drawdowns primarily.

Just for kicks, I decided to try increasing the real estate by a lot and see if it improves the drawdowns. So I increased them to 30% of the overall portfolio.

As you can see, it’s not really an improvement. Drawdowns still over 60%. If drawdowns are 60%, you need returns of 150% to get back to even! So, probably need to avoid this portfolio. Sorry Merriman.

Once again, these are hypothetical results, based on historical returns. And future performance won’t ever be exactly like any past period.

Ivy League Endowments portfolio

Next up, the Ivy League Endowments. You are probably getting used to this, so let’s jump into the results:

So the result is that the improved optimizer resulted in better returns, while risk is still better than the original portfolio, and drawdowns as well. So a good result.

Generally speaking, things that improve returns will probably result in more risk, but not always. We’re shooting for both, but if we have to choose, lower risk is preferable.

As I’ve said before, the risk of the client losing their stomach in the market is worse than just drawdowns!

Coffeehouse Portfolio

The next portfolio was called Coffeehouse and is a good mix of assets. Let’s see what happens with the comparison:

The improved optimizer gave a much better result with this model and backtest. Returns are much better, though a little less than the original. But risk, sharpe, drawdowns, all much better. While risk, sharpe and drawdowns are not as good as the first time we optimized, overall the result is more balanced.

Bill Bernstein’s No Brainer Portfolio

This portfolio is just 4 assets, equally weighted. The optimizer pushed the bonds higher than stocks, and lowered the real estate allocation. Let’s see how it turned out.

As expected, with more bonds, returns should be less normally, while risk should also be less. But since the improved optimizer didn’t allocate so much to bonds as the first time we ran this, the effect was less. This means we have higher returns than before, with still improvement in risk measures. On to the last portfolio.

Harry Browne’s Permanent Portfolio

Like the Bernstein portfolio, this one allocates equally to 4 securities, but it includes gold rather than real estate. The results:

The results are similar to the Bernstein portfolio, and similar compared to the last test. Returns are improved over the last test, though still not as good as the original models. However, once again a nice balanced result. Still reducing the risk of the original portfolio, and less returns but not dramatically so. Overall the client would sleep better at night.

Takeaways And Notes

This time around, results are more balanced, so the improvements are looking promising. Once again, past performance won’t be repeated. With portfolio optimization, we’re looking for diversification that results in lower risk portfolios, and trying not to reduce returns. But we hate drawdowns, since they make it harder to recover. A portfolio that’s underwater for a long time results in an unhappy client!

Machine Learning In Investing

machine learning in investingMachine learning can be used in investing in a variety of ways. This article seeks to show a few of those and describe the basics.

Hedge funds have tried for years to use machine learning (or AI) in their funds. Moving on to today, all quantitative funds use some form of machine learning. Some hedge funds even build around machine learning specifically, spurning human intervention.

Risk Management

Risk usually means standard deviation and positive correlation. However machine learning can approach risk management in a different way. Traditional investment correlation studies the way stocks move together. Machine learning today can look at correlation of risky investments with alternative data points. For example a model can look at how risk of a certain industry’s securities might rise and fall with weather data points, such as rain or temperature.

Simple methods of machine learning can be used to reduce the prediction errors in mean variance portfolio optimization. This helps manage the risk that can be reduced via diversification.

Building Portfolios

When discussing machine learning in investing, it’s natural to think about building portfolios. Since machine learning can assist with portfolio optimization, it can also be used to select investments to make up ideal portfolios.

Price Prediction

Of all the ways to use machine learning in investing, this one has gotten the most attention. This is because everyone is looking for the holy grail, also known as ‘the algorithm that accurately says when to buy and sell a security’. Many quant platforms are built around this concept.

Macro Economic Predictions

In addition to predicting prices, machine learning is also used to predict the state of the economy. Portfolio optimization depends quite a lot on return estimates, and return estimates are only as good as predicted economic conditions. Traditionally, to figure out what the economy was going to do, you would look at oil, interest rates, or similar macro data points. Now, you can throw huge amounts of alternative data at a model designed to look for similarities in past economic periods to current conditions. These models reinforce the estimates more than can be done with simple machine learning. They should result in much better estimates.

A word about Data

In order to use more complicated machine learning models, you need a lot of data. How much, that depends. It’s safe to say that the starting point to do meaningful things with deep learning, is at least a couple terabytes.

Optimizing Some Popular Portfolios

In Forbes I found a post about some ‘expert’ model portfolios. I thought it would be interesting to try optimizing these portfolios using RIAengine smart portfolio optimizer, and seeing what backtests looked like. Here are the results.

Buffet 90/10 Portfolio

The first portfolio is simple, just 90% allocation to an S&P500 index fund (I chose SPY) and a 10% allocation to Bonds, I used SHY. Here is how it went:

Optimized Buffet Portfolio

What happened is the portfolio went from 90% SPY to 98% SHY. We ran a backtest on it using RIAengine backtester, and tested over 5 years including the crash of 2008. This resulted in a portfolio that returned a little less, but had dramatically less risk. It’s about what you’d expect going from mostly stocks to mostly bonds. Sharpe ratios are both negative since neither portfolio outran the risk free rate. While interesting, the optimizer is meant to handle more instruments than 2 in order to work properly.

Merriman Ultimate Buy and Hold Strategy

Let’s move on to the next portfolio in the article, the Merriman Ultimate Buy and Hold Strategy.This portfolio invests pretty much all in stocks, some large, some small, some value, some blend. Some are US and some international. Last, a little real estate gets added. Right off the bat, without some other asset class in there it will be hard to properly diversify. But let’s have a go.

Optimized Merriman Portfolio

If you want an optimal portfolio, you better have different asset classes to work with!

The interesting result is that sitting all in stocks you will have a hard time overcoming drawdowns. In this case, both portfolios had drawdowns of over 60%. To overcome this kind of drawdown your portfolio has to earn 250%! That’s why I like to talk about avoiding drawdowns, instead of maximizing returns. And sharpe ratios are negative because the portfolios didn’t earn more than the risk free rate of around 3%.

Good lesson here is get some more asset classes when optimizing a portfolio so you can take advantage of diversification.

Ivy League Endowments portfolio

Next up, the Ivy League Endowments portfolio. When optimizing a portfolio it’s good to have different asset classes to work with. This one has large cap stocks, bonds, real estate, and it should have some commodities. Since I couldn’t find commodity ETFs that have been around since 2003, I used an energy ETF, IYE. Here are the results:

Optimized Ivy League Endowments portfolio

The optimized portfolio ended up with significantly less risk, though it made half the gains. Drawdowns are lower by 38%. This will help the client hold on when things are going the wrong way. The risk of a 45% drawdown is more than just market risk. The real risk is the client can’t stomach it.

Coffeehouse Portfolio

This portfolio has a lot of equity, and add some bonds and real estate. The results:

Neither portfolio outran the risk free rate. The optimal portfolio had lower risk and drawdowns.

Bill Bernstein’s and Harry Browne’s Portfolios

The last 2 portfolios are similar in that they have 4 assets equally weighted. The results of optimizing these portfolios is a flight into lower risk, leading to much lower drawdowns and also returns.

Takeaways And Notes

All of these tests were run with no constraints or limits on asset weights. Quite often this leads to the lower risk assets getting the lion’s share of overall weight. So it’s usually best to set constraints to balance the results.

Another thing to note is that backtesting is not research. If you use the backtest to help you tweak parameters such as asset weights in optimizing a portfolio, it will almost certainly lead to misleading results. This is because you start to fit the backtest to the data. In this case I’m optimizing the portfolio from 2003 through 2004 (two years of returns is minimum) and running backtests from 2005 through 2010. If I keep tweaking the results so that the backtest performs well from 2005 through 2010, then I’m overfitting to the data.

So use the backtest to get baseline information, but don’t go too far with your tweaking!

RIAengine will be launching the smart portfolio optimizer soon to a small beta group before launching to the public. Sign up to be notified of the launch.

Investment Portfolio Risk, Reduce It Or Risk Losing Clients

Investment Portfolio RiskInvestment portfolio risk can be an uncomfortable topic.

It’s easy to find articles all over the internet that encourage people to sit tight when things are dropping. Warren Buffet’s famous instructions regarding his wife’s trust are to put 90% of it in the S&P500. And since Warren Buffet says to do it, a lot of people think we should all do the same. (side note, with all that money invested, or course Warren Buffet wants others to sit tight! He’s a rational investor, not a charitable organization!)

But is it working? According to Dalbar, the research firm, while the stock market in general has returned just under 10% a year historically, the average investor only earns over 5%! Why is that? The reason comes down to investment portfolio risk. They don’t enjoy the ride as much as the hoped-for destination. So when the portfolio keeps dropping, they begin to doubt they will ever arrive.

So they call you, the advisor. If you are like most financial advisors, you will coach them on hanging in there, after all, the portfolio is designed to help them meet their future goals. But what if they are just uncomfortable with that amount of risk? Then you might lose them with your message, or lose them as a client.

Perhaps a question to ask yourself is, are your model portfolios too risky?

So How Can Your Reduce Investment Portfolio Risk?

Whether you have model portfolios for each individual, or a set of 5 or 10 that you apply based on risk profiles, you can find ways to reduce portfolio risk.

First, when you built the portfolios, you may not have optimized them properly, leading to a lessening of the benefits of diversification. This leads to greater risk than necessary, which leads to uncomfortable clients.

Next,  your allocations might not effectively minimize the risk of the investments you’ve chosen. This is not the same as the first point: This isn’t about asset allocations, rather allocations to individual investments to meet those allocations.

And last, there may be other investments that will, by adding them to the portfolio, reduce risk while again not making a big impact on expected returns.