How to Ensure Your Portfolio Risk Analytics Are Working as Expected

In this post, our research team demonstrates a clever way to backtest forward-looking scenarios commonly used in portfolio risk analysis.

September 04, 2018

This is the second in a series of posts in which our research team leverages new investment risk analytics in Stylus Pro to demonstrate how historical and forward-looking stress tests can provide deeper insight into fund performance across various market regimes and hypothetical scenarios.

Scenario analysis is a form of risk estimation that sits apart from many other measures. Rather than reporting what has happened, scenarios permit us to explore possibilities that may have never occurred and assign a numerical value to a variety of “what if” propositions. Scenario analysis also allows us to estimate how a shock to a variable, be it a market index, risk factor or macro-economic variable, could impact a fund or portfolio.

The result of the analysis can be interpreted as sensitivity to the proposed scenario, also equal to the fund’s projected return under the chosen conditions. This projection is based on the fund’s most recent returns-based style analysis 1exposure estimate for each fund.

These types of analyses can greatly improve the typical manager screening process for both short-term (tactical) and long-term (strategic) portfolio allocations. It is important, however, to ensure the correct factor shocks have been selected and properly backtested before conducting a forward-looking analysis.

Because factor selection and backtesting is so critical to scenario analysis, we will use this post to discuss our approach to this step before we move on to discuss an example of how scenario analysis can improve the fund selection and monitoring process.

Before incorporating a new scenario statistic into the investment process, it’s essential to confirm that inputs are reasonable and that, in general, the scenario works. This can be achieved by applying a simple heuristic, which will test how scenarios could have impacted performance during recently observed moderate shocks. While extreme shocks are rare and may not have occurred within observable history for many funds, moderate shocks do occur with reasonable frequency. Furthermore, these moderate shocks can be observed to move portfolios, even if considered in isolation.

Below we examine the impact of four scenarios on a universe of 338 target-date funds (TDF) with a return history of at least five years in order to estimate sensitivity, or projected return. (If you’d like to read more of our research on target-date funds, you can find it here.)

To backtest each factor shock, we first apply the shock to the month prior to the one in which a recent historical shock actually occurred. We then take that prior month’s sensitivity estimate (again, its projected return) and plot it against the funds’ subsequent monthly returns (during the recent historical shock, or the observed date). If our scenario analysis methodology is sound, our hypothetical return should closely match the historically realized return.

The correlation between each scenario and their respective subsequent shocks indicate the strength of the relationship, the results of which, in our TDF example, are listed in the below table. It is encouraging to notice that all of the scenario estimates have a moderately strong to very strong relationship with the subsequent returns overall.

While these results cannot guarantee that current sensitivity analysis will always be predictive in nature, the strong relationship between expectation and reality in these cases supports the methodology and our selection of individual scenarios. The below chart expands on the relationship summarized by the correlation numbers in the above table.

It’s worth noting that while directional correlation is clear, some of the above cases exhibit a difference in magnitude between expectation and reality. The S&P 500 Index example represents a direct shock to a primary asset exposure for most funds, and so we see a nearly 1:1 relationship between the scenario forecast and historical return.

The relationship between credit spreads and assets is less straightforward. While spreads are negatively associated with corporate bond prices, directly shocking a bond index will have a bigger impact on performance estimates. And while widening credit spreads, which are negatively associated with equity returns, can also be an indicator of market stress, the relationship is even less direct. Similarly, the way in which increases in the strength of the dollar and spikes in Treasury yields propagate through assets must be considered when selecting and backtesting scenarios.

The degree to which projected returns vary across these four scenarios (which is driven by a combination of factor selection and fund asset mixes) underscores the importance of selecting and thoroughly testing the appropriate factor to shock in a forward-looking scenario analysis.

In the next post in our risk series, we will use the same group of 338 TDFs to illustrate how conducting an equity market shock scenario on a group of funds with the same generic risk rating can reveal widely divergent projected returns.


  • 1DISCLAIMER: MPI conducts performance-based analyses and, beyond any public information, does not claim to know or insinuate what the actual strategy, positions or holdings of the funds, portfolios or organizations discussed are, nor are we commenting on the quality or merits of the strategies. This analysis is purely returns-based and does not reflect insights into actual holdings. Deviations between our analysis and the actual holdings and/or management decisions made by funds and/or organizations are expected and inherent in any quantitative analysis. MPI makes no warranties or guarantees as to the accuracy of this statistical analysis, nor does it take any responsibility for investment decisions made by any parties based on this analysis.