Machine-Learned Regime Prediction
May 2, 2025


The Quanted Round-up is a curated summary that covers relevant research on key topics in quantitative financial decision-making.
Highlights
This edition examines how regime prediction is shifting toward more data-driven, interpretable, and context-sensitive approaches. The research highlights a movement away from rigid state definitions toward probabilistic, dynamic, and behaviourally-aware frameworks that better capture the evolving complexity of market cycles.
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios
A. Sinem Uysal & John M. Mulvey
The authors present a machine learning approach to regime-based asset allocation. The framework consists of two primary components: (1) regime modeling and prediction and (2) identifying a regime-based strategy to enhance the performance of a risk parity portfolio. For the former, they apply supervised learning algorithms, including the random forest, based on a large macroeconomic database to estimate the probability of an upcoming recession or a stock market contraction. Out-of- sample tests show the reliability of these predictions, especially for recessions in the United States, over the period 1973 to 2020. The probability estimates are linked to a dynamic investment overlay strategy. The combined approach improves risk-adjusted returns by a substantial amount over nominal risk parity in two-asset and multi- asset test cases, even during rising interest rates in the late 1970s.
Frequency-dependent regime-switching in VAR models
Youngjin Hwang
This study presents a simple frequency-dependent regime- switching vector autoregression (VAR) model, where each regime and its associated parameters in the VAR are characterized by their distinct spectral properties. Empirical applications to several key macroeconomic variables reveal clear frequencydependent switching dynamics, with each regime exhibiting distinctive features regarding spectral properties, volatility, and impulse responses. We compare this model with a conventional regime-switching model (typically studied in the time domain) and highlight several key differences between the two approaches.
Regimes
Amara Mulliner, Campbell R. Harvey, Chao Xia, Ed Fang & Otto Van Hemert
We propose a new systematic method for detecting the current economic regime and show how to use this information for predicting returns. Rather than presupposing a set of possible regimes, we rely on economic state variables and determine for which historical dates the values of these variables were most similar. To establish our position in an asset today, we identify historically similar periods and measure subsequent performance of the asset. If the historical performance is positive, we initiate a long position; conversely, if it is negative, we initiate a short position. We illustrate the efficacy of our method on six common long-short equity factors over 1985-2024. Our results show that using this information our regime classification leads to significant outperformance. Interestingly, we also find important information in what we call anti-regimesperiods in the past that are the most dissimilar to today.
Tactical Asset Allocation with Macroeconomic Regime Detection
Daniel Cunha Oliveira, Dylan Sandfelder, André Fujita, Xiaowen Dong & Mihai Cucuringu
This paper extends the tactical asset allocation literature by incorporating regime modeling using techniques from machine learning. We propose a novel model that classifies current regimes, forecasts the distribution of future regimes, and integrates these forecasts with the historical performance of individual assets to optimize portfolio allocations. Utilizing a macroeconomic data set from the FRED-MD database, our approach employs a modified k-means algorithm to ensure consistent regime classification over time. We then leverage these regime predictions to estimate expected returns and volatilities, which are subsequently mapped into portfolio allocations using various sizing schemes. Our method outperforms traditional benchmarks such as equal-weight, buy-and-hold, and random regime models. Additionally, we are the first to apply a regime detection model from a large macroeconomic dataset to tactical asset allocation, demonstrating significant improvements in portfolio performance. Our work presents several key contributions, including a novel data-driven regime detection algorithm tailored for uncertainty in forecasted regimes and applying the FRED-MD data set for tactical asset allocation.
Explainable AI (XAI) Models Applied to Planning in Financial Markets
Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez & Steve Ohana
Regime changes planning in financial markets is well known to be hard to explain and interpret. Can an asset manager ex- plain clearly the intuition of his regime changes prediction on equity market ? To answer this question, we consider a gradi- ent boosting decision trees (GBDT) approach to plan regime changes on S&P 500 from a set of 150 technical, fundamen-tal and macroeconomic features. We report an improved ac- curacy of GBDT over other machine learning (ML) methods on the S&P 500 futures prices. We show that retaining fewer and carefully selected features provides improvements across all ML approaches. Shapley values have recently been intro- duced from game theory to the field of ML. This approach allows a robust identification of the most important variables planning stock market crises, and of a local explanation of the crisis probability at each date, through a consistent features attribution. We apply this methodology to analyse in detail the March 2020 financial meltdown, for which the model of-fered a timely out of sample prediction. This analysis unveils in particular the contrarian predictive role of the tech equity sector before and after the crash.
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach
Esra Alp Coşkun, Hakan Kahyaoglu & Chi Keung Marco Lau
Overconfidence behavior, one form of positive illusion, has drawn considerable attention throughout history because it is viewed as the main reason for many crises. Investors’ overconfidence, which can be observed as overtrading following positive returns, may lead to inefficiencies in stock markets. To the best of our knowledge, this is the first study to examine the presence of investor overconfidence by employing an artificial intelligence technique and a nonlinear approach to impulse responses to analyze the impact of different return regimes on the overconfidence attitude. We examine whether investors in an emerging stock market (Borsa Istanbul) exhibit overconfidence behavior using a feed-forward, neural network, nonlinear Granger causality test and nonlinear impulse-response functions based on local projections. These are the first applications in the relevant literature due to the novelty of these models in forecasting high-dimensional, multivariate time series. The results obtained from distinguishing between the different market regimes to analyze the responses of trading volume to return shocks contradict those in the literature, which is the key contribution of the study. The empirical findings imply that overconfidence behavior exhibits asymmetries in different return regimes and is persistent during the 20-day forecasting horizon. Overconfidence is more persistent in the low- than in the high-return regime. In the negative interest-rate period, a high-return regime induces overconfidence behavior, whereas in the positive interest- rate period, a low-return regime induces overconfidence behavior. Based on the empirical findings, investors should be aware that portfolio gains may result in losses depending on aggressive and excessive trading strategies, particularly in low-return regimes.
References
Explainable AI (XAI) Models Applied to Planning in Financial Markets. June 2021. Benhamou, E.; Ohana, J.; Saltiel, D.; Guez, B. and Ohana, S. Université Paris-Dauphine Research Paper No. 3862437, Available at SSRN: http://dx.doi.org/10.2139/ssrn.3862437
Frequency-dependent regime-switching in VAR models. January 2025. Hwang, Y. Macroeconomic Dynamics, 29(74). Available at Cambridge: https://doi.org/10.1017/S1365100524000786
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios. March 2021. Uysal, A. S. and Mulvey, J. M. The Journal of Financial Data Science Spring 2021, 3 ( 2) 87 - 108. Available at Portfolio Management Research: https://doi.org/10.3905/jfds.2021.1.057
Regimes. March 2025. Mulliner, A.; Harvey, C.R.; Xia, C.; Fang, E. and van Hemert, O. Available at SSRN: http://dx.doi.org/10.2139/ssrn.5164863
Tactical Asset Allocation with Macroeconomic Regime Detection. March 2025. Oliveira, C, D.; Sandfelder, D.; Fujita, A.; Dong, X. and Cucuringu, M. Available at SSRN: https://doi.org/10.48550/arXiv.2503.11499
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach. January 2023. Coskun, E.A.; Kahyaoglu, H. and Lau, C.K.M Financ Innov 9(30). Available at Springer: https://doi.org/10.1186/s40854-022-00446-2
The Quanted Round-up is a curated summary that covers relevant research on key topics in quantitative financial decision-making.
Highlights
This edition examines how regime prediction is shifting toward more data-driven, interpretable, and context-sensitive approaches. The research highlights a movement away from rigid state definitions toward probabilistic, dynamic, and behaviourally-aware frameworks that better capture the evolving complexity of market cycles.
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios
A. Sinem Uysal & John M. Mulvey
The authors present a machine learning approach to regime-based asset allocation. The framework consists of two primary components: (1) regime modeling and prediction and (2) identifying a regime-based strategy to enhance the performance of a risk parity portfolio. For the former, they apply supervised learning algorithms, including the random forest, based on a large macroeconomic database to estimate the probability of an upcoming recession or a stock market contraction. Out-of- sample tests show the reliability of these predictions, especially for recessions in the United States, over the period 1973 to 2020. The probability estimates are linked to a dynamic investment overlay strategy. The combined approach improves risk-adjusted returns by a substantial amount over nominal risk parity in two-asset and multi- asset test cases, even during rising interest rates in the late 1970s.
Frequency-dependent regime-switching in VAR models
Youngjin Hwang
This study presents a simple frequency-dependent regime- switching vector autoregression (VAR) model, where each regime and its associated parameters in the VAR are characterized by their distinct spectral properties. Empirical applications to several key macroeconomic variables reveal clear frequencydependent switching dynamics, with each regime exhibiting distinctive features regarding spectral properties, volatility, and impulse responses. We compare this model with a conventional regime-switching model (typically studied in the time domain) and highlight several key differences between the two approaches.
Regimes
Amara Mulliner, Campbell R. Harvey, Chao Xia, Ed Fang & Otto Van Hemert
We propose a new systematic method for detecting the current economic regime and show how to use this information for predicting returns. Rather than presupposing a set of possible regimes, we rely on economic state variables and determine for which historical dates the values of these variables were most similar. To establish our position in an asset today, we identify historically similar periods and measure subsequent performance of the asset. If the historical performance is positive, we initiate a long position; conversely, if it is negative, we initiate a short position. We illustrate the efficacy of our method on six common long-short equity factors over 1985-2024. Our results show that using this information our regime classification leads to significant outperformance. Interestingly, we also find important information in what we call anti-regimesperiods in the past that are the most dissimilar to today.
Tactical Asset Allocation with Macroeconomic Regime Detection
Daniel Cunha Oliveira, Dylan Sandfelder, André Fujita, Xiaowen Dong & Mihai Cucuringu
This paper extends the tactical asset allocation literature by incorporating regime modeling using techniques from machine learning. We propose a novel model that classifies current regimes, forecasts the distribution of future regimes, and integrates these forecasts with the historical performance of individual assets to optimize portfolio allocations. Utilizing a macroeconomic data set from the FRED-MD database, our approach employs a modified k-means algorithm to ensure consistent regime classification over time. We then leverage these regime predictions to estimate expected returns and volatilities, which are subsequently mapped into portfolio allocations using various sizing schemes. Our method outperforms traditional benchmarks such as equal-weight, buy-and-hold, and random regime models. Additionally, we are the first to apply a regime detection model from a large macroeconomic dataset to tactical asset allocation, demonstrating significant improvements in portfolio performance. Our work presents several key contributions, including a novel data-driven regime detection algorithm tailored for uncertainty in forecasted regimes and applying the FRED-MD data set for tactical asset allocation.
Explainable AI (XAI) Models Applied to Planning in Financial Markets
Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez & Steve Ohana
Regime changes planning in financial markets is well known to be hard to explain and interpret. Can an asset manager ex- plain clearly the intuition of his regime changes prediction on equity market ? To answer this question, we consider a gradi- ent boosting decision trees (GBDT) approach to plan regime changes on S&P 500 from a set of 150 technical, fundamen-tal and macroeconomic features. We report an improved ac- curacy of GBDT over other machine learning (ML) methods on the S&P 500 futures prices. We show that retaining fewer and carefully selected features provides improvements across all ML approaches. Shapley values have recently been intro- duced from game theory to the field of ML. This approach allows a robust identification of the most important variables planning stock market crises, and of a local explanation of the crisis probability at each date, through a consistent features attribution. We apply this methodology to analyse in detail the March 2020 financial meltdown, for which the model of-fered a timely out of sample prediction. This analysis unveils in particular the contrarian predictive role of the tech equity sector before and after the crash.
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach
Esra Alp Coşkun, Hakan Kahyaoglu & Chi Keung Marco Lau
Overconfidence behavior, one form of positive illusion, has drawn considerable attention throughout history because it is viewed as the main reason for many crises. Investors’ overconfidence, which can be observed as overtrading following positive returns, may lead to inefficiencies in stock markets. To the best of our knowledge, this is the first study to examine the presence of investor overconfidence by employing an artificial intelligence technique and a nonlinear approach to impulse responses to analyze the impact of different return regimes on the overconfidence attitude. We examine whether investors in an emerging stock market (Borsa Istanbul) exhibit overconfidence behavior using a feed-forward, neural network, nonlinear Granger causality test and nonlinear impulse-response functions based on local projections. These are the first applications in the relevant literature due to the novelty of these models in forecasting high-dimensional, multivariate time series. The results obtained from distinguishing between the different market regimes to analyze the responses of trading volume to return shocks contradict those in the literature, which is the key contribution of the study. The empirical findings imply that overconfidence behavior exhibits asymmetries in different return regimes and is persistent during the 20-day forecasting horizon. Overconfidence is more persistent in the low- than in the high-return regime. In the negative interest-rate period, a high-return regime induces overconfidence behavior, whereas in the positive interest- rate period, a low-return regime induces overconfidence behavior. Based on the empirical findings, investors should be aware that portfolio gains may result in losses depending on aggressive and excessive trading strategies, particularly in low-return regimes.
References
Explainable AI (XAI) Models Applied to Planning in Financial Markets. June 2021. Benhamou, E.; Ohana, J.; Saltiel, D.; Guez, B. and Ohana, S. Université Paris-Dauphine Research Paper No. 3862437, Available at SSRN: http://dx.doi.org/10.2139/ssrn.3862437
Frequency-dependent regime-switching in VAR models. January 2025. Hwang, Y. Macroeconomic Dynamics, 29(74). Available at Cambridge: https://doi.org/10.1017/S1365100524000786
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios. March 2021. Uysal, A. S. and Mulvey, J. M. The Journal of Financial Data Science Spring 2021, 3 ( 2) 87 - 108. Available at Portfolio Management Research: https://doi.org/10.3905/jfds.2021.1.057
Regimes. March 2025. Mulliner, A.; Harvey, C.R.; Xia, C.; Fang, E. and van Hemert, O. Available at SSRN: http://dx.doi.org/10.2139/ssrn.5164863
Tactical Asset Allocation with Macroeconomic Regime Detection. March 2025. Oliveira, C, D.; Sandfelder, D.; Fujita, A.; Dong, X. and Cucuringu, M. Available at SSRN: https://doi.org/10.48550/arXiv.2503.11499
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach. January 2023. Coskun, E.A.; Kahyaoglu, H. and Lau, C.K.M Financ Innov 9(30). Available at Springer: https://doi.org/10.1186/s40854-022-00446-2
The Quanted Round-up is a curated summary that covers relevant research on key topics in quantitative financial decision-making.
Highlights
This edition examines how regime prediction is shifting toward more data-driven, interpretable, and context-sensitive approaches. The research highlights a movement away from rigid state definitions toward probabilistic, dynamic, and behaviourally-aware frameworks that better capture the evolving complexity of market cycles.
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios
A. Sinem Uysal & John M. Mulvey
The authors present a machine learning approach to regime-based asset allocation. The framework consists of two primary components: (1) regime modeling and prediction and (2) identifying a regime-based strategy to enhance the performance of a risk parity portfolio. For the former, they apply supervised learning algorithms, including the random forest, based on a large macroeconomic database to estimate the probability of an upcoming recession or a stock market contraction. Out-of- sample tests show the reliability of these predictions, especially for recessions in the United States, over the period 1973 to 2020. The probability estimates are linked to a dynamic investment overlay strategy. The combined approach improves risk-adjusted returns by a substantial amount over nominal risk parity in two-asset and multi- asset test cases, even during rising interest rates in the late 1970s.
Frequency-dependent regime-switching in VAR models
Youngjin Hwang
This study presents a simple frequency-dependent regime- switching vector autoregression (VAR) model, where each regime and its associated parameters in the VAR are characterized by their distinct spectral properties. Empirical applications to several key macroeconomic variables reveal clear frequencydependent switching dynamics, with each regime exhibiting distinctive features regarding spectral properties, volatility, and impulse responses. We compare this model with a conventional regime-switching model (typically studied in the time domain) and highlight several key differences between the two approaches.
Regimes
Amara Mulliner, Campbell R. Harvey, Chao Xia, Ed Fang & Otto Van Hemert
We propose a new systematic method for detecting the current economic regime and show how to use this information for predicting returns. Rather than presupposing a set of possible regimes, we rely on economic state variables and determine for which historical dates the values of these variables were most similar. To establish our position in an asset today, we identify historically similar periods and measure subsequent performance of the asset. If the historical performance is positive, we initiate a long position; conversely, if it is negative, we initiate a short position. We illustrate the efficacy of our method on six common long-short equity factors over 1985-2024. Our results show that using this information our regime classification leads to significant outperformance. Interestingly, we also find important information in what we call anti-regimesperiods in the past that are the most dissimilar to today.
Tactical Asset Allocation with Macroeconomic Regime Detection
Daniel Cunha Oliveira, Dylan Sandfelder, André Fujita, Xiaowen Dong & Mihai Cucuringu
This paper extends the tactical asset allocation literature by incorporating regime modeling using techniques from machine learning. We propose a novel model that classifies current regimes, forecasts the distribution of future regimes, and integrates these forecasts with the historical performance of individual assets to optimize portfolio allocations. Utilizing a macroeconomic data set from the FRED-MD database, our approach employs a modified k-means algorithm to ensure consistent regime classification over time. We then leverage these regime predictions to estimate expected returns and volatilities, which are subsequently mapped into portfolio allocations using various sizing schemes. Our method outperforms traditional benchmarks such as equal-weight, buy-and-hold, and random regime models. Additionally, we are the first to apply a regime detection model from a large macroeconomic dataset to tactical asset allocation, demonstrating significant improvements in portfolio performance. Our work presents several key contributions, including a novel data-driven regime detection algorithm tailored for uncertainty in forecasted regimes and applying the FRED-MD data set for tactical asset allocation.
Explainable AI (XAI) Models Applied to Planning in Financial Markets
Eric Benhamou, Jean-Jacques Ohana, David Saltiel, Beatrice Guez & Steve Ohana
Regime changes planning in financial markets is well known to be hard to explain and interpret. Can an asset manager ex- plain clearly the intuition of his regime changes prediction on equity market ? To answer this question, we consider a gradi- ent boosting decision trees (GBDT) approach to plan regime changes on S&P 500 from a set of 150 technical, fundamen-tal and macroeconomic features. We report an improved ac- curacy of GBDT over other machine learning (ML) methods on the S&P 500 futures prices. We show that retaining fewer and carefully selected features provides improvements across all ML approaches. Shapley values have recently been intro- duced from game theory to the field of ML. This approach allows a robust identification of the most important variables planning stock market crises, and of a local explanation of the crisis probability at each date, through a consistent features attribution. We apply this methodology to analyse in detail the March 2020 financial meltdown, for which the model of-fered a timely out of sample prediction. This analysis unveils in particular the contrarian predictive role of the tech equity sector before and after the crash.
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach
Esra Alp Coşkun, Hakan Kahyaoglu & Chi Keung Marco Lau
Overconfidence behavior, one form of positive illusion, has drawn considerable attention throughout history because it is viewed as the main reason for many crises. Investors’ overconfidence, which can be observed as overtrading following positive returns, may lead to inefficiencies in stock markets. To the best of our knowledge, this is the first study to examine the presence of investor overconfidence by employing an artificial intelligence technique and a nonlinear approach to impulse responses to analyze the impact of different return regimes on the overconfidence attitude. We examine whether investors in an emerging stock market (Borsa Istanbul) exhibit overconfidence behavior using a feed-forward, neural network, nonlinear Granger causality test and nonlinear impulse-response functions based on local projections. These are the first applications in the relevant literature due to the novelty of these models in forecasting high-dimensional, multivariate time series. The results obtained from distinguishing between the different market regimes to analyze the responses of trading volume to return shocks contradict those in the literature, which is the key contribution of the study. The empirical findings imply that overconfidence behavior exhibits asymmetries in different return regimes and is persistent during the 20-day forecasting horizon. Overconfidence is more persistent in the low- than in the high-return regime. In the negative interest-rate period, a high-return regime induces overconfidence behavior, whereas in the positive interest- rate period, a low-return regime induces overconfidence behavior. Based on the empirical findings, investors should be aware that portfolio gains may result in losses depending on aggressive and excessive trading strategies, particularly in low-return regimes.
References
Explainable AI (XAI) Models Applied to Planning in Financial Markets. June 2021. Benhamou, E.; Ohana, J.; Saltiel, D.; Guez, B. and Ohana, S. Université Paris-Dauphine Research Paper No. 3862437, Available at SSRN: http://dx.doi.org/10.2139/ssrn.3862437
Frequency-dependent regime-switching in VAR models. January 2025. Hwang, Y. Macroeconomic Dynamics, 29(74). Available at Cambridge: https://doi.org/10.1017/S1365100524000786
A Machine Learning Approach in Regime-Switching Risk Parity Portfolios. March 2021. Uysal, A. S. and Mulvey, J. M. The Journal of Financial Data Science Spring 2021, 3 ( 2) 87 - 108. Available at Portfolio Management Research: https://doi.org/10.3905/jfds.2021.1.057
Regimes. March 2025. Mulliner, A.; Harvey, C.R.; Xia, C.; Fang, E. and van Hemert, O. Available at SSRN: http://dx.doi.org/10.2139/ssrn.5164863
Tactical Asset Allocation with Macroeconomic Regime Detection. March 2025. Oliveira, C, D.; Sandfelder, D.; Fujita, A.; Dong, X. and Cucuringu, M. Available at SSRN: https://doi.org/10.48550/arXiv.2503.11499
Which return regime induces overconfidence behavior? Artificial intelligence and a nonlinear approach. January 2023. Coskun, E.A.; Kahyaoglu, H. and Lau, C.K.M Financ Innov 9(30). Available at Springer: https://doi.org/10.1186/s40854-022-00446-2