Financial Management Perspective

Financial Management Perspective

Dynamic Portfolio Management in Cryptocurrency Markets: A Deep Reinforcement Learning Approach with a Risk-Sensitive Adaptive Agent Framework

Document Type : Original Article

Authors
1 Faculty of financial & accounting
2 Faculty Member, Faculty of Financial Sciences, Kharazmi University, Tehran, Iran
Abstract
Purpose: This study aims to propose and evaluate a dynamic framework for portfolio optimization in cryptocurrency markets that ensures an effective balance between return maximization and risk control under conditions of high volatility. The primary objective is to develop a risk-sensitive adaptive agent that integrates multiple baseline deep reinforcement learning agents with a dynamic switching mechanism to adjust asset weight allocations over time. This adaptive structure enables conservative behavior during highly volatile market regimes and more aggressive positioning during trending periods. The empirical analysis is conducted on a selected set of five major cryptocurrencies—Bitcoin, Ethereum, Solana, Ripple, and Tether—to assess the effectiveness and practical applicability of the proposed framework in real-world market environments.

Method: Daily data spanning five years were collected from Yahoo Finance and, after data cleaning, were divided into training and testing sets using a 70%–30% split. The state representation comprised asset prices, trading volumes, and commonly used technical indicators. To capture long-term temporal dependencies, state encoding was implemented using a Transformer-based architecture. Two categories of methods were examined: (1) individual deep reinforcement learning agents—namely PPO, A2C, and DQN—each independently managing the portfolio; and (2) a risk-sensitive adaptive framework that dynamically switches among agents by monitoring short-term performance indicators such as rolling cumulative profit and loss and five-day moving volatility. The reward function incorporated risk-sensitive components, including a logarithmic utility of returns, a volatility penalty, and the Calmar ratio. Performance evaluation was conducted using cumulative return, annualized return, Sharpe ratio, Sortino ratio, Calmar ratio, and maximum drawdown.

Findings: On the out-of-sample test dataset, the proposed risk-sensitive adaptive framework consistently outperformed each individual deep reinforcement learning agent. During non-trending market periods, the proposed model achieved higher terminal portfolio value and superior risk-adjusted performance compared to alternative methods. For instance, it recorded a Sharpe ratio of approximately 1.13 and a Sortino ratio of about 1.81. In bullish market regimes, the framework delivered markedly stronger results, achieving a Sharpe ratio of approximately 1.73, a Calmar ratio of around 3.34, and higher annualized returns, such that the final portfolio value during the uptrend was nearly twice that of the best-performing standalone agent. The quantitative results indicate that dynamic switching among aggressive, balanced, and conservative agents enhances the exploitation of favorable market trends while mitigating losses during highly volatile periods. Although a temporary increase in maximum drawdown was observed in certain scenarios, this effect was effectively offset by a reversion mechanism that shifts the agent back toward conservative behavior.

Conclusion: Integrating a Transformer architecture for temporal feature extraction with a risk-sensitive adaptive agent framework provides an effective solution for dynamic portfolio management in cryptocurrency markets, particularly when the objective is to simultaneously enhance returns and control downside risk. The findings indicate that employing a dynamic switching mechanism among multiple deep reinforcement learning agents can significantly improve portfolio performance and stability when facing shifting market regimes. Furthermore, statistical significance analysis using the Wilcoxon Signed-Rank Test across ten independent random seeds confirmed that the proposed model significantly outperformed PPO, A2C, and DQN in all major performance metrics (p < 0.01), demonstrating both robustness and reproducibility of the results.
Keywords

 
Abdi, N., Moradzadehfard, M., Ahmadzadeh, H., & Khoddam, M. (2022). A hybrid model for portfolio optimization based on stock price forecasting with LSTM recurrent neural network using cardinality constraints and multi-criteria decision-making methods. Journal of Financial Management Perspective, 36, 119–143. (in Persian)
Ayatollahi, M.R., & Jafary, M.B. (2024). Deep reinforcement learning and transformers: A novel approach to intelligent trading in Tehran Stock Exchange, The Scientific-Specialized Quarterly Journal of New Technologies in Electrical Engineering and Computer, 4(2), 94–106. (in Persian)
Bahoo, S., Cucculelli, M., Goga, X., & Mondolo, J. (2024). Artificial intelligence in finance: A comprehensive review through bibliometric and content analysis. SN Business & Economics, 4, 23.
Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., & Mordatch, I. (2021). Decision transformer: Reinforcement learning via sequence modeling. Advances in Neural Information Processing Systems, 34, 15084–15097.
Chen, L., et al. (2022). PPO for cryptocurrency portfolio optimization. IEEE Transactions on Neural Networks and Learning Systems, 33(8), 4012–4025.
Cheng, L. C., & Sun, J. S. (2024). Multiagent-based deep reinforcement learning framework for multi-asset adaptive trading and portfolio management. Neurocomputing, 594, 127800.
Cornalba, F., Disselkamp, C., Scassola, D., & Helf, C. (2024). Multi-objective reward generalization: improving performance of Deep Reinforcement Learning for applications in single-asset trading. Neural Computing and Applications, 36, 619–637.
Dalili, S., Rezaei Piteh Novi, Y., Kabiri, M. T., Safari Graili, M., & Arabzadeh, M. (2024). Portfolio Optimization Using Deep Reinforcement Learning Based on Modern Portfolio Theory, Accounting, Finance and Computational Intelligence, 2(4), 276-293. (in Persian)
Deng, Y., Bao, F., Kong, Y., Ren, Z., & Dai, Q. (2016). Deep direct reinforcement learning for financial signal representation and trading. IEEE Transactions on Neural Networks and Learning Systems, 28(3), 653–664.
Du, Y., Tang, K., & Chen, K. (2023). A novel crude oil futures trading strategy based on ensemble deep reinforcement learning. Energy, 285, 128474.
Fabozzi, F. J., et al. (2022). Portfolio Theory and Risk Management. Wiley.
Gholami, N., & Shams Gharneh, N. (2024). Presenting an optimized CNN‑LSTM model for stock price forecasting in the Tehran Stock Exchange. Journal of Financial Management Perspective, 14(45), 123–147. (in Persian)
Jaffri, A., Shirvani, A., Jha, A., Rachev, S., & Fabozzi, F. (2025). Optimizing portfolios with Pakistan-exposed ETFs: Risk and performance insight. Journal of Risk and Financial Management, 18(3), 158. (in Persian)
Koratamaddi, P., Wadhwani, K., Gupta, M., & Sanjeevi, S. (2021). Market sentiment-aware deep reinforcement learning for stock portfolio allocation. Engineering Science and Technology, 24, 848–859.
Lin, K., et al. (2023). Risk-sensitive reinforcement learning for portfolio optimization. Finance Research Letters, 52, 103512.
Lin, Y. C., Chen, C. T., Sang, C. Y., & Huang, S. H. (2022). Multiagent-based deep reinforcement learning for risk-shifting portfolio management. Applied Soft Computing, 123, 108894.
Markowitz, H. (1952). Portfolio selection. Journal of Finance, 7(1), 77–91.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. MIT Press.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30).
Wang, X., & Liu, L. (2025). Risk-sensitive deep reinforcement learning for portfolio optimization. Journal of Risk and Financial Management, 15, 347.
Zare, M. H., Nilchi, M., & Fareed, D. (2020). Comparative evaluation of Markowitz approach with a new hybrid method to create an optimal portfolio using deep DNN learning method and gravitational search algorithm. Journal of Financial Management Perspective, 9(28), 165–188. (in Persian)
Zhang, X., et al. (2021). Deep reinforcement learning for regime-switching markets. Journal of Financial Econometrics, 19(4), 789–810.
Zhang, Y., Zohren, S., & Roberts, S. (2020). Deep reinforcement learning for trading strategies. Quantitative Finance, 20(9), 1459–1473.