An Intelligent Integrated Framework for Return Prediction, Asset Selection, and Portfolio Optimization Based on Ensemble Learning and the Aquila meta-heuristic optimization algorithm

Introduction:Predicting asset returns, managing tail risk, and constructing efficient portfolios are major challenges in financial markets, particularly in highly volatile and non-stationary environments such as the Iranian capital market. Most previous studies have focused on only one component return prediction, asset selection, or portfolio optimization and few have offered an integrated framework capable of performing these tasks simultaneously and intelligently. To address this gap, the present study proposes an innovative framework for intelligent stock portfolio optimization based on ensemble machine learning and the Aquila metaheuristic optimization algorithm, designed to improve return prediction accuracy and reduce tail risk using the Conditional Value-at-Risk (CVaR) criterion. Methods:Daily data from all listed and over-the-counter companies in Iran from April 2013 to October 2024 were used in this study. After data preprocessing and the removal of inconsistent or insufficiently covered symbols, 370 stocks were retained. In the first stage, an ensemble learning model comprising Random Forest, Adaptive Boosting, and Extreme Gradient Boosting was developed. The Aquila metaheuristic algorithm simultaneously performed three tasks: identifying influential features, tuning hyperparameters, and optimizing the weights of the base algorithms. The fitness function combined the mean squared error of stock returns with a penalty for feature set size. The optimized model was then used to predict future returns for all stocks, and based on prediction error, 30 stocks with the highest behavioral stability were selected. In the second stage, these 30 stocks were incorporated into the convex optimization framework of Rockafellar and Uryasev’s CVaR model to determine optimal portfolio weights for target returns of 0.2%, 0.5%, and 0.8%. Portfolio performance was evaluated using the Sortino & Sharp ratios, and the Acerbi–Szekely backtesting procedure was employed to assess the accuracy of tail-risk estimation. Results and discussion: The ensemble learning model enhanced by the Aquila metaheuristic algorithm successfully identified a compact yet highly effective set of price-, trend-, volatility-, and volume-based features, achieving a high level of predictive accuracy. The portfolio constructed with a target return of 0.5% demonstrated the best overall performance, achieving the highest Sortino & Sharp ratios at both the 95% and 99% confidence levels. Compared with an index-based portfolio, this strategy improved risk-adjusted performance by approximately 24%. Additionally, around 76% of the optimal portfolio weight was allocated to only eight top-performing stocks, mainly from the refining, petrochemical, and investment industries, which exhibited low correlation during market stress. The Acerbi–Szekely test confirmed that the CVaR model was well-calibrated and free from risk underestimation. Conclusions:This study introduces the first integrated framework that unifies three core components of financial decision-making and return prediction, intelligent asset selection, and advanced portfolio optimization-within a single, intelligent hybrid system. The results demonstrate that integrating ensemble learning with the Aquila metaheuristic algorithm under the CVaR framework can meaningfully reduce portfolio downside risk while enhancing risk-adjusted returns. This framework offers practical value for investment funds, asset managers, and institutional investors and can be extended to other emerging markets.