arXiv:2402.12381v1 Announce Type: new
Abstract: Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention. Various constrained multi-objective optimization evolutionary algorithms (CMOEAs) have been developed with the use of different algorithmic strategies, evolutionary operators, and constraint-handling techniques. The performance of CMOEAs may be heavily dependent on the operators used, however, it is usually difficult to select suitable operators for the problem at hand. Hence, improving operator selection is promising and necessary for CMOEAs. This work proposes an online operator selection framework assisted by Deep Reinforcement Learning. The dynamics of the population, including convergence, diversity, and feasibility, are regarded as the state; the candidate operators are considered as actions; and the improvement of the population state is treated as the reward. By using a Q-Network to learn a policy to estimate the Q-values of all actions, the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance. The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems. The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.
The Importance of Operator Selection in Constrained Multi-Objective Optimization Evolutionary Algorithms
In recent years, there has been significant interest in solving constrained multi-objective optimization problems using evolutionary algorithms. These algorithms have been developed with various strategies, operators, and constraint-handling techniques. However, the performance of these algorithms can heavily depend on the selection of the operators used.
Operator selection is a challenging task as it requires a deep understanding of the problem at hand. Different problems may require different operators to achieve optimal results. In many cases, it is difficult to manually select suitable operators, especially as the complexity of the problem increases. Therefore, there is a need for automated methods to improve operator selection for constrained multi-objective optimization evolutionary algorithms (CMOEAs).
This work presents a novel approach to operator selection in CMOEAs using Deep Reinforcement Learning (DRL). DRL combines the power of deep neural networks with reinforcement learning techniques to enable adaptive decision-making. In this framework, the dynamics of the population, including convergence, diversity, and feasibility, are considered as the state of the system. The candidate operators are treated as actions, and the improvement of the population state is used as the reward signal.
By training a Q-Network to estimate the Q-values of all possible actions, the proposed approach can dynamically select an operator that maximizes the improvement of the population state based on the current system state. This adaptive operator selection leads to improved algorithmic performance and better optimization results.
What makes this approach particularly interesting is its multi-disciplinary nature. It combines concepts from evolutionary algorithms, optimization, machine learning, and reinforcement learning. By integrating these diverse fields, researchers can harness the power of different techniques and create hybrid approaches that outperform traditional methods.
In the experimental evaluation, the proposed DRL-assisted operator selection framework was embedded into four popular CMOEAs and tested on 42 benchmark problems. The results demonstrated a significant improvement in the performance of these CMOEAs compared to nine state-of-the-art algorithms. The approach not only improved the overall optimization results but also exhibited better versatility in handling various problem types.
This research opens up new possibilities for improving the performance of constrained multi-objective optimization evolutionary algorithms. By leveraging the power of Deep Reinforcement Learning, researchers can tackle complex optimization problems more effectively. This work also highlights the importance of integrating multiple disciplines to create innovative solutions to challenging problems.