Solving an energy resource management problem with a novel multi-objective evolutionary reinforcement learning method
Authors
Matos Cardoso Leite, Gabriel; Jiménez Fernández, Silvia; Salcedo Sanz, Sancho; Gil Marcelino, Carolina; Pedreira, C.E.Identifiers
Permanent link (URI): http://hdl.handle.net/10017/60818DOI: 10.1016/j.knosys.2023.111027
ISSN: 0950-7051
Publisher
Elsevier
Date
2023-09-26Funders
Agencia Estatal de Investigación
Bibliographic citation
Matos Cardoso Leite, G. [et al.], 2023, "Solving an energy resource management problem with a novel multi-objective evolutionary reinforcement learning method", Knowledge-Based Systems, vol. 280, art. no. 111027, pp. 1-19.
Keywords
Multi-objective reinforcement learning
Policy search
Neuroevolution
Microgrids
Energy management
Project
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2020-115454GB-C21/ES/NUEVOS ALGORITMOS NEURO-EVOLUTIVOS PARA CLASIFICACION ORDINAL: APLICACIONES EN CLIMA, ENERGIAS LIMPIAS Y MEDIO AMBIENTE/
info:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica, Técnica y de Innovación 2021-2023/TED2021-131777B-C22
Document type
info:eu-repo/semantics/article
Version
info:eu-repo/semantics/publishedVersion
Publisher's version
https://doi.org/10.1016/j.knosys.2023.111027Rights
Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)
© 2023 The authors
Access rights
info:eu-repo/semantics/openAccess
Abstract
Microgrids have become popular candidates for integrating diverse energy sources into the power grid as means of reducing fossil fuel usage. Energy Resource Management (ERM) is a type of Unit Commitment problem, where a player operates a microgrid with diverse renewable generators integrated with an external supplier. Calculating the economic dispatch of each committed unit on a planning horizon is an NP-hard problem, and therefore, finding an exact solution is difficult. This paper presents a multi-objective solution to the ERM problem from the perspective of battery operation and external supplier dispatch. First, a novel multi-objective decision problem modeling is proposed that considers three objectives: cost, greenhouse gas emissions, and battery degradation. This framework involves a learning agent that controls the depth of discharge of a Lithium-Ion battery. To address the proposed problem, a new multi-objective algorithm called Multi-Objective Evolutionary Policy Search (MEPS) is introduced. The proposed algorithm uses NeuroEvolution of Augmenting Topologies structure to evolve artificial neural networks for estimating action-preference values considering multi-objective rewards. The MEPS performance is evaluated on both standard and newly-proposed benchmark problems, using the hypervolume as the evaluation metric. When compared to standard deep reinforcement learning, results showed that MEPS provides cost-effective, environmentally friendly, and efficient energy storage management solutions. Furthermore, MEPS effectively solves the proposed ERM problem by finding neural networks with a small number of nodes and connections, which are suitable for use in embedded control systems. Overall, MEPS proved to be a promising multi-objective approach in the transition to clean energy resources.
Files in this item
Files | Size | Format |
|
---|---|---|---|
Solving_Leite_Knowledge_Based_ ... | 840.8Kb |
|
Files | Size | Format |
|
---|---|---|---|
Solving_Leite_Knowledge_Based_ ... | 840.8Kb |
|
Collections
- TSENCOM - Artículos [58]