Analysis and Application of Deep Reinforcement Learning in the Context of Energy Economics and Energy Systems

The ongoing transition from fossil fuels to renewable energies has drastically changed the energy sector. Both electricity generation and demand have become significantly more flexible, forcing the entire supply chain to adapt as traditional approaches reach their limits. While Deep Learning (DL) methods have been proposed to address this uncertainty, their practical applicability has only been demonstrated to a limited extent. This thesis extends this research by exploring model-free Deep Reinforcement Learning (DRL) approaches in two energy domain applications. As the first application field, we selected electricity trading, specifically continuous intraday markets, where participants adjust positions up to 30 minutes before delivery. Given the difficulty for traders to capture these market dynamics, DRL is particularly suitable here. We develop an environment incorporating market dynamics and external variables, identifying the short-term price forecast as a key variable alongside transaction price and order books. Thus, we include an extensive study of shallow and DL models for electricity price forecasting in day-ahead and intraday markets. Our DRL agent was successfully tested against rule-based trading algorithms, demonstrating viability, and we provide insights into real-world deployment on the EPEX SPOT exchange. The second application field is topology optimization of power grids. With increasing renewable energy shares, transmission system operators must adjust strategies as redispatch becomes more expensive. Topology optimization offers a promising alternative, though the associated optimization problem is highly complex and scales exponentially with grid size. We analyze DRL's potential for identifying suitable topologies, finding that DRL can propose topology actions faster than rule-based approaches. However, combining DRL agents with rule-based components proves even more beneficial, allowing domain knowledge integration. Since transmission grids are critical infrastructure, we also include a failure analysis of the DRL agents. Both application fields provide practical insights into DRL utilization in the energy domain, and this thesis additionally assesses the general possibilities and limitations of DRL in this field.

Level thesis:degree_level

thesis.doctoral

Grantor dc:publisher

Christian-Albrechts-Universität zu Kiel

Year

2026

Author dc:creator

• Lehna, Malte

Contributors dc:contributor

• Tomforde, Sven
• Nieße, Astrid

• Deep Reinforcement Learning
• Topology Optimization of Power Grids
• Automated Electricity Trading
• Deep Learning
• Machine Learning
• Energy Economics
• Energy Systems

Repository record source_url

https://macau.uni-kiel.de/receive/macau_mods_00008891

OAI record oai:identifier

https://macau.uni-kiel.de/servlets/OAIDataProvider?verb=GetRecord&metadataPrefix=xMetaDissPlus&identifier=oai%3Amacau.uni-kiel.de%3Amacau_mods_00008891