Forecasting and trading optimisation in the day-ahead and balancing market

The transition to sustainable energy sources is a critical challenge of the 21st century. Over the past decade, the adoption of renewable resources such as wind and solar power has surged, driven by policy incentives and decreasing costs. While these measures are essential for decarbonising the energy sector, they introduce challenges such as price volatility, efficient energy management, and curtailments. The rapid expansion of wind and solar power also increases variability in net power demand, posing significant reliability concerns for electricity grids. Addressing these challenges requires advanced tools to predict market dynamics and manage the resulting imbalances effectively. This dissertation responds to these needs by focusing on predicting electricity prices and modelling trading strategies in the Day-Ahead Market (DAM) and Balancing Market (BM), with a primary emphasis on the latter. The DAM is a forward market where electricity is traded a day in advance based on forecasted demand, while the BM operates closer to real-time, allowing for adjustments to address supply-demand imbalances and forecast errors. The BM, characterised by higher volatility than other markets, has become increasingly important for managing forecast errors associated with variable renewable energy. While traditional spot markets like the DAM and Intra-Day Market are well-studied, the BM remains under-explored despite its critical role in balancing real-time supply and demand, particularly in regions where large-scale energy storage solutions are not economically viable, necessitating a real-time balance between production and consumption to minimise curtailments. The research outlines how to approach forecasting the BM, starting with the collection and preprocessing of datasets for both the Irish DAM and BM, accompanied by a detailed time series analysis. Various techniques for forecasting electricity prices are investigated, utilising statistical, Machine Learning, and Deep Learning models. The study identifies the Lasso Estimated AutoRegressive model as the most effective for accurate price prediction, closely followed by Extreme Gradient Boosting and Random Forest. Deep Learning models, while excelling in the DAM forecast, struggle to accurately forecast prices in the BM. Expanding on these insights, the thesis explores the optimisation of trading strategies using quantile-based forecasts. Initially, a trading approach is detailed for executing trades once per day in each market, which is then extended to a heuristic optimisation strategy for higher frequency trading. The research further evolves to a dual-market strategy, holding positions in both markets concurrently, highlighting the advantages of multi-market participation. The outcomes emphasise the pivotal role of increasing trading frequency with large scale energy storage to improve grid reliability and market liquidity. Furthermore, to refine probabilistic price prediction, this thesis evaluates recent adaptations of Conformal Prediction techniques tailored for time series data, including Ensemble Batch Prediction Intervals and Sequential Predictive Conformal Inference. We demonstrate the efficacy of Conformal Prediction methods in achieving validity and providing more reliable prediction intervals compared to conventional approaches like Quantile Regression and Quantile Regression Averaging, which excel in efficiency. We propose an ensemble combining the efficiency of Quantile Regression with the improved coverage of Conformal Prediction techniques, delivering reliable forecasts with narrow intervals, and demonstrating improved financial returns in both markets. This thesis advances electricity price forecasting and trading strategies by developing a transparent framework for probabilistic forecasting and trading in both the DAM and BM. For energy producers, the proposed methods enhance revenue stability by mitigating price volatility and curtailment risks. Grid operators benefit from improved forecast accuracy, enabling proactive grid management, effective scheduling, and reduced reliance on costly balancing measures. Optimised trading strategies leverage the DAM’s forecast-based scheduling and the BM’s real-time adjustments, addressing market-specific challenges while enhancing profitability and liquidity for traders and other market participants. By supporting the integration of renewable energy sources and reducing curtailments, these methods align with global decarbonisation goals and the transition to a low-carbon energy system. This research provides a practical framework for improving market efficiency and resilience, equipping stakeholders with actionable tools to navigate the complexities of renewable integration.

Grantor dc:publisher

University College Cork

Year dc:date.issued

2025

Author dc:creator

• O'Connor, Ciaran

Advisors dc:contributor.advisor

• Prestwich, Steve
• Visentin, Andrea

• Day-ahead market
• Balancing market
• Electricity price forecasting
• Quantile trading
• Probabilistic forecasting

Statement dc:rights

• © 2025, Ciaran O'Connor.

Licence dc:rights.uri

• https://creativecommons.org/licenses/by/4.0/

Language dc:language.iso

en

Handle dc:identifier.uri

https://hdl.handle.net/10468/17912

OAI identifier oai:identifier

oai:cora.ucc.ie:10468/17912