Whether your challenge can be solved by our existing products or requires the development of new analytics solutions and market design approaches, our team is ready to help.
At N-SIDE, accuracy is considered one of the main pillars of forecasting. It's a topic that often raises questions from systems operators, and for good reason - achieving accurate forecasts can be quite challenging. In order to assist power systems professionals, we've put together a brief guide outlining the key concepts to help them get started on this important subject.
As a systems operator, it's crucial to prioritize both accuracy and reliability. While a highly complex algorithm that performs well on benchmarks may seem impressive, it's not practical for production if it's difficult to maintain. Ultimately, accuracy should not be achieved at the expense of practicality and reliability.
At N-SIDE, we believe that having an accurate algorithm is not enough if it solves the wrong problem. Therefore, we always ask ourselves what problem the forecast is intended to solve. For instance, instead of predicting imbalance, we could determine mFRR activation. Similarly, instead of forecasting grid losses and the DA price of electricity, we could predict when to buy energy. Additionally, instead of an HVDC flow forecast, we could forecast ATC (available transfer capacity). In all these cases, we aim to build the business case into the algorithm itself, rather than just minimizing a vanilla Mean Absolute Error. This is made possible by our advanced algorithms.
We distinguish three areas for accuracy: algorithm quality, the data and features of the forecasting system, and the processes used to maintain the system. Let’s consider each in turn:
When people think of forecast accuracy, they usually think of the pure science of data science. It involves staying updated with the latest literature, trying out new ideas, and creating a technically sophisticated product based on advanced mathematical concepts and cutting-edge technologies. At N-SIDE, we have a team of skilled applied mathematicians, physicists, and engineers who have created their own algorithms that have proven to be more effective than commercially available ones for the energy applications we've tested. But we also know that an algorithm by itself won’t give you a good forecast.
To provide good features, having top-quality data is crucial. In the European energy industry, some data streams are required to be publicly accessible by law. Additionally, there are numerous commercial data vendors with relevant and high-quality data, such as weather forecasts for renewable energy production and demand. In addition to mining publicly available sources, we subscribe to various data vendors to make sure our algorithms receive the best data. To transform these data into useful features, we must also process them intelligently. For example, identifying and comprehending long weekends from regular time stamps enhances our ability to predict the demand for a particular region.
At N-SIDE, we have two powerful tools that help us enhance forecast accuracy: transparent forecasting algorithms and probabilistic forecasts. Although these features are typically geared towards making it easier for forecasters to use, they have also given us valuable insight into the reasoning behind our forecasters' predictions, as well as identifying when the model is uncertain. Such insights are critical to improving the model.
Adaptability is another crucial feature for modern energy forecasts. The world, especially the energy world, is changing very fast. In the “good old days”, we could rely on having a 2-year historical data set with which to train our models. Not so today: two years ago is ancient history in Energy, and our forecasting systems must be adaptable enough to cope with that.
Many processes and best practices ensure that the theoretical performance is realized in production. These are things like:
In conclusion, in forecasting, we often find that accuracy is not simply a property of an algorithm, but rather an emergent quality of a well-coordinated system. Striking the right balance between algorithm quality, data and feature selection, and system maintenance processes is crucial to achieving reliable results. This complexity underscores the importance of maintaining a dynamic approach that not only keeps up with the fast-paced energy sector but also anticipates its trajectory. Our understanding of this landscape, combined with a firm commitment to continuous improvement, allows us to consistently deliver practical and reliable forecasts. As we navigate the ever-evolving energy ecosystem, we invite you to join the conversation and explore the future of grid operations.
Grid Forecasting Lead at N-SIDE. Herman holds a PhD in electrical engineering, where he focussed on machine learning on energy datasets. He has been involved in numerous forecasting and other advanced analytics machine learning projects, both as data scientist and as project manager. Herman is currently the product owner for grid forecasting at N-SIDE.
Herman CarstensThe N-SIDE Energy Forecasting Platform enables the development of predictive and prescriptive forecasting services. Whether you’re starting from a demonstrated use case or building a new one, our forecasting services can be easily customized to your specific needs.
read moreGrid operators have some of the world's most demanding service level requirements. And yet, as the complexity of energy systems increase, grid operators are becoming more reliant on intelligent IT systems such as forecasters.
read moreGrid balancing in Europe is evolving, and balancing paradigms such as MARI and PICASSO are changing the calculus for both market participants and systems operators. This webinar will be presented by practitioners who have implemented several balancing analytics and forecast projects at different European TSOs.
watch on demand