Three success factors for grid forecasts

Embarking on the complex journey of grid forecasting can often feel like navigating a labyrinth. Whether you're grappling with congestion, HVDC flow, imbalances, losses or any number of other grid quantities, finding your bearings is no easy task. At N-SIDE have partnered with numerous System Operators across Europe, gaining invaluabe experience and insights into this intricate process. From our experience on these projects, we've identified 3 key success factors that can turn this project into a smooth-sailing voyage:

1. Solve the business problem
2. Security of (data) supply
3. Basic people management for advanced automation

Let's deep-dive into each of them.

1. Solve the business problem

Clarify the problem

Forecasts are made to take decisions. The first key to success is to determine which decision you will take with the information that the forecast provides. Would it not make more sense to forecast that thing directly? Very often value can be unlocked by trying to get as close to your decision as you can get.

Agree on success criteria

When is a forecast good enough? This is an important business and technical question. By setting realistic criteria for solving the business problem early in the process, we can tailor the technical development and determine quickly whether a given forecast is feasible.

In some cases, it is also possible to encode the success criterion into the algorithm itself. We build our own algorithms from scratch exactly for this reason. Instead of trying to optimize the Mean Absolute Error, Root Mean Squared Error, or another generic quantity, it could be beneficial to encode it in your algorithm.

Sketch the business process

The forecast exists as a part of a larger system. We know that there is a downstream part (the decision based on the forecast). But there is also an upstream process. Sketching this out is a tool for surfacing latent knowledge about the process and identifying critical points early in the process.

2. Security of (data) supply

PoC vs Prod

It is one thing to do an offline proof-of-concept or desktop study. For these, the data are often assembled especially for the proof of concept by someone inside the TSO, such as a specialist or an intern. At this stage it is important to consider the feasibility of making these same data sources available real-time in production.

Pipeline stability

The data pipeline is the most common point of failure for a forecaster. The forecast is only as reliable as pipeline that feeds it. We often build multiple fallbacks into our forecasts because we know that you still need to make a decision, but time spent on ensuring pipeline stability often has high returns in production.

Feature persistence

Forecasts are long-term projects. However, we know that the grid changes regularly. If grid features are used in the forecast, it is a good idea to devise a procedure for accounting for these changes to ensure that the forecaster performance does not decay over time as lines are renamed, new ones are commissioned, or old ones are replaced, for example.

3. Basic people management for advanced automation

The points below are project management basics, and for this reason are often ignored, to do the detriment of the project.

Obtain stakeholder buy-in

Do you know how many stakeholders there are for a forecasting project? More than you might think when you start! Ensuring you obtain their buy-in will help to ensure that the project proceeds smoothly and is not delayed unnecessarily.

Assign roles and responsibilities

Through the systems operator forecasting projects that we’ve done, we have identified the roles and responsibilities that need to be assigned for successful project delivery. Some of them include common roles such as the Single Point of Contact (SPOC), project manager, or machine learning engineer. But depending on the nature of the forecast (on-prem or cloud-based), other less common roles might also be needed, such as infra engineer, the business team for negotiating the SLA, etc.

Ensure continuity

Because these projects last a long time, you need redundancy in your teams to ensure business continuity, because invariably something happens to someone. You don’t want to project to stall when someone is on leave.

In the intricate landscape of grid forecasting, each stage - from pinpointing the business issue, to securing data supply, to managing the human facets of high-tech automation - is pivotal. At N-SIDE, we've traversed this terrain alongside various System Operators, gaining firsthand insights and expertise. The knowledge we've accumulated is a valuable tool, primed to guide you towards improved forecasting accuracy and operational efficiency. Keep in mind, while the process may seem complex, having a reliable partner like N-SIDE can transform this challenge into an opportunity for growth and innovation. Let's continue the dialogue, because through collaboration, we can propel your grid operations into the future.