How we measure forecast accuracy
Frontwatt scores every forecast against metered output with the same rules for every plant: we use the forecast that existed at your decision deadline, divide the error by installed capacity and report every missing hour as missing. Where your submitted schedule is available, its error appears next to ours.
Updated
The metrics
- nMAE, normalised mean absolute error: the average of the hourly absolute errors divided by installed capacity. On a 100 MW plant, an nMAE of 5% means the forecast was off by 5 MW in an average hour.
- nRMSE, normalised root mean square error: like nMAE, but errors are squared first, so large misses weigh more.
- Bias: the average of forecast minus actual output. A positive bias means the forecast runs high.
- Range coverage: the share of hours in which actual output fell inside the P10–P90 range. It should be close to 80%.
- Imbalance cost: the deviation priced at the actual imbalance prices. This is the number that matters on your invoice.
Rules that keep the score honest
- Scored at the deadline. Every forecast is stored with its timestamp. The score uses the latest forecast that existed before the decision deadline, for example before the day-ahead market closes, not a later update.
- Same hours for every comparison. Our forecast, your submitted schedule and the baseline are compared on exactly the same hours.
- Gaps stay gaps. Hours without metered data are left out and reported. They are never filled with zeros, and days on which a plant produced nothing at all, such as a hydro plant shut for the season, are not counted as easy wins.
- A baseline for comparison. We also report a simple persistence forecast, the output two days earlier, so you can see how much the weather model adds.
- No cherry-picking. Evaluation periods are set in advance, not chosen after looking at the results.
How a new model replaces the current one
New models first run in the shadow of the live one. A candidate is promoted only when all of these hold:
- at least 60 days of training data, a separate 14-day calibration period and three locked 14-day test periods, covering at least two seasons;
- a separate 14-day live shadow period on forecasts made before the deadline;
- at least 2% lower nMAE than both the current model and the baseline, with a confidence interval that holds up;
- a range coverage close to the target and an imbalance cost no worse than today’s.
If the evidence weakens later, the candidate is withdrawn and the physical model carries on.
Why we do not publish a single accuracy number
Accuracy depends on the technology, the terrain, the season and the comparison window. The same forecast can look excellent in a calm month and poor in a stormy one. A single headline number says little about your plants. We would rather show you daily scores on your own plants, calculated with the rules on this page.