Met the adoption rule set before the experiment: both error scores improved on v4's (v5 − v4 at 97.5%: −0.0133 [−0.0166, −0.0091] on the point-error score, −0.0119 [−0.0147, −0.0084] on the interval score; 1 policy tested against the rule). Development · post-selection
Error scores: error relative to a simple naive forecast, averaged over the test periods; lower is better.
Adoption rule: set on 2026-10-05; its five conditions are in the v5 chapter. Read them
Paired difference:v5's score minus v4's, with the rule's 97.5% interval; below zero favours v5.
Policies: the models and variants tested.
Development · post-selection: development evidence, not a test on new data.
The demo runs the released model, v1. Research generations are not released one by one; only the final model, after its one-shot test and live run, replaces the released one.
Runs in your browser: about 57 MB on a first visit. A forecast appeared after 18.3 s in Chrome 153 on a Mac, public demo, 2026-09-29; last verified 2026-09-29.
Historical forecast · v1Forecast and observed price for a held-out day. This is a historical replay, not a live forecast.
Median forecast80% prediction intervalObserved price
The released model, v1 · released LightGBM, topic by topic: the artifact its one-shot test evaluated and the demo computes.
All topics in full
Data and the information cutoff
What it forecasts. The German–Luxembourg day-ahead price of each hour of the next delivery day, in EUR/MWh: 24 values on most days and 23 or 25 on daylight-saving transition days, issued together as one curve before the 12:00 CET auction gate.
Where the data comes from. One committed hourly snapshot, from 2019-01-01 to 2026-09-06, of the day-ahead price and the day-ahead load forecast, pinned by its SHA-256 hash. It was pulled from SMARD.de, which republishes the ENTSO-E series, and a sample spanning every regime agreed with the ENTSO-E Transparency Platform to the cent. Data: ENTSO-E Transparency Platform; Bundesnetzagentur | SMARD.de — CC BY 4.0.
The information cutoff. Every price-derived input uses only prices of delivery days before the forecast day, and rolling price statistics are frozen at the day before. The day-ahead wind and solar forecast is published after the gate, so the model never uses it. That the load forecast exists before the gate is an assumption, not a measurement.
Fitting, calibration and test data. The model was fitted on data up to 2026-04-07, calibrated once on 2026-04-09..2026-06-07 and tested once on 2026-06-09..2026-09-06. The demo and this page replay the committed snapshot; no live data feed is read.
Preprocessing. A quarter-hour series becomes an hour only from four complete quarter-hours; from 2025-10-01 the hourly price is the mean of four quarter-hour prices. The price is not log-transformed, because it is often negative.
Price regimes and negative prices
Three regimes shape the development period: before 2021-09-01, the energy crisis from 2021-09-01 to 2022-12-31, and from 2023-01-01 a market with frequent negative prices. The development hours split into 4,319, 2,112 and 4,316 hours across them; 465 hours, on 85 days, had a negative price.
Why it matters: the model learns from calm, crisis and negative-price periods together, so its errors and interval coverage are reported per regime, and the calibration's assumption that recent errors resemble future ones breaks when the regime shifts. The day-ahead price floor moved to −600 EUR/MWh from 2026-05-28, an environment shift the frozen model predates.
v1 uses 25 inputs, a catalog frozen before fitting: calendar, holiday and seasonal features; two date-based regime flags; the day-ahead load forecast for the hour and for the day; the price of the same hour one, two and seven days earlier; rolling price statistics over the previous week and month; and a weekly count of negative-price hours.
The inputs, by group, as the model card names them
What was left out, and why. The one candidate domain input, a residual-load proxy built from actual wind and solar generation, had to lower the pooled pinball loss in a comparison fixed before fitting; it did not (+0.371516%). The day-ahead wind and solar forecast is excluded because it is published after the gate; a separate benchmark measured what it would have been worth (-19.4926%).
Why hour-of-day and day-of-week inputs. The price spectrum shows pronounced cycles at 24 and 168 hours, with a 12-hour harmonic, and their amplitude changed in the crisis; the calendar inputs follow that structure rather than assuming it.
Before release. Five development folds of 90 days each, spanning all three regimes, were scored walk-forward: each fold's model was fitted on earlier data only, with a gap of one delivery day before its test block. The similar-day naive, a 168-hour seasonal naive and ridge regression were scored on the same rows. The folds hold 10,747 hours on 448 days, the same hours the research comparison uses.
The one-shot test. After every choice was frozen, the model was fitted once, calibrated once on 2026-04-09..2026-06-07 and frozen; only then was the 90-day holdout, 2026-06-09..2026-09-06, opened and scored, once. Pre-specified one-shot holdout DM test on a fixed 90-day window — confirmatory-style, not power-qualified.
Evidence classes. The development folds also chose the input catalog, so their results are post-selection evidence; the holdout is the only test specified in advance; prospective live evidence is a third class, reserved for a future final model.
Leakage checks, on the browser path. Setting the delivery day's 24 prices to missing changes the forecast by exactly 0.0; raising all 24 D−1 prices by 250 EUR/MWh moves it by 220.9433 EUR/MWh. The model's input schema also refuses the post-gate forecast.
Measured results
The one-shot holdout Confirmatory-style, not power-qualified
On its pre-specified 90-day holdout, v1 beat the similar-day naive: MAE 25.9 against 27.8 EUR/MWh, and mean pinball loss 6.7 against 13.9 EUR/MWh. A one-sided Diebold–Mariano test on the daily pinball-loss vectors gave p < 10⁻⁶; it tests the probabilistic forecast, not the MAE difference.
Its prediction intervals covered 0.4407, 0.7593 and 0.9398 of the holdout's hours at the 50, 80 and 95% levels.
Where it falls short. On the development folds its point error is 28.58% worse than the naive's, pooled over all days (a one-sided test for an improvement gives p = 0.948: no evidence of an advantage). Over the August 2022 peak weeks its 95% interval covered 0.194 of outcomes.
In the shared development comparison, whose error scores divide by the naive's error in each period, v1's development replay scores 1.052 on the point-error score and 0.986 on the interval score, where the naive scores 1.000; lower is better.
The price of the same hour one day earlier dominates the median forecast.
Mean absolute SHAP value, EUR/MWh · median head · the released artifact on fold 5's test block · in-sample diagnostic
The weekly lag, the rolling weekly price quantiles, the two-day lag and the day of the week follow, close to one another.
Which model and rows. These are the released artifact's own SHAP values for its median head on fold 5's test block, 2026-01-08 to 2026-04-07: rows it was fitted on, so an in-sample diagnostic. Fold 5's development model, explained out of sample on the same rows, ranks the inputs almost identically (Spearman rank correlation 0.98; 10 of the top 10 shared), but agreement does not make them the same evidence.
View values, beside the development model's ranking
Mean absolute SHAP value of the median head, EUR/MWh, on fold 5's test block
Rank
Released artifact, in sample
Value
Fold 5 development model, out of sample
Value
1
price_lag_24h
14.638026612371469
price_lag_24h
15.401310383360386
2
price_lag_168h
5.657623951395043
price_lag_168h
5.412601170582895
3
price_roll_q50_168h
4.448508921689905
price_roll_q50_168h
5.1258021429891265
4
price_lag_48h
4.439940498351033
day_of_week
4.510413707759048
5
day_of_week
4.396023958232133
price_lag_48h
4.206478858888543
6
price_roll_q95_168h
4.386255175377148
load_forecast_mw
3.2302870958504752
7
load_forecast_mw
3.418967676456138
day_type
3.1644473239108297
8
local_hour
3.395383482828897
local_hour
3.064186071616306
9
price_roll_mean_720h
3.281260605520439
price_roll_mean_720h
2.9835918587747425
10
day_type
2.982999170265987
price_roll_q95_168h
2.71908082228481
SHAP on the median explains its central tendency, not the interval width, and it is not a test of incremental value: that was the comparison fixed before fitting, in the inputs topic.
Permutation importance shuffles one input at a time and measures how much the median forecast's MAE rises: here for fold 5's development model on its own test block, out of sample. price_lag_24h adds 8.3 EUR/MWh, price_lag_48h 1.8 and day_of_week 1.4. Its ranking agrees with SHAP's at the top and differs in the middle (Spearman rank correlation 0.69).
Permutation importance: rise in the median forecast's MAE when one input is shuffled, EUR/MWh, fold 5's development model on its own test block
Rank
Input
MAE increase
Standard deviation
1
price_lag_24h
8.260802538981292
0.30247307929369166
2
price_lag_48h
1.802510035141463
0.14628203865002448
3
day_of_week
1.443593197772815
0.15776352757618647
4
local_hour
1.1285028713503344
0.09176676071425387
5
price_lag_168h
1.0966840658952974
0.08430715494917224
6
load_forecast_day_mean_mw
0.4105148675733332
0.07627746551533812
7
load_forecast_mw
0.3855651672396828
0.07415806585874446
8
day_type
0.33649415897484614
0.06551227273566475
9
price_roll_std_720h
0.1886270625418504
0.04192072992783573
10
price_roll_q50_168h
0.15114704944396032
0.05026603320434395
Neither ranking measures incremental value or causation; the only incremental-value test was the comparison fixed before fitting.
Scenario sensitivity. The load-forecast control in the replay and the demo multiplies the day's load forecast and holds every other input fixed. Scenario perturbations are ceteris-paribus sensitivity probes and may be out of distribution.
Where it fails: errors by regime
Errors and coverage are worst in the crisis: MAE 141.0 EUR/MWh [123.1, 160.4], with the 95% interval covering 0.555 of 2,112 hours, against 13.3 EUR/MWh and 0.889 before it and 21.6 EUR/MWh and 0.943 after. Over the August 2022 peak weeks the 95% interval covered 0.194 of 408 hours; on negative-price hours, 0.710.
The released recipe on its five development folds, by stratum: MAE in EUR/MWh with a day-block bootstrap 95% confidence interval for thin subsets, and interval coverage as a fraction of hours
Which model. These strata come from the five development folds, each scored by the model fitted for that fold and calibrated on its own window, not by the frozen artifact, whose holdout covers only the most recent regime. Thin subsets carry day-block bootstrap 95% confidence intervals and are read qualitatively. The crisis failure's mechanism is in the limitations: shrinkage toward the training price level, with calibration thresholds estimated at a lower level.
How reliable its intervals are
Calibration raised coverage, which stayed below nominal at every level, on the holdout and on the development folds.
Share of hours inside the interval · nominal level as reference · the frozen artifact on its one-shot holdout, and the development folds
raw heads
after CQR
final
nominal level
The intervals come from nine quantile heads, then conformalized quantile regression (CQR), which shifts them by thresholds calibrated on a recent window, then an isotonic step that keeps the quantiles in order. On the one-shot holdout the 95% interval covered 0.8394 of hours from the raw heads, 0.9398 after CQR and 0.9398 after the isotonic step; on the development folds, 0.6705, 0.8416 and 0.8452.
Coverage is read with width: over the development hours the final intervals averaged 41.5, 75.5 and 130.9 EUR/MWh at the 50, 80 and 95% levels. The holdout report records coverage but not width.
The one hard gate was correctness, not a favourable number: adjacent quantiles crossed 10,158 times from the raw heads and 4,412 times after CQR on the development folds, and 0 times after the isotonic step; 0 on the holdout.
CQR's finite-sample marginal guarantee applies to its output before the isotonic step, and only when past and future errors are exchangeable; regime shifts break that, so every coverage figure here is empirical.
Read a forecast
Forecast for 2026-09-06, a day of the one-shot holdout
Historical replay · EUR/MWh by local hour (Europe/Berlin) · the frozen released model
Median forecastPrediction intervalObserved price
At the selected level, the frozen model's empirical coverage over the holdout was
0.7593.
A scenario is active: the observed price is hidden, because it belongs to the unperturbed day.
How to read it. The line is the median forecast for each local hour; the band is the prediction interval at the level you choose; the dashed line is the price that cleared, an outcome that was never an input. Historical out-of-sample replay — the frozen champion forecasting a 90-day period it never trained on. This is not a live forecast.
The controls. The level selector reads other quantiles of the same forecast, and the frozen model's empirical coverage at that level over the holdout is shown beside the chart. The load-forecast scenario multiplies the day's load forecast with every other input held fixed, and hides the cleared price, which belongs to the unperturbed day.
Computed in your browser. The demo forecasts historical delivery days with the released model's own boosters, so a Python runtime and the model download first, about 57 MB on a first visit; on a committed fixture of 54 delivery days its output equals the frozen artifact's bitwise. Change the interval level or the load scenario there as here.
The main limits, each in full below: the development results are post-selection evidence; the intervals under-cover when the regime shifts, severely in the 2022 crisis; the model is frozen, fitted 152 delivery days before its snapshot ends; the load forecast's availability before the gate is an assumption, not a measurement; and it is a portfolio artifact, not an operations system.
Exchangeability under regime shift.CQR provides finite-sample marginal coverage guarantees under exchangeability. The walk-forward CV mildly violates exchangeability — the crisis regime is not exchangeable with the pre-crisis regime, and the solar-driven negative-price era is not exchangeable with either — so empirical coverage may diverge from nominal on regime-shift folds. This is documented in the reliability diagram (Section 8.4).
Development versus one-shot evidence.The five-fold development results are descriptive post-selection evidence, never confirmatory: those folds also chose the catalog. Only the 90-day holdout was pre-specified and opened once, and on the development folds the point-accuracy DM shows a deficit, not merely the absence of an advantage.
Disclosed assumption — the load forecast.A65/A01 is pre-gate by explicit assumption, not by measurement: both the existence of the delivery-day load forecast before the 12:00 CET gate and its equality to the archived vector used here are assumed, and the regulatory update provision permits later revisions.
Disclosed assumption — the generation archive.A75 aggregate actual generation is used at its current archived values, which may differ from the values visible in real time despite the D-2 boundary.
The measured cost of the strict gate.The strict-gate design has a measured cost rather than an assumed one: the post-gate A69 forecast is worth 19.4926% of pooled raw-head pinball loss, and the project declines to use it.
A two-sided bounded target, live at the floor.The target is two-sided and bounded: the price is routinely negative and has hit the −500 EUR/MWh floor, which truncates the lower conformity residuals, so the lowest intervals under-cover conditionally near the floor.
Coverage divergence.Empirical coverage diverges from nominal: 0.4407 / 0.7593 / 0.9398 against 50 / 80 / 95 %, so the 50 % interval under-covers by roughly six points on the holdout window. On the crisis stratum it does not merely diverge, it collapses: over the August-2022 peak weeks the 95 % interval covered 0.194 of outcomes. The mechanism is measured — that fold's CQR thresholds were estimated on a May-June 2022 calibration window at a ~198 EUR/MWh level and applied to an evaluation block averaging 376 EUR/MWh, and the conformal correction is additive, not multiplicative. This is what a split-conformal guarantee does when exchangeability breaks; it is the defect later generations address (see the v2 and v3 chapters), and it is not fixed in the released v1.
Model staleness, with all four cutoffs.The deployed demo applies a frozen model whose raw-model fit cutoff (2026-04-07) precedes the snapshot cutoff (2026-09-06) by 152 delivery days, with the final calibration window 2026-04-09..2026-06-07 and the holdout window 2026-06-09..2026-09-06 — all four cutoffs published separately because they are four different dates.
The 15-minute MTU averaging choice.From 2025-10-01 an hourly price is the mean of four quarter-hour prices, so every hour-level statistic here — the negative-hour tally included — depends on that averaging choice, and a quarter-hour tally differs.
Scope.This is a portfolio artifact, not an operations system: no retraining schedule, no drift gate, no rollback machinery, no monitoring surface, and no multi-day-ahead forecast.
The day-ahead price floor moved to −600 EUR/MWh from 2026-05-28, an environment shift the frozen model predates.
Run this product
Run the released model on your machine. Install the pinned dependencies with uv sync, then uv run python predict_next_day.py --level 80 --self-check forecasts a delivery day offline from the bundled snapshot and re-proves the gate boundary.
Check its identity. The artifact fingerprint is 57e3ad40a7f48bb7896628ce2e5dec61314dea567da9867aede4bf1b0a10e0fb, the same value in the model card, the one-shot holdout report and the registered model's alias; the snapshot's SHA-256 hash is 7dd2dc73407706ca6bd3c1ad51d201ac0de35eec5ca129ce320cca639f697f00.
Check the browser computation.make wasm rebuilds the browser payload from the committed model, proves its output equal to the frozen artifact's on the fixture and exports the demo; uv run pytest tests/test_22_wasm_equivalence.py -q reruns that proof alone.
The archived report's container instructions are historical: the demo is a static Hugging Face Space that computes in the browser, and publishing it is the Owner's action.
How the models compare
v5 against v4, as a share of v4's error scores: −2% [−3%, −2%] on the point-error score and −2% [−3%, −2%] on the interval score, with 95% confidence intervals.
Development evidence, not a test on new data. Meeting the targets is a development diagnostic, not a product qualification.
Accuracy targets: each error score at least 10% below the strongest benchmark's, set on 2026-09-15; a policy's distance from them is a point comparison.
Daily LEAR: the strongest benchmark, a linear model refitted daily.
Not adopted: the candidates of the two experiments tested against v4 and not adopted, drawn as open circles. Pooled member, averaged: one LightGBM for all 24 hours, averaged over four sizes, in place of v4's three blocks. Pooled pair: v4's normalized and raw-price LightGBM pair, each pooled over all 24 hours. DDNN member on v4: a distributional neural network with one third of the blend's weight. Their cards are under the experiments not adopted between v4 and v5.
12 policies · the same 10,747 historical hours over 448 days · error scores averaged with equal weight over five test periods · lower is better
v5
v4
v3
v2
study arm, or a candidate not adopted
reference
v1
The dashed lines mark the targets: each error score at least 10% below the strongest benchmark, daily LEAR (set 2026-09-15). 25 policies were tested against them; v3 was the first to meet both; 16 later policies met them too, among them v4 and v5. The chart's 12 rows are not that census: 8 of them were tested against the targets, and the others are references that never were: daily LEAR, daily LightGBM, v1's development replay and the similar-day naive.
Each score divides a model's error by that of a simple similar-day forecast in the same period, which therefore scores 1.000. The interval score accounts for both interval width and missed outcomes.
For scale, as mean absolute error per test period: v5 4.8–14.4 EUR/MWh in the four ordinary periods and 45.5 EUR/MWh in the 2022 crisis period; the similar-day naive 8.6–30.5 and 86.9 EUR/MWh. Context only: the error scores rank the policies.
v1 appears here as its development replay; its one-shot holdout results are in the v1 chapter.
Equal-fold error scores (ratio to the naive), the targets, and pooled errors (EUR/MWh), identical hours
Policy
Point-error score (S_MAE)
Interval score (S_WIS)
Both targets
Distance from daily LEAR, point / interval, percent
Pooled MAE, EUR/MWh
Pooled WIS, EUR/MWh
v5 · DDNN-2 added v5
0.5223212760431432
0.4936627890811673
met
−20.60% / −22.74%
17.05033286364895
9.977221704544798
v4 · three-block LightGBM added HGL
0.5356586541153796
0.5055748401472236
met
−18.57% / −20.88%
17.549693593320555
10.23606326165729
v3 · weather features HG
0.5657606376333641
0.532218690181866
met
−13.99% / −16.71%
18.258885500475362
10.665905821725305
v2 · blended LEAR, hour-aware intervals H0
0.6440721386286878
0.6160299894915677
not met
−2.09% / −3.59%
20.23103542154645
12.046459955387254
Pooled member, averaged (not adopted) R
0.5368011399310709
0.5065780356195718
met
−18.40% / −20.72%
17.520429664636808
10.248430843718442
Pooled pair (not adopted) M
0.5343316729501412
0.5048157261017694
met
−18.77% / −21.00%
17.423424196407982
10.205525194847981
DDNN member on v4 (not adopted) v5, CP-23
0.5421044684654983
0.5126648907263633
met
−17.59% / −19.77%
17.960361164832925
10.413018697663293
Daily LEAR (benchmark) B2
0.6578109123747921
0.6389910407326591
—
—
20.984252545162313
12.662319073520665
Normalized LEAR (study) A1
0.6722908121370033
0.646015091607363
not met
2.20% / 1.10%
20.70749196318633
12.449676327150861
Daily LightGBM (benchmark) B3
0.7841363977682992
0.7399052224715813
—
—
26.323933197067298
15.167693659401653
v1 · released LightGBM (development replay) B1
1.051845134839199
0.9856366964174134
—
—
41.743481556831235
25.24792934918561
Similar-day naive (normalizer) B0
1.0
1.0
—
—
32.81010165627617
20.020603986162254
A fair comparison
The five test periods are called folds below; the third covers the 2022 price crisis. These are development results, not a test on new data.
Bootstrap settings and the five historical test periods
Paired intervals come from a moving-block bootstrap with seed 15042, 2,000 replicates and 7-day blocks, resampled identically for every policy. They are confidence intervals of an estimated difference, not forecast intervals. A change's interval as a share of the comparator's score is, from v4 on, the percentile interval of the ratio of the two scores over the same draws; for v2 and v3, it is the difference's interval divided by the comparator's score.
Fold
Delivery days
Days
Hours
Fold 1
2020-07-01 to 2020-09-28
90
2,160
Fold 2
2021-04-01 to 2021-06-29
90
2,159
Fold 3 (the 2022 crisis)
2022-07-01 to 2022-09-28
88
2,112
Fold 4
2025-05-01 to 2025-07-29
90
2,160
Fold 5
2026-01-08 to 2026-04-07
90
2,156
Definitions, the policies tested, and why v1 scores differently in its own report
The point-error score (S_MAE) is the mean over the five historical test periods (folds) of a policy's MAE divided by the similar-day naive's MAE in the same period; the interval score (S_WIS) does the same for the weighted interval score, which rewards narrow intervals and penalizes missed outcomes. MAE uses the emitted median.
The 25 policies tested against the targets, each identity counted once, by decision date. Decided 2026-09-16: Normalized LEAR; Normalized LightGBM; Normalized component mean; 84-day normalized LEAR; Normalized component mean, variant. Decided 2026-09-23: v2 · blended LEAR, hour-aware intervals; Pooled-interval control. Decided 2026-09-24: v3 · weather features. Decided 2026-09-30: v4 · three-block LightGBM added; Pooled LightGBM with weather; Three-block LightGBM; Normalized three-block LightGBM. Decided 2026-10-04: DDNN member on v4; v3 plus a capacity-averaged pooled LightGBM; v3 plus pooled normalized and raw LightGBM; v3 plus a pooled normalized LightGBM; v3 plus a pooled raw LightGBM; v3 plus normalized block LightGBM; v4 with dynamic intervals; v3 with dynamic intervals; DDNN alone; v3 plus a DDNN member. Decided 2026-10-11: v5 · DDNN-2 added; DDNN-2 alone; v3 plus a DDNN-2 member.
Why v1 scores 1.052 in the shared development comparison but 28.58% worse in its own report. Both figures describe the same 448 development days. v1's report compared its daily absolute error with the raw similar-day naive (MAE 32.45 EUR/MWh) and pooled all days, so the 2022 crisis dominates. This comparison gives each of the five folds equal weight, and its naive reference is the forecast's emitted median after the common residual layer (MAE 32.81 EUR/MWh). v1's own error is 41.74 EUR/MWh in both. v1's original nine-quantile pinball is a different score from the seven-quantile interval score used here.
From v4 to v5: adding a distributional neural network
Development · post-selection
Predecessor
v4 · three-block LightGBM added. In September 2026, v4 was adopted in research.
What changed
v5 kept v4's inputs, its two LEAR forecasts and its LightGBM, and added DDNN-2, a distributional neural network that forecasts the whole next day at once, with one sixth of the blend's weight, taken from the LightGBM's third; the hour-aware intervals were re-estimated on the new forecast's own errors.
Comparator
Set in advance: v4 itself, on identical hours, so the protocol's comparator is also the predecessor and the comparison is direct.
Result
Point-error score −2% [−3%, −2%] and interval score −2% [−3%, −2%], as a share of v4's score, with 95% confidence intervals of that ratio from the experiment's own bootstrap draws; at the rule's 97.5% decision level, [−3%, −2%] and [−3%, −2%]. Development evidence after selection, not a test on new data.
Not established
The comparison scores 448 of the 2,654 days of data up to the end of the development period, 2019-01-01 to 2026-04-07; 0 of them fall in October to December, and only 1 winter (December to February) contains any of them. It is the second decision on a distributional neural network over the same five test periods; only a one-shot test on days never used for development or selection protects against that reuse.
Decision
In October 2026, v5 was adopted in research. It met the adoption rule set before the experiment against v4; the released model, v1, did not change. The two study arms, DDNN-2 alone and v3 plus a DDNN-2 member, were built only to show where the change came from, and were never candidates for adoption.
v3 · weather features. In September 2026, v3 was adopted in research.
What changed
v4 kept v3's inputs and its two LEAR forecasts, added a LightGBM forecaster with one third of the blend's weight, and re-estimated the hour-aware intervals on the new forecast's own errors. The gain comes from the combination: on its own, no LightGBM arm showed an improvement over v3, but the blend improves both error scores, because a nonlinear model built from decision trees makes different errors from the two linear forecasts. The LightGBM's split into hour blocks is not shown to contribute.
Comparator
Set in advance: v3 itself, on identical hours, so the protocol's comparator is also the predecessor and the comparison is direct.
Result
Point-error score −5% [−6%, −4%] and interval score −5% [−6%, −4%], as a share of v3's score, with 95% confidence intervals of that ratio from the experiment's own bootstrap draws; development evidence, not a test on new data.
Not established
Separate hour-block models are not shown to help: the three-block against the pooled LightGBM shows no demonstrated joint preference. A gain over the August 2022 peak is not shown either: over those 17 days, v4's point error is higher than v3's.
Decision
In September 2026, v4 was adopted in research. It met the adoption rule set before the experiment against v3; the released model, v1, did not change. The three study arms were built only to separate the change into steps, and were never candidates for adoption.
v2 · blended LEAR, hour-aware intervals. In September 2026, v2 was adopted in research.
What changed
v3 kept v2's blend of two LEAR forecasts and its hour-aware intervals, and added three weather forecasts available before the auction: wind speed at 10 m and 100 m and solar radiation, each with a missing-data indicator.
Comparator
Set in advance: v2 itself, on identical hours, so the protocol's comparator is also the predecessor and the comparison is direct.
Result
Point-error score −12% [−16%, −9%] and interval score −14% [−17%, −11%], as a share of v2's score, with 95% confidence intervals; development evidence, not a test on new data.
Not established
The three weather inputs were tested together, so no single input's contribution is isolated; within the 2022 crisis period the point-error gain is not demonstrated on its own.
Decision
In September 2026, v3 was adopted in research. It met the joint improvement rule against v2 and both accuracy targets.
From v1 to v2: a blended linear model with hour-aware intervals
Development · post-selection
Predecessor
v1 · released LightGBM. In September 2026, v1 was released.
What changed
After v1 failed in the 2022 crisis, v2 replaced its LightGBM quantile model and fixed-window calibration with an equal blend of two LEAR forecasts refitted every day, and intervals built from recent errors by hour of the day.
Comparator
Set in advance: daily LEAR, the strongest benchmark, not the predecessor v1; a pooled-interval control isolated the interval method.
Result
Against daily LEAR: point-error score −2% [−4%, −1%] and interval score −4% [−5%, −2%], as a share of daily LEAR's score, with 95% confidence intervals; development evidence, not a test on new data.
Against the predecessor
No paired interval for v2 against v1 exists: the protocol did not test that pair. As descriptive context on the same development hours, without an interval, the point-error scores are 1.052 for v1 and 0.644 for v2, and the interval scores 0.986 and 0.616. v1's one-shot holdout covers a different window and evidence class and is not compared.
Not established
The gain over daily LEAR is not attributed to hour-aware intervals alone, the blend and the interval layer are not separated, and against its pooled-interval control there is no demonstrated joint preference.
Decision
In September 2026, v2 was adopted in research.v3 was built on it.
No policy met the criteria; the study informed v2's blend of two LEAR forecasts.
Question, result and evidence
Which forecasting approach copes best with the 2022 crisis?
Nine policies on the same hours, none meeting the product criteria. Normalized LEAR cut crisis-window MAE to 49.9 EUR/MWh, but daily LEAR kept the better error scores: 0.658 and 0.639 against 0.672 and 0.646.
Pooled LightGBM member for v4Not adoptedOctober 2026
Both candidates were decisively worse than v4 in one test period, with a 95% interval wholly above zero; the rule allowed none, so v4 was kept as it was.
Question, result and evidence
Can one pooled LightGBM, for all 24 hours, replace v4's three-block LightGBM member without losing accuracy against v4 itself?
Against v4, over the five test periods together, neither candidate's 95% interval lay wholly above zero: the averaged member's paired difference is +0.0011 [−0.0031, +0.0054] in the point-error score and +0.0010 [−0.0028, +0.0049] in the interval score; the pair's −0.0013 [−0.0036, +0.0016] and −0.0008 [−0.0030, +0.0022]. In fold 4, both were decisively worse: point error +0.3 [+0.1, +0.4] and +0.2 [+0.1, +0.3] EUR/MWh, interval score +0.2 [+0.1, +0.2] and +0.1 [+0.024, +0.2] EUR/MWh. Development evidence after selection, not a test on new data.
The improvement over v4 that the rule required was not shown, on either score.
Question, result and evidence
Does a distributional neural network, added to v4 as a one-third member, improve on v4 in point and interval accuracy together?
Against v4, the paired difference is +0.0064 [−0.0016, +0.0149] in the point-error score and +0.0071 [−0.0009, +0.0138] in the interval score, with 95% intervals; in fold 3, the 2022 crisis, its point error was decisively worse, +1.9 [+0.3, +4.4] EUR/MWh. Development evidence after selection, not a test on new data.
Does a distributional neural network that forecasts the whole next day at once, added to v4 with a fixed one-sixth weight, improve both of its error scores?
v5 adds DDNN-2, a distributional neural network: it forecasts the 24 hours of the next day together, each as a full probability distribution, from the same information as v4. It takes half of the third that v4's LightGBM held; the two LEAR forecasts keep two thirds, and the hour-aware intervals are re-estimated on the new forecast's own errors.
v4
two LEAR forecasts, two thirds of the weight
a LightGBM forecaster, one third
hour-aware intervals from recent errors
+
added in v5
DDNN-2: a distributional neural network that forecasts the whole next day at once
half of the LightGBM's third: one sixth of the weight
same information; intervals re-estimated on the new errors
Against v4, the point-error score changed by −2% [−3%, −2%] and the interval score by −2% [−3%, −2%], each as a share of the comparator's score, with 95% confidence intervals; at the rule's 97.5% decision level the intervals are [−3%, −2%] and [−3%, −2%].
Difference in error score, v5 − v4 · paired intervals at 95% and at the rule's 97.5% decision level · identical hours · development, post-selection
All five conditions of the rule set in advance held: at the rule's 97.5% level both paired differences' intervals lie below zero; v5 passed the six screening criteria set before the experiments (the two accuracy targets, interval coverage in each test period, the August 2022 peak against the benchmarks, each test period against the daily benchmarks, and complete forecasts); the evaluation was complete and valid; no test period's paired interval lies wholly above zero; and both scores improved on v4's by more than the practical size set in advance, half a percent.
In October 2026, v5 was adopted in research. It met the adoption rule set before the experiment against v4; the released model, v1, did not change. The two study arms, DDNN-2 alone and v3 plus a DDNN-2 member, were built only to show where the change came from, and were never candidates for adoption.
The blend.v5's central forecast gives v4's two LEAR forecasts, bit for bit, two thirds of the weight, v4's LightGBM one sixth and DDNN-2 one sixth. The weights were fixed before any fit and never tuned. The hour-aware intervals keep v4's recipe, re-estimated on v5's own errors.
DDNN-2. A feed-forward network written in NumPy, on one row per delivery day, with a Johnson SU distribution for each of the 24 hours, and exactly v4's information. Each member trains on up to two years before the forecast day, from 2019-01-01 at the earliest, minus a random fifth of whole weeks held out to stop its training; four configurations, chosen per test period by a search on training data only, times two seeds, are combined by the median of their forecasts.
The study arms: DDNN-2 alone, and DDNN-2 in the LightGBM's place
The study arms.DDNN-2 alone, against v4: −11% [−14%, −7%] on the point-error score and −16% [−19%, −12%] on the interval score, as shares of v4's score: descriptive, never eligible.
v3 plus a DDNN-2 member, DDNN-2 in the LightGBM's place, scored better than v5 itself: v5 minus it is +0.0067 [+0.0033, +0.0097] on the point-error score and +0.0062 [+0.0031, +0.0089] on the interval score, both intervals wholly above zero: with DDNN-2 present, adding the LightGBM member made both scores worse. It too was a study arm, never eligible, so the rule's candidate was v5.
Per-period consistency, including absolute errors
Did v5 help in every test period?
Every test period favours v5: in each, the paired 95% intervals of both errors lie below zero. The largest change is in fold 3, the 2022 crisis: point error −1.5 [−2.3, −0.7] EUR/MWh.
Row
Point error, v5 − v4 EUR/MWh, paired mean daily loss difference
Interval score, v5 − v4 EUR/MWh, paired mean daily loss difference
Mean absolute error per test period: v5 4.8–14.4 EUR/MWh in the four ordinary periods and 45.5 EUR/MWh in the 2022 crisis period, where v4 had 47.0.
The stress period: the 2022 crisis
The August 2022 peak
The August 2022 peak, 17 days inside the crisis period: v5's point error is 46.9 EUR/MWh against v4's 50.1. Descriptive, on a small sample: no inference is drawn.
Coverage and interval width
Did the intervals get more reliable, or only wider?
v5's pooled intervals are narrower than v4's at every level: mean widths 25.3, 54.7 and 100.2 EUR/MWh at the 50%, 80% and 95% levels, against 26.3, 56.1 and 101.9. Its pooled coverage is 0.4916, 0.7923 and 0.9395, against v4's 0.4961, 0.7905 and 0.9392.
Protocol and review
The rule, set on 2026-10-05 and frozen in the protocol before any of its fits: v5 would be adopted only if, against v4, the paired differences' intervals at 97.5% lay below zero for the interval score and at or below zero for the point-error score; it met the six original screening criteria; the evaluation was complete and valid; no test period was decisively worse, that is, none had a 95% paired interval wholly above zero; and both point estimates improved on v4's score by at least half a percent. The 97.5% level splits a 5% error rate across the two attempts the plan allowed; one ran. All five conditions held.
As a diagnostic, v5 meets all six original screening criteria, as v4, DDNN-2 alone and v3 plus a DDNN-2 member do. This is a development diagnostic, not a product qualification.
All 80 frozen causal and integrity controls passed, each negative check paired with a positive one: masking the delivery day and future inputs changed the forecast by exactly zero, and a day-before price change, weather permutations and held-out weeks moved DDNN-2.
No delivery-day outcome is used: the leakage test at every origin
No outcome from the delivery day on is used, at any origin. With every such outcome destroyed, the frozen network refitted at all 636 warm-up and evaluation origins reproduced the committed forecasts bit for bit at 636 of them, over 10,747 hours; a change to the previous day's prices moved them, and a planted one-day leak was caught in every test period. The input vintages are inherited from the earlier experiments, not re-tested here.
Adopted in researchSeptember 2026
v4 · three-block LightGBM added
v3 plus a three-block LightGBM member
Development · post-selection
The question
Does adding a nonlinear forecaster, LightGBM, to v3 improve both of its error scores?
v4 adds a LightGBM forecaster, a nonlinear model built from decision trees, with the same information as v3, to its blend of two linear forecasts, and re-estimates the hour-aware intervals on the new forecast's own errors.
v3
two LEAR forecasts, blended equally
hour-aware intervals from recent errors
+
added in v4
a LightGBM forecaster: nonlinear, built from decision trees
one third of the blend's weight, the LEAR pair two thirds
same information; intervals re-estimated on the new errors
Against v3, the point-error score changed by −5% [−6%, −4%] and the interval score by −5% [−6%, −4%], each as a share of the comparator's score, with 95% confidence intervals.
All four conditions of the rule set in advance held: both paired differences' 95% confidence intervals lie below zero; v4 passed the six screening criteria set before the experiments (the two accuracy targets, interval coverage in each test period, the August 2022 peak against the benchmarks, each test period against the daily benchmarks, and complete forecasts); the evaluation was complete and valid; and no test period's paired interval lies wholly above zero.
In September 2026, v4 was adopted in research. It met the adoption rule set before the experiment against v3; the released model, v1, did not change. The three study arms were built only to separate the change into steps, and were never candidates for adoption.
The blend.v4's central forecast gives v3's two LEAR forecasts, bit for bit, two thirds of the weight, and the mean of the raw and normalized three-block LightGBM forecasts one third. The weights were fixed before any result and never tuned. The hour-aware intervals keep v3's recipe, re-estimated on v4's own errors.
The same information as v3. Every LightGBM arm uses the model comparison study's LightGBM inputs plus the three weather forecasts and their missing-data indicators. The blocks are the night, the solar hours and the shoulder and peak hours, in local time. At every forecast origin each block picks one of 4 model sizes on the training window's last 28 days, by MAE, then refits.
The ladder. Three study arms separate the change into steps, one at a time. Daily LightGBM to the pooled LightGBM adds weather, bundled with the training-only choice of model size and the missing-input rule, so that step is not a weather effect on its own. The pooled to the three-block LightGBM adds the block split alone: the same rows, target, inputs, size grid, selection rule and interval recipe. v4 then blends the block models into v3.
The ladder: what each step added
View values
Row
Point-error score normalized score difference
Interval score normalized score difference
Pooled − daily LightGBM: weather, bundled with size selection and the missing-input rule
The block split (three-block minus pooled LightGBM): point-error score +0.0050 [−0.0084, +0.0142], interval score +0.0063 [−0.0050, +0.0135]: no demonstrated joint preference. It was tested on the raw price only, with one seed.
Each study arm against v3
No study arm on its own shows a demonstrated joint preference over v3: each arm's intervals against v3 span zero, except the three-block LightGBM's interval score, whose interval lies wholly above zero, worse: +0.0218 [+0.0004, +0.0450].
Every test period's point estimate favours v4, and no period's paired interval lies wholly above zero. In fold 3, the 2022 crisis, both intervals cross zero: point error −1.0 [−2.6, +0.6] EUR/MWh.
View values
Row
Point error, v4 − v3 EUR/MWh, paired mean daily loss difference
Interval score, v4 − v3 EUR/MWh, paired mean daily loss difference
Mean absolute error per test period: v4 4.9–15.0 EUR/MWh in the four ordinary periods and 47.0 EUR/MWh in the 2022 crisis period, where v3 had 48.0.
View values
MAE by fold, EUR/MWh
Row
v3
v4
Fold 1
5.253924602538004
4.930960509642647
Fold 2
8.380965193656028
7.713016585113509
Fold 3
48.035385691213605
47.02773874828664
Fold 4
14.636119694466323
13.772374557966877
Fold 5
15.640309999597784
14.950077113978647
WIS by fold, EUR/MWh
Row
v3
v4
Fold 1
3.208235420275873
3.017880239754553
Fold 2
5.13448166723657
4.786404255399961
Fold 3
27.791822265728936
27.01096309573922
Fold 4
8.517781550107456
8.046634538327039
Fold 5
9.052235002444618
8.685815912228607
The stress period: the 2022 crisis
The August 2022 peak: where v4's point error is higher
The protocol's stress period is fold 3, the 2022 crisis. Its peak, delivery 2022-08-15 to 2022-08-31, is the 17 days charted here; their figures are for those days only, not the whole period.
The August 2022 peak, 17 days inside the crisis period: v4's point error is higher than v3's, MAE 50.1 against 47.5 EUR/MWh, and so is its interval score, 28.0 against 27.3 EUR/MWh; 382 against 383 of its 408 hours fell inside the 95% interval. Descriptive, on a small sample: no inference is drawn.
View values
Row
MAE, EUR/MWh EUR/MWh
WIS, EUR/MWh EUR/MWh
v2
52.5098015356535
30.102050231473346
v3
47.52143732035691
27.308312686097484
v4
50.085106925008226
28.02151976713374
Coverage and interval width
Did the intervals get more reliable, or only wider?
v4's pooled intervals are narrower than v3's at every level. Its pooled coverage is 0.4961, 0.7905 and 0.9392 at the 50%, 80% and 95% levels, against v3's 0.4937, 0.7914 and 0.9377. No hour or block effect is claimed.
View values
Coverage (fraction)
Row
v3
v4
50% interval
0.4937191774448683
0.496138457243882
80% interval
0.7913836419465897
0.7904531497161998
95% interval
0.9376570205638783
0.9392388573555411
Mean interval width, EUR/MWh
Row
v3
v4
50% interval
27.142760138629214
26.328242357115077
80% interval
58.80676873570269
56.139984005350115
95% interval
106.24360374406864
101.91577182718189
Protocol and review
The rule, set on 2026-09-29 and frozen in the protocol before any fit: v4 would be adopted only if, against v3, the interval score's paired difference had its upper 95% confidence limit below zero and the point-error score's at or below zero; it met the six original screening criteria; the evaluation was complete and valid; and no test period was decisively worse, that is, none had a paired interval wholly above zero. All four held.
As a diagnostic, v4 meets all six original screening criteria, as v3 does. Among the study arms, the normalized three-block LightGBM meets them; the pooled LightGBM misses criterion 4 (the August 2022 peak against the benchmarks), and the three-block LightGBM misses criteria 4 and 5 (each test period against the daily benchmarks). This is a development diagnostic, not a product qualification.
Run through this experiment's code, v3 reproduced its accepted forecasts bit for bit on all 10,747 hours, so the comparison is with v3 itself.
All 49 frozen causal and integrity controls passed: masking the delivery day or future weather changed the forecast by exactly zero, a day-before price change moved every arm, and the pooled and block arms saw the same rows.
22,260 LightGBM fits in the main run. The time of one daily cycle was measured as a diagnostic, not as a qualification for daily operation.
Adopted in researchSeptember 2026
v3 · weather features
v2 plus three weather forecast inputs
Development · post-selection
The question
Do weather forecasts available before the auction improve v2?
v3 adds weather forecasts available before the auction to v2's inputs; the modelling setup is otherwise unchanged.
v2 inputs
price history
load forecast
calendar
+
added in v3
wind speed at 10 m
wind speed at 100 m
solar radiation
each with a missing-data indicator
Weather inputs lowered both error scores against v2.
Both confidence intervals lie below zero and every test period favours v3; its prediction intervals are narrower in every period, with pooled 95% coverage slightly lower, 0.9377 against 0.9389.
Operational NCEP GFS 0.25°, the 00 UTC run of the day before delivery. Wind components are interpolated to each delivery hour; wind speed is computed per grid cell before a cos(latitude)-weighted average over the fixed box 47–55.25°N, 5.5–15.5°E; radiation is de-averaged from the forecast's running means. Each feature has a missing indicator. A regional weather proxy: not the DE-LU zone's exact outline and not a generation forecast.
Only delivery 2019-01-01 has no weather, structurally: its run would be 2018-12-31, before the input floor. Nothing was imputed from a failed retrieval.
Availability of each run before the D−1 11:00 UTC forecast origin is reconstructed from dated NCEP production records with an assumed dissemination lag. It is not a per-day delivery guarantee.
Per-period consistency, including absolute errors
Did weather help in every test period?
View values
Row
Point error, v3 − v2 EUR/MWh, paired mean daily loss difference
Interval score, v3 − v2 EUR/MWh, paired mean daily loss difference
Absolute errors per test period, for v1, v2 and v3
View values
MAE by fold, EUR/MWh
Row
v1
v2
v3
Fold 1
6.6128886954178006
6.310802110246006
5.253924602538004
Fold 2
20.03914109230744
9.457089743262161
8.380965193656028
Fold 3
140.9928521771404
51.213622921785436
48.035385691213605
Fold 4
20.06527158805811
16.1242511972614
14.636119694466323
Fold 5
23.168349101365912
18.730145716433146
15.640309999597784
WIS by fold, EUR/MWh
Row
v1
v2
v3
Fold 1
4.0026294086922105
3.855935752625375
3.208235420275873
Fold 2
10.824925546830002
5.981432938667355
5.13448166723657
Fold 3
87.93534942326052
30.22126634528976
27.791822265728936
Fold 4
11.370950921093906
9.54679415511936
8.517781550107456
Fold 5
13.470357590829112
11.026057654908808
9.052235002444618
Errors by hour of the day, v2 against v3
Descriptive only: no hour or block effect is claimed.
View values
MAE by local hour, EUR/MWh
Hour
Fold 1v2
Fold 1v3
Fold 2v2
Fold 2v3
Fold 3v2
Fold 3v3
Fold 4v2
Fold 4v3
Fold 5v2
Fold 5v3
0
2.6686801494807924
2.5767688986937745
4.736417350478568
4.618055487701848
23.68954960489116
23.923719695100484
6.603110313748953
7.079885375486371
7.185149904053438
6.842553634286969
1
2.902094238117903
2.798643455225212
5.708219294708601
5.604733794049524
26.496267494374056
26.574240168599484
6.934189358912006
7.317260617798622
7.264586251041968
6.881701046426991
2
3.0764977906566324
2.9826907355197596
6.3723526263010495
6.319339636602996
30.919372633359174
30.73403621983847
7.751303044093423
8.064453389839377
7.321566476507715
6.846153447508501
3
3.0739327267863
2.989659600701974
6.785113145618487
6.804627077805382
35.23576295041502
34.136685508059806
8.601792690076799
8.631016747984388
8.490598513406736
8.016810354851694
4
3.3563863116201924
3.0782443082541153
7.101781194265057
7.150816909441624
33.86621993213467
35.007114730923426
8.887572628034865
8.910108640680432
9.192628511000494
8.703148712752807
5
3.6744599090557006
3.3078545617107946
7.083510882536434
6.742938722200108
35.646506147278494
35.56097906679812
8.9687665967806
8.863389654660516
10.011774484748477
9.34928640134387
6
4.281733854596694
3.7605329742496325
7.668169383796363
7.554087786312033
40.86217330776495
40.320384274942874
11.192506811334518
11.100396074751188
14.265235789327066
13.71752932524898
7
4.949436207236711
4.618963236925001
9.664868524296546
9.53737097113295
42.31691261969251
39.90197640319252
13.705851996639437
13.861480087995485
18.70434353902439
18.28556128431616
8
6.201260035007014
5.6178603006304435
11.75065344232724
11.166917133970504
44.61921513326294
43.9779732630827
14.0045291946085
15.524821572359196
21.171073652276405
19.429740524305636
9
6.577480668072571
5.703211487835789
10.18199523105575
8.782145811277895
48.62665035544608
45.9995960105642
14.56282535795011
15.437823863362333
21.089625806533775
18.868546883911172
10
7.198401204831431
5.785531655646092
10.343532640762259
8.657415741469789
58.9053155520269
54.67515314280536
17.59554322452575
16.952423852874485
22.89014740184511
20.254698996139428
11
7.864262609694391
6.494082087101629
10.729218665584934
9.23742399909659
62.566199075750895
57.923480707080245
19.899481014706446
17.879524957520793
24.40105546457575
19.56871033094505
12
8.605467027636493
6.726226348942776
11.876849173991792
10.379193553249722
67.30803745249746
61.02705816052806
20.970549625320658
19.178653448005875
24.8746832516956
18.68461655147161
13
9.435905866270156
7.588240271463186
14.073619170416181
12.290010608899905
68.25454580170204
63.17494148300592
22.995816939371217
21.604682498098402
25.84153463774318
19.1520548721402
14
10.471356616495282
7.9203275490299925
16.12995040465418
13.876045358198548
64.97074849089528
61.53441182325574
22.894984466627612
21.57046350405213
24.36237302890797
19.292915239570004
15
9.65419835710871
7.132715845085785
15.323672493686901
13.370660104810753
69.15320292350431
62.946517955380244
21.16236493079366
17.801913577559823
25.106356338235614
19.096971769737237
16
8.559783781372094
6.284822110229484
13.260170088329794
11.50391946553396
70.04844382768978
62.89421155327398
19.329766489181246
17.631647633186244
24.463417903345835
18.496417649102433
17
7.523853918960889
5.53471749377755
10.144347807641175
8.387146692284475
65.31890731522691
59.556134844765985
20.78744466378316
17.16501867137699
26.426281383009957
20.934182056833567
18
7.767627058198226
6.232701954186529
8.461137385519386
7.204670020778824
55.40665394841146
52.33307343892348
17.92122069485654
13.442169104112525
28.370239869668453
22.43198404765202
19
11.43430237549882
10.035219588130774
8.503095068824143
7.50667118473512
61.24587005284969
56.64504356616685
19.578584578052926
16.903504145843048
25.123412098934143
21.365935817806477
20
8.340774455347146
7.1855674353611665
8.613091858380093
7.183608808199946
57.04953652261615
52.83117511283188
32.09924186727043
24.801254694307804
20.136061059986957
16.771334919236967
21
5.105592475990739
4.460879696069
7.476684356576993
5.517322440249132
54.53461096769752
50.14203810237024
25.92766042889145
21.576171911281325
17.080123781639074
13.793541694964514
22
4.460963946641686
3.7090314337563193
6.79132359564331
4.966670162537846
55.01337739694193
50.383832766501776
14.249926890109716
11.727020531900274
16.5191712091917
13.222942357264024
23
4.274799061227575
3.5696974323853095
8.156105196041533
6.758466226571282
57.072870616421135
50.645478591134754
10.35699492860362
8.241788112154149
18.725008871476806
14.969250670215398
The stress period: the 2022 crisis
What happened in the 2022 crisis window
The protocol's stress period is fold 3, the 2022 crisis. Its peak, delivery 2022-08-15 to 2022-08-31, is the 17 days charted here; their figures are for those days only, not the whole period. The dashed line marks 95% of the window's hours, the nominal target: a computed reference, not an observed count; there is no coverage guarantee.
During the 2022 price peak, a 17-day window inside the crisis period, MAE went from 52.5 to 47.5 EUR/MWh. Descriptive only; the hours inside the interval are in the chart.
View values
Row
MAE, EUR/MWh EUR/MWh
Hours inside the 95% interval count
v1
275.25954291591773
79
Normalized LEAR (study)
49.876699068332044
378
v2
52.5098015356535
377.0
v3
47.52143732035691
383.0
Coverage and interval width
Did the intervals get more reliable, or only wider?
The prediction intervals are narrower in every period, while pooled 95% coverage is slightly lower, 0.9377 against 0.9389 for v2.
View values
Coverage (fraction)
Row
v1
v2
v3
50% interval
0.36447380664371454
0.4993021308272076
0.4937191774448683
80% interval
0.6280822555131664
0.7906392481622778
0.7913836419465897
95% interval
0.8451660928631246
0.9388666604633852
0.9376570205638783
Mean interval width, EUR/MWh
Row
v1
v2
v3
50% interval
41.53164224298361
29.459177719100104
27.142760138629214
80% interval
75.46902836943482
65.77562220812527
58.80676873570269
95% interval
130.86485779642376
124.63573426269241
106.24360374406864
Protocol and review
As a diagnostic, v3 is the first policy evaluated against the original criteria to meet all six; v2 misses criteria 1 and 2. This is a development diagnostic, not a product qualification.
Frozen causal controls ran on 2020-07-01 and 2025-05-01: masking the delivery day or the future changed the forecast by exactly zero, an available day-before price change moved it, and future weather changed nothing.
One frozen check could not fail: multiplying all weather by a constant is undone by the training-only scaler. It was kept as an invariance check, and two controls that can fail were added, a shift of the target day's weather and a permutation of the training weather. Both passed (repair r13).
The first independent Integration review within this project's process failed: one frozen input's recorded hash described its bytes with Windows line endings, so the reproduction commands refused to run in a clean checkout. The bytes were restored to the recorded identity (repair r14) and a fresh Integration review passed.
The v3 research model's weather data is derived from NCEP GFS 0.25° (NOAA/NWS/NCEP) via NCAR GDEX d084001 (doi:10.5065/D65D8PWK) and NOAA Open Data Dissemination on AWS, modified and aggregated by this project; it is not an official NOAA product.
Adopted in researchSeptember 2026
v2 · blended LEAR, hour-aware intervals
Two LEAR forecasts blended, hour-aware intervals
Development · post-selection
The question
Could a different forecasting model repair v1's failure in the 2022 crisis?
v1 substantially underestimated prices during the 2022 crisis. Recalibrating its intervals improved coverage, but the improvement was insufficient.
v2 combines two LEAR forecasts (LEAR: a regularized linear model fitted for each hour) with intervals that account for the hour of the day. A control uses the same blend with pooled intervals, so the interval methods can be compared.
v1
one LightGBM quantile model
conformal intervals from a fixed calibration window
Against daily LEAR both confidence intervals lie below zero; against its pooled-interval control only the interval score improved, and the point-error interval ends just above zero, at +0.0000039: not equivalence.
What the pooled-interval control established, and the scores against the targets
The pooled-interval control against daily LEAR does not meet the rule: its interval-score interval, [−0.0201, +0.0007], spans zero.
Both v2 arms miss criteria 1–2, the point-error and interval-score thresholds, and meet criteria 3–6.
View values
Row
Point-error score ratio to the similar-day naive (B0)
Interval score ratio to the similar-day naive (B0)
v2 · blended LEAR, hour-aware intervals
0.6440721386286878
0.6160299894915677
Pooled-interval control
0.6457675850846845
0.6287217727210093
Daily LEAR (benchmark)
0.6578109123747921
0.6389910407326591
Normalized LEAR (study)
0.6722908121370033
0.646015091607363
Daily LightGBM (benchmark)
0.7841363977682992
0.7399052224715813
v1 · released LightGBM (development replay)
1.051845134839199
0.9856366964174134
ReleasedSeptember 2026
v1 · released LightGBM
Nine-quantile LightGBM with calibrated intervals
A LightGBM ensemble forecasting nine quantiles for every hour, calibrated into 50 / 80 / 95% prediction intervals, and shipped exactly as it was evaluated.
The one-shot holdout Confirmatory-style, not power-qualified
On its pre-specified 90-day holdout, v1 beat the similar-day naive: MAE 25.9 against 27.8 EUR/MWh, and mean pinball loss 6.7 against 13.9 EUR/MWh. A one-sided Diebold–Mariano test on the daily pinball-loss vectors gave p < 10⁻⁶; it tests the probabilistic forecast, not the MAE difference.
Pre-specified one-shot holdout DM test on a fixed 90-day window — confirmatory-style, not power-qualified.
View values
The one-shot holdout, exact values
Measure
v1
Similar-day naive
MAE, EUR/MWh
25.9078
27.7578
Mean pinball loss, EUR/MWh
6.7083
13.8789
Diebold–Mariano statistic
-8.6798
Diebold–Mariano p-value
1.98e-18
Coverage at 50 / 80 / 95%
0.4407 / 0.7593 / 0.9398
Where it falls short. On the development folds its point error is 28.58% worse than the naive's, pooled over all days (a one-sided test for an improvement gives p = 0.948: no evidence of an advantage). Over the August 2022 peak weeks its 95% interval covered 0.194 of outcomes.
Why: its crisis errors were about price level, not shape. On the crisis window the bias was −269.4 EUR/MWh, and the daily mean-level error was 269.4 EUR/MWh against a within-day shape error of 66.9 EUR/MWh.
The original v1 report (published
2026-09-15), preserved
Archived v1 report, preserved as published. Its container deployment
instructions are historical: the current demo runs in your browser, and this report needs no additional
requests after loading. Current instructions ·
Back to the model comparison
DE-LU day-ahead price forecasting
A probabilistic forecast of the next delivery day's hourly German–Luxembourg
day-ahead electricity price, with calibrated 50 / 80 / 95 % prediction intervals.
Strict-gate by construction: the shipped model uses no input published after the
12:00 CET day-ahead auction gate. Evaluated exactly once, on a holdout pinned before
development.
The shipped model is exactly the model the holdout evaluated: there is no retrain and no re-tune after the result was opened.
This page is static. It renders from files committed in the repository and makes
no runtime call to any external service — no analytics, no fonts, no CDN, no call
to the interactive Space. Everything below, including the chart, works with the network off.
The four cutoffs, stated separately because they are four different dates
Cutoff
Value
snapshot_cutoff
2026-09-06
raw_model_fit_cutoff
2026-04-07
final_calibration_window
2026-04-09..2026-06-07
holdout_window
2026-06-09..2026-09-06
The raw-model fit cutoff precedes the snapshot cutoff by 152 delivery
days (1 + 60 + 1 + 90). That is what shipping the evaluated model costs, stated plainly
rather than apologised for; the deployed demo applies a frozen model and there is
no scheduled refresh.
1 · The data
One committed Parquet snapshot: 67,343 continuous UTC-indexed hourly rows,
delivery 2019-01-01 through 2026-09-06, carrying the day-ahead price (A44), the
day-ahead load forecast (A65/A01), the benchmark-only day-ahead VRE forecast (A69) and aggregate
actual generation (A75). Quarter-hourly feeds aggregate to hours from exactly four complete bins,
never a partial one; the 2025-10-01 switch to a 15-minute price product is handled as the mean of
four quarter-hour prices. The snapshot is pinned by SHA-256
7dd2dc73407706ca6bd3c1ad51d201ac0de35eec5ca129ce320cca639f697f00 and ships inside the container, so the demo never pulls a feed
while you use it.
Data: ENTSO-E Transparency Platform; Bundesnetzagentur | SMARD.de — CC BY 4.0.
2 · Three regimes in one series
The annual mean price swept from about €30/MWh in 2020 to about
€235/MWh in 2022 and back to about €89/MWh in 2025 — a crisis
and a normalization inside one training set. Underneath it, the solar build-out pushed
negative-price hours from 139 (2021) → 69 (2022) →
301 (2023) → 457 (2024) → 576 (2025).
The 2025 count is measured on this snapshot's hourly series. From 2025-10-01 an hour is the mean
of four quarter-hour prices, so any hour-level tally depends on that averaging choice and a
quarter-hour tally differs. The price is routinely negative and has touched the −500 €/MWh
floor, which is why no log or Box-Cox transform is applied to the target: pinball loss, the CQR
score and monotone rearrangement are all location-equivariant and behave correctly through zero.
Two consequences the whole design follows from. Evaluation is stratified across all three
regimes rather than collapsed into a recent tail. And exchangeability — the assumption CQR's
coverage guarantee rests on — is mildly violated by construction, which is disclosed rather than
assumed away.
3 · Feature catalog — frozen before fitting
Two catalogs were frozen before any model ran. base: calendar and regime features,
the A65 load forecast, calendar-day-matched price lags at D−1 / D−2 / D−7, and D-1-frozen rolling
price statistics. base + residual_load_proxy adds exactly one domain feature — a
residual-load proxy built from a 42-complete-delivery-day trailing mean of actual wind and solar
generation ending at D−2, which clears the 12:00 gate with a wide margin.
Pooled observation-weighted raw-head mean pinball loss over the five pinned development
folds, matched on rows, seed, hyperparameters and (zero) tuning budget, unrounded as stored:
Arm
Pooled raw-head mean pinball loss
base
13.015841509664993
base + residual_load_proxy
13.064197422052183
Percentage difference (augmented vs base): +0.371516%.
Selected catalog: base. The rule was fixed before
fitting — the augmented catalog ships only if its unrounded stored pooled loss is lower. It is not,
so the domain feature did not earn its place and the champion ships strict-gate. That is a
reportable result, not a failure, and the comparison was designed so that "no improvement" was a
publishable answer.
3b · Why these seasonal features? (spectral view)
Welch periodogram — Hann window, 50 % overlap.Per-regime overlay.ACF cross-check, time domain.
The Welch periodogram shows pronounced price energy at the 24-hour and 168-hour cycles with a
visible 12-hour harmonic, which is what justifies the catalog's hour-of-day and day-of-week
structure rather than an assumption that electricity "should" be daily-seasonal. The per-regime
overlay shows an elevated broadband floor and altered seasonal amplitude during the crisis: the
spectrum itself is not stationary across regimes, which supports reporting performance across all
three rather than collapsing them into one recent-tail summary. The ACF cross-check confirms the
same daily and weekly recurrence in the time domain, so the conclusion does not rest on a single
estimator. These are price diagnostics only — they neither validate nor justify retaining
residual_load_proxy, which the frozen two-arm comparison alone decides.
4 · Validation design
Expanding-window walk-forward CV with five development folds, each carrying a
one-complete-delivery-day embargo — 23, 24 or 25 rows as DST requires, never
hard-coded to 24 — and a 90-day evaluation block, anchored so the blocks span pre-crisis, the
crisis peak and the post-crisis negative-price era. Five tail partitions were pinned before any
EDA: fold 5, Embargo A, a 60-day final-calibration slice, Embargo B, and the final
90-day holdout.
The target is D+1, anchored at the 12:00 CET gate, because that is what the
day-ahead auction clears. That makes the availability rule stricter than it first looks: the model
emits the whole next-day curve at once, so a row-wise t−1 boundary would admit the
target curve into its own features. The binding rule is per delivery day — for a forecast of day
D, every price-derived feature may consume only prices whose delivery date is earlier
than D, rolling statistics are frozen at the D−1 boundary for the whole curve, and an
unavailable or ambiguous calendar-day match fails closed to null rather than reaching for a
same-day price.
Strict gate. The day-ahead wind and solar forecast (A69) is not published until
after the 12:00 gate, so rather than use it as-archived and disclose that, this project excludes it
from the shipped model entirely. Delivery-day A69 is rejected by the champion's runtime schema.
What the exclusion costs is then measured, in section 5, instead of being waved at.
Two disclosed assumptions, stated as assumptions rather than as measurements:
A65/A01 is pre-gate by explicit assumption, not by measurement: both the existence of the delivery-day load forecast before the 12:00 CET gate and its equality to the archived vector used here are assumed, and the regulatory update provision permits later revisions.
A75 aggregate actual generation is used at its current archived values, which may differ from the values visible in real time despite the D-2 boundary.
5 · Results
Development folds — descriptive post-selection evidence
evidence_class = development_post_selection. These folds were also used for
catalog selection and model development, so their p-values are descriptive, never confirmatory, and
no result gates anything.
Analysis
Comparator
Statistic
p-value
N days
Probabilistic daily-vector pinball
similar-day naive
-2.5517
0.00536
448
Point median absolute error
similar-day naive
1.6228
0.948
448
The point-accuracy test does not merely fail to show an advantage — it shows a
deficit. The test is one-sided (reject if DM < −1.645) and the statistic is
+1.6228, p = 0.948: the champion's median is
28.58% worse against the naive over the development folds. Reported here rather than
omitted or reframed. The
probabilistic win is broad and the point-accuracy win is not: the pooled MAE gap comes almost
entirely from the crisis-peak fold. The mechanism is measured, not assumed: the model did see the
crisis (that fold's training included 5,784 crisis hours at a €173 mean and a €700 maximum), but
61.3% of the evaluation block sits above the 99th percentile of everything it ever saw while
only 2.45% exceeds its maximum — shrinkage toward the training level, not an extrapolation
wall. Persistence has no training distribution and carries the level for free. Note also that
a point forecast scored on pinball is structurally disadvantaged against a nine-quantile model, so
the pinball margin is largely the value of having a predictive distribution.
model
stage
n_obs
mae_p50
mean_pinball
crossing_violations
coverage_50
coverage_80
coverage_95
base
raw
10747
41.76
13.02
10158
0.2257
0.4322
0.6705
base
post_cqr
10747
41.76
11.46
4412
0.3642
0.628
0.8416
base
final
10747
41.74
11.44
0
0.3645
0.6281
0.8452
base_plus_residual_load_proxy
raw
10747
41.17
13.06
10563
0.2004
0.4056
0.6568
base_plus_residual_load_proxy
post_cqr
10747
41.17
11.57
3899
0.3151
0.5949
0.8377
base_plus_residual_load_proxy
final
10747
40.94
11.55
0
0.318
0.5941
0.8401
similar_day_naive
point
10747
32.45
16.23
0
seasonal_naive_168h
point
10747
37.63
18.81
0
ridge
point
10747
28.11
14.05
0
The one-shot holdout — opened exactly once
After the catalog, hyperparameters and every implementation choice were frozen, the nine raw
heads were fit on every eligible row strictly before Embargo A, the four CQR thresholds were
estimated once on the 60-day calibration slice, isotonic-last was attached, and that complete
artifact was frozen. Only then was the holdout opened.
Metric
Champion
Similar-day naive
Difference
MAE (EUR/MWh)
25.9078
27.7578
-6.66%
Mean pinball loss
6.7083
13.8789
-51.67%
Final empirical coverage over 2,160 rows on 90 delivery days:
0.4407 / 0.7593 /
0.9398 at the 50 / 80 / 95 % nominal levels.
Probabilistic daily-vector Diebold–Mariano against the similar-day naive: statistic
-8.6798, p-value 1.98e-18,
standardized effect size -1.0478.
Pre-specified one-shot holdout DM test on a fixed 90-day window — confirmatory-style, not power-qualified.
The result supports a probabilistic-skill claim against the similar-day naive on this window and
nothing wider. It is not a gate, and no superiority claim beyond what it supports is made anywhere
on this page.
What the post-gate forecast would have been worth
A controlled ablation on raw quantile heads, neither arm calibrated, identical in every respect
except the added delivery-day A69 forecast and its named derivatives: pooled mean pinball loss
moves from 13.015841509664993 to 10.478714632475889 —
-19.4926%. So the project ships an honest model and puts a number on the
value of the information it chose not to use.
This uncalibrated raw-head comparison isolates the information content of post-gate A69. It makes no claim about calibrated interval quality and does not make A69 available at the forecast gate.
6–7 · Explainability
SHAP summary, p50 head, scored out of sample on fold 5's evaluation block.
price_lag_24hprice_lag_168h
Top 10 by mean |SHAP|
feature
mean_abs_shap
rank
price_lag_24h
15.4
1
price_lag_168h
5.413
2
price_roll_q50_168h
5.126
3
day_of_week
4.51
4
price_lag_48h
4.206
5
load_forecast_mw
3.23
6
day_type
3.164
7
local_hour
3.064
8
price_roll_mean_720h
2.984
9
price_roll_q95_168h
2.719
10
Top 10 by permutation importance
feature
importance_mean_mae_increase
importance_std
rank
price_lag_24h
8.261
0.3025
1
price_lag_48h
1.803
0.1463
2
day_of_week
1.444
0.1578
3
local_hour
1.129
0.09177
4
price_lag_168h
1.097
0.08431
5
load_forecast_day_mean_mw
0.4105
0.07628
6
load_forecast_mw
0.3856
0.07416
7
day_type
0.3365
0.06551
8
price_roll_std_720h
0.1886
0.04192
9
price_roll_q50_168h
0.1511
0.05027
10
SHAP on the p50 head explains central tendency, not interval width — width is driven by the
inter-quantile spread and the CQR shift. Neither ranking is the incremental-value test; that is the
frozen two-arm comparison in section 3, which selected base.
The two rankings agree at the top and disagree in the middle, which is the ordinary difference
between attribution in expectation and degradation under shuffling.
8 · Regime-stratified error
n_obs on every row. Thin subsets carry day-block bootstrap 95 % confidence
intervals — days, not hours, because the rows of one delivery day share a day effect — and are read
qualitatively.
stratum
n_obs
n_days
mae
mean_pinball
coverage_50
coverage_80
coverage_95
read
mae_ci95_low
mae_ci95_high
pinball_ci95_low
pinball_ci95_high
all validation folds
10747
448
41.74
11.44
0.3645
0.6281
0.8452
quantitative
Dunkelflaute days (A69 stratum)
648
27
38.63
10.49
0.2577
0.4938
0.7716
qualitative (thin subset)
19.9
61.38
5.036
17.11
non-flag days
10099
421
41.94
11.51
0.3713
0.6367
0.8499
quantitative
pre-crisis (< 2021-09-01)
4319
180
13.32
3.348
0.3137
0.5913
0.8889
quantitative
crisis (2021-09-01 .. 2022-12-31)
2112
88
141
39.98
0.2088
0.3527
0.5554
qualitative (thin subset)
123.1
160.4
33.62
46.78
post-crisis (>= 2023-01-01)
4316
180
21.62
5.586
0.4914
0.7996
0.9432
quantitative
negative-price hours
465
85
39.03
11.21
0.2022
0.4323
0.7097
qualitative (thin subset)
33.02
44.89
9.008
13.44
weekday
7630
318
43.37
11.88
0.3541
0.6071
0.833
quantitative
weekend
3117
130
37.75
10.39
0.3898
0.6795
0.8749
quantitative
August-2022 peak weeks
408
17
275.3
87.95
0.01961
0.06618
0.1936
qualitative (thin subset)
236.8
316.7
71.04
106
Dec-2024 Dunkelflaute week (backtest, fold-3 model, raw heads, 2.5y stale)
168
7
75.24
25.79
0.2738
0.5179
0.6667
qualitative (backtest outside every eval block; raw heads, uncalibrated)
43.5
110
10.34
45.67
Coverage collapses on the crisis stratum and on negative-price hours. The bounded target
truncates the lower conformity residuals near the floor, so the lowest intervals under-cover
conditionally. This is disclosed rather than engineered around: a floor-aware tail would reopen
scope this project deliberately closed.
9 · Reliability — three stages
Raw heads → CQR → isotonic last, at all three nominal levels.
stage
nominal
empirical
raw LightGBM
0.5
0.2257
raw LightGBM
0.8
0.4322
raw LightGBM
0.95
0.6705
post-CQR / pre-isotonic
0.5
0.3642
post-CQR / pre-isotonic
0.8
0.628
post-CQR / pre-isotonic
0.95
0.8416
final post-isotonic
0.5
0.3645
final post-isotonic
0.8
0.6281
final post-isotonic
0.95
0.8452
The finite-sample, distribution-free per-interval marginal guarantee attaches to the
post-CQR / pre-isotonic output only. Isotonic is a monotone rearrangement applied
last; whenever it moves an endpoint it perturbs the interval, so the final output's coverage is
reported as empirical. No joint or simultaneous coverage across the four intervals
is claimed.
The one retained hard gate is a correctness property, not a favourable number: zero
quantile-crossing violations after the full pipeline. On the selected
base catalog's development evaluation rows, the raw heads cross on
10,158 adjacent pairs and still cross on
4,412 after CQR; after isotonic the count is
0, and 0 on the holdout. That CQR
alone leaves thousands of crossings is exactly why isotonic is unconditionally last.
CQR provides finite-sample marginal coverage guarantees under exchangeability. The walk-forward CV mildly violates exchangeability — the crisis regime is not exchangeable with the pre-crisis regime, and the solar-driven negative-price era is not exchangeable with either — so empirical coverage may diverge from nominal on regime-shift folds. This is documented in the reliability diagram (Section 8.4).
10 · Next-day forecast
Archived v1 report. The replay below runs in this page; the current interactive demo
runs in your browser, and the container instructions later in this report are historical.
Historical out-of-sample replay — the frozen champion forecasting a 90-day period it never trained on. This is not a live forecast. The delivery day below, 2026-09-06,
falls inside the pinned holdout window 2026-06-09..2026-09-06. The dashed line is the price that
actually cleared — an outcome, never a model input.
Nominal level selected above; the frozen champion's own empirical coverage at that level over
the 90-day holdout was 0.7593.
Scenario perturbations are ceteris-paribus sensitivity probes and may be out of distribution. The scenario holds every other input fixed,
including the price history the lags and rolling statistics are built from, so a large perturbation
asks the model a question it was never trained on. The cleared-price line is hidden while a
scenario is active, because the outcome belongs to the unperturbed day.
Want to drive the model yourself? The
interactive demo — runs in your browser; the first visit downloads about 57 MB runs the champion's own boosters in
your browser, proved bitwise equal to the frozen artifact over 54 delivery days.
This page stays the first touch because it downloads nothing and cannot fail when a CDN does.
11 · Honest limitations
Exchangeability under regime shift. CQR provides finite-sample marginal coverage guarantees under exchangeability. The walk-forward CV mildly violates exchangeability — the crisis regime is not exchangeable with the pre-crisis regime, and the solar-driven negative-price era is not exchangeable with either — so empirical coverage may diverge from nominal on regime-shift folds. This is documented in the reliability diagram (Section 8.4).
Development versus one-shot evidence. The five-fold development results are descriptive post-selection evidence, never confirmatory: those folds also chose the catalog. Only the 90-day holdout was pre-specified and opened once, and on the development folds the point-accuracy DM shows a deficit, not merely the absence of an advantage.
Disclosed assumption — the load forecast. A65/A01 is pre-gate by explicit assumption, not by measurement: both the existence of the delivery-day load forecast before the 12:00 CET gate and its equality to the archived vector used here are assumed, and the regulatory update provision permits later revisions.
Disclosed assumption — the generation archive. A75 aggregate actual generation is used at its current archived values, which may differ from the values visible in real time despite the D-2 boundary.
The measured cost of the strict gate. The strict-gate design has a measured cost rather than an assumed one: the post-gate A69 forecast is worth 19.4926% of pooled raw-head pinball loss, and the project declines to use it.
A two-sided bounded target, live at the floor. The target is two-sided and bounded: the price is routinely negative and has hit the −500 EUR/MWh floor, which truncates the lower conformity residuals, so the lowest intervals under-cover conditionally near the floor.
Coverage divergence. Empirical coverage diverges from nominal: 0.4407 / 0.7593 / 0.9398 against 50 / 80 / 95 %, so the 50 % interval under-covers by roughly six points on the holdout window. On the crisis stratum it does not merely diverge, it collapses: over the August-2022 peak weeks the 95 % interval covered 0.194 of outcomes. The mechanism is measured — that fold's CQR thresholds were estimated on a May-June 2022 calibration window at a ~198 EUR/MWh level and applied to an evaluation block averaging 376 EUR/MWh, and the conformal correction is additive, not multiplicative. This is what a split-conformal guarantee does when exchangeability breaks; it is the defect later generations address (see the v2 and v3 chapters), and it is not fixed in the released v1.
Model staleness, with all four cutoffs. The deployed demo applies a frozen model whose raw-model fit cutoff (2026-04-07) precedes the snapshot cutoff (2026-09-06) by 152 delivery days, with the final calibration window 2026-04-09..2026-06-07 and the holdout window 2026-06-09..2026-09-06 — all four cutoffs published separately because they are four different dates.
The 15-minute MTU averaging choice. From 2025-10-01 an hourly price is the mean of four quarter-hour prices, so every hour-level statistic here — the negative-hour tally included — depends on that averaging choice, and a quarter-hour tally differs.
Scope. This is a portfolio artifact, not an operations system: no retraining schedule, no drift gate, no rollback machinery, no monitoring surface, and no multi-day-ahead forecast.
The day-ahead price floor moved to −600 EUR/MWh from 2026-05-28, an environment shift the frozen model predates.
The holdout is a single contiguous recent period, so it tests generalization to the most recent regime rather than repeated out-of-sample skill. Its partitions were pinned before development and it was evaluated once, after the complete model and calibration pipeline were frozen, with no subsequent tuning. The 90-day length was fixed by partition design, not by a power calculation, so the Diebold–Mariano result is confirmatory-style but not power-qualified. The model shown is exactly the model evaluated — no refit followed the holdout — so its raw-model fit cutoff precedes the snapshot cutoff by 152 delivery days. Sequential lag features may use earlier holdout observations exactly as they would in live forecasting; no holdout outcome entered fitting or a development decision. Enforcement is procedural: this is a solo build and the discipline is disclosed, not cryptographically guaranteed.
12 · Reproducibility
Three public surfaces, all live. Each answers a different
question and each stands on its own. This page is the one that needs no network.
Is the decision trail real, or is this page the only evidence?
anonymous, no sign-in
The decision trail, addressed directly. Every link here was checked from an
unauthenticated client. The DagsHub repository UI redirects an anonymous visitor to a
sign-in page; the .mlflow tracking host does not, which is why every link uses it.
Model registry — the registered champion and its
champion alias
Tracking root — if a deep link ever moves, start here
Runs are named rather than linked by id, because several decision-bearing runs were reproduced and
no single id is canonical — the name is what to search for:
Run name
What it decided
catalog::base
the selected catalog's development arm
catalog::base_plus_residual_load_proxy
the domain-feature arm that lost, by +0.371516%
baseline::similar_day_naive
the headline comparator
baseline::seasonal_naive_168h
the second baseline
baseline::ridge
the third baseline
benchmark::base
the strict-gate arm of the §7.2 benchmark
benchmark::base+a69
the post-gate arm -- what the unavailable forecast is worth
Tracking after v1. Research after v1 will be tracked in a separate `delu-generations` experiment on the same tracking server: every policy evaluated since v1 will appear there once, mirrored from the committed evidence, so v1's record stays exactly as the one-shot holdout left it and the two are never mixed in one experiment.
See how it evolved.
The pickle's bytes are deliberately not the identity to check: MLflow stamps a fresh
UUID and creation time into MLmodel on every save and cloudpickle is not
byte-reproducible, so models/champion/ changes while the model does not. The
fingerprint is computed over the catalog, the feature list, the nine quantiles, the four thresholds
and the nine boosters' own serializations.
Tagged commit. Check out the tagged commit and run uv sync then make train: the champion is rebuilt from the committed snapshot with pinned dependencies and fixed seeds, no extra fetch.
MLflow permalinks. Every decision-bearing run — the three baselines, both catalog candidates, both benchmark arms, the champion's final fit and holdout, and the diagnostics — is public at https://dagshub.com/hrsi56/delu-day-ahead-forecast.mlflow with snapshot hash, code SHA, fold spec, feature list, seed, hyperparameters, metrics and artifact links. Counting runs there will not give you 'one', and it is not meant to: the tracking server shows several runs named champion::final-fit-and-holdout, because the script is deterministic and was re-run while CP-2 was authored — once aborted on the runtime firewall before any outcome was read, and every completed run reproducing the previous run's metrics and artifact fingerprint exactly. “Evaluated exactly once” is a statement about the evaluation decision, not about how many times a deterministic script may be executed — an Integration Critic re-running it from a clean worktree is reproducing the result, not taking a second look at the holdout. No catalog, hyperparameter, threshold or analysis choice was changed after any of them.
The registered champion alias. The champion is registered on that MLflow instance as the model delu-day-ahead-champion under the champion alias, carrying release and lineage tags — release_status=portfolio_release, the source run id, the code commit, the snapshot hash and the four cutoffs. It is portfolio evidence only: the deployed demo loads the bundled artifact and never queries the registry.
Canonical entry point. The static GitHub Pages report at https://hrsi56.github.io/delu-day-ahead-forecast/ is the canonical entry point, and the interactive Space at https://huggingface.co/spaces/Yarden-Viktor/delu-day-ahead-forecast is linked from it.
DuckDB SQL. The hand-authored DuckDB queries in sql/feature_queries.sql express the same calendar-day lag and D-1-frozen rolling semantics as the canonical Python pipeline, and run against the committed Parquet with make sql.
The four cutoffs. All four cutoffs are published separately: snapshot 2026-09-06, raw-model fit 2026-04-07, final calibration 2026-04-09..2026-06-07, holdout 2026-06-09..2026-09-06.
Attribution. Data: ENTSO-E Transparency Platform; Bundesnetzagentur | SMARD.de — CC BY 4.0. Code MIT; the redistributed data stays CC BY 4.0 with attribution, and the trained champion is a derived work of it. The v3 research model's weather data is derived from NCEP GFS 0.25° (NOAA/NWS/NCEP) via NCAR GDEX d084001 (doi:10.5065/D65D8PWK) and NOAA Open Data Dissemination on AWS, modified and aggregated by this project; it is not an official NOAA product. See LICENSE and DATA-LICENSE.md.
Run it yourself. Clone
https://github.com/hrsi56/delu-day-ahead-forecast, then uv sync, then
uv run python predict_next_day.py --level 80 --self-check forecasts a delivery day
offline from the bundled snapshot and re-proves the gate boundary as it goes;
make test runs the invariant suite including the CQR order-statistic fixture;
make sql runs the DuckDB queries; and
docker build -t delu-showcase . && docker run -p 7860:7860 delu-showcase
builds and serves the same image the Space runs, champion and snapshot bundled inside it.
Compute. Apple M3, 16 GB, CPU only, no GPU, $0 run rate; the whole
pipeline — development, benchmark, final fit and diagnostics — runs in well under an hour.
Next: a re-evaluation on every eligible day, new input data, renewable-generation forecasts and model combinations, then an evaluation of the selected model under a frozen protocol and prospective monitoring.
See planned experiments
Each item is subject to the research plan, where its conditions are set out, and would be compared with v5 on the same days (the plan).
A re-evaluation on every eligible day
Question it will test
How do the adopted generations perform when every eligible day is scored, not only the five historical test periods?
Evidence that would decide it
An evaluation protocol fixed before any result, then every generation scored on all eligible days, with the five test periods kept as the historical record
Work item
Re-evaluation
New input data
Question it will test
Do fuel and carbon prices, neighbouring markets, or earlier forecasts that already cover the delivery day add information to v5?
Evidence that would decide it
A provenance, licence and availability check for each source, then one predefined comparison with and without it
Work item
New data
Wind and solar generation forecasts
Question it will test
Does a wind and solar generation forecast add information beyond direct weather?
Evidence that would decide it
A generation forecast available before the auction, then an ablation
Work item
4.4V · VRE
Combining models
Question it will test
Does combining adopted models help?
Evidence that would decide it
A predefined combination test
Work item
4.8 · Recombination
Frozen-protocol evaluation, then prospective monitoring
Question it will test
How does the selected model perform under a frozen evaluation protocol, and then prospectively?
Evidence that would decide it
An evaluation window and an evidence classification, both fixed in the protocol before the test
Work item
4.7T and live · Final evaluation
How the system works, and how to check it
How the system works
Source data and vintagesENTSO-E and SMARD prices and load forecasts; from v3, the GFS run of the day before
Information cutoffForecast-cutoff checks; source-availability assumptions documented
FeaturesCalendar, price lags, load forecast; weather from v3
Model and interval policyv1: LightGBM quantiles, conformal calibration; v2 and v3: blended LEAR, hour-aware intervals; v4: that blend plus a three-block LightGBM; v5: v4 plus a distributional neural network written in NumPy
Evaluation and artifactsFive historical periods, identical hours; committed predictions, metrics and reviews
Report, demo and trackingThis page, the in-browser v1 demo and the MLflow mirror
Live operation · plannedDaily forecasts scored after the fact
Built with Python, entsoe-py, pandas, NumPy, DuckDB, LightGBM, scikit-learn, Parquet, GitHub Actions, GitHub Pages, marimo, Pyodide, Hugging Face Static Space and MLflow on DagsHub.
Checked in code: controls that fail when a feature crosses the forecast cutoff. Assumed and documented: each source's historical availability (the availability assumptions). Dashed outlines mark planned parts.
Rebuild the report from saved evidence
Rebuild the presentation from committed evidence, without fitting models or downloading source data.
Install the pinned dependencies once (this downloads packages), then rebuild:
uv sync
uv run python scripts/rebuild_presentation.py
One measured rebuild
Measured once on 2026-09-25: 9.6 s on the build machine (Darwin arm64, Python 3.13.15), from the release-check record reports/presentation/release-checks/2026-09-25-rebuild.json. It is one measurement, not a benchmark of every revision.
Each experiment below has its own full reproduction. To run the released model, see
Run this product; v1's original instructions, with the historical container,
are in its archive.
Every policy evaluated since v1 is in the MLflow experiment delu-generations, mirrored from the repository's committed evidence; the repository is the source of truth, and this page works without the tracking service.
I led the project's problem definition, evaluation criteria and research direction, and made the decisions on model adoption and product presentation. AI agents assisted with implementation, analysis and documentation. Automated tests and reviews separate from implementation supported verification. I retained responsibility for approving deliverables and publication.
Terms and attribution
Data: ENTSO-E Transparency Platform; Bundesnetzagentur | SMARD.de — CC BY 4.0.
Code MIT; the redistributed data stays CC BY 4.0 with attribution, and the trained champion is a derived work of it. The v3 research model's weather data is derived from NCEP GFS 0.25° (NOAA/NWS/NCEP) via NCAR GDEX d084001 (doi:10.5065/D65D8PWK) and NOAA Open Data Dissemination on AWS, modified and aggregated by this project; it is not an official NOAA product. See LICENSE and DATA-LICENSE.md.
