60e0706736
- Hourly data exporter (InfluxDB → CSV) for prices, P1, irradiance. - LP-based 24h-foresight oracle dispatch with SoC-consistent state engine. - Reverse-engineered thuisbatterijgids.nl formula (matches their quotes to within €0.50 across three battery configs). - Catalog scraper for the 52 batteries on thuisbatterijgids.net via their /wp-json REST endpoint. - Web app (Flask) that ranks every catalog battery by honest payback and contrasts with the store's quote, deployable via the included Procfile.
147 lines
8.2 KiB
Markdown
147 lines
8.2 KiB
Markdown
# Quick index
|
||
|
||
Where each concept lives in the codebase. Update as code is added.
|
||
|
||
## Code map
|
||
|
||
| Concept | Location |
|
||
| --- | --- |
|
||
| InfluxDB → CSV export | `scripts/export_from_influx.py` |
|
||
| Hourly data loader | `pluginbattery.sim.load_hourly` |
|
||
| Battery spec + presets (`ECOFLOW_STREAM_AC`) | `pluginbattery.sim.Battery` |
|
||
| State engine (walks SoC, clamps to limits) | `pluginbattery.sim.simulate` |
|
||
| 24h-foresight oracle (4-var LP per UTC day) | `pluginbattery.sim.oracle_daily_schedule` |
|
||
| NL consumer-price tariff helper | `pluginbattery.sim.apply_nl_tariff` |
|
||
| PV synthesis from horizontal irradiance | `pluginbattery.sim.synthesize_pv` |
|
||
| Driver: compare multiple batteries side-by-side | `scripts/compare_batteries.py` |
|
||
| Driver: run oracle for one battery (default EcoFlow) | `scripts/run_oracle.py` |
|
||
| Driver: honest LP vs cracked store calc, single scenario | `scripts/compare_with_store.py` |
|
||
| Driver: capacity-vs-savings sweep | `scripts/capacity_sweep.py` |
|
||
| Driver: rank every catalog battery by honest payback | `scripts/battery_leaderboard.py` |
|
||
| Catalog scraper (thuisbatterijgids.net REST API) | `scripts/scrape_batteries.py` |
|
||
| **Reverse-engineered store calculator (thuisbatterijgids.nl)** | `pluginbattery.store_calc` |
|
||
| **Web app (Flask): live leaderboard + scenario inputs** | `pluginbattery.web` |
|
||
| LP-side smoke tests | `tests/test_sim.py` |
|
||
| Store-formula regression tests (locked to 3 quotes) | `tests/test_store_calc.py` |
|
||
|
||
## Web app
|
||
|
||
```
|
||
PORT=8765 uv run python -m pluginbattery.web # local dev
|
||
gunicorn pluginbattery.web:app # production (Procfile is set up for h4a)
|
||
```
|
||
|
||
The page renders the default scenario server-side at first load (Marstek Venus B / €124/yr / 3.84 yr) so there's no blank flash. Inputs trigger a POST to `/api/calculate` which returns JSON. Results are cached per scenario hash — the first cold scenario takes ~20 s for 37 unique (cap, power) LP runs; identical re-requests are instant.
|
||
|
||
## Cracked store formula (thuisbatterijgids.nl)
|
||
|
||
Verified against three observed quotes within €0.5 each. Source: `src/pluginbattery/store_calc.py`.
|
||
|
||
```
|
||
hours_to_fill = capacity_kwh / max_charge_kw
|
||
cycles_per_day = min(1.0, 4 hours / hours_to_fill)
|
||
arbitrage = 365 × cycles_per_day × capacity × 0.85 × (avg_retail − avg_EPEX)
|
||
self_consume = 195 × capacity × 0.85 × avg_retail (if PV and no saldering)
|
||
= 0 (if full saldering)
|
||
year1 = arbitrage + self_consume
|
||
payback @ 3% i = log(1 + cost·i/year1) / log(1+i)
|
||
```
|
||
|
||
Where `avg_EPEX ≈ €0.0824/kWh` (their apparent assumed wholesale baseline).
|
||
|
||
The deception is in the `(avg_retail − avg_EPEX)` term: charged kWh are valued at avg wholesale (no VAT, no energy tax), discharged kWh at full retail. No NL supplier will sell you wholesale-priced kWh, so that spread doesn't exist for any real customer. Inflates dynamic-mode savings ~2× for EcoFlow, up to ~3.5× for Marstek 2.5 kW. Fixed-rate PV self-consumption side is honest within ~5%.
|
||
|
||
## Running comparisons
|
||
|
||
```
|
||
# No PV, no saldering (post-2027 future)
|
||
uv run python scripts/compare_batteries.py
|
||
|
||
# 3 kWp PV, no saldering (likely dad's situation 2027+)
|
||
uv run python scripts/compare_batteries.py --pv-kwp 3.0
|
||
|
||
# 3 kWp PV with saldering still active (current 2023-24 reality)
|
||
uv run python scripts/compare_batteries.py --pv-kwp 3.0 --saldering full
|
||
```
|
||
|
||
PV total is auto-calibrated to 900 kWh/kWp/year (NL norm). Override with `--pv-target`.
|
||
|
||
## Run
|
||
|
||
```
|
||
uv sync
|
||
uv run python scripts/run_oracle.py # writes data/processed/oracle_daily.csv
|
||
uv run pytest # smoke tests
|
||
```
|
||
|
||
## Latest oracle results — payback in years
|
||
|
||
Window 2023-09-01 → 2024-09-01 (8745 h). Tariff: `consumer = EPEX × 1.21 + 0.136 EUR/kWh`. PV calibrated to 2700 kWh/yr at 3 kWp. 24h-foresight oracle, daily LP per UTC day.
|
||
|
||
The number depends *strongly* on what you assume about saldering and price level. Same hardware, same data, same dispatch — three very different paybacks:
|
||
|
||
| Scenario | EcoFlow €699 | Marstek 0.8 kW €1339 | Marstek 2.5 kW €1339 |
|
||
| --- | ---: | ---: | ---: |
|
||
| No PV, saldering on/off (same — no PV means no exports either way) | 12.0 | 16.3 | 12.6 |
|
||
| 3 kWp PV, **full saldering** (current 2024 reality) | 13.3 | 18.9 | 14.6 |
|
||
| 3 kWp PV, no saldering, export at raw EPEX (~€0.075/kWh) | 6.8 | 9.7 | 8.2 |
|
||
| 3 kWp PV, no saldering, export = €0 | 5.9 | 7.7 | 6.7 |
|
||
| **3 kWp PV, export = €0, prices × 1.5** *(matches stroomstoring.nl style sales calc)* | **3.96** | **5.81** | **4.51** |
|
||
|
||
The `× 1.5` factor lifts our 2023-24 average consumer price (€0.226) to ≈€0.34, the typical Dutch retail level in 2025. At that price level + zero export compensation, the EcoFlow at €699 pays back in 4.0 years — exactly matching the **stroomstoring.nl** quote of €176/yr / 3.9 years.
|
||
|
||
The simulator gives a different answer than the store **only because of two assumption changes**:
|
||
1. Export rate: the store assumes €0 (no compensation at all), our default uses raw EPEX (~€0.075/kWh).
|
||
2. Price level: the store uses current retail (~€0.34/kWh), our backtest uses what *actually happened* in 2023-24 (~€0.23/kWh).
|
||
|
||
Both positions are defensible — the store's is forward-looking ("buy this today and save under post-saldering 2025+ pricing"), ours is historically grounded ("had we owned this in 2023-24"). Use `--price-mult 1.5 --export-rate 0` on `compare_batteries.py` to reproduce the store quote.
|
||
|
||
Annual electricity bill (no battery), our house, our 2023-24 prices:
|
||
|
||
| Setup | Imports | Exports | Bill |
|
||
| --- | ---: | ---: | ---: |
|
||
| No PV | 7277 kWh | 0 | €1497.38 |
|
||
| 3 kWp PV, full saldering | 5708 | 1125 | €995.86 |
|
||
| 3 kWp PV, no saldering, export = raw EPEX | 5708 | 1125 | €1158.18 |
|
||
| 3 kWp PV, no saldering, export = 0, prices × 1.5 | 5708 | 1125 | €1803.75 |
|
||
|
||
Validation against dad's actual bill (different house, similar 3 kWp PV): dad reported 1976 kWh exported. With ~2700 kWh PV that implies 25–30% self-consumption — typical NL. Our simulation lands at 60% self-consumption because this house has higher base demand to soak up more PV directly.
|
||
|
||
## Raw data files (`data/raw/`)
|
||
|
||
Generated by `scripts/export_from_influx.py`. All hourly, UTC, ISO-8601 timestamps.
|
||
|
||
| File | Column | Unit | Source field on data-vm |
|
||
| --- | --- | --- | --- |
|
||
| `prices_hourly.csv` | `eur_per_kwh` | EUR/kWh, **raw EPEX** (no taxes, no markup) | `epex-price` (market='nl') / 1000 |
|
||
| `p1_hourly.csv` | `power_w` | watts, hour-mean, **household import only** (no PV in this house) | `power-current` |
|
||
| `solar_hourly.csv` | `irradiance_w_m2` | W/m², horizontal (zenith-facing) | `ws1-solarradiation` |
|
||
|
||
Tariff additions (BTW, energy tax, supplier markup, network charges, terugleverkosten, saldering) are layered on downstream (UI / cost calc), not in the export.
|
||
|
||
### Working window
|
||
|
||
**2023-09-01T00:00:00Z → 2024-09-01T00:00:00Z** (8784 hours, 366 days incl. 2024-02-29).
|
||
|
||
Picked over 2024-09→2025-09 because the latter contains a 19-day P1 outage (2025-03-05→23).
|
||
|
||
### Known gaps in the working window
|
||
|
||
| File | Coverage | Largest gap |
|
||
| --- | --- | --- |
|
||
| prices_hourly.csv | 8782/8784 (100.0%) | 1h at each DST transition — ENTSO-E artefact |
|
||
| p1_hourly.csv | 8764/8784 (99.8%) | 12h on 2024-05-09 18:00→2024-05-10 05:00 |
|
||
| solar_hourly.csv | 8765/8784 (99.8%) | 8h overnight 2024-01-04→05 |
|
||
|
||
Drop hours where any signal is missing rather than interpolating, unless an interpolation model is explicitly chosen.
|
||
|
||
### Sign / sense conventions
|
||
|
||
- `power_w`: this house has no PV today, so the field is pure household consumption. Always positive. For "with solar" scenarios, synthesize PV output from `irradiance_w_m2` × assumed kWp/tilt/azimuth and subtract from `power_w` to get net.
|
||
- `eur_per_kwh`: pre-tax EPEX. Genuinely negative in some hours (mean 0.075, min −0.200, max 0.464 over the window). Adding 21% BTW on a negative wholesale price is a tariff modelling decision, not a data one.
|
||
|
||
### Value-range sanity checks (current window)
|
||
|
||
- Prices (EPEX raw): mean 0.075 EUR/kWh, min −0.200, max 0.464.
|
||
- Solar: mean 92.8 W/m², max 762 W/m². Plausible for NL horizontal.
|
||
- P1: mean 833 W, median ~335 W, max 6135 W. Reasonable household consumption.
|