Initial import: home-battery ROI simulator + cracked thuisbatterijgids calc

- Hourly data exporter (InfluxDB → CSV) for prices, P1, irradiance.
- LP-based 24h-foresight oracle dispatch with SoC-consistent state engine.
- Reverse-engineered thuisbatterijgids.nl formula (matches their quotes
  to within €0.50 across three battery configs).
- Catalog scraper for the 52 batteries on thuisbatterijgids.net via their
  /wp-json REST endpoint.
- Web app (Flask) that ranks every catalog battery by honest payback and
  contrasts with the store's quote, deployable via the included Procfile.
This commit is contained in:
Michiel Berger 2026-04-30 13:46:27 +02:00
commit 60e0706736
29 changed files with 30358 additions and 0 deletions

97
CLAUDE.md Normal file
View file

@ -0,0 +1,97 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project intent
Backtest the ROI of a home battery against real historical hourly data. The core question:
> "If I had owned battery X over period Y, what would it have actually saved me?"
First-class scenarios:
- **No solar, just price arbitrage** — charge when day-ahead is cheap, discharge when expensive.
- **With solar** — store surplus generation instead of exporting at low / negative prices.
Two battery archetypes to compare side-by-side, because they have very different economics:
- **Plug-in (AC-coupled, schuko-style)** — small capacity (~2 kWh typical), low continuous output (~800 W), no grid export, simple time-shifting of own consumption only. Cheap, no installer.
- **Installed (hybrid inverter / DC-coupled)** — 515 kWh, 35 kW continuous, can charge from solar DC, can export to grid. Expensive, needs an electrician.
Output should include: euro savings vs. counterfactual, simple payback period, and a sensitivity sweep across battery size / power rating / dispatch strategy.
## Decisions still open — do not silently assume
These materially change the answer. Get the user to nail them down before writing simulation code, or state the assumption loudly in every result.
- **Battery model fidelity.** Round-trip efficiency, depth-of-discharge limit, calendar + cycle degradation, idle/standby draw, inverter efficiency. Start with constants. Only add curves once results turn out sensitive to them.
- **Solar PV system parameters.** Panel kWp, tilt, azimuth, inverter clipping. The horizontal irradiance from `solar_hourly.csv` is the input; PV output is computed downstream from these parameters. Default to a flat-roof horizontal panel until the user specifies otherwise.
## Decisions made
- **Country / market**: Netherlands.
- **Backtest window**: **2023-09-01T00:00:00Z → 2024-09-01T00:00:00Z** (8784 hours, 366 days; 99.5%+ coverage on all three signals). The next year forward had a 19-day P1 outage in March 2025 and was rejected.
- **Household load profile**: real P1 meter log from user's house in InfluxDB. The relevant field (`power-current`) is **import-only** — this house has **no PV today**. For "with solar" scenarios we synthesize PV output from a horizontal irradiance signal × assumed system parameters.
- **Day-ahead prices**: raw EPEX, hourly mean, in EUR/kWh, from user's InfluxDB. **No taxes, BTW, supplier markup, or saldering math is baked into the data** — those are tariff layers added downstream (UI / cost calc). Negative hours are real and preserved.
- **Tariff modelling**: out of scope for this codebase. The simulator outputs energy flows (kWh imported, exported, charged/discharged) per hour. The UI / spreadsheet layered on top translates those into euros under whichever tariff structure is being explored.
- **Solar gap forward**: irradiance sensor offline ~2025-09-09 → ~2026-04-29. Don't pick a window ending after Sep 2025 unless this is fixed.
- **Dispatch strategy**:
1. Build a 24h-perfect-foresight oracle as the ceiling — what's the max possible savings?
2. Build rule-based dispatchers (e.g. price percentile thresholds, SoC bands) and measure how close they get to the oracle.
3. Stretch goal: an LSTM (or simpler) policy trained on historical sequences to beat the rules. Don't pull in PyTorch until the rules have a measured, reproducible gap to close.
## Data sources
The user's home InfluxDB is the *origin* — what actually happened in this house. But the simulator does **not** talk to InfluxDB. The user exports CSVs once and drops them in `data/raw/`; everything downstream reads CSV. That keeps runs reproducible, reviewable in git diffs (for small fixtures), and free of network/auth concerns.
What's known to exist in InfluxDB (verify before assuming):
- Hourly day-ahead electricity prices (>2 years).
- 15-minute electricity prices (~last 4 months).
- P1 smart-meter log (>2 years), 1-min / 10-sec resolution.
- Solar irradiance / PV output, with a gap from ~September 2025 to ~2026-04-29.
Expected raw CSVs (one per signal, UTC timestamps, hourly resampled at the source if possible):
- `data/raw/prices_hourly.csv``timestamp, eur_per_kwh` (or `eur_per_mwh`, document which)
- `data/raw/p1_hourly.csv``timestamp, import_kwh, export_kwh` (or net `consumption_kwh`)
- `data/raw/solar_hourly.csv``timestamp, pv_kwh` (and/or `irradiance_w_m2`)
Document the exact columns and units in `docs/quick-index.md` once the export is in place. Anything resampled or derived from these files goes to `data/processed/` and must be regenerable.
If the InfluxDB export turns out to be tedious to do by hand, write a single one-off script under `scripts/export_from_influx.py` that reads connection details from `.env`. It runs once per refresh, not every simulation run.
## Stack (assumed; change if user objects)
- Python 3.12+, managed with `uv`.
- `pandas` for time series; drop to `numpy` only if the inner loop becomes the bottleneck.
- `pytest` for tests.
- `influxdb-client` only if/when we write the optional one-off export script — not a runtime dep of the simulator.
- No web framework, no ORM, no plotting library yet. Add only when there is a concrete use case.
- No PyTorch / TensorFlow yet. Only after rule-based dispatchers have a measured gap to the oracle that's worth closing.
## Common commands
To be filled in once the project has code. For now:
```
uv sync # install deps once pyproject.toml exists
uv run pytest # run tests
uv run pytest tests/test_dispatch.py::test_name # run a single test
```
## Repo layout (target)
- `src/` — simulation code
- `tests/` — pytest. Unit-test the dispatch logic and tariff calc; one integration test that runs a full year on small fixture data
- `data/raw/` — immutable downloads (gitignored except small fixtures)
- `data/processed/` — derived, regenerable
- `docs/quick-index.md` — pointer to where each concept lives in the code
- `scripts/` — one-off data fetchers (ENTSO-E, KNMI, PVGIS)
## Things to resist (anti-overengineering for this project)
- Don't build a `BatteryStrategy` interface with one implementation. Write the function. Add the interface when the second strategy lands with a name.
- Don't build a tariff DSL or YAML config. Hard-code the NL tariff structure in a function until a second country actually shows up.
- Don't build a CLI framework. `python -m batteryroi.run --year 2024` via `argparse` is enough.
- Don't add caching / parquet / duckdb until a run takes >30s and the user is actively iterating on it.
- Don't add a `Battery` dataclass with builders/factories. Plain kwargs into a function.
- Don't pre-create empty modules "for structure". Files arrive when they have content.