The simulator used to reconstruct "gross household demand" by adding
back a synthesized PV trace (irradiance × kWp peak-match) to the P1 net
meter, then re-subtract a different synthesized PV per scenario. That
reconstruction was leaky — Michiel's horizontal pyranometer is at a
different location and orientation than dad's SE-facing array, so the
synthesis can't reproduce dad's actual production curve. Result: 511
hours of negative "gross demand" and phantom export inflation up to
~6 kW peak in scenarios where pv_kwp ≠ 3.
New shape: simulator works on a single signed signal, raw_demand_kw
(the P1 reading as recorded). No solar synthesis. Whatever the meter
shows is the input.
Concretely:
- sim.py: drop synthesize_pv, reconstruct_gross_demand,
schedule_with_planning_pv, no_foresight_schedule, groundhog_schedule,
_oracle_daily_schedule_legacy. Rename column convention demand_kwh →
raw_demand_kw. Plug-in discharge cap becomes max(0, raw_demand_kw).
- web.py: drop pv_kwp/pv_yield/strategy form params. Demand slider
now applies as an *additive* baseline shift (not multiplicative —
multiplying scaled the export bursts too, which is wrong since dad's
PV stays the same regardless of household consumption). Default
demand_kwh = 2325 (dad's actual full-year net per his quote;
extrapolated 8-month window comes out to ~1515, partial coverage).
Saturation metric now measures (surplus ≥ pc_max), not (charge ≥
pc_max) — the latter conflated arbitrage top-off with power-bottleneck.
- templates/index.html: drop PV input, drop strategy radios, drop
irradiance chart. Modal charts collapsed from 4 to 3: price, net
meter (toggles between with/without battery), SoC.
- app.js: mirror the above, drop pv_kwp/strategy plumbing.
- tests: rebase fixtures on raw_demand_kw, drop synthesize_pv test.
- scripts: drop --pv-kwp/--pv-yield flags throughout, switch column
references to raw_demand_kw.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single algorithm with two passes:
Pass 1 — greedy self-consumption (real-firmware default):
For each hour, charge any surplus, discharge into any demand.
This nails sunny days: battery fills from morning surplus, exports
only after capacity is reached, drains during evening peak.
Pass 2 — daily price-spread arbitrage on the residual capacity:
For each UTC day, repeatedly find the most profitable
cheap-charge → expensive-discharge pair (positive after round-trip
efficiency), execute it, recompute SoC trajectory, repeat until no
profitable cycle remains. Discharge must fit within hourly demand
for plug-in batteries (no grid push). Sees only that day's prices,
matching what a Tibber/Frank-style smart-charging controller does
with day-ahead price visibility.
Effect:
- Sunny weeks (June 17-19): unchanged — greedy fill + evening discharge.
- No-sun weeks (Feb 7+): now generates arbitrage savings instead of €0.
- Mixed weeks: greedy handles surplus, arbitrage handles the rest.
Updated: test_grid_arbitrage_kicks_in_on_no_sun_days replaces the prior
"greedy doesn't arbitrage" test; now verifies the dispatcher charges
cheap hours and discharges expensive ones when no PV is available.
The LP was producing visually unintuitive schedules: when several surplus
hours had identical cost benefit (true with constant export_rate), the
solver picked an arbitrary subset, which the user couldn't read as 'this
is the battery doing its job'. Worse, the daily-LP variant drained to 0
every midnight because it placed zero value on next-day SoC.
Replaced oracle_daily_schedule() with a one-pass greedy dispatcher that
matches what every plug-in battery in the catalog (Marstek, Zendure,
EcoFlow, HomeWizard) actually does in 'self-consumption mode':
for each hour:
if exporting at meter: charge as fast as power+capacity allows
if importing at meter: discharge to cover net demand
Trade-offs:
- Greedy doesn't do grid arbitrage (charge cheap → discharge expensive
without a surplus source). The LP would; real plug-in firmware
doesn't, so greedy is more honest about what dad's battery would do.
- Charts now show 'fill in the morning, overflow at midday' which is
what users expect to see.
- Updated tests: dropped LP-arbitrage assertion, added a greedy-fills-
from-surplus-then-overflows test that locks the new behaviour in.
- DEFAULTS['export_rate'] = 0 (terugleveringskosten can be re-enabled per
scenario via the Advanced field — keeping the headline default simple).
- LP gets a microscopic per-hour penalty (1e-6 € · hour) on charging that
prefers earlier hours, and on discharging that prefers later. Total
effect on annual savings: < €0.001 (well below display rounding). But
visually the battery now fills as soon as surplus arrives, instead of
the LP arbitrarily mixing surplus hours with grid-arbitrage hours when
both give the same daily total.
1. /api/timeline silently defaulted missing export_rate to None → 0,
which broke the LP for dad's terugleveringskosten scenario. With
export=0, charging during surplus has no benefit, so the battery
stayed half-empty while solar exported. Fix: when the param is
absent, fall back to DEFAULTS['export_rate'] (matches the form's
initial value). Explicit blank still means 'use saldering toggle'.
2. uPlot charts had legend disabled. Re-enabled with per-series value
formatters so hovering shows '14:00 NL · −1.70 kWh/h' style values.
Modal:
- Click any battery row to open a modal with three uPlot charts:
irradiance (W/m²), battery SoC (kWh), net meter flow (kWh/h).
- Date picker + prev/next buttons + 1/3/7-day window selector.
- 'Battery on' toggle flips the SoC and grid charts to the
no-battery baseline; summary numbers update accordingly.
- New /api/timeline endpoint returns the full per-hour series for
any (battery, scenario, window).
Data:
- Re-exported prices + solar from data-vm for 2025-01-01 → 2025-09-01
(8 months — solar coverage limit). All three signals now share
real 2025 timestamps; no more month/day alignment fudge.
- Re-processed dad's CSV against the new window so power_w is real
net consumption per actual hour.
- Compute_leaderboard now applies an 8766/window-hours annualisation
factor to year-1 savings, capt kWh, throughput, and total surplus
so payback / 10-yr net stay correct on the shorter window.
- Defaults: demand 1010 (= dad's net over 8 months, scale 1.0),
start 2025-06-15.
Tests pass. Local run: PORT=8765 uv run python -m pluginbattery.web,
click any row.
The number was previously labelled 'total solar surplus' which read like
'total PV produced'. It's actually 'capturable surplus' = the kWh that
would be exported to the grid without a battery, which is less than gross
PV by the directly-self-consumed portion (PV used at the same hour it's
produced never crosses the meter).
For dad's 3 kWp setup: PV produced ≈ 2625 kWh, directly consumed ≈ 750
kWh, exported ≈ 1869 kWh. The battery can only redirect the 1869,
because the 750 doesn't cross the meter.
Renamed heading + extended legend with the explanation.
New columns:
- kWh capt: absolute kWh of solar surplus the battery absorbed
- % capt: that, as a % of total available surplus for the scenario
- % full: of all surplus hours, the % when the battery was already full
- % sat: of all surplus hours, the % when charging was at max kW
The leaderboard heading also shows total scenario surplus in kWh/yr so
the percentages have a denominator.
These make the limits visible: e.g. on dad's data, an 0.8 kW plug-in is
saturated 19% of surplus hours, while a 2.5 kW unit is never saturated
(0%) — but both end up bottlenecked by 'battery full' on multi-day sunny
stretches when demand can't drain it overnight.
Removed: Store €/yr, Store yr, ×over columns. The store_calc module is
still in the codebase and the API still returns the values; the columns
are just not displayed since dad isn't shopping based on the dishonest
quote.
- scripts/import_dad_p1.py converts his 15-min cumulative meter dump
(~/Downloads/P1e-2025-1-1-2026-1-1.csv) into our hourly format.
Net of PV exports — power_w can be negative on surplus hours.
Aligned to our 2023-09 → 2024-09 window by (month, day, hour) so
prices and irradiance stay seasonally matched.
- data/raw/p1_hourly.csv is now dad's data; ours preserved as
p1_hourly_michiel.csv for diff/comparison.
- LP gE upper bound was 'pv[t]'; now max(0, pv − demand). This was
wrong before but invisible because demand was always positive; with
dad's net data demand goes negative and the LP went infeasible.
- New 'Export rate' field in Advanced. Override saldering toggle with
a literal €/kWh — set to −0.106 for terugleveringskosten.
Single sync worker means a slow request to / (compute_leaderboard, ~17s
on dev / longer on h4a's compact CPU slice) blocks every other request,
including the /healthz probe. With 2 worker processes and 4 threads
each, slow requests on / no longer wedge the gunicorn queue, and
/healthz responds promptly even while a /api/calculate is in flight.
- Backend: every row now carries ten_year_profit_eur =
year1 × cumulative inflation factor − battery cost. Negative for
batteries that don't pay back within 10 years (the Powerwall).
- New column shows green for positive net, red for negative.
- Every column header is now clickable: click to sort, click again to
toggle direction. Sort indicator (↑/↓) on the active header.
- Sort + filter selections persist in localStorage independently.
- Default sort stays 'LP yr ascending' (best payback first), but a
single click on '10-yr net' flips to absolute lifetime profit and
larger batteries surface to the top.
- Synthesised one hybrid entry (Tesla Powerwall 3, installed price €9500)
appended to the catalog. Tagged installation: hybrid; LP runs with
allows_export=True for those.
- Replaced single 'plug-in only' checkbox with three category filters:
≤0.8 kW plug-in, ≤2.5 kW plug-in, Hybrid/DC-coupled. Default: all on.
Filter selection persists in localStorage.
- API now tags every row with a category so the JS can filter without
re-fetching. SSR rows carry data-category for first-paint filtering.
- Store quote left blank for hybrids since thuisbatterijgids.net only
models plug-in arithmetic.
- ProcessPoolExecutor with fork context: 4 workers each inherit the
scenario df via copy-on-write. linprog/HiGHS doesn't release the GIL
so threads were useless; processes get clean ~3.5× speedup.
- Cross-scenario LP cache keyed on everything the LP depends on
(demand, retail, PV size, saldering, η_rt) but NOT inflation or cost.
Result: tweaking just inflation re-uses every LP — sub-100ms.
Cold cold scenario went 75s → 17s; inflation tweak went 17s → 10ms.
- Checkbox 'plug-in only (≤ 0.8 kW, no electrician)' filters the leaderboard
client-side and re-picks the top recommendation from visible rows.
Filter state is persisted in localStorage.
- scripts/deploy.sh: idempotent rsync + systemd setup against the
'thuisbatterij' SSH alias. Run after provisioning the h4a workload.
- Hourly data exporter (InfluxDB → CSV) for prices, P1, irradiance.
- LP-based 24h-foresight oracle dispatch with SoC-consistent state engine.
- Reverse-engineered thuisbatterijgids.nl formula (matches their quotes
to within €0.50 across three battery configs).
- Catalog scraper for the 52 batteries on thuisbatterijgids.net via their
/wp-json REST endpoint.
- Web app (Flask) that ranks every catalog battery by honest payback and
contrasts with the store's quote, deployable via the included Procfile.
