fc90e65271
The simulator used to reconstruct "gross household demand" by adding
back a synthesized PV trace (irradiance × kWp peak-match) to the P1 net
meter, then re-subtract a different synthesized PV per scenario. That
reconstruction was leaky — Michiel's horizontal pyranometer is at a
different location and orientation than dad's SE-facing array, so the
synthesis can't reproduce dad's actual production curve. Result: 511
hours of negative "gross demand" and phantom export inflation up to
~6 kW peak in scenarios where pv_kwp ≠ 3.
New shape: simulator works on a single signed signal, raw_demand_kw
(the P1 reading as recorded). No solar synthesis. Whatever the meter
shows is the input.
Concretely:
- sim.py: drop synthesize_pv, reconstruct_gross_demand,
schedule_with_planning_pv, no_foresight_schedule, groundhog_schedule,
_oracle_daily_schedule_legacy. Rename column convention demand_kwh →
raw_demand_kw. Plug-in discharge cap becomes max(0, raw_demand_kw).
- web.py: drop pv_kwp/pv_yield/strategy form params. Demand slider
now applies as an *additive* baseline shift (not multiplicative —
multiplying scaled the export bursts too, which is wrong since dad's
PV stays the same regardless of household consumption). Default
demand_kwh = 2325 (dad's actual full-year net per his quote;
extrapolated 8-month window comes out to ~1515, partial coverage).
Saturation metric now measures (surplus ≥ pc_max), not (charge ≥
pc_max) — the latter conflated arbitrage top-off with power-bottleneck.
- templates/index.html: drop PV input, drop strategy radios, drop
irradiance chart. Modal charts collapsed from 4 to 3: price, net
meter (toggles between with/without battery), SoC.
- app.js: mirror the above, drop pv_kwp/strategy plumbing.
- tests: rebase fixtures on raw_demand_kw, drop synthesize_pv test.
- scripts: drop --pv-kwp/--pv-yield flags throughout, switch column
references to raw_demand_kw.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
105 lines
4.1 KiB
Python
105 lines
4.1 KiB
Python
#!/usr/bin/env python3
|
|
"""Convert dad's 15-min P1 export into our hourly p1 format.
|
|
|
|
Source: ~/Downloads/P1e-2025-1-1-2026-1-1.csv (cumulative T1/T2 import/export
|
|
meter readings, every 15 min, 2025 calendar year, Europe/Amsterdam local time).
|
|
|
|
Output: data/raw/p1_hourly.csv (timestamp UTC, power_w mean over hour). Net
|
|
of PV exports — `power_w` can be negative when his solar exceeded household
|
|
demand. The simulator interprets `raw_demand_kw = power_w / 1000`, and a
|
|
plug-in battery's discharge cap `max(0, raw_demand_kw)` correctly forbids
|
|
discharge during net-export hours (the meter is already flowing the wrong way).
|
|
|
|
The simulator's window is 2023-09-01 → 2024-09-01 UTC. Dad's data is the
|
|
2025 calendar year. We align by (month, day, hour) so seasonal patterns
|
|
(winter peaks, summer surplus) line up with our prices and irradiance.
|
|
Feb 29 in our 2024 leap year falls back to Feb 28 from dad's 2025.
|
|
|
|
Run from project root:
|
|
uv run python scripts/import_dad_p1.py
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import argparse
|
|
from pathlib import Path
|
|
|
|
import pandas as pd
|
|
|
|
|
|
def main() -> None:
|
|
p = argparse.ArgumentParser(description=__doc__)
|
|
p.add_argument("--src", type=Path,
|
|
default=Path.home() / "Downloads/P1e-2025-1-1-2026-1-1.csv")
|
|
p.add_argument("--out", type=Path, default=Path("data/raw/p1_hourly.csv"))
|
|
p.add_argument("--window-start", default="2023-09-01T00:00:00Z")
|
|
p.add_argument("--window-end", default="2024-09-01T00:00:00Z")
|
|
args = p.parse_args()
|
|
|
|
print(f"Reading dad's CSV from {args.src}")
|
|
dad = pd.read_csv(args.src, parse_dates=["time"])
|
|
dad["time"] = dad["time"].dt.tz_localize(
|
|
"Europe/Amsterdam", ambiguous="infer", nonexistent="shift_forward"
|
|
)
|
|
dad = dad.set_index("time").sort_index()
|
|
|
|
# Cumulative net energy (signed). Imports add, exports subtract.
|
|
net_cum = (
|
|
dad["Import T1 kWh"] + dad["Import T2 kWh"]
|
|
- dad["Export T1 kWh"] - dad["Export T2 kWh"]
|
|
)
|
|
|
|
# Sample at exact hour boundaries (the meter records every 15 min, so the
|
|
# H:00 reading exists). Diff to get energy consumed in each hour.
|
|
hourly_stock = net_cum.resample("1h").first()
|
|
hourly_kwh = hourly_stock.diff().dropna()
|
|
# Now hourly_kwh.index[i] = end of the hour just consumed; shift to start.
|
|
hourly_kwh.index = hourly_kwh.index - pd.Timedelta(hours=1)
|
|
hourly_kwh = hourly_kwh.tz_convert("UTC")
|
|
|
|
annual = hourly_kwh.sum()
|
|
print(f"Dad's hourly net (kWh): n={len(hourly_kwh)}, "
|
|
f"annual net = {annual:.0f} kWh "
|
|
f"(min hour {hourly_kwh.min():.2f}, max {hourly_kwh.max():.2f}, "
|
|
f"{(hourly_kwh < 0).sum()} export hours)")
|
|
|
|
# Build (month, day, hour) → kWh lookup. dad's full year covers every key
|
|
# except a possible duplicate around DST (averaged).
|
|
lookup = (
|
|
hourly_kwh.groupby([
|
|
hourly_kwh.index.month,
|
|
hourly_kwh.index.day,
|
|
hourly_kwh.index.hour,
|
|
]).mean()
|
|
)
|
|
|
|
# Build target timestamps for our simulator window.
|
|
target = pd.date_range(args.window_start, args.window_end,
|
|
freq="1h", inclusive="left", tz="UTC")
|
|
out = []
|
|
missing = 0
|
|
for ts in target:
|
|
m, d, h = ts.month, ts.day, ts.hour
|
|
if (m, d, h) in lookup.index:
|
|
kwh = lookup.loc[(m, d, h)]
|
|
elif m == 2 and d == 29 and (2, 28, h) in lookup.index:
|
|
kwh = lookup.loc[(2, 28, h)]
|
|
else:
|
|
missing += 1
|
|
continue
|
|
out.append((ts, kwh * 1000.0)) # kWh per hour → mean watts
|
|
|
|
if missing:
|
|
print(f"WARNING: {missing} target hours had no dad-data match")
|
|
|
|
df_out = pd.DataFrame(out, columns=["timestamp", "power_w"])
|
|
df_out["timestamp"] = df_out["timestamp"].dt.strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
|
|
args.out.parent.mkdir(parents=True, exist_ok=True)
|
|
df_out.to_csv(args.out, index=False)
|
|
print(f"Wrote {len(df_out)} hours → {args.out}")
|
|
print(f" annualised mean: {df_out['power_w'].mean():.0f} W "
|
|
f"({df_out['power_w'].sum() / 1000:.0f} kWh net per year)")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|