comp package

Submodules

comp.constants module

Constants and configuration for the comp (observational comparison) module.

This module validates the historical ADCIRC storm-surge simulations (the SurgeNet training set, published on Hugging Face) against de-tided NOAA CO-OPS tide-gauge observations. See comp.validate.

comp.coops module

NOAA CO-OPS tide-gauge access and de-tiding for the comp module.

Fetches water-level observations from the CO-OPS data-getter API (cached on disk) and produces a de-tided storm-surge residual. The residual is computed by a robust utide harmonic analysis of the storm’s full calendar year (so it does not depend on CO-OPS having published harmonic predictions for the station); if the record is too sparse to fit, it falls back to the CO-OPS predictions product.

comp.coops.fetch_year(station, year)

Hourly water level (MSL) for a calendar year; verified then preliminary.

Return type:: Series

comp.coops.gulf_gauges(box={'lat': (27.3, 30.9), 'lon': (-97.6, -84.0)})

Return CO-OPS water-level stations within box (cached metadata).

Return type:: List[Tuple[str, str, float, float]]

comp.coops.observed_residual(station, lat, year, start, end)

De-tided surge residual over [start, end].

Returns (residual_series, method) where method is "utide", "pred" (CO-OPS predictions fallback) or "none".

Return type:: Tuple[Series, str]

comp.nulltest module

Negative-control / falsification tests for the tide-gauge validation.

The headline skill (clean-pair peak r = 0.89) is only meaningful if it is specific to the right storm, the right gauge, and the right time. This module runs the placebo tests that try to destroy the signal; if the real score sits far outside the null distributions, the skill is a real physical match and not an artefact of autocorrelation, both-signals-being-positive, or threshold tuning.

Tests (peak-level ones need only val_summary.csv; the lag test needs the cached storm netCDFs):

global label permutation – break the sim<->obs pairing entirely; null r ~ 0.

within-storm permutation – shuffle only which gauge matches which, keeping
each storm’s magnitude; isolates spatial skill.

cross-storm same-gauge – sim of storm A vs obs of storm B at the SAME gauge;
measures leftover gauge climatology (some gauges are just surgier), which the true score must beat.

temporal-lag curve – shift the simulated hydrograph by +/- days and
recompute the time-series correlation; real skill peaks sharply at lag 0.

Run:

python -m comp.nulltest                 # peak-level nulls + lag curve (+ figure)
python -m comp.nulltest --no-lag        # skip the netCDF-heavy lag test

comp.nulltest.cross_storm(c, seed=0)

Return type:: Dict

comp.nulltest.lag_curve(storms=None, lags_days=(-3, -2, -1, -0.5, 0, 0.5, 1, 2, 3))

Median time-series correlation over clean pairs as the simulated hydrograph is shifted by each lag (days). Real surge skill peaks sharply at lag 0.

Return type:: DataFrame

comp.nulltest.load_clean()

Return type:: DataFrame

comp.nulltest.main()

Return type:: None

comp.nulltest.perm_global(c, n=5000, seed=0)

Return type:: Dict

comp.nulltest.perm_within_storm(c, n=5000, seed=0)

Return type:: Dict

comp.nulltest.plot(g, w, lag, paths)

Return type:: None

comp.nulltest.run(do_lag=True)

Return type:: None

comp.sensitivity module

Sensitivity of the headline skill to the analysis’ free thresholds.

A score that only looks good at one hand-picked set of cut-offs is not trustworthy. This module re-derives bias / RMSE / r as each knob is varied, to show the result is robust rather than tuned. Two groups of knobs:

clean-filter cut-offs (MIN_OBS_PEAK_M, MAX_TIMING_HR) only re-label existing gauge-storm pairs, so they sweep cheaply straight from val_summary.csv.

node-selection cut-offs (WET_MIN_M, MAX_NODE_DEG, KNN) change which mesh node is sampled and therefore the simulated peak, so they require the cached storm netCDFs. Each storm is loaded ONCE and re-sampled for every combination.

Run:

python -m comp.sensitivity              # cheap filter sweep + node-selection sweep
python -m comp.sensitivity --no-node    # cheap filter sweep only (no netCDF)

comp.sensitivity.filter_sweep()

Return type:: DataFrame

comp.sensitivity.main()

Return type:: None

comp.sensitivity.node_sweep(combos=None, storms=None)

Re-run peak extraction for several (WET_MIN_M, MAX_NODE_DEG, KNN) combos.

Loads each storm netCDF once; re-uses the cached observed residuals. Reports the clean-pair skill for each combo so it can be compared with the default (0.30, 0.12, 60).

Return type:: DataFrame

comp.sensitivity.run(do_node=True)

Return type:: None

comp.validate module

Validate historical ADCIRC surge against de-tided NOAA gauges.

Pipeline, per storm:

download the storm’s netCDF from Hugging Face (HF_REPO);
extract the simulated surge (SSH = WD + DEM) at the nearest wet mesh element centroid (the archived dual-graph node) to each NOAA CO-OPS gauge in the box;
fetch + de-tide the gauge record (comp.coops.observed_residual());
score peak surge (bias/RMSE/correlation, with bootstrap CIs and a within-storm spatial correlation), the full hydrograph (timeseries_skill()), and peak timing; tag “clean” pairs and regenerate the paper figures + LaTeX table.

ADCIRC here is surge-only (no tides), so we always compare against the de-tided observed residual rather than total water level.

Each storm’s de-tided (sim, obs) series are cached as Parquet under data/comp/ts_cache/ on a full sweep (write-through), keyed by the node-selection + de-tiding parameters so the cache self-invalidates if those change. --examples-only then regenerates the example-panel figure from that cache without re-running the (slow) utide de-tiding.

Run:

python -m comp.validate                # full sweep, all STORMS (populates the cache)
python -m comp.validate --storms "Ida 2021" "Katrina 2005"
python -m comp.validate --examples-only         # just the example figure, from cache (fast)
python -m comp.validate --examples-only --refresh-cache   # recompute the series first

comp.validate.add_flags(df)

Return type:: DataFrame

comp.validate.bootstrap_ci(d, n=2000, seed=0)

5-95% percentile CIs for pooled (bias, RMSE, r) by resampling pairs.

Return type:: Dict[str, Tuple[float, float]]

comp.validate.classify_setting(name)

Return type:: str

comp.validate.download_storm(fname)

Return type:: str

comp.validate.latex_table(df, path)

Write the per-storm skill tabular that the appendix inputs.

Emits only the tabular environment (the surrounding table float, caption and label live in paper/appendix.tex), so the prose stays hand-edited while every number is generated – they cannot drift apart. Storms in POOR_SURGE_EVENTS are flagged with a $^{\dagger}$ . Columns: storm, n, peak bias, peak RMSE, peak r, median time-series r.

Return type:: None

comp.validate.load_storm_series(storm, fname, gauges, refresh=False)

Cached accessor for one storm’s {gauge: (sim, obs)} series.

Reads the pickle when present and current; otherwise runs the full validate_storm() (which write-through populates the cache).

Return type:: Dict[str, tuple]

comp.validate.main()

Return type:: None

comp.validate.metrics(d)

(bias, RMSE, r) of simulated vs observed peak surge.

Return type:: Tuple[float, float, float]

comp.validate.plot_examples(panels, paths, ncol=2, refresh=False)

Plot simulated surge vs de-tided observed residual for chosen (storm, gauge).

panels is a list of (storm, gauge_name) tuples. The per-storm series are loaded from the time-series cache (load_storm_series()), so regenerating this figure is instant once the cache exists; pass refresh=True to rebuild it.

Return type:: None

comp.validate.report(df)

Return type:: None

comp.validate.run(storms=None)

Return type:: DataFrame

comp.validate.scatter(df, paths)

Return type:: None

comp.validate.timeseries_skill(sim, obs)

Temporal skill of the simulated surge hydrograph against the observed residual: (corr, rmse, n_overlap) over the gauges’ common time window.

The simulated surge (2-hourly) is linearly interpolated onto the observed (hourly) residual times within the overlap, so this scores the whole storm time series, not just its peak. Returns NaNs if the overlap is too short or either series is flat (correlation undefined).

Return type:: Tuple[float, float, int]

comp.validate.validate_storm(storm, fname, gauges)

Return (rows, series) for one storm. series[name] = (sim, obs).

Return type:: Tuple[List[dict], Dict[str, tuple]]

comp.validate.within_storm_r(d)

Across-gauge correlation of peak surge after removing each storm’s mean, i.e. the spatial skill with between-storm magnitude differences taken out (a stricter test than the pooled r, which the storm spread inflates).

Return type:: float

Module contents

comp: compare historical ADCIRC surge simulations against NOAA tide gauges.

Validates the SurgeNet historical-storm ADCIRC dataset (Hugging Face sdat2/surgenet-train) against de-tided NOAA CO-OPS observations.

Entry point:

python -m comp.validate

comp.metrics(d)

(bias, RMSE, r) of simulated vs observed peak surge.

Return type:: Tuple[float, float, float]

comp.run(storms=None)

Return type:: DataFrame

comp.validate_storm(storm, fname, gauges)

Return (rows, series) for one storm. series[name] = (sim, obs).

Return type:: Tuple[List[dict], Dict[str, tuple]]