Axpo PD Pulse Firewall — Energy Data Hackdays 2026

01Results — every pulse routed in 33 µs

No ground truth exists for this dataset, so the firewall optimises for the safe failure mode: reject only multi-channel-confirmed common-mode noise, keep lone high-SNR pulses, and send everything ambiguous to review instead of guessing.

14.4%

fast-rejected as common-mode interference

40.2%

kept as PD candidates (lone + high SNR)

45.5%

UNKNOWN → heavy stage / expert

4.0%

high-SNR pulses at rejection risk

33 µs

per pulse, whole firewall

~30k/s

pulses/s per core

12.5×

raw-path filter speedup (decimation + ZOH)

ML training required for stage 1

Separated PRPDs. Top-left: everything mixed (the diagnostic problem). Top-right: rejected common-mode — diffuse, low-amplitude, phase-spread background. Bottom-left: PD candidates — coherent phase-locked families near 75°/165°/255°/315°. Bottom-right: UNKNOWN — preserved for expert review, not guessed away.

Before vs after. The deliverable view: one unreadable diagram becomes a clean candidate PRPD plus an auditable rejection panel.

The rules (all explainable, no black box):

REJECT≥2 channel twins within 0.5 µs and amplitude ratio in [⅓, 3] and same polarity — symmetric common-mode; or 1 twin, ratio ≈ 1, low SNR.

KEEPno twin within 0.5 µs and SNR > 3 → PD candidate.

UNKNOWNeverything else — including twin-rich but amplitude-asymmetric high-SNR events (possibly real PD riding on common-mode noise; 1,318 pulses saved by this rule).

02How it works

The interference has physical fingerprints that are cheap to check. The firewall checks them first; only ambiguity flows downstream.

raw 100 MHz ──► filter ──► peak detect ──► pulse windows (401 samples) │ ┌─────────────────────────────────────────┘ ▼ CHEAP FINGERPRINTS (~33 µs/pulse, NumPy only) ├── cross-channel coincidence (0.2 µs) twins within 0.5 µs ├── amplitude symmetry (free) ratio + polarity of twins ├── shape-template similarity (18 µs) 12 prototypes, cosine ├── SNR · width · rise/fall · 6-band spectrum (33 µs total) ▼ ROUTE: common-mode ─► rejected (14.4%) lone + highSNR ─► PD candidate (40.2%) ambiguous ─► UNKNOWN ─► heavy ML / expert (45.5%)

Fingerprint 1 — coincidence

42% of pulses have a twin on another channel within 0.5 µs (chance ≈ 2.6%); twins are 84% same-polarity, amplitude ratio ≈ 0.7, lag ≈ 50 ns → common-mode coupling, not local PD. Matches EPRI two-coupler practice on turbine generators.

Fingerprint 2 — repetition

The median pulse has a near-identical twin (cosine 0.986). Top-10 shape families cover 73% of the corpus — the interference is a small, enumerable zoo.

Fingerprint 3 — clocks

Pulse-rate spectra show lines at ~23–30 kHz on all channels (converter/excitation clocks), and the continuous background carries a 207.5 kHz-spacing comb — a comb-notch target.

03Interference physics — the evidence

Every claim above, measured. These plots are the audit trail.

Inter-arrival times. Hard wall at exactly 30.0 µs = the detector's dead time (3,000 samples) censoring true bursts; burst structure above it.

Rate spectra. Persistent lines at ~23.3, ~23.6 and ~29–30 kHz on every channel — quasi-periodic burst trains, not a global lock.

Near-duplicate structure. Distribution of the best cosine match per pulse vs random pairs: the corpus is full of repeated shapes.

Shape families (k=8). Note C4: 91% Neutral channel, 84% lone — a local family; C2/C6: phase-spread with 48–52% twin rate — common-mode families.

Coincidence overview. 16k lone / 7k one-twin / 3.7k two-twin / 1k three-twin pulses; lone vs twin amplitudes and phases compared.

Compressibility. A small expert-labelled library describes most of the zoo — label families once per machine, not thousands of pulses.

04Raw 100 MHz data — engineering results

The raw measurements arrived mid-hackathon. We reproduced the supplied baseline bit-exactly (31.5 s wall, 2.35 GB peak RSS for both measurements, every regenerated field identical), then used the raw streams for speed and spectra.

×8-decimated + exact-ZOH filter vs the supplied numba filter on a real channel: near-identical waveforms. Position error of matched pulses: 30 ns. Recommendation: two-pass — decimated detector + exact re-filter of ±5 µs event windows.

Raw pulse-window band profiles. Even unfiltered windows are >99% below 2.5 MHz — the analogue front-end sets the band; raw pulse spectra add little.

Discovery — switching comb in the continuous background. Median block spectrum (transients suppressed) shows persistent tones at 329.6 / 537.1 / 744.6 / 952.1 / 1,159.6 / 1,392.2 kHz — constant 207.5 kHz spacing. Natural target for a comb-notch / adaptive filter. Question posed to Axpo: what switches at ~207.5 kHz?

05The expert prior — "PD where dV/dt is higher"

The challenge owner told us PD preferentially occurs where the voltage derivative is higher — near the zero crossings. We measured it.

Measured verdict: lone high-SNR pulses do cluster at 0°/180° (both polarities) — directionally consistent. But mean |cos φ| is ≈ 0.62 for every route and polarity-vs-slope agreement is 51% (chance): within a single 0.32 s record the prior does not separate routes, so it is used as a review-ranking prior (UNKNOWN triage exported ranked by prior × amplitude) and a family-plausibility feature — not as a hard routing rule.

06Experiment registry

Every experiment logged with method, data, runtime and result — the full trail is experiments/results.csv plus one self-contained script each.

#	Method	Runtime	Key result
01	Inter-arrival + rate spectra	11.0 s	23–30 kHz lines all channels; 30 µs dead-time wall
02	Fine periodicity + lead/lag	7.6 s	Near-clock pulses 8–15%; twins lag 50 ns
03	Near-duplicate + k-means sweep	6.2 s	Median twin cosine 0.986; k=4 silhouette 0.357
04	Families × coincidence	17.1 s	C4 lone-neutral family vs phase-spread common families
05–06	Firewall v1/v2 + template library	15.2 s	Twin-count alone too blunt; v2 safer routing
07	Family components + twin polarity	7.7 s	Top-10 families = 73%; twins 84% same-polarity
08	Firewall v3 FINAL	2.6 s	14.4% / 40.2% / 45.5%; high-SNR risk 4.0%
09	Raw-path: exact state-space + decimation	—	×8 + ZOH = 12.5× cheaper, 96.5% retention (synthetic)
10	Raw-path validation, real channels	12 s	87–97% match, 30 ns error; two-pass recommended
11	Raw spectra + background tones	4 s	Front-end band-limited; 207.5 kHz comb discovery
12	Turbo context checks	8 s	50.0 Hz / 3,000 rpm confirmed; asymmetry ≈ 1.0
13	dV/dt expert prior test	0.7 s	No route separation → review-ranking prior

07Honest caveats

What this system does not claim.

No ground truth existed

All routes await expert judgement on the separated PRPD panels. The UNKNOWN share (45.5%) is the price of not guessing.

Machine dependent by nature

PD patterns are machine specific (this machine: turbo, 50 Hz / 3,000 rpm). Interference physics transfers; the shape-family library is re-learned per machine in minutes.

Detection artefacts

30 µs dead time and the 5 mV threshold censor the underlying signal; both are baseline parameters an Axpo review should revisit.

Amplitudes on the fast path

The ×8-decimated detector needs per-band calibration or the two-pass exact re-filter before production amplitudes are trusted.