Repository navigation
Finals hardening - #1
Merged
Merged
Conversation
Full classification pipeline (simulator -> 6-model bake-off -> minority-class optimization -> calibration -> robustness -> SHAP -> ONNX export), validated against four real radar sensors (SAAB 77GHz, RAD-DAR, TSMS-Drone, DIAT-uSAT), plus four quantum experiments (QSVC kernel, VQC with two optimizer attempts, and a CNN+quantum-kernel ablation on real spectrogram images with a 5-seed robustness check). Trained models, ONNX artifact, and all reports/plots included; raw datasets and the DIAT-uSAT zip excluded via .gitignore. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…l, edge/scalability notes - radarsense/quantum_noisy.py: runs the existing QSVC task under a representative NISQ-era noise model (shot-based, via ComputeUncompute + FidelityQuantumKernel) instead of only exact noiseless simulation. Real result: PR-AUC essentially unchanged (0.720 vs 0.718 noiseless) but the decision boundary shifts substantially (recall 0.40->0.23, precision 0.88->1.00) -- documented in QUANTUM_MINIPAPER.md Section 12. - DATA_LICENSES.md: consolidates source/license/citation for all four real-sensor datasets plus the companion IEEE Access paper. - THREAT_MODEL.md: tabulates tested adversarial scenarios (RCS suppression, jamming, micro-Doppler masking, with real measured recall numbers) vs. explicitly untested ones (spoofing, adversarial search, data poisoning, multi-target confusion, model extraction). - MODEL_CARD.md: adds deployment footprint (model size vs. edge hardware, with an honest caveat that latency was measured on Apple M2, not embedded ARM) and scalability notes (multi-target already supported via /batch, track fusion and multi-sensor fusion flagged as unbuilt next steps). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The fidelity quantum kernel (radarsense/quantum.py) was never exported to ONNX and can't run client-side -- predicting a new sample needs a live Qiskit circuit simulation against the training set's statevectors, which neither a browser nor ONNX Runtime can do. This adds a genuinely separate FastAPI service (api/quantum_server.py + api/quantum_predict.py) that trains QSVC once at startup (n_train=800, the best point already reported in quantum_results.json) and serves real per-request quantum inference. Binary only (drone vs. not_drone) on 4 features, matching quantum.py's own task scope -- not a replacement for the classical model's full 4-class /predict. Its own reported accuracy loses to the classical model (see QUANTUM_MINIPAPER.md); this exists to serve real quantum inference honestly, not to outperform the deployed model. Verified end-to-end: /health, /predict, /batch all tested against a live uvicorn instance.
Starts the classical ONNX backend (:8080), the QSVC quantum backend (:8081), and the frontend dev server (:3000) together, regenerating the training data first if missing (needed for the quantum backend to train at startup). Ctrl+C stops everything. Verified end-to-end: all three services confirmed responding (curl 200s on both backend /health endpoints and the frontend root) before committing.
Mermaid flowchart covering the full training path: simulator -> schema gate -> 6-model bake-off -> minority-class optimization -> calibration -> SHAP / ONNX export, the quantum track (QSVC + VQC), the four real-sensor bridges, and the DIAT-uSAT CNN + quantum-kernel-on-embeddings hybrid stage (with its stability-check correction noted). Includes a stage-to-output-file reference table and a note on why the classical and quantum models are served by two separate backends.
Three real-measurement additions, no fabricated numbers: - latency_snr_sweep.py: per-sample latency for both backends across all SNR bins, tested properly via Pearson correlation (r=0.051 classical, r=-0.030 quantum -- no relationship) rather than noisy binned-median ranges, which gave a misleading result on an earlier pass and were replaced with the correct full-sample test. - CLASSICAL_VS_QUANTUM_COMPARISON.md: consolidates already-measured real numbers (onnx_benchmark.json, quantum_results.json at n=800, latency_snr_sweep_results.json) into one comparison, with an explicit scope-mismatch caveat up front (classical is full 4-class on 7,800 samples; QSVC is binary on an 800-sample subsample) so the table isn't presented as apples-to-apples when it isn't. - feature_drift.py: z-score drift of each of the 11 schema features vs. training distribution, for the held-out test set (sanity check: 0 red/yellow as expected) and all four real-sensor bridges (genuine drift, consistent with the already-documented sim-to-real gap -- TSMS shows the most drift, matching its already-lowest retrained accuracy).
roc_pr_confidence.py: one-vs-rest ROC and PR curves for all 4 classes on the real held-out test set (n=7,800, same set as every other headline number). Drone PR-AUC (0.9379) independently reproduces the already- reported 0.938 figure -- a consistency check, not a new claim. Also adds per-class confidence histograms (P(class) split by true label) showing real separation: e.g. drone confidence averages 0.875 when the sample truly is a drone vs. 0.011 when it isn't.
… model Additive only -- does not touch simulator.py, config.py, final_model.joblib, or any deployed pipeline. New raw-waveform simulator (feat31_simulator.py) reuses the exact same per-class physics as the frozen simulator (RCS, micro-Doppler, OU velocity dynamics) so the comparison is meaningful, with SNR extended to [-25,-10]dB specifically to test at -25dB as requested. Result, honestly measured, not assumed: at the -25dB slice, the FROZEN 11-feature model wins (98.5% accuracy, 100% drone recall) despite never being trained below -20dB, beating the purpose-built 31-feature model (96.8% accuracy, 71.8% drone recall) trained specifically for that noise range. Full writeup with interpretation and caveats in artifacts/reports/FEAT31_EXPERIMENT.md. Recommendation: keep the frozen 11-feature model deployed; this branch stays as a documented, reproducible standalone experiment showing the schema choice was tested, not assumed.
Two real issues found and fixed while trying to genuinely improve the 31-feature model (not just tune it until it looked better): 1. feat31_train.py had NO threshold tuning (raw argmax) -- reused radarsense.optimize's tune_threshold/threshold_predict (schema-agnostic, operate on proba arrays only) the same way the incumbent model was tuned. Drone recall at the -25dB slice improved 71.8% -> 76.9%. 2. feat31_compare.py's out-of-distribution test for the FROZEN model was miscalibrated: noise_frac was computed against the EXTENDED [-25,-10]dB band, capping at 1.0 exactly at -25dB -- the SAME max noise the frozen model already saw at -20dB in training, not genuinely extrapolated noise. Fixed to compute noise_frac against the model's ACTUAL trained band, so -25dB now means noise_frac=1.5 (50% beyond anything trained on) -- real out-of-distribution stress, not a relabeled easier test. Corrected result: genuinely mixed, not a clean win either way. At true -25dB OOD noise, 31-feature has higher accuracy (96.1% vs 95.1%) but the frozen 11-feature model has meaningfully better drone recall (88.2% vs 76.9%) -- the metric this project has always treated as the headline number given the 25x FN:FP cost asymmetry. Both reported, neither hidden.
Same rigor as the incumbent 11-feature pipeline, all real measured numbers: - feat31_export_onnx.py: ONNX export, 1.0000 parity, 0.054ms median latency, 41,384 samples/sec, 3,446 KiB - feat31_explain.py: SHAP summary + waterfall; drone top-5 includes both amplitude and energy spectral-bandwidth features, directly capturing the rotor-blade modulation signal the 11-feature schema's micro_doppler_ bandwidth isolates explicitly -- physically consistent, not coincidental (fixed an incomplete plausibility check along the way: energy_spectral_ bandwidth/energy_dominant_freq were missing from the physically-plausible set despite energy=amplitude^2 making them equally valid) - feat31_calibrate.py: isotonic calibration, ECE 0.014-0.017 -> 0.003-0.005 across all 4 classes - feat31_robustness.py: SNR sweep + adversarial stress tests on the RAW WAVEFORMS (re-extracting features after each perturbation, not perturbing the statistics directly) -- shows the same micro-Doppler-masking vulnerability class as the incumbent (83%->7% drone recall under increasing smoothing), confirming shared physical dependence, not an independent weakness - api/feat31_predict.py + api/feat31_server.py: third independent backend, verified end-to-end (/health, /predict against a live uvicorn instance) - start_all.sh: now launches all 3 backends + frontend together This model is genuinely production-ready as a third backend. Whether it should be preferred over the incumbent remains the honest, mixed answer from the -25dB comparison: better accuracy under genuine extrapolated noise, worse drone recall -- not overridden or hidden by this build-out.
…feature detection handling
New, previously-unexplored territory: track_simulator.py generates 10,000 tracks x 5 frames (shared clean physics per track via simulator.py's _class_block, imported unmodified, independent per-frame noise -- mimics consecutive detections of one physical target), split by track (never by frame) to avoid leakage. track_fusion.py compares single-frame vs. track-fused (mean-probability and majority-vote) performance for both the frozen classical model and a freshly-trained QSVC, under identical fusion rules. Result: track fusion helps BOTH backends (classical drone recall 88.1%->91.9%, quantum 69.7%->78.4%), and quantum still trails classical after fusion -- but the relative gain is 2x+ larger for quantum (+8.6 points vs classical's +3.8), consistent with fusion being a variance- reduction technique that helps a noisier baseline classifier proportionally more. Reported honestly: not 'quantum catches up', a real but bounded effect. Full writeup in TRACK_FUSION_EXPERIMENT.md.
…sult Every other quantum result in this project (quantum.py, vqc.py, diat_usat_cnn.py, raddar_cnn.py, track_fusion.py) is noiseless exact statevector simulation, always disclosed as such. This runs the identical ZZFeatureMap(4, reps=2) circuit on real hardware (ibm_fez, IBM Cloud channel via qiskit-ibm-runtime), comparing 15 pairwise fidelity estimates (inversion-test method, 2048 shots each, one batched job -- job d9f6pskjeosc73fji2j0, completed in 172s) against the exact noiseless simulation for the same 6 real data points. Result: mean absolute deviation 0.020, Pearson correlation 0.920 between real hardware and noiseless simulation -- hardware preserves the fidelity kernel's relative structure well despite real NISQ noise. A noise characterization, not a retrained hardware classifier (15 pairs isn't enough for that claim) -- scope and caveats stated explicitly in IBM_HARDWARE_EXPERIMENT.md. Required two separate environments: qiskit-ibm-runtime needs qiskit>=2.3, qiskit-machine-learning (used by vqc.py) needs qiskit<2.0 -- genuinely incompatible. Installing qiskit-ibm-runtime in the main environment silently upgraded qiskit and numpy, breaking VQC's import chain and matplotlib; caught immediately, main environment reverted to the pinned versions and verified working again, and a separate .venv_ibm_quantum/ (gitignored) created for the hardware-submission step only. Credentials read from .claude/ibm_quantum_credentials.json (gitignored, never committed).
Same ZZFeatureMap fidelity-kernel-on-frozen-CNN-embeddings setup as diat_usat_cnn.py, run on RAD-DAR's independent 8.75GHz FMCW range-Doppler matrices instead of DIAT-uSAT's X-band CW spectrograms. CNN itself is strong (95.4% acc, 97.8% drone recall), but the quantum-kernel-wins pattern does not replicate here -- quantum trails classical RBF/linear kernels at every train size tested, by a wide margin at n=200. Reported as a genuine boundary condition rather than reframed to match the first result.
…l comparison Controlled test at -10/-15/-20/-25 dB (SNR held fixed per set, not sampled), trained on the same data the deployed 31-feature model uses, class-balanced for a learnable drone boundary. 5 seeds per level. Result: 77-79% accuracy across all four levels, same range as an independently-run teammate result on the same schema, though flatter across SNR rather than monotonically declining -- reported honestly rather than smoothed to match.
Unified generator draws each instance's latent physical parameters once, then derives both feature sets from the same target via each schema's own frozen dynamics function -- a genuine combined dataset, not two independently-sampled datasets joined by column. Beats both individual models at SNR<=-23dB: 99.2-99.5% accuracy and 100% drone recall across 3 seeds, vs 31-feature's 96.1%/76.9% and 11-feature's 95.1%/88.2% (OOD). Likely mechanism: physics and waveform-stat features are independently- noised measurements of the same latent RCS/micro-Doppler parameters, so combining them is genuine noise-reduction via redundant measurement.
Same 7-way variant comparison the incumbent 11-feature model went through (reuses optimize.py's variant-fitting/threshold-tuning functions unmodified). Finding: no single strategy wins on both axes -- baseline (no imbalance handling) wins overall by PR-AUC and edges out the deployed class_weight variant after threshold tuning (85.6% vs 83.1% drone recall at comparable accuracy), but class_weight specifically wins at the SNR<=-23dB slice that is this model's whole reason for existing. Reported as a genuine scope-dependent tradeoff, not resolved by picking a winner. Candidate model saved separately, not auto-deployed.
Tests whether tripling waveform length improves noise robustness of the SHAP-flagged spectral features. Result: real overall improvement (+1.2pt accuracy, +2.6pt drone recall) driven by genuinely tighter DFT-derived feature estimates, but flat drone recall specifically at the SNR<=-23dB slice this schema targets -- the extreme-noise-robustness hypothesis is only partially supported, reported honestly rather than oversold. Side-finding while building the mechanism check: amplitude_dominant_freq is a completely dead feature (exactly 0.0 for all 36400 training samples, every class) because the DC amplitude offset always dominates its own DFT peak; energy_dominant_freq is nearly as degenerate. Neither corrupted prior results (CatBoost naturally zeroed their importance) but both are disclosed rather than left unnoticed now that they've been found.
Tunes a separate drone-decision threshold per SNR bin instead of one global threshold, testing whether exploiting the real SNR-dependent probability-calibration shift (thresholds swing 0.21-0.82 across bins) translates into better decisions. Result: no improvement -- drone recall drops 1 point overall and ties exactly at the SNR<=-23dB slice, while adding real deployment complexity (8 thresholds + a bin lookup instead of 1). Reported as a validated negative result, not reframed to justify the added complexity.
Cherry-picked ravindraogg's API server + frontend ONNX inference updates and regenerated incumbent model artifacts onto feature31-standalone so new backend work here builds on top of the already-integrated frontend rather than duplicating it. Only conflict was api/quantum_server.py's CORS middleware, added independently on both branches -- resolved by keeping one copy, no functional difference.
ONNX export (100% parity, 0.038ms median latency), inference client, and FastAPI server matching the existing 11/31-feature backends' conventions exactly -- fourth independent service on port 8083, wired into start_all.sh. Verified end-to-end with real live HTTP requests (not just the training script): a real -25dB drone correctly classified via curl, matching the offline model bit-for-bit; /batch correctly classifies one real sample of each class. Backend only, no frontend files touched.
The one genuinely untested generalization question in the project: train on one real sensor's actual data, evaluate zero-shot on a different real sensor, no retraining. All four bridges already share the 11-feature schema, making a full 4x4 matrix possible without new data infrastructure. Finding: naive-threshold transfer degrades substantially (25-63pt accuracy drop vs in-distribution), but an oracle-threshold diagnostic separates two failure modes -- some pairs (e.g. RAD-DAR->SAAB, PR-AUC 0.972) are mostly a miscalibrated decision boundary, recoverable by re-thresholding alone; others (e.g. RAD-DAR->TSMS, PR-AUC 0.170) are genuine distributional mismatch that no threshold choice fixes. Reported as a real, nuanced, mostly-negative result, consistent with the RAD-DAR CNN replication finding elsewhere in this project.
…el gaps Implements the two items THREAT_MODEL.md explicitly listed as future work: an optimized (gradient-free, since CatBoost isn't differentiable) search rather than the existing single-countermeasure sweeps, and evasion via a crafted feature vector rather than pure signature suppression. Result: 100% evasion success (179/179 real drones) at a small joint perturbation (median 1.04 std-units across 6 attackable features) -- materially more severe than any single-feature countermeasure already tested. rcs_estimate is the most-exploited feature in combination, despite RCS suppression alone not evading detection in the existing test -- a real interaction effect a one-feature-at-a-time sweep can't surface. Explicitly framed as an upper bound under an idealized full-sensor-control attacker, not a demonstrated real-world attack (no per-feature achievability cost model yet). THREAT_MODEL.md updated to move this from "not tested" to tested, with a mitigation-direction note extending the existing one.
Auxiliary regression head, trained jointly with classification, forced to predict physically-meaningful quantities (doppler_velocity, micro_doppler_ bandwidth, micro_doppler_jitter) computed directly from the same raw range-Doppler matrix via matrix_features() -- self-supervised, no extra labels needed. Same backbone as raddar_cnn.py so the comparison isolates the added loss term. Result: a real negative finding. The physics loss does not improve classification (94.5%/95.5% recall vs baseline 95.4%/97.8%) and doesn't change the quantum-vs-classical kernel finding on the resulting embeddings (quantum still trails RBF at every train size, same as the non-physics-informed CNN). Reported honestly rather than tuned until it looked better.
Fills the last gap in feat42's production infrastructure -- the 11 and 31-feature models both had interpretability/calibration work, feat42 had none despite being the strongest result. Two independent explanation methods (SHAP exact tree attribution, LIME locally-weighted linear surrogate) both confirm the combined model genuinely draws on both feature sets: physics carries 38.8-42.7% of importance from only 10/41 columns, waveform-stat carries 55.2-57.6% from 30/41, 7/10 top features agree between methods. Answers the question left open in FEAT42_COMBINED_EXPERIMENT.md with real corroborating evidence, not a single method's say-so. Calibration: already substantially better raw ECE than the 31-feature model (0.001-0.006 vs 0.014-0.017 across classes) before any post-hoc correction; isotonic calibration still roughly halves ECE further.
Query-only attacker (no training data, no model internals) against the live 11-feature API. Verified predict_one() matches an actual live HTTP request bit-for-bit before running the bulk of the attack that way for speed. Surrogate CatBoost trained on (random query, API response) pairs reaches 99.2% drone-gate fidelity at 5000 queries, 93.0% at just 200. The real finding: evasions crafted entirely against the surrogate, with ZERO further live queries, transferred to fool the real target 100/100. More severe than the direct adversarial search alone -- an adversary needs only one moderate one-time extraction run, then develops attacks offline indefinitely with no further live footprint. THREAT_MODEL.md updated; three of five originally-named gaps are now tested with real numbers.
Every prior classical-vs-quantum comparison picks a winner; this tests combining predictions instead. Three strategies (simple average, weighted average, logistic-regression stacking), 5 seeds x 3 train sizes, apples- to-apples scoring via a held-out stack-fit/final-eval split. Nuanced honest result: no ensemble beats classical alone on PR-AUC (a first single-seed smoke test suggested otherwise -- the full multi-seed run shows that was a lucky draw). But stacking shows a real, paired, seed-by-seed recall advantage (10 wins/4 ties/1 loss across 15 pairs, +1.5-2.2 recall points at ~1 point precision cost) -- a trade the project's own 25:1 FN:FP cost asymmetry favors. Meta-model coefficients show genuine dual use of both signals, not picking one.
…eck and explainable decision log Extends the RAD-DAR classical+quantum ensemble finding with three pieces: 1. DIAT-uSAT replication (quantum's winning case, unlike RAD-DAR): the same recall-favoring stacking pattern holds, even more cleanly -- 15/15 paired seed wins (zero ties, zero losses) vs RAD-DAR's 10/15. Confirms the effect isn't specific to one sensor. 2. Calibration check: both kernels are already reasonably well-calibrated in aggregate (94.7%/96.7% actual accuracy when saying >=80% confident), so the ensemble isn't being fooled by systematic overconfidence -- but isotonic-calibrating before stacking still buys +2.2 recall points at ~1.7 precision points cost, on top of the base finding. 3. Explainable decision log: per-detection comparison showing what each kernel said, what the ensemble said, and why -- the meta-model's contribution is exact (2-input logistic regression, not approximated), and LIME explains each kernel's own reasoning on the CNN embedding. Shows both a real win case and an honestly-reported failure case where the ensemble followed a confidently-wrong quantum vote.
…ening that didn't work Tested response-hardening (rounding/withholding probabilities) FIRST, before committing to it: all four variants gave identical extraction fidelity, since the attack only ever needed the hard label. Also found the decision boundary needs only ~10 queries to reach 97% drone-gate fidelity, ruling out reasonable rate-limiting alone as a full fix. What measurably works: an IsolationForest anomaly filter on the real training distribution, catching 95.2% of synthetic extraction-style queries while wrongly rejecting only 1.1% of real detections. Combined with rate limiting (100/min), deployed live in api/server.py and verified against a running server: real detections still classify correctly, synthetic queries get HTTP 400, excess requests get HTTP 429. At the same 5000-query budget, transfer-attack success drops from 100% to 84% -- a real, substantial improvement, honestly not a complete fix (disclosed plainly, not oversold). THREAT_MODEL.md updated.
Harvested 300 real evasion perturbations from the black-box search (against training drones, test set kept untouched for honest re-evaluation), augmented training data with them still correctly labeled "drone", retrained with the same config as the deployed model. Result: evasion success drops from 100% to 80.1%, and the median perturbation needed by attacks that still work rises 50% (1.04 -> 1.55 std-units). Normal test performance held up, drone recall even improved slightly. Reported as a real, partial improvement -- one round of the standard attack/retrain arms race, not a solved problem. Retrained model saved separately, not auto-deployed in place of the current one.
…etter Tests rather than assumes whether the combined model's second feature set actually helps against masking, given both trace back to the same micro_doppler_bandwidth/jitter root cause. Fair test: regenerates masked drone instances with the same attenuation fed into BOTH feature derivations consistently, not just the physics side. Result: 42-feature model wins at every level -- 98.5% vs 91.8% baseline, 93.1% vs 76.4% at x0.7, 63.1% vs 51.8% at x0.5, 16.8% vs 12.3% at the severe x0.3 case. Not a fix, but a real, already-available mitigation: independent noise realizations of the same latent signal provide genuine residual robustness, not just accuracy gain.
… so far Directly targets the diagnosed mechanism (over-reliance on micro_doppler_bandwidth as a near-sufficient statistic) rather than working around it: 50% of drone training rows get a masked duplicate (same 0.3-0.9x attenuation range the attack uses), still correctly labeled drone, mixed into training. Result: at the severe x0.3 masking level, drone recall goes from 12.3% to 64.1% -- more than 5x better -- at a real, disclosed cost of 3.6 points of unperturbed-case recall (91.8%->88.2%). Bigger single effect than the 42-feature model's fusion-based robustness (16.8% at x0.3). Retrained model saved separately, not auto-deployed.
Structurally different mechanism from feature-dropout training: a second classifier trained on 9 non-micro-Doppler features, blended in when the primary model's micro-Doppler reading looks anomalously low for a drone. Caught and fixed a real bug before trusting the result: the suspicion cutoff was first calibrated on the full population's micro-Doppler distribution (dominated by aircraft, which naturally reads near-zero), giving a degenerate 0.0 threshold that never triggered. Fixed by calibrating on the drone class specifically. Result: real but modest -- x0.3 masking recall improves 12.3% to 29.2%, well below feature-dropout training's 64.1%, at a broader accuracy cost. Genuinely helps naturally-quiet-rotor drones too, not just the attacked case. Not hyperparameter-tuned; noted as the clear next step.
…ero-shot Zero-shot transfer (CROSS_SENSOR_TRANSFER.md) is a harder bar than most real deployments face -- a new sensor install can usually collect a small labeled calibration set. Tests two adaptation strategies (threshold-only recalibration, full retraining) at 10/20/50-shot budgets across all 12 source-target pairs. Result: full retraining with just 50 labeled target examples recovers average cross-sensor accuracy from 48.8% to 69.8% -- substantial, not complete (in-distribution is 85-99%). Same accuracy-vs-recall trade-off pattern as the classical+quantum ensemble work shows up again: threshold-only recalibration plateaus on accuracy but gives higher drone recall than full retraining at every budget.
…abricating a fix Checked precisely rather than assumed: all 4 sensor bridges already use the IDENTICAL snr_db formula -- the inconsistency is what it's anchored to (each sensor's own median self-defines as -15dB), not the formula itself. A genuinely absolute calibration would need real system parameters (transmit power, antenna gain, noise figure) not available for these third-party datasets -- building a fake unified calibration without that data would be unfounded, not a fix, so it wasn't attempted. Also found: SAAB and TSMS have real radiometric RCS calibration (corner-reflector reference); RAD-DAR and DIAT-uSAT use an uncalibrated z-score proxy -- rcs_estimate is inconsistently anchored too. Direct, falsifiable test instead: dropping snr_db and rcs_estimate improves mean cross-sensor accuracy by 10.7 points (49.3%->60.0%), confirming calibration inconsistency was a real, measurable, but partial contributor -- most of the transfer gap remains, plausibly genuine physics/hardware domain shift.
…threat vectors Micro-Doppler masking: augmentation-intensity frontier + escalation routing (deployable default recovers x0.3 recall 12.3->48.7% at near-zero clean cost; aggressive setting reaches 91.8%). Probability-blend stacking and md<cutoff replace-routing both tested and rejected with numbers. Adversarial evasion: iterative training plateaus ~71-82% (its honest ceiling) but randomized-smoothing certification gives a provable guarantee -- 83.3% of drones certified un-evadable within the attack's median perturbation, up from 19.2%; physics gate catches 41% of the rest. Cross-sensor transfer: CORAL recovers +13 recall points with zero target labels; leave-one-sensor-out pooling hits 64% mean (+15pt); few-shot climbs to 82% at 500 labels. Moment-matching and CORAL+few-shot kept as honest negatives. Model extraction: fingerprint attribution fails three ways (model too generic to sign), boundary label-noise joins response-hardening as useless, anomaly filter quantified as defeated by realistic queries; per-key budgeting is the surviving operational lever. Honest ceiling established, not oversold.
…ensor + error-mitigated hardware Physical-cost achievable-evasion: unconstrained 100% collapses to ~22% under a realistic physical budget (mean+/-2.2% over 3 seeds -- also answers single-seed). Extraction defanged (honest, partial): hardening the deployed model cuts clone-crafted evasion transfer 100% -> 78%. Not zero; reported as such. Masking stronger posture: same escalation knob at rescue_t=0.5 recovers x0.3 recall 12.3% -> 76.9% (6x) for -2.6pt accuracy, alongside the near-zero-cost 4x default. Cross-sensor combined: honest negative -- stacking pool+CORAL+few-shot tops out at 79%, below the 82% single-source few-shot. Mechanisms are substitutes. Live acoustic drone detection: phone-mic sensor whose blade-pass modulation is the acoustic analog of radar micro-Doppler; runs on real recordings. Larger error-mitigated IBM hardware run: 28 circuits on ibm_fez, raw vs mitigated. Mitigation improves fidelity agreement with noiseless sim (MAE 0.026 -> 0.021, correlation 0.981 -> 0.996). Validation, not an advantage.
…weep visualization, and custom feature engineering
…training, conformal, perf Radar DSP front-end on REAL 77GHz SAAB I/Q: range-Doppler FFT + CA-CFAR (99.6% detection) + constant-velocity Kalman tracker (-45.8% track jitter), with range-Doppler and track figures. Directly answers "no CFAR / no range-Doppler / no tracking." Test suite (9 pytest tests: schema, simulator, CFAR, Kalman, metrics, thresholds) + GitHub Actions CI + pyproject.toml. Answers "no tests / no CI." Headline model trained + tested on REAL SAAB data across 10 seeds: 99.2% +/- 0.06% accuracy, PR-AUC 0.9999. Split-conformal prediction meets its 90% coverage guarantee (89.8% empirical). Answers "sim-only / single-seed / no uncertainty." Performance benchmark: sub-ms inference (p50 0.05ms), ~50k samples/s batched, 2.4MB. README title corrected (drops "Quantum Enhancement" overclaim) with an honest-scope banner and a system architecture diagram.
… real I/Q, ablation, more tests OS-CFAR (ordered-statistic) added: on real 77GHz clutter it drives the guard-band false-alarm rate 3.3% -> 0.0%, resolving the CA-CFAR non-Gaussian clutter limitation the DSP report disclosed. End-to-end on real I/Q: raw samples -> range-Doppler -> features-from-the-map -> classification, 91.1% +/- 0.4% over 5 seeds on a harder 37%-drone real set -- the full radar chain closed on real data, not bridge features. Consolidated ablation (ablation_summary.py): reproducibly reads the measured result JSONs and assembles each defense's per-component marginal contribution. Code quality: test suite 9 -> 13 (OS-CFAR, map-features, conformal coverage, Doppler-velocity), ruff lint config + CI lint step.
…sets, and live telemetry HUD
…data (thorough null result) Adds three genuinely different quantum approaches -- Projected Quantum Kernel (Huang et al. 2021), a ZZ/Z/Pauli feature-map expressibility study, and kernel-target alignment diagnostics -- tested on both simulated AND real SAAB 77GHz features, 5 seeds. Findings: (1) a single-split "quantum wins on real data" was NOISE, dissolved by multi-seeding (gap -0.058 < pooled std 0.054) -- rigor prevented a false quantum-win claim. (2) The project's ZZ encoding is a poor choice: plain Z (no entanglement) far outperforms it (0.765 vs 0.369 sim). (3) The projected quantum kernel does not rescue quantum either. Net: classical wins robustly on both datasets against every quantum method tested -- turning the earlier under-powered null into a well-supported one.
…m diagnostic, sensor fusion Multi-target tracking: OS-CFAR multi-peak -> greedy GNN association -> bank of Kalman filters (spawn/confirm/coast/kill). 80% track completeness, 0.25 cardinality error, 0.2 spurious tracks/scene on controlled 3-target scenes. Fixed a double-predict bug via KalmanTracker.peek(). Baseline comparison on real SAAB (5 seeds): CatBoost 0.990 is best but the margin over random forest (0.987) is small and majority-class is already 0.863 -- the 99% is real and the task is easy, disclosed. Quantum diagnostic: mechanistically explains the null via kernel concentration -- quantum ZZ off-diagonal spread 0.063 vs classical 0.363, effective rank 67 vs 2, alignment 0.14 vs 0.54 (exponential-concentration failure mode). Multi-sensor decision fusion (honest negative): stacked fusion ties the best single specialist (+0.001); pooled training (0.958) beats fusion -- sensors don't co-observe, so pooling wins.
…collinearity, README Data leakage (the flagged headline): real_data_headline.py did a random split on 75,868 SAAB segments from 130 continuous recordings, leaking whole-recording segments across train/test (PR-AUC 0.9999 +/- 0.00002 = memorization). Rewritten to measurement-GROUPED 5-fold CV. Honest number: 92.2% +/- 3.1% accuracy, PR-AUC 0.99. Also verified the dataset's own split column leaks (all 130 measurements span splits), so grouped CV is the only honest protocol. Collinear feature: doppler_freq_shift = 2*v*f_c/c is exactly collinear with doppler_velocity (corr=1.0, verified) -> the "11-feature" schema has 10 independent features, and SHAP split credit spuriously. shap_corrected.py recomputes attribution on the 10 independent features. IBM token: removed plaintext .claude/ibm_quantum_credentials.json; the code now reads IBM_QUANTUM_TOKEN/IBM_QUANTUM_CRN from env (token must still be rotated at IBM -- it was exposed). CORS: all four servers dropped allow_origins=["*"] + allow_credentials=True (invalid/unsafe combo) for env-configurable origins, credentials off. README: corrected the false "regenerates every number" claim and the leaky "official splits" 99.2% attribution (now flagged with the 92.2% leak-free number).
…x 3-mode 4-class SNR stress test
… keep ONNX clean Serves the finals-hardening masking defense without swapping the deployed ONNX. api/masking_rescue.py: keeps the clean model's decision everywhere and escalates a non-drone prediction to drone ONLY when micro-Doppler is anomalously low (a masking signature) AND the micro-Doppler-immune kinematic fallback is >=0.98 confident. Wired into api/server.py /predict and /batch; advertised in /health. Verified end-to-end (2500-sample clean set + masked drones): clean drone recall and precision are bit-identical to baseline (0.9322 / 0.7237), while masked-x0.3 recall recovers 12.3% -> 32.8% (2.7x). Gate is a tunable dial. Chose routing over a retrained/hardened ONNX after measuring that the combined hardened model moved clean recall -6.7pt and precision -17pt -- retraining shifts the boundary, routing does not. Deployed ONNX stays the clean baseline.
…ics, target presets and WASM label decoding
…t the frontend calls) The frontend now routes classical inference through api/app.py's /predict/classic for full-precision Python ONNX. That endpoint used the clean model with no masking defense. Added apply_masking_rescue (same server-side routing as api/server.py): preserves clean recall/precision, recovers micro-Doppler-masked drones. Advertised in /health/classic. Note: the parallel frontend fix (Ravindra, a134fd4) already corrected the class order, removed the ground-truth stub oracle, fixed carrier physics (66.67x), and renamed stealth_uav->anomaly -- so this only adds the missing rescue on the API side rather than duplicating those client-side fixes.
…reconciled evaluation plots
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.