Skip to content

[WIP] feat(hardware): add daily hardware aggregates recomputed from raw - #2037

Draft
alanpeixinho wants to merge 1 commit into
mainfrom
feat/hardware-daily-aggregates
Draft

[WIP] feat(hardware): add daily hardware aggregates recomputed from raw#2037
alanpeixinho wants to merge 1 commit into
mainfrom
feat/hardware-daily-aggregates

Conversation

@alanpeixinho

@alanpeixinho alanpeixinho commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

What it is

Adds daily hardware aggregate tables (hardware_daily_builds, hardware_daily_tests) built from raw checkouts / builds / tests.

  • recompute_hardware_daily: idempotent per-day rebuild (DELETE + INSERT). Uses temp tables so the raw join runs once per day. Advisory locks avoid concurrent collisions. If raw is pruned but aggregates
    exist, keeps the stale rows.
  • Cron: hourly --days=1 (today), every 6h --days=7 — no 60-day batch.
  • prune_db: can prune aggregates by checkout_day independently of raw.
  • update_db / seed_test_data: snapshot/restore and seed support for the new tables.
  • hardware_status: unchanged — still written and read as before.

Closes #2034.

How to test

migrate

Run local database and apply migration

manual recompute

python manage.py recompute_hardware_daily --day=2025-07-18
python manage.py recompute_hardware_daily --days=7

verify rows

Check that the aggregate counts for builds, boots and tests match the sum of the raw tables (builds and tests).

prune dry-run (aggregates only)

python manage.py prune_db --older-than "30 days" \ --tables hardware_daily_builds,hardware_daily_tests --dry-run

@alanpeixinho
alanpeixinho force-pushed the feat/hardware-daily-aggregates branch 2 times, most recently from 931ec7b to 39d0d5b Compare August 17, 2026 21:19
- Add daily aggregate tables and recompute_hardware_daily (idempotent, locked,
  keeps rows when raw is pruned)
- Materialize day_tests once per day so both aggregate inserts share one raw scan
- Cron: hourly for today, every 6h for the last 7 days (drop the 60-day run)
- Wire prune_db, seed_test_data and update_db for the new tables
- Leave hardware_status written and read as before

Closes #2034

Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>
@alanpeixinho
alanpeixinho force-pushed the feat/hardware-daily-aggregates branch from 39d0d5b to 27eb2e6 Compare August 17, 2026 21:59
@alanpeixinho alanpeixinho changed the title feat(hardware): add daily hardware aggregates recomputed from raw [WIP] feat(hardware): add daily hardware aggregates recomputed from raw Aug 17, 2026
@alanpeixinho
alanpeixinho marked this pull request as ready for review August 17, 2026 22:10
@alanpeixinho
alanpeixinho marked this pull request as draft August 17, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HW-list: split hardware_status (migration + aggregations)

1 participant