Halyna R.

[halyna1995]

What I built

• Monthly Airflow DAG in dags/taxi_pipeline.py
• Strict task sequence: ingest_taxi_month → dbt_run → dbt_test
• Monthly partition selection based on the Airflow logical date
• catchup=False
• Two retries with a retry delay
• Week 10 reference dbt project in include/dbt_project/
• Seven-month backfill for January–July 2024
• Repeated backfill with identical SQL row counts, confirming idempotency
• Operational instructions in RUNBOOK.md
• Assignment summary and evidence in ASSIGNMENT_REPORT.md
• AI usage documentation in AI_ASSIST.md
• Screenshots and SQL evidence in the evidence/ folder

How to review

• DAG implementation: dags/taxi_pipeline.py
• Operational instructions: RUNBOOK.md
• Backfill and idempotency results: ASSIGNMENT_REPORT.md
• AI assistance documentation: AI_ASSIST.md
• Screenshots: evidence/

How to run

From a clean clone, with Docker Desktop and the Astro CLI installed:

astro dev start

Create the azure_pg connection in the Airflow UI:

Admin → Connections

Then run the DAG integrity test:

astro dev pytest tests/test_dag_integrity.py --args "-v"

For a manual run, open taxi_pipeline, click Trigger, and use:

{
  "target_date": "2024-01-01"
}

To run the seven-month backfill:

astro dev run backfill create --dag-id taxi_pipeline --from-date 2024-01-01 --to-date 2024-07-31 --max-active-runs 1

To repeat the backfill for the idempotency check:

astro dev run backfill create --dag-id taxi_pipeline --from-date 2024-01-01 --to-date 2024-07-31 --max-active-runs 1 --reprocess-behavior completed

What reviewers should see

• DAG visible in Airflow without import errors
• Graph order: ingest_taxi_month → dbt_run → dbt_test
• Successful local manual run
• Successful dbt_run
• Successful dbt_test
• Seven successful monthly backfill runs
• Identical row counts after repeating the backfill
• Evidence and screenshots in the evidence/ folder

Test results

• test_dag_integrity.py: all tests passed
• DAG import errors: none
• Manual run: successful
• Backfill: seven runs completed successfully
• Repeated backfill: row counts remained identical

Shared Airflow deployment

Shared Airflow deployment: https://github.com/lassebenni/c55-shared-airflow/pull/10/changes

Known limitations / out of scope

• Shared Airflow deployment proof is not included yet.
• PostgreSQL credentials are configured through the Airflow UI and are not committed to the repository.

Self-check

• bash .hyf/test.sh passes
• astro dev pytest tests/test_dag_integrity.py --args "-v" passes
• catchup=False is configured
• retries are configured
• no credentials are committed
• seven-month backfill evidence is included
• repeated backfill row counts are included
• RUNBOOK.md is completed
• AI_ASSIST.md is completed
• ASSIGNMENT_REPORT.md is completed

[github-actions]

📝 HackYourFuture auto grade

Assignment Score: 100 / 100 ✅

Status: ✅ Passed
Minimum score to pass: 60
🧪 The auto grade is experimental and still being improved

Test Details

=== Week 12 Autograder — Orchestrated Pipeline ===
  ✓ PASS  found dags/taxi_pipeline.py
  ✓ PASS  found tests/test_dag_integrity.py
  ✓ PASS  found requirements.txt
  ✓ PASS  found RUNBOOK.md
  ✓ PASS  found ASSIGNMENT_REPORT.md
  ✓ PASS  found AI_ASSIST.md
  ✓ PASS  Level 1: required files (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: no NotImplementedError stubs left
  ✓ PASS  dags/taxi_pipeline.py: defines a @dag with tasks
  ✓ PASS  Level 2: DAG implemented (15/15 pts)
  ✓ PASS  dags/taxi_pipeline.py: ingest, dbt_run, and dbt_test all present
  ✓ PASS  dags/taxi_pipeline.py: tasks are chained with >>
  ✓ PASS  Level 3: sequential tasks (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: dbt runs through uvx (works on the image's Python 3.14)
  ✓ PASS  dags/taxi_pipeline.py: retry behaviour configured (retries=...)
  ✓ PASS  Level 4: uvx dbt + retries (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: partition derived from the logical date
  ✓ PASS  dags/taxi_pipeline.py: catchup=False and max_active_runs set
  ✓ PASS  Level 5: parameterized runs (15/15 pts)
  ✓ PASS  RUNBOOK.md: filled in (2905 chars, no TODO left)
  ✓ PASS  AI_ASSIST.md: filled in (1020 chars, no TODO left)
  ✓ PASS  ASSIGNMENT_REPORT.md: filled in (3598 chars, no TODO left)
  ✓ PASS  screenshots: found 5 image file(s) (need ≥3 for Graph + Grid/run + task log)
  ✓ PASS  Level 6: documentation + screenshots (10/10 pts)

Score: 100 / 100  (passing: 60)  pass=true

Reminder: screenshot *content*, shared-Airflow deploy proof, and before/after
row counts are still teacher-reviewed. Autograder green is not a pass — a
high static score is necessary but not sufficient.