Mohamad bader AA

[noneeeed]

What I built

• Airflow DAG: dags/taxi_pipeline.py (schedule, ingest_taxi_month → dbt_run → dbt_test, partition from logical date, retries)
• Week 10 dbt project dropped into include/dbt_project/
• Backfill + idempotency evidence (Task 5)
• Operational notes: RUNBOOK.md
• Assignment report: ASSIGNMENT_REPORT.md
• (Target tier) namespaced DAG deployed to the shared class Airflow
• AI usage: AI_ASSIST.md

How to review

• DAG: read dags/taxi_pipeline.py (schedule, task order, catchup=False, retries, partition date).
• Evidence: backfill / idempotency notes in ASSIGNMENT_REPORT.md; operations in RUNBOOK.md.
• AI usage: AI_ASSIST.md.

How to run

From a clean clone, with the Astro CLI installed:

astro dev start
astro dev pytest tests/test_dag_integrity.py --args "-v"

Then open the UI URL, add the azure_pg connection (Admin → Connections), unpause taxi_pipeline, and trigger a run for a real month (e.g. 2024-01-01).

Prerequisite: your Week 10 dbt project in include/dbt_project/ and the azure_pg connection to the shared Postgres.

What reviewers should see (expected results)

Fill in what your run actually produces:

• test_dag_integrity.py result: <e.g. all pass>
• Task order in the Graph view: <e.g. ingest → dbt_run → dbt_test>
• Backfill: <e.g. 7 runs green, re-run is idempotent (same row count)>
• Deployed DAG name (Target tier): <e.g. taxi_pipeline_>

Known limitations / out of scope

-(requires az login first), then astro dev run connections add azure_pg --conn-uri "$PG_URL". | [Make sure the PGURL is populated from UI by clicking left panel Admin> connections OR http://c55-data-week-12.localhost:6563/connections]

Self-check

• [ ✅] bash .hyf/test.sh passes
• [ ✅]astro dev pytest tests/test_dag_integrity.py passes
• [ ✅] catchup=False and retries are set
• [ ✅] No credentials committed (connections added in the UI, not in code)
• [ ✅] Backfill / idempotency evidence is in ASSIGNMENT_REPORT.md

[lassebenni]

Reopening to re-trigger grade workflow after checkout fix on main.

[lassebenni]

Autograder note (teacher)

The CI score is a static smoke test only (file presence + code patterns + docs/screenshot files). It does not grade:

• whether screenshots show a real green Graph / Grid / task log / shared-UI run
• before/after backfill row counts
• shared-Airflow deploy proof (merged PR URL + tagged screenshot)

Autograder 100 ≠ assignment pass. Teachers review those from your PR against the Week 12 deliverables. Please make sure ASSIGNMENT_REPORT.md is fully filled (no TODOs), commit at least 3 screenshot images, and include shared-deploy evidence when the class VM is up.

[github-actions]

📝 HackYourFuture auto grade

Assignment Score: 100 / 100 ✅

Status: ✅ Passed
Minimum score to pass: 60
🧪 The auto grade is experimental and still being improved

Test Details

=== Week 12 Autograder — Orchestrated Pipeline ===
  ✓ PASS  found dags/taxi_pipeline.py
  ✓ PASS  found tests/test_dag_integrity.py
  ✓ PASS  found requirements.txt
  ✓ PASS  found RUNBOOK.md
  ✓ PASS  found ASSIGNMENT_REPORT.md
  ✓ PASS  found AI_ASSIST.md
  ✓ PASS  Level 1: required files (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: no NotImplementedError stubs left
  ✓ PASS  dags/taxi_pipeline.py: defines a @dag with tasks
  ✓ PASS  Level 2: DAG implemented (15/15 pts)
  ✓ PASS  dags/taxi_pipeline.py: ingest, dbt_run, and dbt_test all present
  ✓ PASS  dags/taxi_pipeline.py: tasks are chained with >>
  ✓ PASS  Level 3: sequential tasks (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: dbt runs through uvx (works on the image's Python 3.14)
  ✓ PASS  dags/taxi_pipeline.py: retry behaviour configured (retries=...)
  ✓ PASS  Level 4: uvx dbt + retries (20/20 pts)
  ✓ PASS  dags/taxi_pipeline.py: partition derived from the logical date
  ✓ PASS  dags/taxi_pipeline.py: catchup=False and max_active_runs set
  ✓ PASS  Level 5: parameterized runs (15/15 pts)
  ✓ PASS  RUNBOOK.md: filled in (5574 chars, no TODO left)
  ✓ PASS  AI_ASSIST.md: filled in (11037 chars, no TODO left)
  ✓ PASS  ASSIGNMENT_REPORT.md: filled in (1233 chars, no TODO left)
  ✓ PASS  screenshots: found 5 image file(s) (need ≥3 for Graph + Grid/run + task log)
  ✓ PASS  Level 6: documentation + screenshots (10/10 pts)

Score: 100 / 100  (passing: 60)  pass=true

Reminder: screenshot *content*, shared-Airflow deploy proof, and before/after
row counts are still teacher-reviewed. Autograder green is not a pass — a
high static score is necessary but not sufficient.

[jrnhofman]

You didn't fill in the PR template, please do so next time.

[jrnhofman]

don't think you need most of the files in include/dbt_project/
of this for the airflow to work!

[jrnhofman]

what does 'its linear and not exponential' mean in this context?

[jrnhofman]

This is about what date you run with (logical date). The question is how you scheduled your workflow (in your case monthly, with backfill = False) and explain why.