Skip to content

feat: scaffold Week 12 assignment (Airflow orchestrated pipeline) - #2

Merged
lassebenni merged 1 commit into
mainfrom
feat/scaffold-week12-airflow
Jul 13, 2026
Merged

feat: scaffold Week 12 assignment (Airflow orchestrated pipeline)#2
lassebenni merged 1 commit into
mainfrom
feat/scaffold-week12-airflow

Conversation

@lassebenni

Copy link
Copy Markdown
Collaborator

Summary

Scaffolds the Week 12 Airflow orchestrated-pipeline assignment (curriculum Ch11) as an Astro project. Students implement dags/taxi_pipeline.py: a scheduled DAG with catchup=False + a tag, three sequential tasks (ingest_taxi_month → dbt_run → dbt_test) where dbt runs via uvx --python 3.11 (plain dbt crashes on the image's Python 3.14), logical-date partitioning, and retries. Provided: the Ch6 tests/test_dag_integrity.py, a Dockerfile (Astro Runtime 3.3), requirements.txt (providers, no dbt), and RUNBOOK / ASSIGNMENT_REPORT / AI_ASSIST templates.

⚠️ Supersedes PR #1 for this repo. PR #1 scaffolds a Metabase + Streamlit dashboarding assignment, which does not match the current Week 12 (Airflow) curriculum. That dashboarding scaffold belongs to a different week — please redirect or close #1.

Autograder verified end-to-end

.hyf/test.sh is a static 100-pt ladder (pass = 60), reusing grader_lib.sh. No Airflow install needed — pure bash/sed/grep, so it runs in the shared HYF auto-grade CI.

Score Gate
20 Required files exist
+15 DAG implemented (no NotImplementedError; @dag + tasks)
+20 Three tasks ingest → dbt_run → dbt_test, chained with >>
+20 dbt via uvx; retries configured
+15 Logical-date partition ({{ ds }}/logical_date); catchup=False
+10 RUNBOOK.md + AI_ASSIST.md filled (no leftover TODO)

Test plan

  • Bare scaffold: 25/100, pass=false (only files-exist + DAG structure)
  • Full solution (snapshot DAG + filled docs): 100/100, pass=true
  • Stub + solution DAGs py_compile clean; test.sh bash -n clean
  • Guidance kept out of grep-target literals (docstrings/comments don't pre-satisfy checks)

🤖 Generated with Claude Code

Replaces the raw template with an Astro-project scaffold for the Week 12
Airflow assignment (Ch11). Students implement dags/taxi_pipeline.py:
schedule + catchup=False + tag, three sequential tasks
(ingest_taxi_month -> dbt_run -> dbt_test) with dbt run via
uvx --python 3.11, logical-date partitioning, and retries.

- dags/taxi_pipeline.py: starter, parses but task bodies raise
  NotImplementedError; guidance kept out of grep-target literals.
- tests/test_dag_integrity.py: provided (Ch6 DagBag test), all tiers.
- Dockerfile (Astro Runtime 3.3), requirements.txt (providers, no dbt),
  include/dbt_project/ README, .env.example, RUNBOOK/ASSIGNMENT_REPORT/
  AI_ASSIST templates.
- .hyf/test.sh: static 100-pt ladder (pass=60), reuses grader_lib.sh.
  Verified end-to-end: bare scaffold 25/100 fail, full solution 100/100 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UEFNwdbVCzZbqVbCCCqJd6
@lassebenni
lassebenni merged commit 22e5748 into main Jul 13, 2026
@lassebenni
lassebenni deleted the feat/scaffold-week12-airflow branch July 13, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant