Halyna R.

.dockerignore

@@ -0,0 +1,8 @@
+astro
+.git
+.env
+airflow_settings.yaml
+logs/
+.venv
+airflow.db
+airflow.cfg

.hyf/grader_lib.sh

@@ -7,11 +7,9 @@
 # and a set of common static-analysis checks derived from recurring
 # PR review patterns across cohort c55.
 #
-# blocker(): use for leaked-secret findings (a committed profiles.yml/.env,
-# a hardcoded password/connection string). It behaves like fail() for the
-# printed report, but also flips a flag that forces write_score() to report
-# pass=false regardless of the earned point total -- a leaked secret must
-# be fixed before the PR can pass, it cannot be "pointed around."
+# blocker(): use for findings that must fail the PR regardless of points
+# (leaked secrets, missing required evidence like screenshots). Behaves like
+# fail() in the printed report, but forces write_score() to pass=false.
 
 _grader_details=()
 _grader_blocker=false
@@ -38,7 +36,7 @@ write_score() {
   [[ "$score" -ge "$passing" ]] && pass_flag="true"
   if [[ "$_grader_blocker" == true ]]; then
     pass_flag="false"
-    echo "🚫 A blocker was found (leaked secret) -- forcing pass=false regardless of score." >&2
+    echo "🚫 A blocker was found -- forcing pass=false regardless of score." >&2
   fi
   cat > "$outfile" << JSON
 {

.hyf/test.sh

@@ -3,8 +3,9 @@
 # The DAG needs a running Astro/Airflow stack and a live Azure PostgreSQL
 # connection that CI cannot reach, so this checks file presence and code
 # patterns in dags/taxi_pipeline.py and the docs. The actual green run,
-# Screenshot files are presence-checked; content, backfill idempotency, and
-# shared-Airflow deploy are reviewed by a teacher.
+# Screenshot files are required (≥3): missing screenshots force pass=false.
+# Content of those shots, backfill idempotency, and shared-Airflow deploy
+# are still reviewed by a teacher.
 # Total points: 100. Passing score: 60.
 set -euo pipefail
 
@@ -218,9 +219,10 @@ shot_count=$(
 if [[ "$shot_count" -ge 3 ]]; then
   l6=$((l6 + 3)); pass "screenshots: found ${shot_count} image file(s) (need ≥3 for Graph + Grid/run + task log)"
 elif [[ "$shot_count" -gt 0 ]]; then
-  fail "screenshots: only ${shot_count} image file(s) — commit at least 3 (local Graph, green Grid/run, one task log; add shared-UI shot when the VM is up)"
+  # Screenshots are required evidence for teacher review — cannot pass without them.
+  blocker "screenshots: only ${shot_count} image file(s) — commit at least 3 (local Graph, green Grid/run, one task log; add shared-UI shot when the VM is up)"
 else
-  fail "screenshots: none found — commit Graph, Grid/run, and task-log images into the PR (any folder)"
+  blocker "screenshots: none found — commit Graph, Grid/run, and task-log images into the PR (any folder). Screenshots are required; a high code score without them still fails."
 fi
 score=$((score + l6))
 pass "Level 6: documentation + screenshots ($l6/10 pts)"

AI_ASSIST.md

@@ -1,12 +1,18 @@
 # AI assistance log
 
 <!-- Document at least one point where you used an LLM on this assignment.
-     Never paste connection strings, passwords, or real data. Fill in each field. -->
+     Never paste connection strings, passwords, or real data. Replace TODO. -->
 
 ## Use 1
 
-**Prompt I sent:** _Replace this section._
+**Prompt I sent:**
 
-**What the model answered:** _Replace this section._
+Docker Desktop was stuck on “Starting the Docker Engine”, and Windows showed that the paging file was too small. I asked what safe steps I could try without deleting my project data.
 
-**What I kept, changed, or discarded, and why:** _Replace this section._
+**What the model answered:** 
+
+The model suggested restarting Windows, shutting down WSL, stopping unused Docker containers, and starting only one Astro project. It also warned me not to reset Docker to factory settings or remove Docker volumes because this could delete local Airflow data and settings.
+
+**What I kept, changed, or discarded, and why:** 
+
+I followed the safe steps: I restarted the computer, opened only the current project, and stopped containers from older projects. I did not uninstall Docker or delete any Docker volumes because I wanted to keep my local Airflow history and connection settings. I checked the result myself by running `docker ps` and reopening the Airflow interface. No passwords, connection strings, or private data were shared with the model.

RUNBOOK.md

@@ -1,22 +1,44 @@
 # RUNBOOK
 
-<!-- Fill in every section below. Another student should be able to
+<!-- Replace every TODO with real content. Another student should be able to
      operate your DAG from this file alone, without reading your Python. -->
 
 ## How to trigger the DAG manually
 
-_Replace this section._
+1. Start the local stack from the project root with `astro dev start`.
+2. Open the Airflow UI and confirm that the `azure_pg` connection exists.
+3. Open `taxi_pipeline`, unpause it, and click **Trigger**.
+4. In Trigger Options, pass a real past logical date such as `2024-01-01`.
+   If the UI does not expose a logical-date field, use this run configuration:
+   `{"target_date": "2024-01-01"}`.
+5. Open the new run and confirm the order `ingest_taxi_month → dbt_run → dbt_test`.
+
+Do not trigger the DAG for a future or unpublished TLC month. Such a run returns HTTP 403 because the parquet file does not exist yet.
 
 ## How to run a backfill
 
-_Replace this section._
+The DAG uses a monthly schedule, so seven assignment runs require seven months. In PowerShell run this as one line, without Bash backslashes:
+
+```powershell
+astro dev run backfill create --dag-id taxi_pipeline --from-date 2024-01-01 --to-date 2024-07-31 --max-active-runs 1
+```
+
+If the DAG is paused, create the backfill first and then unpause it in the UI. Wait until all runs finish before repeating the range. To repeat completed dates for the idempotency proof, use:
+
+```powershell
+astro dev run backfill create --dag-id taxi_pipeline --from-date 2024-01-01 --to-date 2024-07-31 --max-active-runs 1 --reprocess-behavior completed
+```
 
 ## How to inspect task logs
 
-_Replace this section._
+Open `taxi_pipeline → Runs → the failed run → the red task → Logs`. Read upward from the final generic message until the first concrete exception, such as `HTTPError`, `DatabaseError`, `Compilation Error`, or `Env var required but not provided`. Also check **Rendered Templates** for `dbt_run` and `dbt_test` to confirm the resolved database host, user, database, and schema. Passwords must remain redacted.
 
 ## Top 3 likely failures and first response
 
-1. _Replace: symptom, first check, fix_
-2. _Replace this section._
-3. _Replace this section._
+1. **`ingest_taxi_month` fails with HTTP 403/404.** Check the requested URL and logical date in the log. A future month or mistyped path is deterministic, so retries will not fix it. Trigger/backfill a published month and correct the URL or date range.
+2. **PostgreSQL connection or permission failure.** Confirm `AIRFLOW_STUDENT` in `.env`, then verify `azure_pg` in Admin → Connections. The login and schema name must match, for example role `halyna` writes to `airflow_halyna`. Do not commit credentials.
+3. **`dbt_run` fails.** Read the first dbt `Runtime Error` or `Compilation Error`, not only the final Bash exit code. Confirm `include/dbt_project/profiles.yml` exists, required `PG_*` variables are rendered, and `dbt deps` runs before `dbt run`.
+
+## Safe recovery and escalation
+
+Clear and retry one task only for a transient network or database interruption. Use a backfill after a code or business-logic fix that affects several partitions. If the shared scheduler, shared `azure_pg` connection, or shared VM is broken, do not edit shared settings; report the issue to the teacher.

dags/exampledag.py

@@ -0,0 +1,98 @@
+"""
+## Astronaut ETL example DAG
+
+This DAG queries the list of astronauts currently in space from the
+Open Notify API and prints each astronaut's name and flying craft.
+
+There are two tasks, one to get the data from the API and save the results,
+and another to print the results. Both tasks are written in Python using
+Airflow's TaskFlow API, which allows you to easily turn Python functions into
+Airflow tasks, and automatically infer dependencies and pass data.
+
+The second task uses dynamic task mapping to create a copy of the task for
+each Astronaut in the list retrieved from the API. This list will change
+depending on how many Astronauts are in space, and the DAG will adjust
+accordingly each time it runs.
+
+For more explanation and getting started instructions, see our Write your
+first DAG tutorial: https://www.astronomer.io/docs/learn/get-started-with-airflow
+
+![Picture of the ISS](https://www.esa.int/var/esa/storage/images/esa_multimedia/images/2010/02/space_station_over_earth/10293696-3-eng-GB/Space_Station_over_Earth_card_full.jpg)
+"""
+
+from airflow.sdk import Asset, dag, task
+from pendulum import datetime
+import requests
+
+
+# Define the basic parameters of the DAG, like schedule and start_date
+@dag(
+    start_date=datetime(2025, 4, 22),
+    schedule="@daily",
+    doc_md=__doc__,
+    default_args={"owner": "Astro", "retries": 3},
+    tags=["example"],
+)
+def example_astronauts():
+    # Define tasks
+    @task(
+        # Define an asset outlet for the task. This can be used to schedule downstream DAGs when this task has run.
+        outlets=[Asset("current_astronauts")]
+    )  # Define that this task updates the `current_astronauts` Asset
+    def get_astronauts(**context) -> list[dict]:
+        """
+        This task uses the requests library to retrieve a list of Astronauts
+        currently in space. The results are pushed to XCom with a specific key
+        so they can be used in a downstream pipeline. The task returns a list
+        of Astronauts to be used in the next task.
+        """
+        try:
+            r = requests.get("http://api.open-notify.org/astros.json")
+            r.raise_for_status()
+            number_of_people_in_space = r.json()["number"]
+            list_of_people_in_space = r.json()["people"]
+        except Exception:
+            print("API currently not available, using hardcoded data instead.")
+            number_of_people_in_space = 12
+            list_of_people_in_space = [
+                {"craft": "ISS", "name": "Oleg Kononenko"},
+                {"craft": "ISS", "name": "Nikolai Chub"},
+                {"craft": "ISS", "name": "Tracy Caldwell Dyson"},
+                {"craft": "ISS", "name": "Matthew Dominick"},
+                {"craft": "ISS", "name": "Michael Barratt"},
+                {"craft": "ISS", "name": "Jeanette Epps"},
+                {"craft": "ISS", "name": "Alexander Grebenkin"},
+                {"craft": "ISS", "name": "Butch Wilmore"},
+                {"craft": "ISS", "name": "Sunita Williams"},
+                {"craft": "Tiangong", "name": "Li Guangsu"},
+                {"craft": "Tiangong", "name": "Li Cong"},
+                {"craft": "Tiangong", "name": "Ye Guangfu"},
+            ]
+
+        context["ti"].xcom_push(
+            key="number_of_people_in_space", value=number_of_people_in_space
+        )
+        return list_of_people_in_space
+
+    @task
+    def print_astronaut_craft(greeting: str, person_in_space: dict) -> None:
+        """
+        This task creates a print statement with the name of an
+        Astronaut in space and the craft they are flying on from
+        the API request results of the previous task, along with a
+        greeting which is hard-coded in this example.
+        """
+        craft = person_in_space["craft"]
+        name = person_in_space["name"]
+
+        print(f"{name} is currently in space flying on the {craft}! {greeting}")
+
+    # Use dynamic task mapping to run the print_astronaut_craft task for each
+    # Astronaut in space
+    print_astronaut_craft.partial(greeting="Hello! :)").expand(
+        person_in_space=get_astronauts()  # Define dependencies using TaskFlow API syntax
+    )
+
+
+# Instantiate the DAG
+example_astronauts()