[FEATURE]: Fall back to the variable default when its SetVariable task doesn't run #194

Is there an existing issue for this?

• I have searched the existing issues

Problem statement

When a converted job reads an ADF variable, the read points straight at the task that set it. For example, the Switch in all-activity-types becomes:

- task_key: switch_case_running
  condition_task:
    op: EQUAL_TO
    left: '{{tasks.set-variable.values.status}}'
    right: running

Notebook reads work the same way: dbutils.jobs.taskValues.get(taskKey='<set-variable task>', key='<variable>').

This breaks when the SetVariable task might not run. In ADF, if the SetVariable is on a branch that wasn't taken, a later read returns the variable's default value. In the converted job, the reader has no fallback, so it either fails or gets an empty value. The _init_<variable> task that holds the default exists, but nothing reads from it.

#151 prunes _init_* tasks that nothing reads. That's correct today, since no reader points at an init. But keeping the init wouldn't fix this either: readers would still need to fall back to it.

Proposed Solution

1. Translate reads with a fallback. When a reader's SetVariable might not run (it's on a conditional branch, or behind a failure path), read the SetVariable value if it's there and otherwise read the _init_<variable> value.
2. Leave pruning as it is. Once readers point at the init, the reachability check from Fix three ADF -> Databricks control-flow / codegen bugs #151 keeps the init automatically, and inits that are really unused still get pruned.
3. Add a test case with a reader behind an untaken branch.

Open question to settle first: what Databricks does when a task reads a task value from an upstream task that was skipped or excluded (does taskValues.get / {{tasks.<key>.values.<name>}} return a default, or raise?). The answer decides how the fallback is written.

Additional Context

Example 1: reader after an If Condition branch

IfCondition is_weekend
  true  -> SetVariable mode = "full"      (variable default: "incremental")
  false -> (nothing)
Notebook load  (reads @variables('mode'), depends on IfCondition)

• ADF: on a weekday, load reads "incremental" (the default).
• Converted job today: load reads {{tasks.<set-mode>.values.mode}}. That task was excluded, so there's no value. _init_mode holds "incremental", but nothing reads it, so Fix three ADF -> Databricks control-flow / codegen bugs #151 prunes it.
• Expected: load falls back to _init_mode and reads "incremental".

Example 2: Switch with one case setting the variable

Switch region
  case "us"  -> SetVariable target = "us_catalog"   (variable default: "shared_catalog")
  default    -> Wait
Notebook publish  (reads @variables('target'))

• ADF: for any region other than us, publish reads "shared_catalog".
• Converted job today: publish reads from the us case's SetVariable, which didn't run.
• Expected: publish reads "shared_catalog" from _init_target.

Example 3: SetVariable on a failure path

Copy ingest
  on failure -> SetVariable status = "failed"   (variable default: "ok")
Notebook notify  (reads @variables('status'), runs on completion)

• ADF: when ingest succeeds, notify reads "ok".
• Converted job today: notify reads from the failure-path SetVariable, which didn't run.
• Expected: notify reads "ok" from _init_status.

Not covered here: ADF can run SetVariable and its reader at the same time (for example set-variable and switch in all-activity-types), so the reader might see either value. The converted job always waits for SetVariable. That's a separate gap.

Found while reviewing #151.