Normalize missing schedule job file errors to marshmallow ValidationError

[Copilot]

Creating a schedule with a non-existent create_job file path currently surfaces a deeply nested, multi-schema error blob. The root cause is FileRefField leaking raw FileNotFoundError, which causes UnionField to continue schema fallthrough and aggregate unrelated errors.

• Schema error handling

  • Updated FileRefField._deserialize in sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py to catch filesystem exceptions from load_file(path) and re-raise as marshmallow.ValidationError.
  • This keeps missing-file failures in the normal validation path and avoids traceback-style error aggregation across unrelated union alternatives.

• Schedule regression coverage

  • Added invalid schedule config:
    • sdk/ml/azure-ai-ml/tests/test_configs/schedule/invalid/hello_cron_schedule_with_missing_job_file.yml
  • Added unit test in sdk/ml/azure-ai-ml/tests/schedule/unittests/test_schedule_schema.py:
    • test_load_schedule_with_missing_create_job_file
    • Asserts error includes No such file or directory and the missing filename.
    • Asserts noisy unrelated schema messages are not present.

try:
    data = load_file(path)
except (FileNotFoundError, OSError) as e:
    raise ValidationError(f"No such file or directory: {path}") from e

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• centralus-2.in.applicationinsights.azure.com
  • Triggering command: /home/REDACTED/bin/azsdk /home/REDACTED/bin/azsdk ingest-telemetry --client-type copilot-cli --event-type skill_invocation --session-id 6caa9bff-e084-4c8f-affd-c426ef4c1912 --skill-name find-package-skill (dns block)
• pypi.org
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/python python -m pip install pytest-mock (dns block)
• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Problem

When a user creates a schedule with a wrong/non-existent pipeline.yml path, the resulting error message is an unreadable, deeply nested JSON blob that is confusing and unhelpful. For example:

{
  "create_job": [
    { "_schema": ["In order to specify an existing jobs, please provide either of the following prefixed with 'azureml:'..."] },
    { "_schema": ["Not supporting non file for create_job"] },
    { "job": [{ "_schema": ["..."] }, ["No such file or directory: CLI\\Pipeline\\..\\pipeline.yml"]] },
    { "type": ["Value 'pipeline' passed is not in set ['command']"], "job": [...] },
    { "job": [...], "type": ["Value 'pipeline' passed is not in set ['spark']"], "inputs": ["Unknown field."] }
  ]
}

Root Cause

In sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py, the FileRefField._deserialize method calls load_file(path), which raises a raw FileNotFoundError (an OS exception) when the file doesn't exist:

def _deserialize(self, value, attr, data, **kwargs):
    if isinstance(value, str) and not value.startswith(FILE_PREFIX):
        base_path = Path(self.context[BASE_PATH_CONTEXT_KEY])
        path = Path(value)
        if not path.is_absolute():
            path = base_path / path
            path.resolve()
        data = load_file(path)   # <-- raises raw FileNotFoundError, NOT a ValidationError!
        return data
    raise ValidationError(f"Not supporting non file for {attr}")

Because FileNotFoundError is not a marshmallow.ValidationError, the UnionField._deserialize handles it differently (via a separate except (FileNotFoundError, TypeError) branch that formats a traceback string) instead of treating it as a clean validation failure. This causes UnionField to fall through and try all alternative schema options (ArmStr, CreateJobFileRefField), collecting all their unrelated error messages. The outer JobScheduleSchema then tries all create_job schema variants (PipelineCreateJobSchema, CommandCreateJobSchema, SparkCreateJobSchema), each repeating the same cascade.

Fix

Wrap the load_file(path) call in FileRefField._deserialize with a try/except to catch FileNotFoundError/OSError and re-raise as a clean marshmallow.ValidationError:

File: sdk/ml/azure-ai-ml/azure/ai/ml/_schema/core/fields.py

Change FileRefField._deserialize from:

def _deserialize(self, value, attr, data, **kwargs):
    if isinstance(value, str) and not value.startswith(FILE_PREFIX):
        base_path = Path(self.context[BASE_PATH_CONTEXT_KEY])
        path = Path(value)
        if not path.is_absolute():
            path = base_path / path
            path.resolve()
        data = load_file(path)
        return data
    raise ValidationError(f"Not supporting non file for {attr}")

To:

def _deserialize(self, value, attr, data, **kwargs):
    if isinstance(value, str) and not value.startswith(FILE_PREFIX):
        base_path = Path(self.context[BASE_PATH_CONTEXT_KEY])
        path = Path(value)
        if not path.is_absolute():
            path = base_path / path
            path.resolve()
        try:
            data = load_file(path)
        except (FileNotFoundError, OSError) as e:
            raise ValidationError(f"No such file or directory: {path}") from e
        return data
    raise ValidationError(f"Not supporting non file for {attr}")

Expected Outcome

Once FileNotFoundError is wrapped as a ValidationError:

• UnionField will catch it as a normal validation failure immediately, without falling through to other schema alternatives
• The user will see a single, clear error: "No such file or directory: CLI\Pipeline\..\pipeline.yml" instead of the deeply nested multi-schema error dump
• Add or update unit tests in sdk/ml/azure-ai-ml/tests/ to verify that providing a non-existent file path in a schedule's create_job produces a clean, single-message ValidationError with the "No such file or directory" message

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: https://msdata.visualstudio.com/Vienna/_workitems/edit/3957893 can you analye the task and get the requirement here

Assistant: I'm unable to access Azure DevOps work items at https://msdata.visualstudio.com/Vienna/_workitems/edit/3957893 as it's an internal Microsoft URL that requires authentication and is not publicly accessible.

To help you analyze the task, please share the work item details directly in the chat — such as the title, description, acceptance criteria, or any relevant notes — and I'll be happy to assist from there.

User: ```
Following is the error message when you create schedule with wrong pipeline.yml path

{
"create_job": [
{
"_schema": [
"In order to specify an existing jobs, please provide either of the following prefixed with 'azureml:':\n1. The full ARM ID for the resource, e.g.azureml:/subscriptions/<subscript...

This pull request was created from Copilot chat.