Bind L2 compositional subtasks to the parent's instance - #160
Merged
Merged
Conversation
CompositionalTask.setup_goal rebinds every subtask's instance and per-task user credentials to the parent's, but that loop sat inside the level-3 branch. At L2, a subtask built without the parent's instance (e.g. the order subtask in navigate_and_do.py, or the Filter*ListTask subtasks in dash_do_filter.py) kept its own SNowInstance(), an independent random pick from the instance pool with admin credentials. The browser acted on the parent's instance while the subtask validated and tore down on another one, causing IndexError in list validation and 404s deleting sc_request records, and leaving orphaned records behind. CI never saw it because it pins a single instance via SNOW_INSTANCE_URL. Running the rebinding at every level fixes all affected L2 tasks at once. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stubs subtasks that start bound to other instances, runs the real CompositionalTask.setup_goal at L2 and L3, and checks every subtask ends up on the parent's instance with the per-task user's credentials. Fails at L2 on the pre-fix code; needs no ServiceNow instance, so it runs in the fast CI job. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 51382 | Triggered | Generic Password | df59370 | tests/test_compositional_binding.py | View secret |
| 51383 | Triggered | Authentication Tuple | df59370 | tests/test_compositional_binding.py | View secret |
| 51383 | Triggered | Authentication Tuple | df59370 | tests/test_compositional_binding.py | View secret |
🛠 Guidelines to remediate hardcoded secrets
This is a non-blocking security notice.
A plain-text credential was located within this pull request. Before proceeding, you should understand the context of what this credential provides access to, and the implications of leaving the secret in-place.
Steps from here:
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
Questions? Email devsecops@servicenow.com
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CompositionalTask.setup_goalrebinds every subtask'sinstanceand per-task user credentials to the parent's, but that loop sat inside theif self.level == 3:branch. At L2 it never ran, so any subtask built without the parent's instance kept its ownSNowInstance(): an independent random pick from the instance pool, with admin credentials.The browser acts on the parent's instance, but such a subtask validates and tears down on its own. Affected call sites include every
NavigateAndOrder*Task(the order subtask gets the rawinstancearg, i.e.None) and all fourFilter{Asset,Hardware,Incident,User}ListTasksubtasks indash_do_filter.py(noinstance=at all). Symptoms:IndexErrorattasks/list.py:939/:950: the filter subtask looks up the task's own agent sys_ids on the wrong instance and indexes an empty result.HTTPError 404deletingsc_request/<sys_id>inOrderHardwareTask.teardown: the request only exists on the parent's instance. This also hides the real failure, since the order validation had already looked on the wrong instance.The fix runs the existing rebinding loop at every level, splitting the L3 block so the L3-only tasks are still appended before it and set up after it. One file, no per-call-site changes. No
CompositionalTaskis ever nested inside another (checked statically, including classes passed as*_class=and built later), so the parent always has its_base_user_*fields when the loop runs.Why this went unnoticed
unit_tests.ymlexcludes thepricycompositional tests on PRs, and its weekly job that does run them expectsSNOW_INSTANCE_*secrets that aren't configured (they resolve to''), so it has been failing for months without running anything. The daily pool workflow (instance_pool_ci.yml) runs only L1 tests, each given an explicit instance. The mismatch needs a multi-instance pool and unpinned construction, and gets more likely as instances are added.test_cheat_compositional's@retryhas noretry=predicate, so it retries on any exception (despite logging "Retrying due to a TimeoutError"), up to 5 times. Each retry redraws both instances, so a test only fails if every attempt mismatches: about 13% of affected tests on a 3-instance pool. Before this fix, the affected filter tasks were the slowest L2 tests (400–700s) because they were being retried.Real benchmark runs are exposed as well: tasks are registered without
task_kwargsand browsergym callstask_entrypoint(seed=seed), so any run against the pool that doesn't pass or pin an instance can grade L2 subtasks on the wrong instance.Evidence
workarenapublic23, delete sent toworkarenapublic25). The record was absent on the target instance and had nosys_audit_deleteentry. For the filter failures, the missing sys_id was always the test's own agent.NavigateAndOrder*tasks bound parent and subtask to different instances on a 6-instance pool.setup()check shows every subtask on the parent's instance and running as the per-task user.Test plan
All runs unpinned, against the 6-instance pool, with
-n 5 --slowmo 1000:*FilterAssetListTaskandNavigateAndOrder*L2 task, their infeasible variants, and 6 L3 regression checks. The same families failed 17/18 before the fix, run without retries.tests/test_compositional_binding.py: fails at L2 on the pre-fix code, passes with the fix.black --checkclean on the changed files.Follow-ups (not in this PR)
test_cheat_compositional's retry toTimeoutError, as its log message implies. Retrying every exception hides nondeterministic bugs like this one, and each failed attempt leaves orphaned records.First.Last.NNNNusers plus their incidents, assets and requests. These need a separate cleanup.🤖 Generated with Claude Code