Skip to content

Commit 1428a23

Browse files
anish353claude
andcommitted
[SDK-7461] fix(setup-env): deliver BUILD_RUN_IDENTIFIER, ask the API when the actor is unknown, stop failing silently
Three changes to the BrowserStack re-run delivery path. 1. BROWSERSTACK_BUILD_RUN_IDENTIFIER has been dropped since 2026-05-07. The rebuild/details response carries three variables; the APS-19076 allowlist (8ade0a3) listed only two plus BUILD_NAME, so the identifier is filtered out and only a core.warning marks it. Verified against the live API. The SDKs send it as build_run_identifier in the build-start payload, which files a re-run as an attempt of its parent build rather than an unrelated build. PR #85's own body predicted this: "If the rerun API legitimately sets additional names in production, the list will need to grow." 2. An unreported GITHUB_TRIGGERING_ACTOR no longer aborts delivery. That variable comes from the runner binary, so a self-hosted runner can simply not set it. The actor check is only a cheap pre-filter — rebuild/details is the authority and returns no variables when BrowserStack did not trigger the re-run. Bailing on an absent actor turned a working re-run into a full-suite run on an otherwise correctly configured workflow. We now warn and ask the API instead. 3. Every remaining delivery failure names itself. With github-token at its 'none' default the job log was byte-identical to a healthy run, and the CLI then runs the full spec set because BROWSERSTACK_RERUN_TESTS is absent — so "all my tests ran again" was undiagnosable. checkIfBStackReRun now names the missing input, a human-triggered re-run says so, and an API failure is a warning rather than info. Attempt 1 stays silent: an ordinary run is not a degraded re-run. Verified end-to-end through the built dist into the real CLI across all eight delivery states, and reproduced against a real reported failure: the build ran 11 sessions, the targeted spec folder holds 11 specs, and the real CLI dispatches 11 with the list absent and 3 with it present. 47 tests passing (4 new), eslint clean, dist rebuilt with ncc (reproducible). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 1092dd1 commit 1428a23

4 files changed

Lines changed: 146 additions & 13 deletions

File tree

‎setup-env/config/constants.js‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,14 @@ module.exports = {
2424
// Object.keys(...).forEach(core.exportVariable, ...) call let any caller
2525
// who could influence the API response inject arbitrary env vars into
2626
// the workflow runner (CVSS 9.3 - env-var injection).
27+
// Must list every name the rebuild/details response actually carries; a name missing
28+
// here is dropped silently from the runner's perspective (only a ::warning:: in the log).
2729
ALLOWED_RERUN_ENV_VARS: [
2830
'BROWSERSTACK_RERUN',
2931
'BROWSERSTACK_RERUN_TESTS',
3032
'BROWSERSTACK_BUILD_NAME',
33+
// SDK-7461: the SDKs read this as build_run_identifier in the build-start payload;
34+
// without it a re-run is filed as an unrelated build run instead of an attempt of its parent.
35+
'BROWSERSTACK_BUILD_RUN_IDENTIFIER',
3136
],
3237
};

‎setup-env/dist/index.js‎

Lines changed: 37 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,15 @@ module.exports = {
3030
// Object.keys(...).forEach(core.exportVariable, ...) call let any caller
3131
// who could influence the API response inject arbitrary env vars into
3232
// the workflow runner (CVSS 9.3 - env-var injection).
33+
// Must list every name the rebuild/details response actually carries; a name missing
34+
// here is dropped silently from the runner's perspective (only a ::warning:: in the log).
3335
ALLOWED_RERUN_ENV_VARS: [
3436
'BROWSERSTACK_RERUN',
3537
'BROWSERSTACK_RERUN_TESTS',
3638
'BROWSERSTACK_BUILD_NAME',
39+
// SDK-7461: the SDKs read this as build_run_identifier in the build-start payload;
40+
// without it a re-run is filed as an unrelated build run instead of an attempt of its parent.
41+
'BROWSERSTACK_BUILD_RUN_IDENTIFIER',
3742
],
3843
};
3944

@@ -34661,20 +34666,45 @@ class ActionInput {
3466134666
}
3466234667

3466334668
async checkIfBStackReRun() {
34664-
// Ensure rerunAttempt is a number and greater than 1
34669+
// Attempt 1 is an ordinary run, not a re-run — stay silent, this is not a failure.
3466534670
if (!this.rerunAttempt || Number(this.rerunAttempt) <= 1) {
3466634671
return false;
3466734672
}
3466834673

34669-
// Ensure runId, repository, username, and accessKey are valid
34670-
if (!this.runId || !this.repository || this.repository === 'none'
34671-
|| !this.githubToken || this.githubToken === 'none' || !this.username || !this.accessKey) {
34674+
// Past this point GitHub re-ran the workflow, so the failed-test list was meant to be
34675+
// delivered. Every bail below silently degrades the re-run into a full-suite run, which
34676+
// is indistinguishable from correct behaviour unless we say so here (SDK-7461).
34677+
const missing = [];
34678+
if (!this.githubToken || this.githubToken === 'none') missing.push("the 'github-token' input");
34679+
if (!this.runId) missing.push('GITHUB_RUN_ID');
34680+
if (!this.repository || this.repository === 'none') missing.push('GITHUB_REPOSITORY');
34681+
if (!this.username) missing.push("the 'username' input");
34682+
if (!this.accessKey) missing.push("the 'access-key' input");
34683+
34684+
if (missing.length) {
34685+
core.warning(`This is re-run attempt ${this.rerunAttempt}, but BrowserStack cannot deliver the failed-test list because ${missing.join(', ')} ${missing.length > 1 ? 'are' : 'is'} not set. Every test will run again instead of only the failed ones. Pass github-token to this action to enable re-running only failed tests.`);
3467234686
return false;
3467334687
}
3467434688

3467534689
const triggeringActor = process.env.GITHUB_TRIGGERING_ACTOR;
34690+
34691+
// The actor check is only a cheap pre-filter; the rebuild/details endpoint is the
34692+
// authority on whether BrowserStack triggered this re-run, and it returns no variables
34693+
// when it did not. GITHUB_TRIGGERING_ACTOR comes from the runner binary, so a
34694+
// self-hosted runner can simply not set it — bailing there would turn a working re-run
34695+
// into a full-suite run on an otherwise correct setup (SDK-7461). Ask the API instead.
34696+
if (!triggeringActor) {
34697+
core.warning(`This is re-run attempt ${this.rerunAttempt} and the runner did not report GITHUB_TRIGGERING_ACTOR, so BrowserStack cannot pre-confirm that it triggered this re-run — asking BrowserStack directly instead. This variable is set by the runner itself; on a self-hosted runner, updating the runner restores the faster check.`);
34698+
return true;
34699+
}
34700+
3467634701
core.info(`Triggering actor is - ${triggeringActor}`);
34677-
return triggeringActor === this.githubApp;
34702+
if (triggeringActor !== this.githubApp) {
34703+
core.info(`This re-run was started by '${triggeringActor}', not by the BrowserStack GitHub App ('${this.githubApp}'), so there is no failed-test list to apply and every test will run again. Re-runs started from the BrowserStack dashboard run only the failed tests; check that the BrowserStack GitHub App is installed on ${this.repository}.`);
34704+
return false;
34705+
}
34706+
34707+
return true;
3467834708
}
3467934709

3468034710
async setBStackRerunEnvVars() {
@@ -34710,7 +34740,8 @@ class ActionInput {
3471034740
});
3471134741
}
3471234742
} catch (error) {
34713-
core.info(`Error setting BrowserStack rerun environment variables: ${error.message}`);
34743+
// Swallowing this as info hid a total delivery failure behind a normal-looking log.
34744+
core.warning(`Could not fetch the failed-test list from BrowserStack (${error.message}). Every test will run again instead of only the failed ones.`);
3471434745
}
3471534746
}
3471634747
}

‎setup-env/src/actionInput/index.js‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -94,20 +94,45 @@ class ActionInput {
9494
}
9595

9696
async checkIfBStackReRun() {
97-
// Ensure rerunAttempt is a number and greater than 1
97+
// Attempt 1 is an ordinary run, not a re-run — stay silent, this is not a failure.
9898
if (!this.rerunAttempt || Number(this.rerunAttempt) <= 1) {
9999
return false;
100100
}
101101

102-
// Ensure runId, repository, username, and accessKey are valid
103-
if (!this.runId || !this.repository || this.repository === 'none'
104-
|| !this.githubToken || this.githubToken === 'none' || !this.username || !this.accessKey) {
102+
// Past this point GitHub re-ran the workflow, so the failed-test list was meant to be
103+
// delivered. Every bail below silently degrades the re-run into a full-suite run, which
104+
// is indistinguishable from correct behaviour unless we say so here (SDK-7461).
105+
const missing = [];
106+
if (!this.githubToken || this.githubToken === 'none') missing.push("the 'github-token' input");
107+
if (!this.runId) missing.push('GITHUB_RUN_ID');
108+
if (!this.repository || this.repository === 'none') missing.push('GITHUB_REPOSITORY');
109+
if (!this.username) missing.push("the 'username' input");
110+
if (!this.accessKey) missing.push("the 'access-key' input");
111+
112+
if (missing.length) {
113+
core.warning(`This is re-run attempt ${this.rerunAttempt}, but BrowserStack cannot deliver the failed-test list because ${missing.join(', ')} ${missing.length > 1 ? 'are' : 'is'} not set. Every test will run again instead of only the failed ones. Pass github-token to this action to enable re-running only failed tests.`);
105114
return false;
106115
}
107116

108117
const triggeringActor = process.env.GITHUB_TRIGGERING_ACTOR;
118+
119+
// The actor check is only a cheap pre-filter; the rebuild/details endpoint is the
120+
// authority on whether BrowserStack triggered this re-run, and it returns no variables
121+
// when it did not. GITHUB_TRIGGERING_ACTOR comes from the runner binary, so a
122+
// self-hosted runner can simply not set it — bailing there would turn a working re-run
123+
// into a full-suite run on an otherwise correct setup (SDK-7461). Ask the API instead.
124+
if (!triggeringActor) {
125+
core.warning(`This is re-run attempt ${this.rerunAttempt} and the runner did not report GITHUB_TRIGGERING_ACTOR, so BrowserStack cannot pre-confirm that it triggered this re-run — asking BrowserStack directly instead. This variable is set by the runner itself; on a self-hosted runner, updating the runner restores the faster check.`);
126+
return true;
127+
}
128+
109129
core.info(`Triggering actor is - ${triggeringActor}`);
110-
return triggeringActor === this.githubApp;
130+
if (triggeringActor !== this.githubApp) {
131+
core.info(`This re-run was started by '${triggeringActor}', not by the BrowserStack GitHub App ('${this.githubApp}'), so there is no failed-test list to apply and every test will run again. Re-runs started from the BrowserStack dashboard run only the failed tests; check that the BrowserStack GitHub App is installed on ${this.repository}.`);
132+
return false;
133+
}
134+
135+
return true;
111136
}
112137

113138
async setBStackRerunEnvVars() {
@@ -143,7 +168,8 @@ class ActionInput {
143168
});
144169
}
145170
} catch (error) {
146-
core.info(`Error setting BrowserStack rerun environment variables: ${error.message}`);
171+
// Swallowing this as info hid a total delivery failure behind a normal-looking log.
172+
core.warning(`Could not fetch the failed-test list from BrowserStack (${error.message}). Every test will run again instead of only the failed ones.`);
147173
}
148174
}
149175
}

‎setup-env/test/actionInput/index.test.js‎

Lines changed: 72 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -241,6 +241,51 @@ describe('Action Input operations for fetching all inputs, triggering validation
241241
expect(result).to.be.false;
242242
delete process.env.GITHUB_TRIGGERING_ACTOR;
243243
});
244+
245+
it('Falls through to the API when the runner reports no triggering actor (SDK-7461)', async () => {
246+
// Self-hosted runners set GITHUB_TRIGGERING_ACTOR only from a certain runner version.
247+
// The actor check is a pre-filter, not the authority — bailing here turned a working
248+
// re-run into a full-suite run on an otherwise correct setup.
249+
const coreWarningStub = sinon.stub(core, 'warning');
250+
const actionInput = new ActionInput();
251+
delete process.env.GITHUB_TRIGGERING_ACTOR;
252+
253+
const result = await actionInput.checkIfBStackReRun();
254+
255+
// eslint-disable-next-line no-unused-expressions
256+
expect(result).to.be.true;
257+
sinon.assert.calledWith(coreWarningStub, sinon.match(/did not report GITHUB_TRIGGERING_ACTOR/));
258+
sinon.assert.calledWith(coreWarningStub, sinon.match(/asking BrowserStack directly/));
259+
sinon.assert.neverCalledWith(coreWarningStub, sinon.match(/GitHub App is installed/));
260+
});
261+
262+
it('Warns which input is missing when a re-run cannot be delivered (SDK-7461)', async () => {
263+
// Without this the job log for a degraded re-run is byte-identical to a healthy
264+
// run, so "all tests ran again" is undiagnosable from the customer's side.
265+
const coreWarningStub = sinon.stub(core, 'warning');
266+
const actionInput = new ActionInput();
267+
actionInput.githubToken = 'none';
268+
269+
const result = await actionInput.checkIfBStackReRun();
270+
271+
// eslint-disable-next-line no-unused-expressions
272+
expect(result).to.be.false;
273+
sinon.assert.calledWith(coreWarningStub, sinon.match(/github-token/));
274+
sinon.assert.calledWith(coreWarningStub, sinon.match(/Every test will run again/));
275+
});
276+
277+
it('Explains a human-triggered re-run rather than failing silently (SDK-7461)', async () => {
278+
const coreInfoStub = sinon.stub(core, 'info');
279+
const actionInput = new ActionInput();
280+
process.env.GITHUB_TRIGGERING_ACTOR = 'someHuman';
281+
282+
const result = await actionInput.checkIfBStackReRun();
283+
284+
// eslint-disable-next-line no-unused-expressions
285+
expect(result).to.be.false;
286+
sinon.assert.calledWith(coreInfoStub, sinon.match(/not by the BrowserStack GitHub App/));
287+
delete process.env.GITHUB_TRIGGERING_ACTOR;
288+
});
244289
});
245290

246291
context('Set BrowserStack Rerun Environment Variables', () => {
@@ -320,13 +365,39 @@ describe('Action Input operations for fetching all inputs, triggering validation
320365
});
321366

322367
it('Handles errors when BrowserStack API fails', async () => {
368+
// SDK-7461: a delivery failure is a warning, not info — as info it was
369+
// indistinguishable from a healthy run in the job log.
370+
const coreWarningStub = sinon.stub(core, 'warning');
323371
const actionInput = new ActionInput();
324372
axiosGetStub.rejects(new Error('API failed'));
325373

326374
await actionInput.setBStackRerunEnvVars();
327375

328-
sinon.assert.calledTwice(core.info);
376+
sinon.assert.calledWith(coreWarningStub, sinon.match(/API failed/));
377+
sinon.assert.calledWith(coreWarningStub, sinon.match(/Every test will run again/));
329378
sinon.assert.neverCalledWith(core.exportVariable, sinon.match.any, sinon.match.any);
330379
});
380+
381+
it('Exports BROWSERSTACK_BUILD_RUN_IDENTIFIER from the API response (SDK-7461)', async () => {
382+
// Regression: the rebuild/details response carries this alongside RERUN/RERUN_TESTS,
383+
// and the SDKs send it as build_run_identifier to link a re-run to its parent build
384+
// run. It was absent from ALLOWED_RERUN_ENV_VARS, so the allowlist dropped it.
385+
const actionInput = new ActionInput();
386+
axiosGetStub.resolves({
387+
data: {
388+
data: {
389+
variables: {
390+
BROWSERSTACK_RERUN: 'true',
391+
BROWSERSTACK_RERUN_TESTS: 'cypress/tests/A/spec1.ts,cypress/tests/B/spec2.ts',
392+
BROWSERSTACK_BUILD_RUN_IDENTIFIER: '1784546348645-29728341550',
393+
},
394+
},
395+
},
396+
});
397+
398+
await actionInput.setBStackRerunEnvVars();
399+
400+
sinon.assert.calledWith(core.exportVariable, 'BROWSERSTACK_BUILD_RUN_IDENTIFIER', '1784546348645-29728341550');
401+
});
331402
});
332403
});

0 commit comments

Comments
 (0)