[githooks] Add pre-push hook to prevent pushing evaluation test commits - #12663
[githooks] Add pre-push hook to prevent pushing evaluation test commits#12663reidbaker-agent wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new pre-push Git hook to prevent pushing commits authored or committed with evaluation credentials (Eval Author or eval-author@example.com). It adds the PrePushCommand implementation, registers it in the githooks utility, documents it in the README, and integrates a check for its existence into the readiness checker. Feedback on the changes suggests parsing the git log output to check only the Author and Committer fields rather than the entire commit line to avoid false positives when forbidden credentials are mentioned in commit messages, along with adding a corresponding test case.
|
Keeping in draft until presubmits pass. If they pass then this is ready for review. |
25bb98f to
5413c73
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces a new pre-push Git hook designed to prevent pushing commits authored or committed with evaluation test credentials. It implements the PrePushCommand in the githooks tool, integrates the hook into the CI pipeline, updates the readiness checker to verify its presence, and adds corresponding unit tests and documentation. Feedback on the changes suggests a safer handling of logResult.stderr to prevent printing 'null' if the stderr property is null.
There was a problem hiding this comment.
Code Review
This pull request adds a new pre-push Git hook to prevent pushing commits with evaluation test credentials, along with associated unit tests, CI configuration updates, and documentation. The reviewer feedback suggests reading the pushed refs from standard input instead of HEAD to prevent false positives, updating the unit tests to mock this stream, and outputting installation instructions in the readiness checker if the hook is missing.
| PrePushCommand({ | ||
| Future<ProcessResult> Function( | ||
| String executable, | ||
| List<String> arguments, { | ||
| String? workingDirectory, | ||
| })? | ||
| processRunner, | ||
| }) : processRunner = processRunner ?? Process.run; | ||
|
|
||
| /// The process runner injected for testing. | ||
| final Future<ProcessResult> Function( | ||
| String executable, | ||
| List<String> arguments, { | ||
| String? workingDirectory, | ||
| }) | ||
| processRunner; |
There was a problem hiding this comment.
Add an optional stdIn parameter to the constructor and store it as a field. This allows the command to read the refs being pushed from standard input while remaining fully testable by injecting a mock stream in unit tests.
PrePushCommand({
Future<ProcessResult> Function(
String executable,
List<String> arguments,
{String? workingDirectory},
)?
processRunner,
Stream<List<int>>? stdIn,
}) : processRunner = processRunner ?? Process.run,
stdIn = stdIn ?? stdin;
/// The process runner injected for testing.
final Future<ProcessResult> Function(
String executable,
List<String> arguments,
{String? workingDirectory},
)
processRunner;
/// The standard input stream injected for testing.
final Stream<List<int>> stdIn;| Future<bool> run() async { | ||
| print('Running pre-push validation...'); | ||
|
|
||
| final ProcessResult logResult = await processRunner('git', <String>[ | ||
| 'log', | ||
| '-n', | ||
| '20', | ||
| '--format=%h%x00%an%x00%ae%x00%cn%x00%ce%x00%s', | ||
| ]); | ||
|
|
||
| if (logResult.exitCode != 0) { | ||
| print('Failed to check git commit history.'); | ||
| final String stderr = logResult.stderr?.toString().trim() ?? ''; | ||
| if (stderr.isNotEmpty) { | ||
| print(stderr); | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| final String stdoutStr = logResult.stdout?.toString() ?? ''; | ||
| final List<String> commitEntries = stdoutStr | ||
| .split('\n') | ||
| .map((String line) => line.trim()) | ||
| .where((String line) => line.isNotEmpty) | ||
| .toList(); | ||
|
|
||
| for (final entry in commitEntries) { | ||
| final List<String> fields = entry.split('\u0000'); | ||
| if (fields.length < 6) { | ||
| continue; | ||
| } | ||
| final String sha = fields[0]; | ||
| final String authorName = fields[1]; | ||
| final String authorEmail = fields[2]; | ||
| final String committerName = fields[3]; | ||
| final String committerEmail = fields[4]; | ||
| final String subject = fields[5]; | ||
|
|
||
| if (authorName == evalAuthorName || | ||
| authorEmail == evalAuthorEmail || | ||
| committerName == evalAuthorName || | ||
| committerEmail == evalAuthorEmail) { | ||
| print(''' | ||
| Pre-push check failed: Found commit(s) authored or committed with evaluation test credentials: | ||
| $sha | Author: $authorName <$authorEmail> | Committer: $committerName <$committerEmail> | $subject | ||
|
|
||
| Evaluation test commits must not be pushed. Clean or rebase your branch before pushing. | ||
| To bypass this check, push with --no-verify.'''); | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| print('Pre-push validation passed.'); | ||
| return true; | ||
| } |
There was a problem hiding this comment.
Instead of checking the currently checked-out branch (HEAD), read the refs being pushed from stdIn (which Git passes to the pre-push hook). This avoids false positives when pushing a different branch than HEAD or when deleting a branch. Additionally, use case-insensitive comparison for emails to make the check more robust.
@override
Future<bool> run() async {
print('Running pre-push validation...');
final List<String> shasToCheck = [];
final bool canReadStdin = stdIn is! Stdin || !(stdIn as Stdin).hasTerminal;
if (canReadStdin) {
try {
final String input = await stdIn.transform(utf8.decoder).join();
for (final String line in input.split('\n')) {
final List<String> parts = line.trim().split(' ');
if (parts.length >= 4) {
final String localSha = parts[1];
if (localSha != '0000000000000000000000000000000000000000') {
shasToCheck.add(localSha);
}
}
}
} catch (_) {
// Fallback to HEAD if reading stdin fails.
}
}
if (shasToCheck.isEmpty) {
shasToCheck.add('HEAD');
}
for (final String sha in shasToCheck) {
final ProcessResult logResult = await processRunner('git', <String>[
'log',
'-n',
'20',
'--format=%h%x00%an%x00%ae%x00%cn%x00%ce%x00%s',
sha,
]);
if (logResult.exitCode != 0) {
print('Failed to check git commit history for $sha.');
final String stderr = logResult.stderr?.toString().trim() ?? '';
if (stderr.isNotEmpty) {
print(stderr);
}
return false;
}
final String stdoutStr = logResult.stdout?.toString() ?? '';
final List<String> commitEntries = stdoutStr
.split('\n')
.map((String line) => line.trim())
.where((String line) => line.isNotEmpty)
.toList();
for (final entry in commitEntries) {
final List<String> fields = entry.split('\u0000');
if (fields.length < 6) {
continue;
}
final String commitSha = fields[0];
final String authorName = fields[1];
final String authorEmail = fields[2];
final String committerName = fields[3];
final String committerEmail = fields[4];
final String subject = fields[5];
if (authorName == evalAuthorName ||
authorEmail.toLowerCase() == evalAuthorEmail ||
committerName == evalAuthorName ||
committerEmail.toLowerCase() == evalAuthorEmail) {
print('''
Pre-push check failed: Found commit(s) authored or committed with evaluation test credentials:
$commitSha | Author: $authorName <$authorEmail> | Committer: $committerName <$committerEmail> | $subject
Evaluation test commits must not be pushed. Clean or rebase your branch before pushing.
To bypass this check, push with --no-verify.''');
return false;
}
}
}
print('Pre-push validation passed.');
return true;
}| import 'dart:io'; | ||
| import 'package:args/command_runner.dart'; |
There was a problem hiding this comment.
| final String? repoRoot = _findRepoRoot(workspaceRoot); | ||
| if (repoRoot != null) { | ||
| final File prePushFile = _fileSystem.file( | ||
| _fileSystem.path.join(repoRoot, 'script', 'githooks', 'pre-push'), | ||
| ); | ||
| if (!prePushFile.existsSync()) { | ||
| _log('Error: Git pre-push hook is missing at "${prePushFile.path}".'); | ||
| return false; | ||
| } | ||
| } |
There was a problem hiding this comment.
Print the git hooks installation instructions when the pre-push hook file is missing. This matches the behavior when core.hooksPath is incorrect and helps the user resolve the issue quickly.
final String? repoRoot = _findRepoRoot(workspaceRoot);
if (repoRoot != null) {
final File prePushFile = _fileSystem.file(
_fileSystem.path.join(repoRoot, 'script', 'githooks', 'pre-push'),
);
if (!prePushFile.existsSync()) {
_log('Error: Git pre-push hook is missing at "${prePushFile.path}".');
final String githooksDir = _fileSystem.path.join(repoRoot, 'script', 'githooks');
_log(
'To install git hooks, run:\n'
' (cd $githooksDir && dart pub get && dart run bin/install_hooks.dart)',
);
return false;
}
}| import 'dart:io'; | ||
|
|
||
| import 'package:githooks/src/pre_push_command.dart'; | ||
| import 'package:test/test.dart'; |
There was a problem hiding this comment.
Add import 'dart:convert'; to support encoding/decoding standard input streams in the unit tests.
| import 'dart:io'; | |
| import 'package:githooks/src/pre_push_command.dart'; | |
| import 'package:test/test.dart'; | |
| import 'dart:convert'; | |
| import 'dart:io'; | |
| import 'package:githooks/src/pre_push_command.dart'; | |
| import 'package:test/test.dart'; |
| PrePushCommand createCommand( | ||
| String gitLogOutput, { | ||
| int exitCode = 0, | ||
| String? stderr = '', | ||
| List<List<String>>? capturedArgs, | ||
| }) { | ||
| return PrePushCommand( | ||
| processRunner: | ||
| (String executable, List<String> arguments, {String? workingDirectory}) async { | ||
| capturedArgs?.add(arguments); | ||
| if (executable == 'git' && arguments.contains('log')) { | ||
| return ProcessResult(0, exitCode, gitLogOutput, stderr); | ||
| } | ||
| return ProcessResult(0, 0, 'Success', ''); | ||
| }, | ||
| ); | ||
| } |
There was a problem hiding this comment.
Update createCommand to accept and pass stdIn to PrePushCommand, defaulting to an empty stream to prevent hanging in tests.
PrePushCommand createCommand(
String gitLogOutput, {
int exitCode = 0,
String? stderr = '',
List<List<String>>? capturedArgs,
Stream<List<int>>? stdIn,
}) {
return PrePushCommand(
processRunner:
(String executable, List<String> arguments, {String? workingDirectory}) async {
capturedArgs?.add(arguments);
if (executable == 'git' && arguments.contains('log')) {
return ProcessResult(0, exitCode, gitLogOutput, stderr);
}
return ProcessResult(0, 0, 'Success', '');
},
stdIn: stdIn ?? Stream<List<int>>.empty(),
);
}|
Dang it some of these gemini suggestions look good. Your feedback is still welcome but I have more work to do here. |
This work was pulled out of #12624 (comment) because we decided that a pre push git hook was a better tool than using an eval and skill.
Agent authored description.
Adds a deterministic
pre-pushGit hook underscript/githooksthat inspects the last 20 commits for evaluation credentials (Eval Authorandeval-author@example.com) and aborts the push if detected.Also updates the
check-readinessskill to verify that thepre-pushhook script exists inscript/githooks.Pre-launch Checklist