Skip to content

[githooks] Add pre-push hook to prevent pushing evaluation test commits - #12663

Draft
reidbaker-agent wants to merge 6 commits into
flutter:mainfrom
reidbaker:pre-push-eval-commit-hook
Draft

[githooks] Add pre-push hook to prevent pushing evaluation test commits#12663
reidbaker-agent wants to merge 6 commits into
flutter:mainfrom
reidbaker:pre-push-eval-commit-hook

Conversation

@reidbaker-agent

@reidbaker-agent reidbaker-agent commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

This work was pulled out of #12624 (comment) because we decided that a pre push git hook was a better tool than using an eval and skill.


Agent authored description.
Adds a deterministic pre-push Git hook under script/githooks that inspects the last 20 commits for evaluation credentials (Eval Author and eval-author@example.com) and aborts the push if detected.

Also updates the check-readiness skill to verify that the pre-push hook script exists in script/githooks.

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [relevant style guides].
  • I signed the [CLA].
  • The title of the PR matches the expected standard.
  • All existing and new tests are passing.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new pre-push Git hook to prevent pushing commits authored or committed with evaluation credentials (Eval Author or eval-author@example.com). It adds the PrePushCommand implementation, registers it in the githooks utility, documents it in the README, and integrates a check for its existence into the readiness checker. Feedback on the changes suggests parsing the git log output to check only the Author and Committer fields rather than the entire commit line to avoid false positives when forbidden credentials are mentioned in commit messages, along with adding a corresponding test case.

Comment thread script/githooks/lib/src/pre_push_command.dart Outdated
Comment thread script/githooks/test/pre_push_command_test.dart
Comment thread script/githooks/lib/src/pre_push_command.dart
Comment thread script/githooks/lib/src/pre_push_command.dart Outdated
Comment thread script/githooks/test/pre_push_command_test.dart
Comment thread script/githooks/test/pre_push_command_test.dart
@reidbaker reidbaker added the CICD Run CI/CD label Aug 27, 2026
@flutter-dashboard flutter-dashboard Bot removed the CICD Run CI/CD label Aug 27, 2026
@reidbaker reidbaker added the CICD Run CI/CD label Aug 27, 2026
@flutter-dashboard flutter-dashboard Bot removed the CICD Run CI/CD label Aug 27, 2026
@reidbaker reidbaker added the CICD Run CI/CD label Aug 27, 2026
@reidbaker
reidbaker marked this pull request as draft August 27, 2026 21:32
@reidbaker

Copy link
Copy Markdown
Contributor

Keeping in draft until presubmits pass. If they pass then this is ready for review.

@flutter-dashboard flutter-dashboard Bot removed the CICD Run CI/CD label Aug 28, 2026
@reidbaker
reidbaker force-pushed the pre-push-eval-commit-hook branch from 25bb98f to 5413c73 Compare August 28, 2026 14:13
@reidbaker reidbaker added the CICD Run CI/CD label Aug 28, 2026
@reidbaker
reidbaker marked this pull request as ready for review August 28, 2026 14:17

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new pre-push Git hook designed to prevent pushing commits authored or committed with evaluation test credentials. It implements the PrePushCommand in the githooks tool, integrates the hook into the CI pipeline, updates the readiness checker to verify its presence, and adds corresponding unit tests and documentation. Feedback on the changes suggests a safer handling of logResult.stderr to prevent printing 'null' if the stderr property is null.

Comment thread script/githooks/lib/src/pre_push_command.dart Outdated
@flutter-dashboard flutter-dashboard Bot removed the CICD Run CI/CD label Aug 28, 2026
@reidbaker reidbaker added the CICD Run CI/CD label Aug 28, 2026
@reidbaker
reidbaker marked this pull request as draft August 28, 2026 20:00
@reidbaker
reidbaker marked this pull request as ready for review August 28, 2026 22:03

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds a new pre-push Git hook to prevent pushing commits with evaluation test credentials, along with associated unit tests, CI configuration updates, and documentation. The reviewer feedback suggests reading the pushed refs from standard input instead of HEAD to prevent false positives, updating the unit tests to mock this stream, and outputting installation instructions in the readiness checker if the hook is missing.

Comment on lines +27 to +42
PrePushCommand({
Future<ProcessResult> Function(
String executable,
List<String> arguments, {
String? workingDirectory,
})?
processRunner,
}) : processRunner = processRunner ?? Process.run;

/// The process runner injected for testing.
final Future<ProcessResult> Function(
String executable,
List<String> arguments, {
String? workingDirectory,
})
processRunner;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Add an optional stdIn parameter to the constructor and store it as a field. This allows the command to read the refs being pushed from standard input while remaining fully testable by injecting a mock stream in unit tests.

  PrePushCommand({
    Future<ProcessResult> Function(
      String executable,
      List<String> arguments,
      {String? workingDirectory},
    )?
    processRunner,
    Stream<List<int>>? stdIn,
  }) : processRunner = processRunner ?? Process.run,
       stdIn = stdIn ?? stdin;

  /// The process runner injected for testing.
  final Future<ProcessResult> Function(
    String executable,
    List<String> arguments,
    {String? workingDirectory},
  )
  processRunner;

  /// The standard input stream injected for testing.
  final Stream<List<int>> stdIn;

Comment on lines +52 to +106
Future<bool> run() async {
print('Running pre-push validation...');

final ProcessResult logResult = await processRunner('git', <String>[
'log',
'-n',
'20',
'--format=%h%x00%an%x00%ae%x00%cn%x00%ce%x00%s',
]);

if (logResult.exitCode != 0) {
print('Failed to check git commit history.');
final String stderr = logResult.stderr?.toString().trim() ?? '';
if (stderr.isNotEmpty) {
print(stderr);
}
return false;
}

final String stdoutStr = logResult.stdout?.toString() ?? '';
final List<String> commitEntries = stdoutStr
.split('\n')
.map((String line) => line.trim())
.where((String line) => line.isNotEmpty)
.toList();

for (final entry in commitEntries) {
final List<String> fields = entry.split('\u0000');
if (fields.length < 6) {
continue;
}
final String sha = fields[0];
final String authorName = fields[1];
final String authorEmail = fields[2];
final String committerName = fields[3];
final String committerEmail = fields[4];
final String subject = fields[5];

if (authorName == evalAuthorName ||
authorEmail == evalAuthorEmail ||
committerName == evalAuthorName ||
committerEmail == evalAuthorEmail) {
print('''
Pre-push check failed: Found commit(s) authored or committed with evaluation test credentials:
$sha | Author: $authorName <$authorEmail> | Committer: $committerName <$committerEmail> | $subject

Evaluation test commits must not be pushed. Clean or rebase your branch before pushing.
To bypass this check, push with --no-verify.''');
return false;
}
}

print('Pre-push validation passed.');
return true;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Instead of checking the currently checked-out branch (HEAD), read the refs being pushed from stdIn (which Git passes to the pre-push hook). This avoids false positives when pushing a different branch than HEAD or when deleting a branch. Additionally, use case-insensitive comparison for emails to make the check more robust.

  @override
  Future<bool> run() async {
    print('Running pre-push validation...');

    final List<String> shasToCheck = [];

    final bool canReadStdin = stdIn is! Stdin || !(stdIn as Stdin).hasTerminal;
    if (canReadStdin) {
      try {
        final String input = await stdIn.transform(utf8.decoder).join();
        for (final String line in input.split('\n')) {
          final List<String> parts = line.trim().split(' ');
          if (parts.length >= 4) {
            final String localSha = parts[1];
            if (localSha != '0000000000000000000000000000000000000000') {
              shasToCheck.add(localSha);
            }
          }
        }
      } catch (_) {
        // Fallback to HEAD if reading stdin fails.
      }
    }

    if (shasToCheck.isEmpty) {
      shasToCheck.add('HEAD');
    }

    for (final String sha in shasToCheck) {
      final ProcessResult logResult = await processRunner('git', <String>[
        'log',
        '-n',
        '20',
        '--format=%h%x00%an%x00%ae%x00%cn%x00%ce%x00%s',
        sha,
      ]);

      if (logResult.exitCode != 0) {
        print('Failed to check git commit history for $sha.');
        final String stderr = logResult.stderr?.toString().trim() ?? '';
        if (stderr.isNotEmpty) {
          print(stderr);
        }
        return false;
      }

      final String stdoutStr = logResult.stdout?.toString() ?? '';
      final List<String> commitEntries = stdoutStr
          .split('\n')
          .map((String line) => line.trim())
          .where((String line) => line.isNotEmpty)
          .toList();

      for (final entry in commitEntries) {
        final List<String> fields = entry.split('\u0000');
        if (fields.length < 6) {
          continue;
        }
        final String commitSha = fields[0];
        final String authorName = fields[1];
        final String authorEmail = fields[2];
        final String committerName = fields[3];
        final String committerEmail = fields[4];
        final String subject = fields[5];

        if (authorName == evalAuthorName ||
            authorEmail.toLowerCase() == evalAuthorEmail ||
            committerName == evalAuthorName ||
            committerEmail.toLowerCase() == evalAuthorEmail) {
          print('''
Pre-push check failed: Found commit(s) authored or committed with evaluation test credentials:
  $commitSha | Author: $authorName <$authorEmail> | Committer: $committerName <$committerEmail> | $subject

Evaluation test commits must not be pushed. Clean or rebase your branch before pushing.
To bypass this check, push with --no-verify.''');
          return false;
        }
      }
    }

    print('Pre-push validation passed.');
    return true;
  }

Comment on lines +7 to +8
import 'dart:io';
import 'package:args/command_runner.dart';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add import 'dart:convert'; to support decoding and parsing the standard input stream when reading the refs being pushed.

Suggested change
import 'dart:io';
import 'package:args/command_runner.dart';
import 'dart:convert';
import 'dart:io';
import 'package:args/command_runner.dart';

Comment on lines +149 to +158
final String? repoRoot = _findRepoRoot(workspaceRoot);
if (repoRoot != null) {
final File prePushFile = _fileSystem.file(
_fileSystem.path.join(repoRoot, 'script', 'githooks', 'pre-push'),
);
if (!prePushFile.existsSync()) {
_log('Error: Git pre-push hook is missing at "${prePushFile.path}".');
return false;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Print the git hooks installation instructions when the pre-push hook file is missing. This matches the behavior when core.hooksPath is incorrect and helps the user resolve the issue quickly.

    final String? repoRoot = _findRepoRoot(workspaceRoot);
    if (repoRoot != null) {
      final File prePushFile = _fileSystem.file(
        _fileSystem.path.join(repoRoot, 'script', 'githooks', 'pre-push'),
      );
      if (!prePushFile.existsSync()) {
        _log('Error: Git pre-push hook is missing at "${prePushFile.path}".');
        final String githooksDir = _fileSystem.path.join(repoRoot, 'script', 'githooks');
        _log(
          'To install git hooks, run:\n' 
          '  (cd $githooksDir && dart pub get && dart run bin/install_hooks.dart)',
        );
        return false;
      }
    }

Comment on lines +5 to +8
import 'dart:io';

import 'package:githooks/src/pre_push_command.dart';
import 'package:test/test.dart';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add import 'dart:convert'; to support encoding/decoding standard input streams in the unit tests.

Suggested change
import 'dart:io';
import 'package:githooks/src/pre_push_command.dart';
import 'package:test/test.dart';
import 'dart:convert';
import 'dart:io';
import 'package:githooks/src/pre_push_command.dart';
import 'package:test/test.dart';

Comment on lines +12 to +28
PrePushCommand createCommand(
String gitLogOutput, {
int exitCode = 0,
String? stderr = '',
List<List<String>>? capturedArgs,
}) {
return PrePushCommand(
processRunner:
(String executable, List<String> arguments, {String? workingDirectory}) async {
capturedArgs?.add(arguments);
if (executable == 'git' && arguments.contains('log')) {
return ProcessResult(0, exitCode, gitLogOutput, stderr);
}
return ProcessResult(0, 0, 'Success', '');
},
);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update createCommand to accept and pass stdIn to PrePushCommand, defaulting to an empty stream to prevent hanging in tests.

    PrePushCommand createCommand(
      String gitLogOutput, {
      int exitCode = 0,
      String? stderr = '',
      List<List<String>>? capturedArgs,
      Stream<List<int>>? stdIn,
    }) {
      return PrePushCommand(
        processRunner:
            (String executable, List<String> arguments, {String? workingDirectory}) async {
              capturedArgs?.add(arguments);
              if (executable == 'git' && arguments.contains('log')) {
                return ProcessResult(0, exitCode, gitLogOutput, stderr);
              }
              return ProcessResult(0, 0, 'Success', '');
            },
        stdIn: stdIn ?? Stream<List<int>>.empty(),
      );
    }

@reidbaker

Copy link
Copy Markdown
Contributor

Dang it some of these gemini suggestions look good. Your feedback is still welcome but I have more work to do here.

@reidbaker
reidbaker self-requested a review August 28, 2026 22:31
@reidbaker
reidbaker marked this pull request as draft August 28, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants