Skip to content

Speed up the RSpec suite: parallelize CI, then shrink the JS system-spec slice #7173

Description

@compwron

Baseline (measured, not guessed)

The CI rspec step on main takes 8.4 min, in one process, with no parallelism (bundle exec rspec).

Local timings by slice:

Slice Examples Time Per example
spec/system :js 368 498s 1.35s
spec/requests 932 50s 0.05s
spec/system (rack_test) 388 36s 0.09s
spec/models 666 22s 0.03s

JS system specs are ~82% of the wall clock for ~16% of the examples, and they ran at 28% CPU — the process sits idle waiting on Chrome. That single fact sets the priority order: parallelize first, then shrink the JS set.

Inside the JS slice:

  • spec/system/accessibility/axe_spec.rb — 45 examples / 118s (24% of JS time)
  • spec/system/casa_cases/edit_spec.rb — 26 examples / 61s
  • spec/system/typeahead_controls_spec.rb1 example / 30s
  • spec/system/layouts/flashes_spec.rb — 4 examples / 18.5s

Two things checked and ruled out as meaningful levers: Prosopite's per-example scan costs only ~4% (measured A/B on spec/models, 28.4s vs 29.7s), and Build App + assets:precompile is 0.2 min. Neither is worth touching early.

Phase 1 — turn on parallelism (config only, no spec edits)

The repo is already fully wired for parallel_tests and just doesn't use it in CI. The gem is in the Gemfile, and every per-process collision point is already parameterized by TEST_ENV_NUMBER:

  • config/database.yml:17 — per-process database name
  • config/storage.yml:3 — per-process Active Storage root
  • spec/rails_helper.rb:132 — per-process Capybara server port
  • spec/support/download_helpers.rb:3 — per-process download dir
  • spec/spec_helper.rb:18 — per-process SimpleCov command_name

Someone built this and it fell out of CI. Tasks:

  • In .github/workflows/rspec.yml, replace bundle exec rake db:create db:schema:load with bundle exec rake parallel:create parallel:load_schema, and bundle exec rspec with bundle exec parallel_test spec --type rspec -n 4.
  • Cache tmp/parallel_runtime_rspec.log across runs so --group-by runtime balances groups. Without it, whichever group holds axe_spec becomes the critical path.
  • Add a SimpleCov merge step — bump SimpleCov.merge_timeout and collate the per-process resultsets before the qlty upload, or the reported coverage will silently drop.

ubuntu-latest on a public repo is 4 vCPU. Because the JS specs are browser-wait-bound, 4 processes should scale close to linearly: ~8.4 min → ~3 min. Memory headroom needs a check — 4 concurrent headless Chromes in 16 GB should be fine, but watch it.

Phase 2 — shrink the JS slice

  • Audit the :js tags. 368 JS examples is far more than a Hotwire app of this size needs. Every one converted to rack_test goes 1.35s → 0.09s (15×). Highest-yield item on the list, but it's per-file judgment work, so no payoff number until the audit is done.
  • Move spec/system/accessibility/axe_spec.rb into its own CI job. 118s of full page-load + axe-core runs, independent of everything else. Running it concurrently takes ~2 min off the critical path at zero risk.
  • Fix the flakes, then cut retry: 3. spec/support/rspec_retry.rb retries every :js example up to 3× in CI, so the slowest specs are also the ones that can triple in cost. spec/system/learning_hours/date_range_spec.rb failed 3 examples in a local run and is burning retries in CI right now. Fix it, then drop to retry: 1.
  • Fix spec/system/typeahead_controls_spec.rb. One example, 30s, ~20 let! records, two explicit sleeps and a Timeout.timeout(6) poll loop. Split it per control and replace the manual polling with Capybara's own waiting.

Phase 3 — keep it from regressing

  • Add config.profile_examples = 10 behind an env var so the next slow spec is visible without a manual profiling run.
  • Set .prosopite_ignore directories to :off rather than :log_only. spec/system is on that list, so its N+1s can never fail the build, yet the slowest 82% of the suite still pays the scan cost. Only worth ~4% though — do it last.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions