Skip to content

chore(test): upgrade the mock collector and drop the sw_fork_support seed workaround - #410

Merged
wu-sheng merged 1 commit into
masterfrom
chore/upgrade-mock-collector
Aug 2, 2026
Merged

chore(test): upgrade the mock collector and drop the sw_fork_support seed workaround#410
wu-sheng merged 1 commit into
masterfrom
chore/upgrade-mock-collector

Conversation

@wu-sheng

@wu-sheng wu-sheng commented Aug 1, 2026

Copy link
Copy Markdown
Member

Why

tests/plugin/docker-compose.base.yml pinned the mock collector at 7f20775e, built in October 2022 — 13 commits behind the tool's master.
More importantly, that image contains the first-insert race in SegmentItems/LogItems fixed by apache/skywalking-agent-test-tool#65: when two processes report their first segment for the same service name concurrently, the collector silently discards one while both reporters receive HTTP 200. sw_fork_support (added in #409) hits this by design — its forked parent and child share a service name — and had to work around it by seeding the service name with a parent-only /ping request and polling /receiveData until the collector had registered it.

What

  • Repin to c6b91a0eaef16d427268e9806d5df65949e2a9bf, the current mock-collector image, which carries that fix.
  • Remove the workaround entirely: the /ping route, the seed-and-poll step and the extra expected segment are gone, so sw_fork_support is back to asserting exactly the cross-fork trace it exists for (parent entry/exit -> child entry with CrossProcess ref), plus the agent-restart log assertions.
  • Fix the one incompatibility the upgrade exposed, in sw_loguru (see below).

The one incompatibility, and why it is confined to sw_loguru

The upgrade picks up every validator change made since 2022. Exactly one affected us:
LogAssert now sorts both the expected and the actual logs by their body text before comparing them pairwise (apache/skywalking-agent-test-tool#59). The sort uses the raw expected string, so a matcher expression such as text: not null participates in the sort as the literal "not null". sw_loguru matches its two logging-module records that way, while the records themselves led with the default layout's timestamp — so the real records sorted to the front and their placeholders to the back, inverting the pairing and failing every field. Verified by A/B: the same test passes with the old image pinned and fails with the new one.
Fixed by pinning a log layout that leads with the logger name, which sorts after the placeholders and orders the two records deterministically (CRITICAL before ERROR), plus reordering the expected entries to match. SWFormatter is still exercised; only the field order within the layout changes.
sw_loguru/expected.data.yml is the only expected file in the whole tests/plugin/ tree with a non-empty logItems: block, so no other test can be affected by that change.
Note for future test authors: with this collector, not null placeholders inside logItems are order-significant, because they sort as their literal string.

Verification

  • CI on this PR: 75 pass, 0 fail across Python 3.10-3.14 (plugin, unit and e2e).
  • Local sweeps of tests/plugin/web (14 tests) and tests/plugin/data (13 test dirs) against the new image found no other drift, with every existing expected.data.yml unchanged.
  • The stricter span validation (Add Rabbitmq Plugin #53) produced zero failures across tests covering Entry/Exit/Local spans, cross-process refs and Http/Database/Cache/MQ layers. A negative control confirmed the new validation is genuinely enforcing: mutating componentId, operationName or a tag value in a passing expected file is rejected with HTTP 500.

The pinned mock-collector image dated from October 2022, 13 commits behind
the tool's master. Move it to c6b91a0e, which carries the fix for the
first-insert race in SegmentItems/LogItems
(apache/skywalking-agent-test-tool#65): when two processes reported their
first segment for the same service name concurrently, the collector
silently dropped one while both reporters got HTTP 200.

sw_fork_support worked around that race by seeding the service name with a
parent-only /ping request and waiting for the collector to register it
before triggering the concurrent parent/child reports. The collector no
longer needs the help, so the endpoint, the seed step and the extra
expected segment are gone and the test is back to asserting exactly the
cross-fork trace it is about.

The upgrade also picks up the validator changes made since 2022, one of
which affected us. LogAssert now sorts both the expected and the actual
logs by their body text before comparing them pairwise
(apache/skywalking-agent-test-tool#59), and the sort uses the raw expected
string, so a matcher such as `text: not null` participates as the literal
"not null". sw_loguru matches its two logging-module records that way
while the records themselves led with the default layout's timestamp, so
they sorted first and their placeholders last, inverting the pairing.
Pin a layout that leads with the logger name, which sorts after the
placeholders and orders the two records deterministically, and reorder the
expected entries to match; SWFormatter is still exercised. Its
expected.data.yml is the only one in the tree with a non-empty logItems
block, so no other test is affected.

Also refresh CLAUDE.md: supported Python and grpcio floor, the current
plugin list, the agent's fork/prefork lifecycle, and the plugin-test
validation notes.
@wu-sheng
wu-sheng force-pushed the chore/upgrade-mock-collector branch from 1e103ac to d899034 Compare August 1, 2026 15:46
@wu-sheng wu-sheng added the test test label Aug 1, 2026
@wu-sheng wu-sheng added this to the 1.3.0 milestone Aug 1, 2026
@wu-sheng
wu-sheng requested review from Copilot and kezhenxu94 August 2, 2026 00:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR modernizes the plugin-test harness by upgrading the pinned mock collector image and removing a now-unnecessary sw_fork_support warm-up workaround that existed to avoid a first-insert race in older collectors. It also adjusts sw_loguru’s log reporter layout and expected log ordering to remain compatible with newer collector log-validation behavior, and updates contributor documentation accordingly.

Changes:

  • Repin the mock collector image in tests/plugin/docker-compose.base.yml to a newer SHA.
  • Remove the /ping seed route + polling logic from sw_fork_support, and update its expected segments accordingly.
  • Stabilize sw_loguru log assertions by pinning SW_AGENT_LOG_REPORTER_LAYOUT and reordering/updating logItems expectations; update CLAUDE.md with current support/lifecycle notes.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/plugin/web/sw_fork_support/test_fork_support.py Drops the /ping seed-and-poll workaround; relies on the normal prepare() call to hit /users before validation.
tests/plugin/web/sw_fork_support/services/app.py Removes the /ping endpoint so the test asserts only the intended cross-fork /users trace.
tests/plugin/web/sw_fork_support/expected.data.yml Removes the /ping segment and updates segmentSize accordingly.
tests/plugin/docker-compose.base.yml Repins the mock collector image to the newer SHA referenced in the PR description.
tests/plugin/data/sw_loguru/expected.data.yml Updates log expectations and documents the new order significance of logItems sorting by body text.
tests/plugin/data/sw_loguru/docker-compose.yml Pins SW_AGENT_LOG_REPORTER_LAYOUT to keep reported log text ordering deterministic under the new collector behavior.
CLAUDE.md Updates project documentation (Python/grpcio floors, fork lifecycle notes, validator behavior notes, plugin list count).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@wu-sheng
wu-sheng merged commit e78d0cf into master Aug 2, 2026
219 of 224 checks passed
@wu-sheng
wu-sheng deleted the chore/upgrade-mock-collector branch August 2, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants