[ISSUE #5411] Serve the legacy SDK gRPC protocol on the v2 runtime (port 10205 bridge) + connector plugin tests 23/23 - #5412
Merged
Conversation
…ime (port 10205 bridge) EventMeshGrpcServer binds PublisherService/ConsumerService/HeartbeatService on eventmesh.grpc.port (opt-in, 0/-1 = off) and maps every call onto the v2 UniIngressService pipeline (WAL at-least-once, shared retry/DLQ): - publish/batchPublish/publishOneWay/batchPublishOneWay -> ingress.publish (topic = subject) - requestReply -> ingress.request with the legacy TTL attribute - webhook subscribe -> WebHookChannel target; CLUSTERING->LOAD_BALANCE, BROADCASTING->BROADCAST - subscribeStream (bidi) -> GrpcStreamChannel push target, ACKs ride back on the stream - heartbeat -> GrpcClientRegistry TTL refresh + reaper unsubscribes stale clients GrpcCloudEventMapper holds the proto<->v2 CloudEvent mapping + legacy Response envelope. ruleGrpcProtocolHidden grants the sanctioned runtime.grpc.. adapter (TCP-rule precedent).
…real-SDK integration GrpcCloudEventMapperTest covers every proto attribute flavor + the 3-key response envelope. GrpcLegacyBridgeIntegrationTest boots the bridge over the in-memory storage and drives it with the real EventMeshGrpcProducer/Consumer (the SDK as its own conformance suite): publish, batch publish, stream subscription round-trip.
Hermetic where possible: webhook sinks (dingtalk/http/knative/lark/slack/wechat/ wecom/chatgpt) verified against a local capturing HTTP server; mcp (JSON-RPC envelope), openfunction (Ce-* context headers), prometheus (single merged push), spring (EventForwarder contract) exercised with their exact wire contracts; webhook sources (10 plugins) POST their native callback payloads through the lazy-bound hook port; prometheus/spring sources cover scrape + buffer drains; external-client plugins (canal/jdbc/mongodb/rabbitmq/redis/s3/pravega) get contract-level tests (init config-parse where lazy, commit no-op) since their backends cannot boot in CI. Closes the "4 of 23 plugins carry the only unit tests" gap tracked under the apache#5296 review.
…or Runtime test status configuration.md flips eventmesh.grpc.port from RESERVED to served (opt-in, 1.x default 10205); protocols.md gains the legacy-gRPC mapping table section; both READMEs note the served bridge and update the Connector Runtime row to 23/23 plugin test coverage (drops the stale "4 of 23" wording).
Replace the vague "GA criteria tracked under the apache#5296 architecture review" wording with the actual remaining GA blockers: per-plugin real-backend integration tests and the connector-runtime HA story. Also reference apache#5412 for the 23/23 unit-test coverage.
…nd 23/23 tests - deployment.md: connector-runtime status drops the stale "only 4 of 23 plugins carry unit tests" (now 23/23 since apache#5412; real-backend ITs stay the GA gate) - architecture/overview.md: protocol-status bullet + legacy-SDK paragraph mention the served opt-in gRPC bridge; Key-classes table gains the EventMeshGrpcServer entry - quickstart/getting-started.md: port list mentions the opt-in 10205 legacy gRPC bridge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #5411.
The v2 runtime served HTTP + CloudEvents but kept
eventmesh.grpc.port=10205RESERVED — the legacy SDK gRPC surface (EventMeshGrpcProducer/EventMeshGrpcConsumer, protos ineventmesh-common) was fully preserved yet had nothing to connect to. This PR serves it as a compatibility bridge onto the v2 ingress path (no second messaging engine), and — per the request on the issue — folds in the connector-plugin test work that closes the "4 of 23 plugins carry the only unit tests" gap.Changes
1. Legacy gRPC bridge (
eventmesh-runtime/.../grpc/)EventMeshGrpcServer— gRPCServerBuilderoneventmesh.grpc.port(1.x default 10205,0/unset = disabled, opt-in like the WS port), graceful shutdown with the app lifecycle.publish/batchPublish/publishOneWay/batchPublishOneWay→ proto CloudEvent mapped and persisted viaUniIngressService.publish(topic = protosubject).requestReply→UniIngressService.request, TTL attribute drives the timeout; replies map back to proto.subscribe→WebHookChannelpush target + per-topic v2 subscriptions.subscribeStream(bidi) →GrpcStreamChannelpush target pumping the v2 dispatcher into the stream; ACKs ride back as stream replies (emdeliveryidecho).heartbeat→GrpcClientRegistryTTL refresh; a reaper evicts (and unsubscribes) stale clients.GrpcCloudEventMapper— pure proto ↔ v2io.cloudevents.CloudEventmapping + the legacyResponseenvelope (statuscode/responsemessage/time, the exact 3 keys the SDK parses).CLUSTERING→LOAD_BALANCE,BROADCASTING→BROADCAST; clientId derived fromconsumerGroup+env+idc.EventMeshApplication.withGrpcBridge(port)+-Deventmesh.grpc.portinmain(); runtime gainsio.grpc:grpc-netty-shaded/protobuf/stub1.68.0 (aligned with eventmesh-common).ruleGrpcProtocolHiddennow exempts the sanctionedorg.apache.eventmesh.runtime.grpc..adapter (same precedent the TCP rule grantsruntime..), package-info doc updated.2. Tests
GrpcCloudEventMapperTest— hermetic: every proto attribute flavor round-trips; response envelope contract.GrpcLegacyBridgeIntegrationTest— boots the bridge over in-memory storage and drives it with the real legacy SDK (EventMeshGrpcProducerpublish / batch publish,EventMeshGrpcConsumerstream subscription round-trip). The SDK is its own conformance suite, per the issue.poll()CloudEvents;commitno-op) — their backends can't boot in CI.3. Docs
docs/quickstart/configuration.md:eventmesh.grpc.portRESERVED → served (opt-in).docs/feature/protocols.md: new §1.1 legacy-gRPC bridge mapping table + non-goals.README.md/README.zh-CN.md: migration note gains "gRPC bridge served since Serve the legacy SDK gRPC protocol on the v2 runtime (port 10205 compatibility bridge) #5411"; Connector Runtime row updated to 23/23 plugin test coverage (drops the stale "4 of 23" wording, per the linked request); the tracking note now points to the remaining GA criteria under the [Architecture Review] Consolidate EventMesh boundaries and production-grade consistency #5296 review.Verification
gradlew test checkstyleMain checkstyleTest— runtime 340/340 (incl. 8 new gRPC tests), architecture-guard green, checkstyle 0 violations across all modules; 18/19 connector plugin modules green (81 plugin tests total).eventmesh-connector-filemodule tests fail only on the dev machine withFailed to delete temp directory— a local DLP/ACL artifact (verified with a standalone Java probe: even never-opened temp files are delete-denied on this host). These tests are untouched by this PR and pass on CI/Linux.Follow-ups (not in this PR)
subscribeStreamreplies.