Skip to content

HDDS-11853. Avoid redundant UUID conversion in XceiverClientRatis hot path - #11247

Open
lpavanvenkat wants to merge 1 commit into
apache:masterfrom
lpavanvenkat:HDDS-11853
Open

lpavanvenkat wants to merge 1 commit into
apache:masterfrom
lpavanvenkat:HDDS-11853

Conversation

@lpavanvenkat

@lpavanvenkat lpavanvenkat commented Sep 16, 2026

Copy link
Copy Markdown

What changes were proposed in this pull request?

What and why
XceiverClientRatis.sendCommandAsync called RatisHelper.toDatanodeId(reply.getReplierId()) on every Ratis reply, which internally parses a UUID string via UUID.fromString — only to look up a DatanodeDetails object that is already available from the pipeline. This is redundant work on the hot write path.
Changes
In hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/XceiverClientRatis.java:

  • A Map<String, DatanodeDetails> field peerIdToDatanode is pre-built at construction time from pipeline.getNodes(), keyed by each datanode's UUID string.
  • addDatanodetoReply(String, XceiverClientReply) is added to do a direct map lookup instead of calling RatisHelper.toDatanodeId (which called UUID.fromString) on every reply.
  • sendCommandAsync is updated to call addDatanodetoReply(reply.getReplierId(), ...) directly, removing the intermediate UUID serverId variable.
    The map is always complete: Ozone Ratis pipelines have fixed membership — when a node fails, SCM closes the pipeline and creates a new one. The Ratis peer ID is always datanode.getUuidString() (RatisHelper.toRaftPeerIdString), the exact key used in the map. A miss is structurally impossible.
    No changes to RatisHelper, DatanodeID, or any other class.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11853

How was this patch tested?

  • Unit tests: mvn -pl :hdds-client test — 148 tests, all passed.
  • Checkstyle: ./hadoop-ozone/dev-support/checks/checkstyle.sh — 0 violations across all 58 modules.
  • Integration test: mvn -pl :ozone-integration-test test -Dtest=TestCommitInRatis — 2 tests passed.
  • Verified via temporary LOG.info that every addDatanodetoReply call during TestCommitInRatis produced hit=true with zero fallback UUID parses.
    No UI changes. No Protobuf changes.

@ashishkumar50

Copy link
Copy Markdown
Contributor

@lpavanvenkat please enable CI workflow in your fork branch.

@lpavanvenkat

Copy link
Copy Markdown
Author

I enabled the CI workflow on my forked branch.

@adoroszlai adoroszlai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lpavanvenkat for the patch.

}
asyncReply.setLogIndex(reply.getLogIndex());
addDatanodetoReply(serverId, asyncReply);
addDatanodetoReply(reply.getReplierId(), asyncReply);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reply's list of datanodes is used only for watchForCommit commands, to mark failed nodes:

private void checkReply(XceiverClientReply reply) {
if (reply == null) {
return;
}
final List<DatanodeDetails> dnList = reply.getDatanodes();
if (dnList.isEmpty()) {
return;
}
LOG.warn("Failed to commit BlockId {} on {}. Failed nodes: {}",
blockID, xceiverClient.getPipeline(), dnList);
failedServers.addAll(dnList);

and

XceiverClientReply reply = bufferFull ?
commitWatcher.watchOnFirstIndex() :
commitWatcher.watchOnLastIndex();
if (reply != null) {
List<DatanodeDetails> dnList = reply.getDatanodes();
if (!dnList.isEmpty()) {
Pipeline pipe = xceiverClient.getPipeline();
LOG.warn("Failed to commit BlockId {} on {}. Failed nodes: {}",
blockID, pipe, dnList);
failedServers.addAll(dnList);

But here the node is added for all kinds of commands and regardless of its result.

Question: is it even necessary to add the datanode here? (I tried removing it and CI passed.)

CC @jojochuang @szetszwo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants