Skip to content

HDDS-16405. ReplicationManagerReport should not be used for transient state - #11230

Open
sodonnel wants to merge 2 commits into
apache:masterfrom
sodonnel:HDDS-16405
Open

sodonnel wants to merge 2 commits into
apache:masterfrom
sodonnel:HDDS-16405

Conversation

@sodonnel

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

In a recent change ReplicationManagerReport gained an extra field to track the end health state of a container when it is processed. While this resulted in a smaller code change, this not how RMReport is supposed to be used and it was intended to have aggregated stats, not per request stats. We already have the ContainerCheckRequest object which has all details of a request and any result can be set in there.

This change removes the transient per container state from RMReport and moves the result into ContainerCheckRequest where it is better suited. It also centralizes the report "increment and sample" so the state and increment are performed together with a single call from the handlers.

Also added a new test in TestContainerCheckRequest to ensure that when calling request.setHealthState() it updates the report and sets the heath result, which avoids having to update all the sets that assert on the report state, as that would have made this change a lot larger.

This change is needed before #11199 can move forward as it starts to build on the pattern added to the report.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16405

How was this patch tested?

Existing tests and a new unit test.

Copilot AI lite review requested due to automatic review settings September 11, 2026 09:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The moderate, three-vote finding leaves the read-only classification path with stale health state.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

HDDS-16405 moves transient container health state from ReplicationManagerReport into ContainerCheckRequest while preserving aggregate reporting.

Changes:

  • Centralizes health-state updates and report sampling.
  • Updates replication health handlers to use request state.
  • Removes transient report state and adds unit coverage.
File summaries
File Summary
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/TestContainerCheckRequest.java Tests request health-state and report synchronization.
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/replication/health/TestQuasiClosedStuckReplicationCheck.java Removes obsolete report reset.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ReplicationManager.java Uses request health state during processing. Moderate finding (3 votes): the read-only path can leave stale health state because the result is not copied back.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/VulnerableUnhealthyReplicasHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/RatisUnhealthyReplicationCheckHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/RatisReplicationCheckHandler.java Uses centralized request health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/QuasiClosedStuckReplicationCheck.java Uses centralized request health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/QuasiClosedContainerHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/OpenContainerHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/EmptyContainerHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/ECReplicationCheckHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/ECMisReplicationCheckHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/ClosingContainerHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/health/ClosedWithUnhealthyReplicasHandler.java Uses request-based health-state updates.
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/replication/ContainerCheckRequest.java Stores per-request health state and updates report statistics.
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManagerReport.java Removes transient per-container health-state storage.
Review details
  • Files reviewed: 16/16 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@F64116045 F64116045 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sodonnel for this refactor. Really appreciate your help with the #11199.
LGTM

@sodonnel
sodonnel requested a review from devmadhuu September 16, 2026 09:59

@devmadhuu devmadhuu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sodonnel for the patch. This looks better by refactoring. Just few nits. Please check.

* Simple class to wrap the parameters needed to check a container's health
* in ReplicationManager.
*/
public final class ContainerCheckRequest {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nit and cosmetic , so that no confusion. Earlier the scope of this object was just to carry request params through the chain, but now it is also holding the container healthstate which gets updated at every exec of handler. You can decide if we should rename to something like: ContainerCheckContext, if not in this PR, follow up PR ?

}
// Apply final health state from report to container
containerInfo.setHealthState(report.getContainerHealthState());
containerInfo.setHealthState(checkRequest.getHealthState());

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just caught up in this PR, though this is old code behavior. I think we should protect this container health state from being overwritten by readonly path which usually called by decommission / maintenance workflow calls, so there could be a race flip flop for the value. How about if we protect with if (!readOnly)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devmadhuu I originally did guard this with !readonly, but as copilot highlighted above (and I confirmed) that caused tests you added to fail. Thinking about your earlier change which added this logic, will it do any harm beyond those tests if we add the !readonly guard?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants