BUG: Use label integers instead of float to count pixels - #6759
BUG: Use label integers instead of float to count pixels#6759thomas-albrecht wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Thank you for contributing a pull request! 🙏
Welcome to the ITK community! 🤗👋☀️
We are glad you are here and appreciate your contribution. Please keep in mind our community participation guidelines. 📜
More support and guidance on the contribution process can be found in our contributing guide. 📖
This is an automatic message. Allow for time for the ITK community to be able to read the pull request and comment
on it.
|
|
TODOs: merge into one commit, add commit message prefix as per ghostflow. Preferably before merging, latest upon merge. |
|
Thank you for identifying the issue and contributing. I started to look into this and then saw you PR here. Here are some additional AI recommendation I found: The PR correctly fixes InitializePriorProbabilities() but leaves the same float-counting bug unfixed in InitializeConfusionMatrixArrayFromVoting(): This is arguably the more impactful site because these counts initialize the confusion matrices used throughout all EM iterations. A consistent fix using a local std::vector<size_t> (or Array2D) for counting, followed by normalization into TWeights, should be applied there as well, mirroring the pattern used in InitializePriorProbabilities(). Additional observations: EM M-step accumulation unaddressed — m_UpdatedConfusionMatrixArray[k][j][ci] += W[ci] still accumulates TWeights (float) sums over all pixels. For large 3D images this is a secondary but real precision issue; using double internally for m_UpdatedConfusionMatrixArray (which is private, so no API impact) would resolve it. ITK style nit — std::vector<size_t> is fine, but SizeValueType (ITK's canonical pixel-count type, defined as uint64_t) would be more idiomatic. |
|
Cool, thanks for the feedback. Of course initially I wanted to be as surgical as possible and only change the things that directly affected the problem I found instead of giving the class a general overhaul. But all your points are valid, am I "allowed" to go ahead and make the changes myself? |
|
That can also be a follow-up PR. |
|
I made the changes and pushed them already. |
Float precision is lost when incrementing a float counter beyond 2^24 (~16.7M). Replace float-based pixel counting with integer accumulators in InitializePriorProbabilities and InitializeConfusionMatrixArrayFromVoting. Use double for m_UpdatedConfusionMatrixArray to avoid accumulation loss in the EM M-step.
8ada944 to
29d724e
Compare
| const SizeValueType numCols = static_cast<SizeValueType>(this->m_TotalLabelCount); | ||
| Array2D<SizeValueType> counts(numRows, numCols); | ||
|
|
||
| for (unsigned int k = 0; k < numberOfInputs; ++k) |
There was a problem hiding this comment.
Here's an AI description why the code does the same thing:
The old code uses two separate loops over k:
First loop: count into the float matrix
Second loop: normalize each row by its sum
The new code merges both steps into one loop over k: for each input k, it counts into the integer matrix, then immediately normalizes that input's confusion matrix before moving to the next k.This is valid because there's no cross-dependency between inputs — the confusion matrix for input k is populated and normalized entirely independently of any other input k'. The old code's separation into two loops was unnecessary; it just happened to be written that way.
dzenanz
left a comment
There was a problem hiding this comment.
I made through review earlier, and a glancing review after the scope increase. Everything looked OK. Is this functionality being exercised by any tests?
|
It looks to me like there are extensive tests in |
The original count uses ++ on a
WeightsType, which defaults to float, to count pixels.When there are more than ~ 16.7 million pixels to be counted, which happens quickly in 3D images (times number of inputs), the ++ has no more effect as 16,777,217 has the same float representation as 16,777,216.
The fix is simple: Do the counting in integers (
size_t) and then the final operationin float (
WeightsType).PR Checklist
Refer to the ITK Software Guide for
further development details if necessary.