Skip to content

refactor(connectors): extract row-processing logic into reusable process_row method#3198

Open
atharvalade wants to merge 1 commit intoapache:masterfrom
atharvalade:fix/deduplicate-row-processing-logic
Open

refactor(connectors): extract row-processing logic into reusable process_row method#3198
atharvalade wants to merge 1 commit intoapache:masterfrom
atharvalade:fix/deduplicate-row-processing-logic

Conversation

@atharvalade
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #3173

Rationale

Row-processing logic lived inline in poll_tables, making it impossible to reuse across future parallel/chunked polling paths without duplication.

What changed?

The column-iteration, value-extraction, JSON-payload construction, and ProducedMessage assembly were tightly coupled inside poll_tables's inner loop (~80 lines). Any new polling variant (parallel, chunked) would need to copy-paste this block.

Extracted a process_row method that accepts a RowProcessingConfig (table, tracking/pk columns, format flags) and returns a ProcessedRow (message + offset + pk). The poll_tables loop now delegates to this single-responsibility method in ~12 lines.

Local Execution

  • Passed
  • Pre-commit hooks ran

AI Usage

  1. Opus 4.6
  2. used for exploration and implementation guidance
  3. Verified via cargo check, clippy -D warnings, fmt --check, all 19 unit tests passing, CI lint scripts passing
  4. Yes, all code can be explained

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 0% with 87 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.22%. Comparing base (611fca0) to head (9cc08ad).

Files with missing lines Patch % Lines
core/connectors/sources/postgres_source/src/lib.rs 0.00% 87 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             master    #3198       +/-   ##
=============================================
- Coverage     74.10%   19.22%   -54.88%     
  Complexity      943      943               
=============================================
  Files          1159     1157        -2     
  Lines        102033    90344    -11689     
  Branches      79083    67394    -11689     
=============================================
- Hits          75607    17371    -58236     
- Misses        23765    72575    +48810     
+ Partials       2661      398     -2263     
Components Coverage Δ
Rust Core 0.95% <0.00%> (-74.38%) ⬇️
Java SDK 60.14% <ø> (ø)
C# SDK 69.38% <ø> (ø)
Python SDK 81.43% <ø> (ø)
Node SDK 91.53% <ø> (ø)
Go SDK 39.43% <ø> (ø)
Files with missing lines Coverage Δ
core/connectors/sources/postgres_source/src/lib.rs 40.78% <0.00%> (-26.40%) ⬇️

... and 647 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Row processing logic duplicated three times

1 participant