Skip to content

[iceberg] Retry transient REST DNS failures - #9903

Merged
JingsongLi merged 1 commit into
apache:masterfrom
arthurgaubil:arthur.gaubil/iceberg-rest-dns-retry
Sep 17, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
arthurgaubil:arthur.gaubil/iceberg-rest-dns-retry

Conversation

@arthurgaubil

@arthurgaubil arthurgaubil commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Purpose

Retry Iceberg REST catalog DNS lookup failures with configurable exponential backoff. Five retries with a one-second initial delay are used by default; setting metadata.iceberg.unknown-host-retry.max-retries to 0 preserves immediate failure. Exhausted retries still propagate so publication cleanup and durable recovery remain safe.

Tests

mvn -o -Ppaimon-iceberg,fast-build -pl paimon-iceberg -DwildcardSuites=none -Dtest=IcebergRestMetadataCommitterTest test

All 30 tests passed, including transient recovery, exponential delay, disabled retries, exhaustion, and non-DNS failures. Generated configuration documentation was refreshed with mvn -o package -Pgenerate-docs -pl paimon-docs -DskipTests.

@arthurgaubil arthurgaubil changed the title [ADPS-1364][iceberg] Retry transient REST DNS failures [iceberg] Retry transient REST DNS failures Sep 16, 2026
@arthurgaubil
arthurgaubil force-pushed the arthur.gaubil/iceberg-rest-dns-retry branch from 829c990 to 769e3f5 Compare September 16, 2026 17:22
@JingsongLi

Copy link
Copy Markdown
Contributor

Retrying a transient DNS failure on the commit path is reasonable, and the cause-chain walk correctly handles wrapped exceptions.

One note for the record: with the defaults (max-retries 5, initial delay 1 s, doubling), a genuinely dead host pins the commit thread for ~31 s (1+2+4+8+16) before failing. That is a reasonable trade-off and both knobs are configurable, but worth spelling out in the option docs so commit latency is not surprising. Also, this intentionally only covers UnknownHostException and not other transient connection errors — fine as a narrow scope, just worth documenting.

@JingsongLi
JingsongLi merged commit 586fb0d into apache:master Sep 17, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants