Skip to content

Spark 4.2: Fix view rename target namespace - #18255

Open
manuzhang wants to merge 3 commits into
apache:mainfrom
manuzhang:spark-4.2-view-rename-target-namespace
Open

manuzhang wants to merge 3 commits into
apache:mainfrom
manuzhang:spark-4.2-view-rename-target-namespace

Conversation

@manuzhang

Copy link
Copy Markdown
Member

Problem

Renaming a qualified view to an unqualified target moves the view into the session's current namespace instead of keeping it in the source view's namespace.

USE catalog.other_ns;
ALTER VIEW source_ns.v RENAME TO v2;   -- v2 lands in other_ns, not source_ns

Root cause

ExtendedDataSourceV2Strategy resolves the rename target with Spark3Util.catalogAndIdentifier before deciding whether the target is actually unqualified. For a single-part target name that helper fills in the current namespace, so the identifier handed to IcebergRenameV2ViewExec already carries the session namespace.

IcebergRenameV2ViewExec does handle an unqualified target correctly — it fills the namespace in from the source view — but that branch is unreachable because the namespace is never empty by the time it runs.

Changes

  • Resolve single-part rename targets with an empty namespace, letting IcebergRenameV2ViewExec apply the source view's namespace. This matches Spark's v2 RenameTableExec.
  • Resolve multi-part targets relative to the source catalog, keeping the existing cross-catalog move rejection.
  • Add a regression test covering a qualified source view renamed to an unqualified target while the session is using another namespace.

Tests

The new test fails on main for all three catalog configurations and passes with the fix. TestViews runs 294 tests with no failures.


AI Disclosure

  • Model: Claude Opus 5
  • Platform/Tool: Claude Code
  • Human Oversight: partially reviewed
  • Prompt Summary: port the Spark 4.1 view rename namespace fix to Spark 4.2, with a regression test.

🤖 Generated with Claude Code

@github-actions github-actions Bot added the spark label Sep 25, 2026
@manuzhang
manuzhang force-pushed the spark-4.2-view-rename-target-namespace branch from 95f29e4 to 3e63160 Compare September 27, 2026 14:18
@manuzhang
manuzhang requested a review from szehon-ho September 30, 2026 03:49
manuzhang and others added 3 commits October 2, 2026 09:51
ExtendedDataSourceV2Strategy resolved the rename target with
Spark3Util.catalogAndIdentifier before deciding whether the target was
unqualified. For a single-part target name that helper fills in the
current namespace, which can differ from the source view namespace, so
the renamed view moved to the session namespace.

Resolve single-part targets by leaving the namespace empty, which
IcebergRenameV2ViewExec already fills in from the source view, and
resolve multi-part targets relative to the source catalog so the
existing cross-catalog move rejection still applies. This matches
Spark's v2 RenameTableExec.

Generated-by: Claude Code
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Keep the comment to what an unqualified target means, and document in
Spark3Util where a name without a namespace is placed.

Generated-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants