[SPARK-59582][SQL] Include original JDBC engine SQLSTATE in external engine syntax errors

[alekjarmov]

What changes were proposed in this pull request?

Surface the remote JDBC engine SQLSTATE on JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR as a structured message parameter, labeled separately from Spark's own SQLSTATE.

Why are the changes needed?

Improves observability by improving error grouping.

Does this PR introduce any user-facing change?

Yes. The JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR message now includes the remote SQLSTATE.
Previously:

JDBC external engine syntax error. The error was caused by the query <jdbcQuery>. <externalEngineError>.

Now:

JDBC external engine syntax error. The error was caused by the query <jdbcQuery>. <externalEngineError>. External engine SQLSTATE: <externalEngineSqlState>.

Spark's own SQLSTATE for this condition remains 42000.

How was this patch tested?

Updated SharedJDBCIntegrationSuite so the SPARK-52184 wrapping test asserts externalEngineSqlState is present and matches SQLException.getSQLState from the cause (or unknown).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Cursor Grok 4.6

[uros-b]

Some additional nits: PR body says “groupation”; the before/after examples lost / because GitHub treats them as HTML, please wrap them in backticks.

Also, the duplicated throw sites could move into a small helper (QueryCompilationErrors / QueryExecutionErrors), but that is pre-existing and not blocking.

[uros-b]

The change adds externalEngineSqlState to both JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR throw sites in JDBCRDD and templates it as:

JDBC external engine syntax error with SQLSTATE <externalEngineSqlState>. The error was caused by the query <jdbcQuery>. <externalEngineError>.

Keeping Spark’s own SQLSTATE at 42000 and putting the remote value in messageParameters is the correct design. Putting that value in the message as bare “SQLSTATE” fights Spark’s existing pretty-printer, which already appends Spark’s SQLSTATE:

SparkThrowableHelper.scala
Ln 78–86
def formatErrorMessage(
    errorClass: String,
    displayMessage: String,
    sqlState: String,
    context: String): String = {
  val displaySqlState = if (sqlState == null) "" else s" SQLSTATE: $sqlState"
  val displayQueryContext = (if (context.isEmpty) "" else "\n") + context
  val prefix = if (errorClass.startsWith("_LEGACY_ERROR_")) "" else s"[$errorClass] "
  s"$prefix$displayMessage$displaySqlState$displayQueryContext"
}

On Postgres (42601) the user-visible line becomes two different SQLSTATEs:

[JDBC_EXTERNAL_ENGINE_SYNTAX_ERROR.DURING_OUTPUT_SCHEMA_RESOLUTION] JDBC external engine 
syntax error with SQLSTATE 42601. The error was caused by the query .... <engine message>. The error occurred during output schema resolution. SQLSTATE: 42000

That undercuts the grouping/observability goal: log parsers and humans cannot tell which SQLSTATE is Spark’s and which is the engine’s.

Suggested template:

JDBC external engine syntax error. The error was caused by the query <jdbcQuery>. <externalEngineError>. External engine SQLSTATE: <externalEngineSqlState>.

The placeholder has to stay in the template (tests reject unused parameters unless the class is allowlisted). Only the label needs to change.

[alekjarmov]

Done

[uros-b]

Test is in the right suite, but weak. SharedJDBCIntegrationSuite is the existing SPARK-52184 coverage and will run in CI because this PR touches docker-integration-tests. Please tighten it:

• assert Spark SQLSTATE is still 42000 (sqlState = Some("42000"))
• avoid asInstanceOf[SQLException] on getCause
• [\s\S]* accepts an empty value; the extra assert is what actually checks the parameter

An H2 test in JDBCSuite would cover this without Docker; not required if the Docker job is green.