[SPARK-58382][SQL] binaryFile archive support via wholeFile option

[akshatshenoi-db]

What changes were proposed in this pull request?

This PR lets format("binaryFile") read tar/zip/7z archives, extending the archive-reader
series (SPARK-57135 CSV, SPARK-57419 JSON, SPARK-57478 text, SPARK-57479 XML, SPARK-57481 Avro).
It is gated by a new binaryFile option wholeFile (default true) and the existing
spark.sql.files.archive.reader.enabled flag (default false).

• wholeFile=true (default): the archive is read as a single record (its raw bytes) -- unchanged
  from today's non-archive behavior.
• wholeFile=false: one row is emitted per inner entry. content holds the entry's unpacked
  bytes, path is <archive>!/<entryName>, length is the entry's size, and modificationTime
  is the parent archive's (archive entry timestamps are optional, so the parent's is used).

BinaryFileFormat mixes in SupportsArchiveFormat. On the wholeFile=false archive path it
streams each entry via SupportsArchiveFormat.readArchiveEntries and builds one row per entry from
a synthetic per-entry FileStatus, reusing the same row-builder (BinaryFileFormat.parse) as the
non-archive path so length/modtime filter pushdown applies per entry.

To expose the entry's size to consumers, the readArchiveEntries parseEntry callback now
receives the ArchiveEntry instead of just the entry name; the existing CSV and localizeEntries
callers are updated to read entry.getName.

Known limitation (zip, until ZipFile): the streaming ZipArchiveInputStream reports an entry's
getSize as -1 when the size is only in a trailing data descriptor, so for such entries the
per-entry length column is -1 and the spark.sql.sources.binaryFile.maxLength guard
(status.getLen > maxLength) is skipped, i.e. the entry is read unbounded. tar and 7z report real
sizes and are unaffected. This closes when zip reads move to ZipFile, which exposes entry sizes
from the central directory up front.

Why are the changes needed?

binaryFile is the natural way to ingest opaque/blob files, and archives (tar/zip/7z) are a common
packaging for large numbers of such files. This lets an archive be read as a directory of its
entries without unpacking it to disk first, completing the archive-reader feature across the
file-based formats.

Does this PR introduce any user-facing change?

Yes, gated behind spark.sql.files.archive.reader.enabled (default false). When enabled,
spark.read.format("binaryFile").option("wholeFile", "false").load(<archive>) returns one row per
inner entry. With the flag off, or with wholeFile=true (the default), behavior is unchanged.

How was this patch tested?

New BinaryFileArchiveReadBase with BinaryFileTarArchiveReadSuite,
BinaryFileZipArchiveReadSuite, and BinaryFileSevenZArchiveReadSuite, covering: whole-file vs
per-entry reads, per-entry content/path/length, empty archives, hidden-entry skipping,
spark.sql.sources.binaryFile.maxLength enforcement, length filter pushdown, single-partition
scans, and corrupt-archive handling under ignoreCorruptFiles. The per-entry length assertions
are skipped for zip because the streaming ZipArchiveInputStream reports a data-descriptor entry's
size as -1; they re-enable when zip reads move to ZipFile.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code

[akshatshenoi-db]

This PR's entrySizeKnown = false override for zip (in BinaryFileArchiveReadBase / the binaryFile zip suite) exists only because streaming ZipArchiveInputStream can't report an entry's size up front. #57588 (SPARK-58394) migrates zip reads to random-access ZipFile, which exposes entry sizes from the central directory — so once #57588 lands, the entrySizeKnown override and its assume(entrySizeKnown) guards here can be dropped, letting the per-entry length tests run for zip too. Marking this dependent on #57588 for that gate removal (the gate lives only in this PR, so #57588 doesn't touch it).