fix(docx): read content controls and tracked revisions instead of skipping them

[L4XB]

Problem

partition_docx() reads a document through python-docx. python-docx looks only at the w:p and w:tbl children of the body, a table cell, a header or a footer, and at the runs and hyperlinks directly inside a paragraph. Content one level deeper is skipped without a warning:

• Content controls (w:sdt): the paragraphs, tables, table rows and runs held in them. Forms, templates and cover pages use them, and so do the document properties Word inserts through Quick Parts (feat/docx-content-control-fields #3553).
• Tracked changes (w:ins, w:moveTo): text inserted or moved with track changes on and not yet accepted (docx: partitioner finds text nested in revision-marks #1821).
• Other run wrappers: runs in custom XML markup, smart tags, simple fields and bidirectional spans (w:customXml, w:smartTag, w:fldSimple, w:dir, w:bdo).

I built a probe document with one marker in each place and ran it through partition_docx(). A check means the marker is in the output:

marker	main	this PR
plain paragraph	✓	✓
paragraph in a block content control	–	✓
paragraph in block w:customXml	–	✓
inline content control	–	✓
tracked insertion	–	✓
simple field result	–	✓
smart tag	–	✓
hyperlink	✓	✓
table cell	✓	✓
deleted text, should stay out	–	–

Change

DocxPartitionerOptions.document now runs _unwrap_nested_content() right after loading, on the main document part and on every header and footer part. The function replaces each of these wrappers with its content, in place. The rest of the partitioner then sees that content as ordinary paragraphs, rows, cells and runs, and needs no change of its own. That covers body and section iteration, tables and text_as_html, headers and footers, hyperlinks, emphasis, rendered page breaks and pictures.

This is the "accept all revisions before partitioning" step proposed in #1821, extended to the other wrappers:

• w:ins and w:moveTo are unwrapped, so the inserted or moved-in text is read.
• w:del and w:moveFrom stay where they are. python-docx skips them there, so deleted and moved-away text stays out. The result is the text Word shows after "Accept All Changes".
• w:sdt is replaced by the children of its w:sdtContent, at whatever level it sits: in the body, in a table (a repeating-section row), in a row (a cell), or in a paragraph (inline).
• w:customXml, w:smartTag, w:fldSimple, w:dir and w:bdo are unwrapped.

Two content controls keep today's behaviour:

• The automatic table of contents (w:docPartGallery = "Table of Contents") stays unread, as it is now.
• A control still showing its placeholder text (w:showingPlcHdr), such as "Click or tap here to enter text.", is dropped. That text only says what to type, so it is not document content.

Where these elements can appear, the analysis #1821 asked for:

• Content controls and custom XML: at block, row, cell and run level.
• w:ins and w:moveTo: at run level. There are also empty marks in w:rPr, w:trPr and w:numPr that hold no content, so removing an empty one changes nothing that is read.
• Smart tags, simple fields, w:dir and w:bdo: at run level.
• Headers and footers can hold all of them. Word's built-in headers and cover pages hold document properties as content controls, which is why header and footer parts are included.

Performance. Each part gets one iter() filtered by tag.

document	load	the new pass	partition_docx()
20,000 paragraphs, none of these elements	20 ms	0.05 ms
20,000 paragraphs, each with a tracked insertion	31 ms	111 ms	70 s

Testing

There are three new tests in test_unstructured/partition/test_docx.py. Each builds its document in tmp_path, like the merged-cell tests:

• test_partition_docx_reads_content_nested_in_content_controls_and_revisions covers:
  • a block-level control and block-level custom XML;
  • an inline control;
  • an insertion next to a deletion that holds a tab;
  • a move-from and move-to pair;
  • a simple field, a smart tag and inline custom XML;
  • w:dir and w:bdo;
  • an empty w:ins in paragraph-mark properties.
• test_partition_docx_reads_content_controls_in_tables_headers_and_footers covers a control in a cell, a row wrapped in a control (checked in text and in text_as_html), and controls in the header and the footer.
• test_partition_docx_leaves_out_the_table_of_contents_and_placeholder_text checks that the table-of-contents control and a placeholder stay out.

Results:

• On main, the first two tests fail. The third passes, because it pins the current behaviour. With the change, all 88 tests in test_docx.py pass.
• 14 single mutations each fail the test written for them:
  • content controls not unwrapped;
  • each of the seven other wrappers left in place;
  • the table of contents read;
  • placeholder text kept;
  • headers and footers skipped;
  • a control's content dropped;
  • the pass not run.
• ruff check and ruff format --check from ruff 0.15.10, the locked version, are clean.
• Also run: test_unstructured/chunking (405 passed, 31 skipped), test_unstructured/common/test_html_table.py (51 passed) and test_unstructured/documents/test_elements.py (64 passed).
• No example document in example-docs/ contains any of these elements; I checked all 204 word/*.xml parts. So no fixture or ingest output changes.

Not run here:

• test_doc.py needs LibreOffice; without it, it fails the same 20 tests on main.
• test_auto.py needs pdf2image.
• scripts/version-sync.sh needs GNU sed. CHANGELOG.md and __version__.py both read 0.27.9-dev0.

#2944 describes a document property that vanishes from the output. If that property was inserted through Quick Parts > Document Property, it is a content control and this PR fixes it. I could not confirm that without the file.