Repository navigation
@W-24630680: Wire and package embedded Excel/CSV datasources for data apps - #21
Open
patrickg2014 wants to merge 21 commits into
Open
patrickg2014 wants to merge 21 commits into
patrickg2014 wants to merge 21 commits into
Conversation
package_twbx.py wrote only the .twb and Packages/ into the .twbx, so a data app whose .twb references an embedded datasource file (for example Data/Datasources/Sample - Superstore.xls) packaged and published cleanly but had no data at runtime. Also pack Data/ and Resources/ at the archive root when the workspace has them. The .twb is still written first, OS cruft is still skipped, the Packages/ and .trex checks are unchanged, and nothing else at the workspace root is packed. SKILL.md's Package section says so.
jarhun88
reviewed
Oct 22, 2026
jarhun88
reviewed
Oct 22, 2026
Address review feedback on package_twbx.py: - Pack only Resources/.agents/ (the agent-docs folder that travels with the workbook) instead of all of Resources/. - Skip any file or dir whose name starts with "." below the packed dirs, and list each one on stderr with the reason. SKILL.md says to relay those lines to the user. - Fail when the .twb references a Data/ file (connection filename or dbname) that is missing from the workspace. - Follow symlinked dirs, with a guard against symlink loops.
patrickg2014
changed the title
patrickg2014
changed the title
CSV, JSON, and shapefile connections split the path across two attributes (directory='Data' filename='x.csv'), so the missing-file check only saw 'x.csv' and let a workbook with no embedded data package cleanly. Join directory and filename before checking. Excel and Hyper keep the whole path in one attribute and are unchanged.
…ckaging The scaffold-data-app template now ships Data/.gitkeep and Resources/.agents/README.md, so the plugin no longer owns that layout. - package_twbx.py: pack a dir only when it holds a packed file, so the template's Data/ (only .gitkeep) stays out of the .twbx; skip .gitkeep silently. - wire_datasource.py: add "type": "file" descriptors for Excel (excel-direct) and CSV (textscan) files under Data/, wired as federated datasources alongside published ones in a single run. - SKILL.md: describe the template's Data/ and Resources/.agents/, the embedded file wiring steps, and the packaging rules.
- package_twbx.py: cut only symlink cycles, so an alias no longer hides the real path; check .twb references against the packed names (catches case mismatches macOS hides); fail cleanly on dangling or unreadable files; write to a temp file so a failure never truncates an existing .twbx; never pack the output into itself. - wire_datasource.py: double ] in field names; reject connectionName hashes that aren't letters and digits. - References: cover embedded Excel/CSV files (snapshot, local introspection).
A followed link could pull any file on the machine (an SSH key, a PAT file) into a .twbx that gets published. package_twbx.py now skips and reports every symlink below the packed dirs (and a linked Data/ or Resources/.agents/), and fails if the .twb or Packages/ is one; a .twb reference to a skipped link fails as missing. wire_datasource.py rejects a symlinked data file up front.
Packaging now packs only the Data/ files the .twb references and reports any other file there as skipped. A symlink pointing at the output is reported like any other link. apply_plan.py refuses a template containing symlinks, since safe_join only checks path text. Removes em dashes added on this branch.
Merged
7 of 13 tasks
patrickg2014
changed the title
…d wiring's symlink check
… doubling, and overlapping tests
…ct collect_packed_dirs Each embedded format is a class listing its extensions, so adding one means one class. The skill docs no longer name formats, and package_twbx.py's collect/verify step is its own function.
jarhun88
approved these changes
Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A data app can embed an Excel or CSV file in its
.twbxinstead of querying a published datasource, but the skill couldn't build one:wire_datasource.pyonly wired published datasources, andpackage_twbx.pydroppedData/andResources/.agents/.Change
wire_datasource.py: new"type": "file"descriptor wires an embedded.xls/.xlsx/.xlsm(one sheet) or.csvunderData/. Can be mixed with published datasources.package_twbx.py: packsData/andResources/.agents/at the archive root. UnderData/, only files the.twbreferences; everything else, symlinks, and hidden files are skipped and listed on stderr. Fails if a referenced file is missing.apply_plan.py: rejects a template containing symlinks.Pairs with tableau/tableau-mcp#1118 (adds the two folders to the template).
Testing
query-datasourceand in the browser.