Skip to content

[skills] Record downloaded-skill attribution in ~/.ucode/skills.json - #584

Open
xsh310 wants to merge 1 commit into
skills-picker-skills-fqnfrom
skills-download-attribution
Open

xsh310 wants to merge 1 commit into
skills-picker-skills-fqnfrom
skills-download-attribution

Conversation

@xsh310

@xsh310 xsh310 commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

What

Track which on-disk skill directories ucode downloaded from Unity Catalog, in a
single manifest at ~/.ucode/skills.json, so ucode can tell a downloaded skill
from a user-authored one and (in a later PR) remove downloads by their UC schema.

Skill attribution design doc:
https://doc-center-6051921418418893.staging.aws.databricksapps.com/d/hbzhsbid

Why

The download path kept no record of what it wrote or where it came from, so there
is no safe way to remove a downloaded skill without risking a same-named skill the
user authored themselves. This is the foundation (PR 1 of the attribution work): it
populates the manifest but adds no user-facing command yet.

Changes

  • New skills_state.py owns the manifest and its read/write/query/reconcile logic.
    Records are keyed by (metastore_id, fqn, base); query helpers are
    list_downloaded, attribution_for_dir, records_for_schema, and forget.
  • record_downloads reconciles superseded records on write: it drops the stale
    record for a directory just overwritten (keeping the freshly written files) and
    deletes the orphaned old directory when the same skill's bundle name changed.
  • SkillRef now also carries metastore_id, skill_id, and uc_update_time,
    read for free from the ListSkills/GetSkill response ucode already fetches.
  • The three download flows (whole-schema, named FQNs, picker) and the managed-launch
    download record each skill after it reaches disk.

Kept separate from state.json so a state-version change and ug revert leave it
untouched.

Testing

uv run ruff check, ruff format --check, ty check src, and uv run pytest all
pass (the pre-existing test_e2e_user_agent / test_claude_smart_routing_v2
failures also fail on the base). New tests/test_skills_state.py covers the manifest
round-trip, corrupt/version handling, the queries, and all three reconciliation
cases; tests/test_skills_download.py gains coverage that a download records
attribution and that SkillRef captures the UC metadata.

Stack

Part of gh stack #576, on top of #583 (skills-picker-skills-fqn).

This pull request and its description were written by Isaac.

@xsh310
xsh310 added this pull request to stack #576 September 12, 2026 18:01
Comment thread src/ucode/skills_state.py Outdated
Comment thread src/ucode/skills_state.py Outdated
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 8a9dc5e to edc3e41 Compare September 12, 2026 18:21
@xsh310
xsh310 marked this pull request as ready for review September 12, 2026 19:25
@xsh310
xsh310 force-pushed the skills-download-attribution branch from edc3e41 to 909f2e0 Compare September 13, 2026 00:22
@xsh310
xsh310 marked this pull request as draft September 13, 2026 00:23
@xsh310
xsh310 force-pushed the skills-download-attribution branch 2 times, most recently from 537a36c to 4753d1c Compare September 13, 2026 01:04
@xsh310
xsh310 marked this pull request as ready for review September 13, 2026 01:10
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 4753d1c to 5321bb1 Compare September 14, 2026 03:08
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 5321bb1 to 5c975fd Compare September 14, 2026 20:09
Comment thread src/ucode/skills_state.py
return kept


def record_downloads(installs: list[SkillInstall]) -> None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: the manifest is a read-modify-write (_load → mutate → _save) with no locking. Two concurrent ucode invocations (e.g. a download in one shell, a remove in another) can interleave so the second clobbers the first's update — a just-recorded skill silently vanishes from attribution. atomic_write_json keeps the file well-formed but doesn't prevent the lost update. Rare in practice; worth a file lock or a re-read-before-write if concurrent runs are plausible.

@xsh310
xsh310 force-pushed the skills-download-attribution branch from 5c975fd to 91e6254 Compare September 14, 2026 22:32
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 91e6254 to 41caa3f Compare September 14, 2026 23:20
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 41caa3f to e16350f Compare September 15, 2026 00:53
@xsh310
xsh310 force-pushed the skills-download-attribution branch from e16350f to 4eff582 Compare September 15, 2026 01:17
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 4eff582 to 10176dd Compare September 15, 2026 01:41
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 10176dd to 196a594 Compare September 15, 2026 03:30
@xsh310
xsh310 force-pushed the skills-download-attribution branch from 196a594 to ad4c659 Compare September 15, 2026 03:40
@xsh310
xsh310 force-pushed the skills-download-attribution branch from ad4c659 to e89c0ed Compare September 15, 2026 04:42
Track which on-disk skill directories came from Unity Catalog, so ucode can
tell a downloaded skill from a user-authored one and, later, remove downloads
by their UC schema. A new skills_state module owns the manifest and its
read/write/query/reconcile logic; the download flows record each skill after
it reaches disk. SkillRef now also carries the metastore id, skill id, and UC
update time, all read from the ListSkills/GetSkill response it already fetches.

The manifest is written atomically (temp file + os.replace, via a new
config_io.atomic_write_json), so a crash or concurrent writer can't truncate it.
An unparseable manifest is quarantined (renamed aside) and warned rather than
read as empty, so one bad byte can't let the next write silently erase every
tracked skill. The file carries a schema version; cross-version handling beyond
ignoring an unrecognized one is deferred until the schema first changes.

Kept separate from state.json so a state-version change and ug revert leave it
untouched. No user-facing command yet; ug skill remove wiring follows.

Co-authored-by: Isaac <no-reply@databricks.com>
@xsh310
xsh310 force-pushed the skills-download-attribution branch from e89c0ed to d8c9073 Compare September 15, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants