fix(cli): migrate config from bare file to XDG-compliant path - #269
fix(cli): migrate config from bare file to XDG-compliant path#269Padraic Shafer (padraic-shafer) wants to merge 14 commits into
Conversation
The CLI config was written to ~/.config/nsls2 as a plain file, colliding with other nsls2 tools that need ~/.config/nsls2/ as a directory. New location: $XDG_CONFIG_HOME/nsls2/api/cli.toml (falls back to ~/.config on POSIX, %APPDATA% on Windows). If the legacy bare file exists and is writable on first use, it is moved automatically to the new path so existing settings (base_url, token) are preserved. Non-writable legacy files emit a warning and are left in place; the tool continues normally. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟡 Changes recommended
The migration uses Path.replace() (can fail across filesystems) and the new filename suggests TOML while the code still reads/writes INI via configparser, both of which are likely to cause user-impacting issues.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates the CLI’s config storage to avoid the historical collision where ~/.config/nsls2 was used as a plain file, preventing other nsls2 tools from using ~/.config/nsls2/ as a directory. It introduces an XDG-compliant per-app config file path and attempts a one-time migration of the legacy config file on first read.
Changes:
- Move config location to an XDG-compliant path (
$XDG_CONFIG_HOMEon POSIX,%APPDATA%on Windows). - Add legacy-config migration logic that relocates the old bare-file config into the new directory structure.
- Trigger migration automatically on config reads, with warnings (stderr) on migration failures.
File summaries
| File | Description |
|---|---|
src/nsls2api/cli/settings.py |
Implements new config path resolution and legacy config migration on first use. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Stage the legacy file to a temp sibling (via tempfile.mkstemp) to free the 'nsls2' name before creating the directory tree, then move it into place. On failure, roll back to the original location; if rollback also fails, the warning points at the temp path for manual recovery. All manual-migration hints now use the correct two-step (mv .bak && mkdir -p && mv) sequence instead of the previously impossible self-referential mv. Add tests/cli/test_settings.py covering path resolution, set/get round-trip, and legacy migration (self-collision regression, no-op cases, non-writable warn-and-proceed, and read()-triggered auto-migration). Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
Config uses configparser, which reads/writes INI format, not TOML. Rename cli.toml -> cli.ini so the extension reflects the actual on-disk format. No on-disk cli.toml files exist yet, so no back-compat migration is required. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟡 Changes recommended
There are confirmed backward-compatibility and error-path issues in the migration/read logic that can lose existing settings and emit incorrect recovery guidance.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
src/nsls2api/cli/settings.py:39
- PR description/title say the new config location is
$XDG_CONFIG_HOME/nsls2/api/cli.toml, but the implementation (and tests) usecli.iniviaconfigparser. Please reconcile this (either update the PR description/docs, or switch the code to TOML and adjust migration/tests accordingly) to avoid confusing users and future maintainers.
"""Get the configuration file path ($XDG_CONFIG_HOME/nsls2/api/cli.ini).
Respects XDG_CONFIG_HOME if set; falls back to ~/.config on POSIX and
%APPDATA% on Windows.
"""
if sys.platform == "win32":
appdata = os.environ.get("APPDATA", "")
base = Path(appdata) if appdata else Path.home() / "AppData" / "Roaming"
else:
xdg = os.environ.get("XDG_CONFIG_HOME", "").strip()
base = Path(xdg) if xdg else Path.home() / ".config"
return base / "nsls2" / "api" / "cli.ini"
- Files reviewed: 2/3 changed files
- Comments generated: 4
- Review effort level: Lite
…llback R3: restructure the migration except-block to correctly distinguish three failure cases: (1) staging failed (legacy intact) — clean up the empty temp file, report settings remain at legacy; (2) post-staging failure with rollback — report settings remain at legacy; (3) rollback also failed — report settings at temp path for manual recovery. The previous code fell into case (3) even when legacy was intact, leaking a temp file and giving wrong recovery guidance. R4: read() now falls back to reading the legacy bare file when the new config is absent (e.g. migration was skipped because legacy was not writable), so existing base_url/token are never silently dropped. R1/EXDEV: the final tmp→new move now uses shutil.move() (copy+unlink on cross-device moves) so migration works when $XDG_CONFIG_HOME is on a different filesystem than ~/.config. The staging step (same-directory legacy→tmp) remains a Path.replace() as it cannot cross filesystems. R5: add TestGetFilepathWindowsSimulated covering the win32 branch of get_filepath() with and without APPDATA, runnable on all platforms via sys.platform patching. R6: correct _patch_home docstring — it only patches Path.home(); env vars are controlled per-test. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
|
Follow-up addressing the Copilot review (commit 68436a4):
|
There was a problem hiding this comment.
🟡 Changes recommended
The current migration failure handling can delete the only copy of the user’s config on certain errors, and a couple of new tests contain assertions that will fail as written.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (1)
src/nsls2api/tests/cli/test_settings.py:193
- After a successful migration triggered by
Config.read(), the legacy path should no longer be a file (it becomes a directory for the new layout).assert not legacy.exists()will fail because the directory exists.
assert not legacy.exists()
new = tmp_path / ".config" / "nsls2" / "api" / "cli.ini"
assert new.exists()
- Files reviewed: 2/3 changed files
- Comments generated: 3
- Review effort level: Lite
Use a `staged` bool flag (set after legacy.replace(tmp) succeeds) instead of `legacy.exists()` to discriminate staging-failed vs later-failed in migrate_legacy_config(). The old check was fooled when new.parent.mkdir() created ~/.config/nsls2/ as a directory, causing tmp.unlink() to delete the only copy of user settings. Also fix two contradictory test assertions (assert not legacy.exists() vs assert legacy.is_dir()), remove unused MagicMock import (F401), and add a regression test for the data-loss scenario. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟡 Changes recommended
The migration rollback logic can’t reliably restore the legacy file after directory creation on POSIX, and several warnings include POSIX-only manual commands that are misleading on Windows.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (5)
Previously missed (2) — in code that hasn't changed since the last review.
src/nsls2api/tests/cli/test_settings.py:339
- This test’s docstring/comments claim there is “no migration trigger”, but
Config.read()always callsmigrate_legacy_config(), so the test usually exercises the migration path rather than the legacy-read fallback. Patchmigrate_legacy_configto a no-op so the test reliably covers the intended fallback behavior.
src/nsls2api/cli/settings.py:84 - The manual migration hint uses POSIX shell commands (
mv,mkdir -p), which will be confusing/incorrect for Windows users who are most likely to hit the legacy-file migration path (older versions wrote~/.config/nsls2even on Windows). Consider using a platform-neutral step list (or a Windows-specific hint) in the warning text.
This issue also appears in the following locations of the same file:
- line 107
- line 125
- line 136
src/nsls2api/cli/settings.py:109
- This warning’s manual migration hint uses POSIX shell commands (
mv,mkdir -p), which are not appropriate on Windows. A platform-neutral step list avoids shell-specific instructions.
f"To migrate manually:\n"
f" mv '{legacy}' '{legacy}.bak' && "
f"mkdir -p '{new.parent}' && mv '{legacy}.bak' '{new}'",
src/nsls2api/cli/settings.py:127
- This warning repeats a POSIX-only manual migration command sequence (
mv,mkdir -p). Prefer a platform-neutral step list (or a Windows-specific hint) so the message is accurate across supported platforms.
f"To migrate manually:\n"
f" mv '{legacy}' '{legacy}.bak' && "
f"mkdir -p '{new.parent}' && mv '{legacy}.bak' '{new}'",
src/nsls2api/cli/settings.py:137
- The recovery hint uses POSIX shell commands (
mkdir -p,mv), which are not valid in the default Windows shells. Consider a platform-neutral step list to avoid misleading instructions.
f"To recover:\n"
f" mkdir -p '{new.parent}' && mv '{tmp}' '{new}'",
- Files reviewed: 2/3 changed files
- Comments generated: 1
- Review effort level: Lite
…hints On POSIX, shutil.move failure after new.parent.mkdir() left ~/.config/nsls2 as a directory, causing tmp.replace(legacy) to raise ENOTEMPTY/EISDIR — rollback never succeeded. Fix: rmtree(new.parent) + legacy.rmdir() before tmp.replace(legacy) clears the empty dirs we created, restoring the legacy file reliably. Also replace POSIX-only mv/mkdir -p hints in all four warning messages with platform-neutral numbered step lists (addresses Windows users). Fix tests: assert legacy.is_file() after rollback, patch migrate_legacy_config to no-op in R4 fallback test, update hint-text assertions. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
|
Addressed in 7100155 (suppressed Copilot comments from the previous review round):
|
There was a problem hiding this comment.
🟡 Changes recommended
The rollback logic can recursively delete an existing $XDG_CONFIG_HOME/nsls2/api directory on migration failure, which is a real data-loss risk.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 2/3 changed files
- Comments generated: 2
- Review effort level: Lite
Replace the staged-rename + rmtree-rollback migration with a safer copy-first approach: - shutil.copy2(legacy, tmp) preserves content before any destructive op - legacy.unlink() frees the name only when new is under legacy (no-XDG collision), since tmp already holds a full copy - On shutil.move failure the legacy name is free; restore via move(tmp, legacy) - shutil.rmtree removed entirely — destination tree is never touched - XDG case: legacy left untouched until after new file is confirmed written Fix permission pre-check: os.access(legacy.parent, W_OK|X_OK) matches what rename/unlink actually require (parent dir write+execute, not file write). A read-only file in a writable parent now migrates successfully. Update tests: copy2 staging, parent-dir chmod, new tests for read-only file migration and XDG destination-tree preservation on failure. Vestigial staged-flag / rmtree / rollback comments removed. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
The new_under_legacy check was lexical (legacy in new.parents), so XDG_CONFIG_HOME values that resolve to ~/.config via a different spelling (trailing slash, '..' segment, or a symlink) were incorrectly treated as the no-collision XDG branch, causing migration to fail silently when mkdir encountered the legacy file blocking the 'nsls2' directory name. Fix: compare resolved paths so any XDG spelling that resolves to ~/.config takes the collision branch (unlink legacy before mkdir, restore from tmp copy on failure) — identical mechanism and effect to the no-XDG default. Add parametrized regression tests covering exact, trailing-slash, and '..' forms of XDG=~/.config, plus a move-failure/restore test. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟡 Changes recommended
The migration failure-recovery branch can incorrectly delete the temp copy (and fail to restore legacy) after mkdir() recreates the legacy path as a directory, risking config loss and likely breaking the added rollback tests.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 2/3 changed files
- Comments generated: 2
- Review effort level: Lite
The recovery branch in migrate_legacy_config() reused the fragile `not legacy.exists()` check to decide whether to restore the legacy file. When shutil.move failed _after_ new.parent.mkdir() had run, mkdir had already created a directory at the legacy path (collision case), so `legacy.exists()` returned True and the restore was skipped — discarding the tmp copy and misreporting the config location. Same class of bug as round 3, resurfaced in the error path. Fix: before mkdir, record exactly which ancestor directories don't yet exist (innermost-first). On OSError, rmdir only those empty directories (never recursive), then restore based on `legacy_removed` rather than `legacy.exists()`. Also fix two test regressions on commit 3890662: - test_no_op_when_new_already_exists: removed the contradictory _make_legacy() call; the new.exists() short-circuit fires before any legacy handling so the test only needs `new` in place. - test_migrates_when_xdg_equals_config_dir[/../.config]: compare new_path.resolve() to the expected resolved path — XDG values with '..' segments produce an unresolved return value. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟢 Approval recommended
The migration logic is covered by thorough tests (including failure/rollback cases), and the implementation matches the stated behavior without introducing evident regressions.
Review details
- Files reviewed: 2/3 changed files
- Comments generated: 0 new
- Review effort level: Lite
Both test_shutil_move_failure_restores_legacy and
test_xdg_equals_config_dir_move_failure_restores_legacy patched
shutil.move with a blanket side_effect=OSError("EXDEV"), which caused
every shutil.move call — including the recovery restore — to raise.
This made it impossible for the restore branch to succeed, causing
legacy.is_file() to be False even after the product-side fix in
e26f965.
Replace the blanket mock with a closure (_move_fails_once) that raises
only on the first call and delegates to the real shutil.move on
subsequent calls, accurately simulating a cross-device rename failure
on the migration move while allowing the restore move to complete.
Assisted-by: claude-opus-4-8
Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🟢 Approval recommended
The migration logic is well-covered by tests and appears safe, with only a minor warning-message clarity adjustment requested.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/nsls2api/cli/settings.py:85
- In the "legacy parent not writable" warning, step 1 currently tells the user to move/rename the legacy file even though the message just stated that the containing directory is not writable (so rename/move in-place will fail). Consider updating step 1 to first instruct making the directory writable (or running with appropriate permissions) before attempting the rename.
- Files reviewed: 2/3 changed files
- Comments generated: 0 new
- Review effort level: Lite
settings.py shrinks from 303 → 127 lines. All migration and legacy
fallback logic moves to a new isolated module so that future removal
of legacy support is a single-file deletion:
New: src/nsls2api/cli/settings_migration.py
- legacy_filepath() — canonical legacy path
- migrate_legacy_config(config_filepath) — one-time migration
- read_legacy_fallback(config, config_filepath) — back-compat read
Changed: src/nsls2api/cli/settings.py
- Remove _legacy_filepath() and migrate_legacy_config() from Config
- Remove shutil/tempfile imports (no longer needed here)
- Import settings_migration; rewrite read() to call the three
helpers explicitly with config_filepath = cls.get_filepath()
New: src/nsls2api/tests/cli/test_settings_migration.py
- TestMigrateLegacyConfig and TestReadLegacyFallback moved here
- Patch targets retargeted to nsls2api.cli.settings_migration.*
- Local copies of _make_legacy/_patch_home (intentionally
self-contained for symmetrical deletion later)
Changed: src/nsls2api/tests/cli/test_settings.py
- Remove the two moved test classes
- Remove unused _make_legacy helper
- Update module docstring
No behaviour change; all existing tests preserved.
Assisted-by: claude-opus-4-8
Assisted-by: claude-sonnet-4-6
Refactor of PR changesAfter responding to multiple rounds of copilot review, the migration logic for the config filepath grew quite heavy...nearly 70% of settings.py was devoted to migration, even though migration has a finite lifespan. Extracting this new code into settings_migration.py makes the deprecation path a single-file deletion later, and keeps settings.py focused. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🔵 Needs a closer look
Config.get_filepath() should treat whitespace-only APPDATA as unset on Windows to avoid producing an invalid base path.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
src/nsls2api/cli/settings.py:36
- On Windows,
APPDATAis treated as set even if it contains only whitespace, which would produce a config path under a literal' 'directory. Since the POSIX branch already treats a blankXDG_CONFIG_HOMEas unset, it’d be more robust/consistent tostrip()APPDATAas well before deciding the base path.
- Files reviewed: 4/5 changed files
- Comments generated: 0 new
- Review effort level: Lite
get_filepath() stripped XDG_CONFIG_HOME on POSIX but not APPDATA on Windows, so a whitespace-only APPDATA produced a config path under a literal ' ' directory. Strip APPDATA and fall back to ~/AppData/Roaming when blank, matching the POSIX branch behaviour. Adds test_windows_blank_appdata_falls_back to TestGetFilepathWindowsSimulated, symmetric to the existing test_blank_xdg_config_home_falls_back. Assisted-by: claude-opus-4-8 Assisted-by: claude-sonnet-4-6
There was a problem hiding this comment.
🔵 Needs a closer look
There is a confirmed correctness issue in Config.read() using exists() instead of is_file(), and additional path-hardening fixes are needed to avoid writing config to unintended relative locations.
Review details
Suppressed comments (3)
Previously missed (2) — in code that hasn't changed since the last review.
src/nsls2api/cli/settings.py:36
APPDATAis used directly as aPatheven if it is a relative path. A misconfigured relativeAPPDATAwould cause the CLI to write config relative to the current working directory rather than a user config directory.
This issue also appears on line 38 of the same file.
src/nsls2api/cli/settings.py:56
Config.read()checksconfig_filepath.exists()rather thanis_file(). If the path exists but is not a regular file (e.g., a directory namedcli.ini),configparser.ConfigParser.read()will silently skip it and the legacy fallback will not run, returning an empty config even when the legacy file still has settings.
src/nsls2api/cli/settings.py:39
XDG_CONFIG_HOMEis used directly as aPatheven if it is a relative path. A relative value would cause config to be written relative to the current working directory, which is surprising and can break tooling that expects absolute XDG paths.
xdg = os.environ.get("XDG_CONFIG_HOME", "").strip()
base = Path(xdg) if xdg else Path.home() / ".config"
- Files reviewed: 4/5 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
I'm declining to handle copilot's muted suggestion of handling the pathological case where someone has set XDG_CONFIG_HOME or APPDATA variables to a relative path. Standard accepted practice is that their values must be absolute paths. |
There was a problem hiding this comment.
🔵 Needs a closer look
There are unhandled OSError cases in the migration flow (tempfile creation and post-success legacy cleanup) that can still break normal CLI commands despite the intended “warn and continue” contract.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
src/nsls2api/cli/settings_migration.py:86
tempfile.mkstemp(...)can raiseOSError(disk full, permissions, etc.) before entering thetry:that handles migration failures. That exception would currently propagate and can break normal CLI commands, contradicting the function contract to warn and returnNoneon migration failure. Wrap the mkstemp/close step in atry/except OSErrorand emit the same warning+manual migration instructions used for other failure modes.
src/nsls2api/cli/settings_migration.py:204- If legacy deletion fails after a successful migration in the non-collision (XDG elsewhere) case,
legacy.unlink(missing_ok=True)will raise and can break the CLI even though the new config was written correctly. This cleanup step should be best-effort: catchOSError, warn to stderr, and continue returningnew.
- Files reviewed: 4/5 changed files
- Comments generated: 0 new
- Review effort level: Lite
The CLI config was historically written to ~/.config/nsls2 as a plain file, colliding with other nsls2 tools that need ~/.config/nsls2/ as a directory.
New location: $XDG_CONFIG_HOME/nsls2/api/cli.ini (falls back to ~/.config on POSIX, %APPDATA% on Windows).
If the legacy bare file exists and is writable on first use, it is moved automatically to the new path so existing settings (base_url, token) are preserved. Non-writable legacy files emit a warning and are left in place; the tool continues normally.
Testing