`OPT_OUT_INSTRUMENTATION` is a no-op when agent subshells don't inherit the env var (Hermes, Codex exec mode, MCP servers, etc.) — request a user-level config fallback

[christian-pohl]

Summary

The toolkit currently respects only OPT_OUT_INSTRUMENTATION=true for disabling telemetry. That works for shells that inherit the user's exported env, but several supported agent runtimes spawn skill scripts in fresh subshells that don't (e.g. Hermes' terminal tool, Codex's non-interactive exec mode, MCP-server child processes). For those hosts, the documented opt-out is effectively unreachable, and telemetry continues to POST to https://shopify.dev/mcp/usage.

Repro (Hermes; same shape for any non-env-inheriting host)

1. Install the plugin via .hermes-plugin/install.sh (clone into ~/.hermes/repos/shopify-ai-toolkit/, symlink into ~/.hermes/plugins/).
2. Set OPT_OUT_INSTRUMENTATION=true in ~/.bashrc (interactive shells pick it up).
3. Invoke a skill whose bash hook runs in a non-interactive non-login subshell.
4. Observe: hooks/scripts/track-telemetry.sh reads ${OPT_OUT_INSTRUMENTATION:-} → empty in the subshell → if [ "$OPT_OUT" = "true" ] is false → POST goes out to shopify.dev with skill name, version, client, session id, tool use id (and on Claude Code, the verbatim user prompt from the per-session stash).

Expected

A user who has clearly opted out (OPT_OUT_INSTRUMENTATION=true anywhere they can reasonably set it) should never see outbound telemetry events, regardless of how the agent forks its tool subshells.

Suggested fix

Add a user-level config-file fallback to every spot that reads the env var. The file is read only if the env var is unset — env still wins, so existing CI/tests don't change. The file location follows XDG:

• macOS/Linux: ~/.config/shopify-ai-toolkit/opt-out
• Windows: %APPDATA%\shopify-ai-toolkit\opt-out

File content: a single line. true (with optional surrounding whitespace) → opt out. Anything else → opt in. File missing or unreadable → fall through to env-var behavior, so this is strictly additive.

Bash (apply after OPT_OUT="${OPT_OUT_INSTRUMENTATION:-}" in hooks/scripts/track-telemetry.sh and every skills/*/scripts/track-telemetry.sh):

# Opt-out fallback: env var first, then user config file (XDG).
# Env still wins — CI/test harnesses that set the var are unaffected.
if [ -z "$OPT_OUT" ]; then
  for _opt_out_file in \
    "${XDG_CONFIG_HOME:-$HOME/.config}/shopify-ai-toolkit/opt-out" \
    "${HOME}/Library/Application Support/shopify-ai-toolkit/opt-out"; do
    if [ -f "$_opt_out_file" ]; then
      _opt=$(tr -d '[:space:]' < "$_opt_out_file" 2>/dev/null || true)
      [ "$_opt" = "true" ] && OPT_OUT="true"
      break
    fi
  done
  unset _opt _opt_out_file
fi

Node (replace the body of isInstrumentationDisabled in scripts/log_skill_use.mjs, scripts/search_docs.mjs, scripts/validate.mjs):

function isInstrumentationDisabled() {
  try {
    if (process.env.OPT_OUT_INSTRUMENTATION === "true") return true;
    // Opt-out fallback: env var first, then user config file (XDG / OS-conventional).
    // Env still wins — CI/test harnesses that set the var are unaffected.
    const fs = require("fs"), path = require("path"), os = require("os");
    const candidates = [
      process.env.XDG_CONFIG_HOME && path.join(process.env.XDG_CONFIG_HOME, "shopify-ai-toolkit", "opt-out"),
      path.join(os.homedir(), ".config", "shopify-ai-toolkit", "opt-out"),
      process.platform === "darwin" && path.join(os.homedir(), "Library", "Application Support", "shopify-ai-toolkit", "opt-out"),
      process.platform === "win32" && path.join(process.env.APPDATA || path.join(os.homedir(), "AppData", "Roaming"), "shopify-ai-toolkit", "opt-out"),
    ].filter(Boolean);
    for (const p of candidates) {
      if (fs.existsSync(p)) {
        const s = fs.readFileSync(p, "utf8").trim();
        if (s === "true") return true;
        break; // explicit "false" / anything else: respect it, don't fall through
      }
    }
  } catch {}
  return false;
}

PowerShell (hooks/scripts/track-telemetry.ps1 and per-skill copies) — analogous; happy to follow up with a separate PR if preferred.

Why this is additive, not breaking

• The env var still wins. Any CI/test that sets OPT_OUT_INSTRUMENTATION=true keeps working unchanged.
• The file is only consulted when the env var is unset. Servers and CI typically set the var, so they never see a behavior change.
• For users who never create the file, behavior is identical to today.
• For users who create the file, opt-out now works in subshell contexts where it previously didn't.

Related

• Surface telemetry disclosure in README and plugin.json #23 (open, by @tobinsouth) — improving the disclosure of telemetry in README + plugin.json. This issue is orthogonal: even with perfect disclosure, the opt-out itself is currently unreachable on several supported hosts. Happy to coordinate / cross-link once one of these lands.
• Anthropic's plugin-policy scan already flags this plugin: shopify-ai-toolkit FAILS policy — has_undisclosed_telemetry=true (see anthropics/claude-plugins-official Actions runs). A user-level opt-out that actually works in subshells would let downstream security reviewers tick that box without forcing users to fork the plugin.

Note on submission

I'm filing this as an issue rather than a PR per CONTRIBUTING.md ("we don't accept pull requests"). I'm happy to:

• iterate on the suggested fix in this thread;
• test any patch you cut against Hermes / Codex / Claude Code / Cursor / VS Code Copilot;
• maintain a downstream patch in the meantime.

Environment

• Plugin version: 1.4.1 (current main, commit a8e87a7cff153479eb77230d9c232484a1f3062f)
• Tested on: Hermes WebUI on Debian 13
• Affected scripts (in the current tree): 21 track-telemetry.sh files + 34 isInstrumentationDisabled definitions in skills/*/scripts/*.mjs