feat(tree-sitter-bash): add a pure-TypeScript bash parser and an agent-core-v2 bashParser service#2016
feat(tree-sitter-bash): add a pure-TypeScript bash parser and an agent-core-v2 bashParser service#2016sailist wants to merge 6 commits into
Conversation
Add packages/tree-sitter-bash with the SyntaxNode tree model (UTF-16 code-unit offsets, tree-sitter-bash named-node type names), a parse budget (deadline + node cap) with an aborted ParseResult variant, and a placeholder parse() to be replaced by the real lexer/parser.
Cover the permission-analysis grammar subset: lists, pipelines, commands, words, quotes, expansions, command/process substitution, subshells, the full redirect operator set, heredocs and comments, with tree-sitter-bash named-node type names. Long scan loops check the parse deadline via budget.progress() so large literals cannot exhaust the node cap; parse depth is bounded on all recursion chains.
Add compound commands (if/while/until/for/c-style-for/select/case/ function/compound/do_group), test commands with reference-exact extglob/regex right-hand-side rules, a Pratt expression engine for arithmetic expansion shared across arith/c-for/test modes, arrays and subscripts, declaration/unset commands, ansi-c/translated strings and brace expressions. Reserved words are recognized only in statement position. Case-aware balanced scanning keeps command substitutions intact around case items, expression leftovers are kept as ERROR nodes instead of dropped, and recursion depth is bounded per chain (substitution 150, parse 500, lexer scan 1024).
…erf suites Turn the ad-hoc wasm comparison work into permanent infrastructure: a differential helper pinning reference-equivalent and known-difference samples against the real tree-sitter-bash wasm (478 match / 79 known-diff fixtures plus the official v0.25.0 corpus), a three-way consistency check between fixtures, the known-difference registry and the README, deterministic seeded fuzz with tree-integrity assertions, and performance smoke tests. Converges 15 further divergence groups found by systematic probing and documents the rest; the full suite is 853 tests in ~4s.
Wrap the pure @moonshot-ai/tree-sitter-bash package as
IBashParserService (L1, no dependencies): parse(source, options)
returns a wire-safe BashParseResult whose nodes drop the cyclic
parent link so trees can cross the RPC boundary. Budget exhaustion
surfaces as { ok: false, reason: 'aborted' }, malformed input as
hasError — never a throw.
🦋 Changeset detectedLatest commit: a9ded49 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a9ded4966c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| startIndex: node.startIndex, | ||
| endIndex: node.endIndex, | ||
| isNamed: node.isNamed, | ||
| children: node.children.map(snapshot), |
There was a problem hiding this comment.
When callers parse a long but budget-acceptable arithmetic expression, e.g. echo $((1+1+...)) with a few thousand operands and a larger timeoutMs, the parser returns a deeply left-nested binary_expression tree under the default maxNodes; this recursive map(snapshot) then overflows the JS call stack and makes IBashParserService.parse throw RangeError instead of returning hasError/aborted. Since this service is meant to be the wire-safe RPC adapter, the DTO conversion needs the same iterative approach as materialize.
Useful? React with 👍 / 👎.
1 participant
Related Issue
None — the problem is explained below.
Problem
The Bash tool's permission matching currently treats the entire command string as a single glob subject, so a compound command like
git status && rm -rf /cannot be split and matched per command. The repo had no bash parser at all, and the off-the-shelf option (web-tree-sitter+ wasm) brings wasm asset loading and runtime-specific packaging concerns.What changed
New
packages/tree-sitter-bash— a pure-TypeScript bash parser (zero runtime deps, no wasm).node.text === source.slice(start, end).{ ok: false, reason: 'aborted' }and malformed input returnshasError: true— parsing never throws, so callers can always degrade safely.New
bashParserdomain inpackages/agent-core-v2.IBashParserService:parse(source, options)adapts the package into the DI container and returns a wire-safe DTO (cyclicparentlinks dropped) so results can cross the RPC boundary. No consumers yet — per-command permission matching is the intended follow-up.Repo wiring:
flake.nixworkspace lists, rootAGENTS.mdproject map, and a changeset (internal capability, no user-facing behavior change).This approach fits Kimi Code because it avoids wasm/native loading in the Node runtime, keeps a deterministic failure mode for hostile input, and keeps the parser free of safety policy so permission rules can evolve in the consuming domains.
Checklist
gen-changesetsskill, or this PR needs no changeset.gen-docsskill, or this PR needs no doc update.