Add timeout_ms to fd, walk, ls, and async path functions

DEV.md

@@ -40,12 +40,12 @@ The GitHub workflow builds wheels for Python 3.10-3.13 on Linux and macOS and pu
 
 Paths in `fd`, `walk`, `rg`, and `rg_iter` results are relative to the requested root and use `/` separators. Traversal uses `ignore::WalkParallel`, so result order is not part of the API contract. Search results are structured rows; collected result lists use rg-style `str()` and notebook display. `SearchLine.lnhash` is computed with the same CRC-32-based line-content hash format as exhash (`lineno|hash|`, low 16 bits of CRC-32 over the line's UTF-8 bytes); `lnhashs=True` only changes row display, not `line_number` or matching behavior. Path regexes filter returned/searched paths; `skip_dir` and `skip_dir_re` prune traversal through `ignore::WalkBuilder::filter_entry`. Depth, size, symlink, filesystem, hidden, and ignore options are direct `ignore::WalkBuilder` settings. `rg_iter` exposes the same parallel search stream that `rg` collects by default; `paths=True` and `count=True` consume that stream with different reducers. Binary files and invalid UTF-8 are skipped for now.
 
-Streaming engine: `walk.rs` owns the generic machinery. `StreamIter<T>` is the worker-thread-plus-bounded-channel iterator (`sync_channel(8192)`, so producers block rather than buffer without limit when a consumer lags), and `spawn_walk` owns the shared scaffold: walker config, panic catching, cancel flag, and worker thread. `rg_iter` (`T = SearchLine`), `block_iter` (`T = SearchBlock`), and `nb_iter` (`T = NbCell`) plug entry closures into that engine. Block search reads each file once, searches it once, groups nonblank lines into blocks, maps matching lines to their blocks, and expands context by block index.
+Streaming engine: `walk.rs` owns the generic machinery. `StreamIter<T>` is the worker-thread-plus-bounded-channel iterator (`sync_channel(8192)`, so producers block rather than buffer without limit when a consumer lags), and `spawn_walk` owns the shared scaffold: walker config, panic catching, cancel flag, and worker thread. `rg_iter` (`T = SearchLine`), `block_iter` (`T = SearchBlock`), `nb_iter` (`T = NbCell`), and `find_iter` (`T = String`, the path walk) plug entry closures into that engine. Block search reads each file once, searches it once, groups nonblank lines into blocks, maps matching lines to their blocks, and expands context by block index.
 Each `SearchBlock` carries numeric boundaries plus hashes for its first and last source lines. Python keeps both and chooses the displayed address without another file read.
 
-Async API: `fda`, `rga`, `rga_iter`, `nbrga`, and `nbrga_iter` wrap the corresponding private core operations. `rga(summary=True)` uses `_core.block_search_async`; ordinary `rga` uses `_core.rg_async`. Each collected core function takes a Python callback, runs on Rust threads through the generic `stream_async` helper, and delivers with one GIL attach at the end. Iterator forms use `stream_iter_async` and attach once per batch. The Python side settles an `asyncio.Future` or feeds an `asyncio.Queue` via `loop.call_soon_threadsafe`; no Python thread blocks and `asyncio.to_thread` is not involved. `AsyncHandle.cancel()` sets the same atomic flag used by the Rust iterators.
+Async API: `fda`, `fda_iter`, `rga`, `rga_iter`, `nbrga`, and `nbrga_iter` wrap the corresponding private core operations. `rga(summary=True)` uses `_core.block_search_async`; ordinary `rga` uses `_core.rg_async`. Each collected core function takes a Python callback, runs on Rust threads through the generic `stream_async` helper, and delivers with one GIL attach at the end. Iterator forms use `stream_iter_async` and attach once per batch. The Python side settles an `asyncio.Future` or feeds an `asyncio.Queue` via `loop.call_soon_threadsafe`; no Python thread blocks and `asyncio.to_thread` is not involved. `AsyncHandle.cancel()` sets the same atomic flag used by the Rust iterators.
 
-Truncation is recorded on collected results: `max_results` sets `stop_reason="max_results"`, and `timeout_ms` on `rg`/`rga`/`nbrg`/`nbrga` sets `stop_reason="timeout"`. `SearchResults`, `BlockResults`, `PathResults`, and `NbResults` share this through `_Results`; `complete` means `stop_reason is None`. In block summary mode, `max_results` counts matching blocks and keeps their block context. `count=True` returns a plain int, so it rejects timeouts and block summary mode.
+Truncation is recorded on collected results: `max_results` sets `stop_reason="max_results"`, and `timeout_ms` on `rg`/`rga`/`nbrg`/`nbrga`/`fd`/`fda`/`walk`/`ls` sets `stop_reason="timeout"`. `SearchResults`, `BlockResults`, `PathResults`, and `NbResults` share this through `_Results`; `complete` means `stop_reason is None`. In block summary mode, `max_results` counts matching blocks and keeps their block context. `count=True` returns a plain int, so it rejects timeouts and block summary mode.
 
 Path results are `FileEntry` rows: a `str` subclass carrying the walk root, so paths stay plain strings for compatibility while stat info loads lazily (one cached `os.lstat` per entry, read only on attribute access). The wrapping happens at result construction on the Python side; Rust still streams plain strings. `PathResults.__repr__` shows an `ls -l`-style listing capped at `MAX_REPR` rows, so a huge result never stats everything, while `str()` stays one plain path per line. `ls` is `fd` with shell-style defaults (one level, dirs, ignore rules off), re-sorted with `stop_reason` preserved.
 

README.md

@@ -63,6 +63,8 @@ pip install rgapi
 
 `ls` lists like the shell command: it is `fd` with defaults flipped to one level (`max_depth=1`), directories included, ignore rules off, and results sorted by name. `hidden=True` is `ls -a`, and every `fd` filter still applies.
 
+`fd_iter` is the lazy form of `fd`, yielding `FileEntry` paths as the walk finds them, and takes every `fd` filter. It has no `timeout_ms`, since a consumer that stops asking for paths ends the walk itself.
+
 `path_re` and `skip_path_re` are regex filters on slash-separated relative paths. They filter returned paths or searched files, but do not control traversal. `skip_dir` uses glob syntax to prune matching directory subtrees, and `skip_dir_re` does the same with regex.
 
 `rg` and `rg_iter` return structured rows rather than raw CLI text. They accept the same `include`, `exclude`, `glob`, `ext`, `path_re`, `skip_path_re`, `skip_dir`, `skip_dir_re`, `min_depth`, `max_depth`, `max_filesize`, `follow_links`, and `same_file_system` filters as `fd`. Each row is a `SearchLine` with:
@@ -80,7 +82,7 @@ matches      list of (start, end) byte offsets for match rows
 
 `SearchLine` has a structured `repr`, an rg-style `str` (the `line` is truncated to 120 chars with a trailing `…` for display; `repr` and `asdict()` keep the full line), and `SearchLine.asdict()` returns row fields as a plain Python dict. Pass `rg(..., lnhashs=True)` or `rg_iter(..., lnhashs=True)` to show `lnhash` addresses instead of line numbers in row display while keeping `line_number` available. `rg(..., paths=True)` returns unique matched paths, and `rg(..., count=True)` returns the total number of match spans. `paths` and `count` cannot both be set.
 
-`fd`, `walk`, `ls`, and `rg(..., paths=True)` return `PathResults`, a list of `FileEntry` rows. A `FileEntry` is a `str` subclass holding the relative path, so all string uses keep working, and it stats itself lazily on first access: `stat` is a cached `os.lstat` result (`None` if the path has vanished), with `size`, `mtime`, and `is_dir` derived from it. A `PathResults` displays as an `ls -l`-style long listing, capped at `rgapi.MAX_REPR` rows with a final `… N more` line, so stats are read only for displayed rows; `str()` is still one plain path per line, and `list(res)` shows plain paths. `rg(..., timeout_ms=200)` stops the search at the deadline and returns whatever was collected by then. Results record how they ended: `stop_reason` is `None` for a complete result, `"max_results"` when truncated by `max_results`, or `"timeout"` when a deadline hit, and `complete` is true when `stop_reason` is `None`. `count=True` returns a plain int, which cannot carry the flag, so it rejects `timeout_ms`.
+`fd`, `walk`, `ls`, and `rg(..., paths=True)` return `PathResults`, a list of `FileEntry` rows. A `FileEntry` is a `str` subclass holding the relative path, so all string uses keep working, and it stats itself lazily on first access: `stat` is a cached `os.lstat` result (`None` if the path has vanished), with `size`, `mtime`, and `is_dir` derived from it. A `PathResults` displays as an `ls -l`-style long listing, capped at `rgapi.MAX_REPR` rows with a final `… N more` line, so stats are read only for displayed rows; `str()` is still one plain path per line, and `list(res)` shows plain paths. `rg(..., timeout_ms=200)` and `fd(..., timeout_ms=200)` stop at the deadline and return whatever was collected by then; `walk`, `ls`, and the async forms take it too. Results record how they ended: `stop_reason` is `None` for a complete result, `"max_results"` when truncated by `max_results`, or `"timeout"` when a deadline hit, and `complete` is true when `stop_reason` is `None`. `count=True` returns a plain int, which cannot carry the flag, so it rejects `timeout_ms`.
 
 `before_context`, `after_context`, and `context` are like `rg -B`, `rg -A`, and `rg -C`. Files containing NUL bytes or invalid UTF-8 are skipped.
 
@@ -133,7 +135,7 @@ Notebook walking, parsing, and matching all happen in parallel in Rust, in the s
 
 ## Async
 
-`fda`, `rga`, and `nbrga` are awaitable twins of `fd`, `rg`, and `nbrg`, and `rga_iter` and `nbrga_iter` are async generators that yield rows as the search finds them. All take the same arguments and return the same types as their sync counterparts.
+`fda`, `rga`, and `nbrga` are awaitable twins of `fd`, `rg`, and `nbrg`, and `fda_iter`, `rga_iter` and `nbrga_iter` are async generators that yield rows as the search finds them. All take the same arguments and return the same types as their sync counterparts.
 
 ```python
 from rgapi import fda, rga, rga_iter

python/rgapi/__init__.py

@@ -123,12 +123,14 @@ def walk(
     skip_dir:str|list|None=None, # Directory glob or globs to prune
     skip_dir_re:str|None=None, # Directory regex used to prune traversal
     files:bool=True, # Include files in results
-    dirs:bool=False # Include directories in results
+    dirs:bool=False, # Include directories in results
+    timeout_ms:int|None=None, # Cancel the walk after this long and return partial results
 ) -> PathResults:
     "Walk a directory and return relative file and/or directory paths."
     rt = _fs_path(root)
-    return PathResults(_fe(_core.walk(rt, hidden, ignore, max_depth, min_depth, max_filesize, follow_links,
-        same_file_system, path_re, skip_path_re, _listify(skip_dir), skip_dir_re, files, dirs), rt))
+    paths, timed_out = _core.walk(rt, hidden, ignore, max_depth, min_depth, max_filesize, follow_links,
+        same_file_system, path_re, skip_path_re, _listify(skip_dir), skip_dir_re, files, dirs, timeout_ms)
+    return _mk_results(PathResults, _fe(paths, rt), False, timed_out)
 
 
 def _walk_args(
@@ -160,11 +162,26 @@ def fd(
     files:bool=True, # Include files in results
     dirs:bool=False, # Include directories in results
     show_target:bool=False, # Append `-> target` to symlink rows in the display
+    timeout_ms:int|None=None, # Cancel the walk after this long and return partial results
     **kwargs
 ) -> PathResults:
     "Find paths with fd-style filters and gitignore support."
     rt = _fs_path(root)
-    return PathResults(_fe(_core.find(rt, pattern, *_walk_args(**kwargs), files, dirs), rt, show_target))
+    paths, timed_out = _core.find(rt, pattern, *_walk_args(**kwargs), files, dirs, timeout_ms)
+    return _mk_results(PathResults, _fe(paths, rt, show_target), False, timed_out)
+
+
+@delegates(_walk_args)
+def fd_iter(
+    root:str|Path=".", # Directory or file to walk (expands `~`)
+    pattern:str|None=None, # Smart-case regex matched against each basename
+    files:bool=True, # Include files in results
+    dirs:bool=False, # Include directories in results
+    **kwargs
+):
+    "Walk lazily, yielding `FileEntry` paths as they are found; early exit stops the walk."
+    rt = _fs_path(root)
+    return _fe(_core.find_iter(rt, pattern, *_walk_args(**kwargs), files, dirs), rt)
 
 
 @delegates(fd)
@@ -206,11 +223,30 @@ async def fda(
     pattern:str|None=None, # Smart-case regex matched against each basename
     files:bool=True, # Include files in results
     dirs:bool=False, # Include directories in results
+    timeout_ms:int|None=None, # Cancel the walk after this long and return partial results
     **kwargs
 ) -> PathResults:
     "Async `fd`: find paths on Rust threads without blocking the event loop."
     rt = _fs_path(root)
-    return PathResults(_fe(await _acall(_core.find_async, rt, pattern, *_walk_args(**kwargs), files, dirs), rt))
+    paths, timed_out = await _acall(_core.find_async, rt, pattern, *_walk_args(**kwargs), files, dirs, timeout_ms)
+    return _mk_results(PathResults, _fe(paths, rt), False, timed_out)
+
+
+@delegates(_walk_args)
+async def fda_iter(
+    root:str|Path=".", # Directory or file to walk (expands `~`)
+    pattern:str|None=None, # Smart-case regex matched against each basename
+    files:bool=True, # Include files in results
+    dirs:bool=False, # Include directories in results
+    batch_max:int=512, # Largest batch of paths delivered to the event loop at once
+    **kwargs
+):
+    "Async `fd_iter`: yield `FileEntry` paths as they are found; early exit stops the walk."
+    rt = _fs_path(root)
+    async with aclosing(_abatches(_core.find_iter_async, batch_max, rt, pattern,
+        *_walk_args(**kwargs), files, dirs)) as batches:
+        async for paths in batches:
+            for p in _fe(paths, rt): yield p
 
 
 
@@ -418,4 +454,4 @@ def search_path(
 from .block import BlockResults, SearchBlock, _block_post
 from .nb import NbCell, NbResults, nbrg, nbrg_iter, nbrga, nbrga_iter, search_nb
 
-__all__ = [ "RgIter", "fd", "fda", "ls", "rg", "rga", "rg_iter", "rga_iter", "nbrg", "nbrg_iter", "nbrga", "nbrga_iter" ]
+__all__ = [ "RgIter", "fd", "fd_iter", "fda", "fda_iter", "ls", "rg", "rga", "rg_iter", "rga_iter", "nbrg", "nbrg_iter", "nbrga", "nbrga_iter" ]

src/lib.rs

@@ -13,7 +13,7 @@ pub use search::{
     compile_regex, rg, rg_iter, search_path, search_text, MatchSpan, RgIter, RgOptions, SearchKind,
     SearchLine,
 };
-pub use walk::{find, find_cancelable, FindOptions, StreamIter};
+pub use walk::{find, find_iter, FindIter, FindOptions, StreamIter};
 
 #[derive(Debug, Clone)]
 pub struct RgApiError {