Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
c3a2a1a
merge
wangxingjun778 Oct 13, 2026
950b941
ok Merge branch 'feat/mhs' of github.com:modelscope/leapflow into fea…
wangxingjun778 Oct 15, 2026
acd9464
ok Merge branch 'feat/mhs' of github.com:modelscope/leapflow into fea…
wangxingjun778 Oct 15, 2026
78a2563
remove unused content
wangxingjun778 Oct 16, 2026
6a5aea3
refactor and enhance hardware framework
wangxingjun778 Oct 19, 2026
e5b133d
add enhance hardware skills
wangxingjun778 Oct 20, 2026
1093d18
ok Merge branch 'feat/mhs' of github.com:modelscope/leapflow into fea…
wangxingjun778 Oct 20, 2026
ade894a
refactor hardware drivers
wangxingjun778 Oct 22, 2026
6da5b04
bump version
wangxingjun778 Oct 22, 2026
cb8866b
refine hardware skills and support for new sdk for piper-x
wangxingjun778 Oct 23, 2026
9186fea
refactor and refine hardware for specific sys topology
wangxingjun778 Oct 24, 2026
8a8d51e
refactor & enhance hardware module and add multi-source for media sig…
wangxingjun778 Oct 26, 2026
42dcf8a
feat(vision): share camera capture and harden preview lifecycle
wangxingjun778 Oct 27, 2026
304d5fc
add new module: LeapCalib
wangxingjun778 Oct 27, 2026
30740ec
enhance LeapCalib
wangxingjun778 Oct 30, 2026
9191107
chore: normalize author identity via .mailmap
wangxingjun778 Nov 5, 2026
0bcc6ac
ok Merge branch 'feat/mhs' of github.com:modelscope/leapflow into fea…
wangxingjun778 Nov 5, 2026
e1c1170
chore(repo): ignore .qoder and normalize mislabeled author identity
wangxingjun778 Nov 5, 2026
d11b604
chore(repo): drop obsolete mailmap entry for the rewritten identity
wangxingjun778 Nov 5, 2026
8cd639c
refactor leapcalib
wangxingjun778 Nov 12, 2026
d20bde2
fix leapd arm halt issue
wangxingjun778 Nov 12, 2026
faf1c17
grasp box success
wangxingjun778 Nov 12, 2026
0916784
fix worker leakage
wangxingjun778 Nov 13, 2026
dbb8eef
add leapboard stop and manage
wangxingjun778 Nov 13, 2026
2c5420c
add plan-time reach pre-check to motion.sequence
wangxingjun778 Nov 13, 2026
36104c7
refine pipeline for NL control devices
wangxingjun778 Nov 13, 2026
86404e5
fix hang issue and real jetson board deployment
wangxingjun778 Nov 16, 2026
2669bb8
fix tui terminal crash
wangxingjun778 Nov 16, 2026
577426a
fix vision capabilities
wangxingjun778 Nov 16, 2026
32281f8
ok Merge branch 'main' of github.com:modelscope/leapflow into feat/mhs
wangxingjun778 Nov 16, 2026
524841a
bump version
wangxingjun778 Nov 16, 2026
6f87152
fix hw_skill_run args issue
wangxingjun778 Nov 16, 2026
5eaffb1
fix piper-x halt policy and add hold-in-place policy
wangxingjun778 Nov 17, 2026
5857393
fix tui crash; enhance grasp async-batch perception
wangxingjun778 Nov 17, 2026
55c4408
fix config health classification and guard live preview lifecycle
wangxingjun778 Nov 17, 2026
ebef553
fix candidates and leapboard monitor
wangxingjun778 Nov 17, 2026
33090e4
ok Merge branch 'main' of github.com:modelscope/leapflow into feat/mhs
wangxingjun778 Nov 18, 2026
bfe94fd
bump version
wangxingjun778 Nov 18, 2026
b50bb59
feat: licensed hardware observation policy and graceful node stop
wangxingjun778 Nov 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,42 @@ export LEAPFLOW_FLEET_NODES='[{"node_id":"edge01","endpoint":"tcp://<ip>:8767","
leap daemon restart && leap hw list
```

**Stop the edge node** (run on the node, with the same user and profile):

```bash
leap hw stop # stop the sole local node and its workers
leap hw stop --node-id edge01 # select a node if several are running
leap hw stop --timeout 60 --json # wait longer; structured outcome
```

`stop` uses the owning systemd user unit when present, preventing its restart
policy from immediately relaunching the node; otherwise it sends SIGTERM to the
identified `serve` process. It waits for the node and observed workers to exit.
Repeated stops return `not_running`; ambiguous targets are refused. Discovery is
local and profile-scoped, and the host daemon/TUI and unrelated services remain
running. On Linux, legacy nodes without lifecycle records can also be discovered;
on macOS, the node must have been started by a version that writes these records.
A timeout returns failure without sending SIGKILL. A systemd unit can independently
force termination at its own `TimeoutStopSec`; an abnormal unit result is reported
as unclean, not as a successful graceful stop.

Stopping the service is **not hardware power-off or a parking operation**:
- Graceful shutdown cancels runs/leases, attempts declared actuator halts, and
releases camera streams, previews, and device connections.
- Piper halt attempts a powered hold-in-place; closing CAN sends no disable,
reset, parking move, or gripper-open command. An enabled arm may remain energized
and a gripper may retain its last command. Actual posture, holding torque, and
object retention after disconnect are **not verified**; other hardware follows
its driver's and firmware's shutdown behavior.
- Node-side monitoring/deadman and host remote control/estop through this node are
unavailable after exit. Keep the hardware attended and follow its physical safety
procedures. Host fleet health changes on its configured heartbeat schedule.

The command prints these consequences and distinguishes cleanup completion from
physical verification (`hardware_state_verified: false`). For a supervised node,
resume with `systemctl --user start <owning-unit>`; otherwise reuse the original
`leap hw serve ...` command. Stopping a unit does not disable its boot activation.

</details>

<details>
Expand Down
13 changes: 13 additions & 0 deletions src/leapflow/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,19 @@ def main(argv: list[str] | None = None) -> int:
"--log-file", default="", help="Append and size-rotate the node log here (config hardware.node_log_file)"
)
hw_serve.add_argument("--verbose", action="store_true", help="Log at DEBUG level")
from leapflow.hardware.lhp.node_lifecycle import DEFAULT_STOP_TIMEOUT_S

hw_stop = hw_sub.add_parser(
"stop", parents=[hw_json], help="Gracefully stop a local LHP node and its workers",
description=(
"Stop the local hw serve node in the active profile, through its supervisor "
"when present. This is not hardware power-off: Piper holding torque/grip may "
"remain; physical state is unverified after disconnect. Node monitoring and "
"remote control stop, and camera streams are released. No automatic SIGKILL."
),
)
hw_stop.add_argument("--node-id", default="", help="Select a local node (env LEAPFLOW_LHP_NODE_ID); required if ambiguous")
hw_stop.add_argument("--timeout", type=float, default=DEFAULT_STOP_TIMEOUT_S, help="Seconds to wait for node, workers, and supervisor to stop (default: %(default)s)")
# Verification subcommands
hw_verify = hw_sub.add_parser("verify", parents=[hw_json], help="Verify a device declaration (digest-bound)")
hw_verify.add_argument("device", help="Device id to verify")
Expand Down
80 changes: 78 additions & 2 deletions src/leapflow/cli/commands/hardware.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
import json
import logging
import sys
from dataclasses import replace
from dataclasses import asdict, replace
from pathlib import Path
from typing import Any

Expand Down Expand Up @@ -132,6 +132,8 @@ async def _dispatch(args: argparse.Namespace, action: str) -> int:
return _run_offline(action, args, json_mode)
if action == "serve":
return await _run_serve(args)
if action == "stop":
return await _run_stop(args, json_mode)
if action == "validate" and getattr(args, "workcells", None) is not None:
from leapflow.cli.commands.hardware_workcells import run_workcell_validation

Expand Down Expand Up @@ -188,6 +190,51 @@ async def _run_serve(args: argparse.Namespace) -> int:
raise


async def _run_stop(args: argparse.Namespace, json_mode: bool) -> int:
"""Stop only the selected local node; never initialize a second hardware owner."""
import math
import os

from leapflow.hardware.lhp.node_lifecycle import (
DEFAULT_STOP_TIMEOUT_S, NODE_ID_ENV, StopResult, discover_nodes, stop_node,
)
from leapflow.hardware.lhp.protocol import is_valid_device_id

node_id = str(getattr(args, "node_id", "") or os.environ.get(NODE_ID_ENV, "")).strip()
timeout_s = float(getattr(args, "timeout", DEFAULT_STOP_TIMEOUT_S))
if sys.platform == "win32":
return _emit_result("stop", StopResult(
ok=False, status="unsupported_platform", detail="Graceful node stop is supported on POSIX systems only.",
).to_dict(), json_mode)
if (node_id and not is_valid_device_id(node_id)) or not math.isfinite(timeout_s) or timeout_s <= 0:
return _emit_result("stop", StopResult(
ok=False, status="invalid_args", detail="Use a valid --node-id and a positive, finite --timeout.",
).to_dict(), json_mode)
try:
settings = load_config()
nodes = await asyncio.to_thread(discover_nodes, settings.runtime_dir, node_id)
except (OSError, ValueError) as exc:
logger.error("Local node discovery failed", exc_info=True)
return _emit_result("stop", StopResult(
ok=False, status="discovery_failed", detail=f"Cannot discover local nodes: {type(exc).__name__}; no stop requested.",
).to_dict(), json_mode)
if not nodes:
return _emit_result("stop", StopResult(
ok=True, status="not_running", detail="No matching local node is running in this profile; no service action taken.",
).to_dict(), json_mode)
if len(nodes) != 1:
candidates = ", ".join(f"{node.node_id or '(unknown id)'} (pid {node.pid})" for node in nodes)
return _emit_result("stop", StopResult(
ok=False, status="ambiguous_node",
detail=f"Multiple local nodes match: {candidates}. Select one with --node-id or the correct profile; no stop requested.",
).to_dict(), json_mode)
if not json_mode:
print("Warning: stopping the node cancels work and disconnects hardware. This is NOT power-off; physical arm/grip state is unverified afterward.", flush=True)
print(f"Waiting up to {timeout_s:g}s for {nodes[0].node_id} and its workers to stop...", flush=True)
result = await asyncio.to_thread(stop_node, nodes[0], timeout_s=timeout_s)
return _emit_result("stop", result.to_dict(), json_mode)


# ── Read / estop plane ─────────────────────────────────────────────────────


Expand Down Expand Up @@ -331,6 +378,26 @@ async def _run_sampling_control(


def _emit_result(action: str, result: dict[str, Any], json_mode: bool) -> int:
if action == "stop":
# FailureEnvelope belongs at the application boundary, not in hardware.
from leapflow.engine.recovery.failure_envelope import (
FailureEnvelope, FailureSource, Recoverability, SideEffectState,
)

failure = None
if not result.get("ok"):
envelope = FailureEnvelope.create(
source=FailureSource.SYSTEM, category="node_lifecycle",
failure_class="NodeStopFailure", failure_code=result["status"],
message=result["detail"], recoverability=Recoverability.USER_FIXABLE,
side_effect_state=SideEffectState.UNKNOWN if result.get("via") else SideEffectState.NONE,
)
failure = {
**asdict(envelope), "source": envelope.source.value,
"recoverability": envelope.recoverability.value,
"side_effect_state": envelope.side_effect_state.value,
}
result = {**result, "failure": failure}
if json_mode:
print(json.dumps(result, indent=2, default=str, ensure_ascii=False))
else:
Expand All @@ -339,6 +406,14 @@ def _emit_result(action: str, result: dict[str, Any], json_mode: bool) -> int:


def _render(action: str, result: dict[str, Any]) -> None:
if action == "stop":
print("LEAP Hardware — stop")
print(f" {result.get('status')}: {result.get('detail', '')}")
for consequence in result.get("consequences", ()):
print(f" Warning: {consequence}")
if result.get("survivors"):
print(f" Worker processes still running: {result['survivors']}")
return
print(f"{_CYAN}LEAP Hardware \u2014 {action}{_RESET}")
if not result.get("ok"):
_fail(str(result.get("error") or result.get("code") or "command failed"))
Expand Down Expand Up @@ -1007,7 +1082,7 @@ async def _run_commission(args: argparse.Namespace, json_mode: bool) -> int:


def _print_usage() -> None:
print("Usage: leap hw {list|describe|read|status|estop|pause|resume|replay|serve|verify|validate|permit-key|init|probe|commission} [--json]")
print("Usage: leap hw {list|describe|read|status|estop|pause|resume|replay|serve|stop|verify|validate|permit-key|init|probe|commission} [--json]")
print()
print("Inspect hardware and intervene in it directly.")
print()
Expand All @@ -1021,6 +1096,7 @@ def _print_usage() -> None:
print(" resume <device> Resume daemon sampling for a device")
print(" replay <path> Replay a raw NDJSON segment through the event detector")
print(" serve Serve this machine as an LHP fabric node (edge runtime)")
print(" stop [--node-id ID] Stop the local node and workers; not hardware power-off")
print(" verify <device> Write a digest-bound verification entry for a device")
print(" validate [device] Validate declarations against their verification entries")
print(" permit-key Show the permit issuer public key and key_id")
Expand Down
2 changes: 1 addition & 1 deletion src/leapflow/cli/commands/slash_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ def build_config_payload(ctx: "Context", args: str = "") -> dict[str, Any]:
"""Execute a config command and return a serializable payload."""
from leapflow.config_service import ConfigService

service = ConfigService(ctx.settings)
service = ConfigService(ctx.settings, hardware_registry=getattr(ctx, "_hardware_registry", None))
try:
from leapflow.utils.shell_lex import split_args

Expand Down
15 changes: 14 additions & 1 deletion src/leapflow/cli/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -1133,7 +1133,20 @@ def reload_runtime_config_if_changed(self, *, force: bool = False) -> bool:
return False

previous = self.settings
updated = self._load_runtime_settings_from_files()
try:
updated = self._load_runtime_settings_from_files()
except ValueError:
logger.warning("Invalid runtime configuration; retaining last valid settings", exc_info=True)
self._config_signature = signature
return False
registry = getattr(self, "_hardware_registry", None)
if registry is not None and updated.hardware_observation != previous.hardware_observation:
try:
registry.reconfigure_observation(updated.hardware_observation)
except ValueError:
from dataclasses import replace
logger.warning("Invalid hardware observation reload; retaining last valid policy", exc_info=True)
updated = replace(updated, hardware_observation=previous.hardware_observation)
self._config_signature = self._runtime_config_signature(updated)
llm_changed = (
previous.llm_api_key != updated.llm_api_key
Expand Down
6 changes: 6 additions & 0 deletions src/leapflow/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,8 @@ class Settings:
# enforces (drive temperature, supply voltage, advisory fault flags). Breaches
# are reported to the board with backoff and never gate a command.
hardware_advisory_limits: str = ""
# JSON operational observation policy; never changes writes or task accuracy.
hardware_observation: str = "{}"
# LHP edge-node process log (`leap hw serve`). With a file the node appends and
# size-rotates it itself (max_mb x backups), so restarts keep earlier runs and
# the disk stays bounded; empty writes to stderr only. Read at node start.
Expand Down Expand Up @@ -1569,6 +1571,9 @@ def _build_settings_from_env(
ViewPriorSettings.from_json(hardware_view_priors)
hardware_alert_policies = os.getenv("LEAPFLOW_HARDWARE_ALERT_POLICIES", "").strip()
hardware_advisory_limits = os.getenv("LEAPFLOW_HARDWARE_ADVISORY_LIMITS", "").strip()
hardware_observation = os.getenv("LEAPFLOW_HARDWARE_OBSERVATION", "{}").strip()
from leapflow.hardware.observation import ObservationSettings
ObservationSettings.parse(hardware_observation)
hardware_node_log_file = os.getenv("LEAPFLOW_HARDWARE_NODE_LOG_FILE", "").strip()
hardware_node_log_level = (
os.getenv("LEAPFLOW_HARDWARE_NODE_LOG_LEVEL", "INFO").strip() or "INFO"
Expand Down Expand Up @@ -2139,6 +2144,7 @@ def _tuple_env(key: str, default: tuple) -> tuple:
hardware_view_priors=hardware_view_priors,
hardware_alert_policies=hardware_alert_policies,
hardware_advisory_limits=hardware_advisory_limits,
hardware_observation=hardware_observation,
hardware_node_log_file=hardware_node_log_file,
hardware_node_log_level=hardware_node_log_level,
hardware_node_log_max_mb=hardware_node_log_max_mb,
Expand Down
27 changes: 25 additions & 2 deletions src/leapflow/config_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,13 @@ class ConfigSnapshot:
"else deletes from that table. Read when sampling starts, so a change needs a "
"daemon restart."
),
"hardware.observation": (
"JSON report-only policy with defaults and exact overrides (device_id/channel_id). "
"soft_min/soft_max and margin_floor/margin_max are in channel units; confirm_s/recover_s "
"require continuous valid evidence. Adaptive defaults off and requires margin_max. "
"Hard envelope and task accuracy are unchanged. Valid updates hot-reload; invalid "
"updates retain the last admitted policy. Channel bounds are checked at hardware admission."
),
"hardware.raw_segment_mb": (
"Size at which a raw sample file is closed and a new segment started. A finished "
"segment is written once, so its indexed size and TTL are accurate and old data "
Expand Down Expand Up @@ -823,8 +830,9 @@ def _profile_file_for_section(section: str) -> str:
class ConfigService:
"""Read and mutate LeapFlow configuration through layout-owned paths."""

def __init__(self, settings: Settings) -> None:
def __init__(self, settings: Settings, *, hardware_registry: Any = None) -> None:
self._settings = settings
self._hardware_registry = hardware_registry
self._secrets = ScopedSecretResolver(settings.layout, settings.profile_layout)

def snapshot(self) -> ConfigSnapshot:
Expand Down Expand Up @@ -886,6 +894,19 @@ def set(
if spec.secret:
return self._set_secret_backed_field(spec, str(value), scope=scope)
coerced = _coerce_value(value, spec.value_type)
if normalized == "hardware.observation":
from leapflow.hardware.observation import ObservationSettings
policy = ObservationSettings.parse(coerced)
if self._hardware_registry is not None:
contexts = self._hardware_registry.contexts()
else:
# Offline CLI validation reads declarations only, never discovers or opens devices.
from leapflow.hardware.providers.yaml_provider import YamlContextProvider
from leapflow.hardware.registry import HardwareSettings
hardware = HardwareSettings.from_settings(self._settings)
yaml_config = next((config for kind, config in hardware.providers if kind == "yaml"), None)
contexts = YamlContextProvider(yaml_config).discover() if yaml_config else ()
policy.validate_contexts(contexts)
if normalized.startswith("hardware.skills_planning_"):
from leapflow.hardware.skills.planning import PlanningSettings

Expand Down Expand Up @@ -1034,7 +1055,9 @@ def _field_view(self, spec: ConfigFieldSpec) -> ConfigFieldView:


def _with_metadata(spec: ConfigFieldSpec) -> ConfigFieldSpec:
if spec.section in _RESTART_REQUIRED_SECTIONS:
if spec.key == "hardware.observation":
reload_semantics = "yes"
elif spec.section in _RESTART_REQUIRED_SECTIONS:
reload_semantics = "restart-required"
elif spec.section in _PARTIAL_RELOAD_SECTIONS:
reload_semantics = "partial"
Expand Down
4 changes: 3 additions & 1 deletion src/leapflow/daemon/monitor_coordinator.py
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,9 @@ def _register_hardware_producer(self, ctx: Any, settings: Any) -> None:
from leapflow.hardware.observability import HardwareObservationProducer

monitors.producers.register(
HardwareObservationProducer(lambda: getattr(ctx, "_hardware_registry", None))
HardwareObservationProducer(lambda: getattr(ctx, "_hardware_registry", None),
finding_store=monitors.finding_store,
run_db=getattr(ctx, "_run_deferred_db", None))
)
except Exception:
logger.debug("daemon: hardware observability producer unavailable", exc_info=True)
Expand Down
Loading
Loading