Support serial dependency ordering for actions and targets (#552) - #557
Support serial dependency ordering for actions and targets (#552)#557lodyai[bot] wants to merge 40 commits into
Conversation
Reviewer's GuideAdds manifest-level dependency_order support, threads it through IR to Ninja generation, and implements staged Ninja dyndep bundles plus atomic sidecar materialization so serial dependency lists run in declaration order while preserving a single Ninja scheduler and parallel behaviour for other branches. Sequence diagram for serial dependency Ninja bundle generation and executionsequenceDiagram
actor User
participant Runner as runner.generate_ninja
participant NinjaGen as ninja_gen.generate_bundle
participant Dyndep as process.materialize_dyndep_files
participant Ninja
User->>Runner: netsuke build / clean / generate
Runner->>NinjaGen: generate_bundle(graph)
NinjaGen-->>Runner: GeneratedNinja (build_file, dyndep_files)
Runner->>Dyndep: materialize_dyndep_files(cli, bundle.dyndep_files())
Dyndep-->>Runner: dyndep sidecars materialized
Runner->>Ninja: invoke with bundle.build_file()
Ninja-->>User: serial deps run in order, parallel elsewhere
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
e1cef57 to
7ed4cc8
Compare
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. src/runner/process/dyndep_files.rs Comment on lines +114 to +163 fn write_atomic(dir: &Dir, rel: &Utf8Path, content: &str) -> Result<()> {
let temp = unique_temp_name(rel);
let mut options = OpenOptions::new();
options.write(true).create_new(true);
let mut file = match dir.open_with(&temp, &options) {
Ok(file) => file,
Err(err) if err.kind() == std::io::ErrorKind::AlreadyExists => {
// Another process won the race for our temporary name; verify the
// final path and treat matching content as success.
return match read_verified(dir, rel, content)? {
ReadOutcome::Matching => Ok(()),
ReadOutcome::Mismatch => Err(anyhow!(
localization::message(keys::RUNNER_IO_DYNDEP_CORRUPT)
.with_arg("path", rel.as_str())
)),
ReadOutcome::Missing => Err(anyhow!(
localization::message(keys::RUNNER_IO_DYNDEP_RACE)
.with_arg("path", rel.as_str())
)),
};
}
Err(err) => {
return Err(err).with_context(|| {
localization::message(keys::RUNNER_IO_DYNDEP_WRITE).with_arg("path", rel.as_str())
});
}
};
file.write_all(content.as_bytes()).with_context(|| {
localization::message(keys::RUNNER_IO_DYNDEP_WRITE).with_arg("path", rel.as_str())
})?;
file.flush().with_context(|| {
localization::message(keys::RUNNER_IO_DYNDEP_WRITE).with_arg("path", rel.as_str())
})?;
file.sync_all().with_context(|| {
localization::message(keys::RUNNER_IO_DYNDEP_WRITE).with_arg("path", rel.as_str())
})?;
// Rename is relative to the same directory; `rename` replaces an existing
// destination, so if another process already wrote the final file, the
// atomic replace yields content identical to ours.
if let Err(err) = dir.rename(&temp, dir, rel) {
// The final file may have appeared via a concurrent writer; verify it.
if read_verified(dir, rel, content)? != ReadOutcome::Matching {
return Err(err).with_context(|| {
localization::message(keys::RUNNER_IO_DYNDEP_RENAME).with_arg("path", rel.as_str())
});
}
drop(dir.remove_file(&temp));
}
Ok(())
}❌ New issue: Bumpy Road Ahead |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Comment on lines +123 to +135 fn parallel_edges_produce_no_sidecars() -> Result<()> {
let graph = graph_with_edge(parallel_edge("all", &["dep1", "dep2"]))?;
let bundle = generate_bundle(&graph)?;
ensure!(
!bundle.build_file().contains("ninja_required_version"),
"parallel bundle must not emit a version floor"
);
ensure!(
bundle.dyndep_files().is_empty(),
"parallel graph must produce no sidecars"
);
Ok(())
}❌ New issue: Code Duplication |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Comment on lines +156 to +230 pub fn generate_bundle(graph: &BuildGraph) -> Result<GeneratedNinja, NinjaGenError> {
reject_reserved_paths(graph)?;
let serial_present = graph_requires_dyndep(graph);
let mut out = String::new();
if serial_present {
writeln!(out, "ninja_required_version = 1.10\n")?;
}
let mut actions: Vec<_> = graph.actions.iter().collect();
actions.sort_by_key(|(id, _)| *id);
for (id, action) in actions {
use crate::ninja_gen::NamedAction;
writeln!(out, "{}", NamedAction { id, action })?;
}
let mut edges: Vec<_> = graph.targets.values().collect();
edges.sort_by_key(|a| path_key(&a.explicit_outputs));
let mut seen: HashSet<String> = HashSet::new();
let mut stages = SerialStages::default();
for edge in edges {
let key = path_key(&edge.explicit_outputs);
if !seen.insert(key.clone()) {
continue;
}
let action =
graph
.actions
.get(&edge.action_id)
.ok_or_else(|| NinjaGenError::MissingAction {
id: edge.action_id.clone(),
message: localization::message(keys::NINJA_GEN_MISSING_ACTION)
.with_arg("id", &edge.action_id),
})?;
let requires_gates =
edge.dependency_order == DependencyOrder::Serial && edge.implicit_deps.len() > 1;
if requires_gates {
let mut added = Vec::new();
render_serial_block(edge, &mut out, &mut stages, &mut added)?;
let mut aggregate = edge.clone();
aggregate.implicit_deps = added;
aggregate.dependency_order = DependencyOrder::Parallel;
writeln!(
out,
"{}",
crate::ninja_gen::DisplayEdge {
edge: &aggregate,
action_restat: action.restat,
}
)?;
} else {
writeln!(
out,
"{}",
crate::ninja_gen::DisplayEdge {
edge,
action_restat: action.restat,
}
)?;
}
}
if !graph.default_targets.is_empty() {
let mut defs = graph.default_targets.clone();
defs.sort();
writeln!(out, "default {}", join(&defs))?;
}
Ok(GeneratedNinja {
build_file: out,
dyndep_files: stages.dyndep_files,
})
}❌ New issue: Complex Method |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
There was a problem hiding this comment.
Sorry @leynos, your pull request is larger than the review limit of 150000 diff characters
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
Summary
Implementation
Tests and documentation
WalkthroughThis change adds opt-in serial dependency ordering for actions and targets. It propagates ChangesSerial dependency ordering
Suggested labels: Suggested reviewers: Poem
Caution Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional.
❌ Failed checks (1 error, 1 warning, 2 inconclusive)
✅ Passed checks (16 passed)
📋 Issue PlannerLet us write the prompt for your AI agent so you can ship faster (with fewer bugs). View plan for ticket: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
9cac0aa to
a576350
Compare
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Code Duplicationsrc/runner/process/dyndep_files.rs: What lead to degradation?The module contains 2 functions with similar structure: create_temp_file,open_existing_sidecar Why does this problem occur?Duplicated code often leads to code that's harder to change since the same logical change has to be done in multiple functions. More duplication gives lower code health. How to fix it?A certain degree of duplicated code might be acceptable. The problems start when it is the same behavior that is duplicated across the functions in the module, ie. a violation of the Don't Repeat Yourself (DRY) principle. DRY violations lead to code that is changed together in predictable patterns, which is both expensive and risky. DRY violations can be identified using CodeScene's X-Ray analysis to detect clusters of change coupled functions with high code similarity. Read More |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. String Heavy Function Argumentssrc/runner/process/dyndep_files_tests.rs: What lead to degradation?In this module, 46.2% of all arguments to its 21 functions are strings. The threshold for string arguments is 39.0% Why does this problem occur?String is a generic type that fail to capture the constraints of the domain object it represents. In this module, 46 % of all function arguments are string types. How to fix it?Heavy string usage indicates a missing domain language. Introduce data types that encapsulate the semantics. For example, a user_name is better represented as a constrained User type rather than a pure string, which could be anything. |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
@coderabbitai please stop whining about changes on main: Out of Scope Changes check |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '=== Current branch and remotes ==='
git status --short
git branch --show-current
git remote -v | head -n 4
printf '%s\n' '=== PR metadata ==='
gh pr view 557 --repo leynos/netsuke --json baseRefName,headRefName,baseRefOid,headRefOid,url \
--jq '{url, baseRefName, headRefName, baseRefOid, headRefOid}'
base_ref=$(gh pr view 557 --repo leynos/netsuke --json baseRefName --jq '.baseRefName')
printf '\n=== Diff against origin/%s for warned files ===\n' "$base_ref"
git diff --no-ext-diff -- .gitignore Makefile "origin/$base_ref"...HEAD || true
printf '\n=== .vtcode and DYLINT_TOML provenance ===\n'
git log -n 5 --oneline -- .gitignore Makefile
rg -n -C2 '(^\.vtcode/$|DYLINT_TOML|lint-whitaker)' .gitignore Makefile || true
printf '\n=== Values on origin/%s ===\n' "$base_ref"
git show "origin/$base_ref:.gitignore" 2>/dev/null | rg -n '^\.vtcode/$' || true
git show "origin/$base_ref:Makefile" 2>/dev/null | rg -n -C2 '(DYLINT_TOML|lint-whitaker)' || trueLength of output: 5142
PR 🐇 ✏️ Learnings added
You are interacting with an AI system. |
Align the design sequence diagram with the runner capability-scoped publication contract and record the verified review finding in the issue plan.
Delete the unused `RUNNER_IO_DYNDEP_RACE` key and its catalogue entries. Concurrent publication now reports only errors backed by reachable runtime paths, while verified matching and mismatch outcomes remain unchanged.
|
@coderabbitai review |
✅ Action performedReview finished.
|
Describe reserved graph namespaces across every user path field and clarify when dyndep sidecars are materialized by each CLI command. Record the verified documentation review and leave the already-correct design gate wording unchanged.
Move the computed edge key into deduplication directly and retain the documented compatibility boundary for content-addressed sidecars.
Keep the serial dyndep migration guidance compliant with the repository documentation spelling policy.
|
@coderabbitai Have the following failed checks now been resolved? If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures. Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat o11y, code safety, documentation and validation coverage as in scope). ❌ Failed checks (1 error, 1 warning)
|
Summary
Implements the approved staged-Ninja-dyndep design for issue #552. Actions and
targets can declare
dependency_order: serialwhile preserving one Ninjascheduler, shared-work reuse, failure short-circuiting, and unrelated-branch
concurrency.
Closes #552.
User documentation
dependency_order: parallel | serialfor actions and targets inthe users' guide, with a complete executable manifest.
depsare ordered;independently reachable and unrelated work remains concurrent.
.netsuke/serialand.netsuke/dyndepnamespaces.developer, repository-layout, roadmap, contents, and living ExecPlan records.
Review walkthrough
parallel.Validation
make check-fmt: passed.make typecheck: passed.make lint: passed, including Whitaker.make test: passed; 1,939 tests passed, one skipped, and doctests passed.make markdownlint: passed.make nixie: passed.coderabbit review --agent: completed with zero actionable findings.References