Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
succeeds, stage `notes` in this repository and commit/push its new gitlink. A failed
push must never leave a published parent pointer to unavailable notes.
- The README contains the complete command sequence. Keep original histories and
resolve divergence without overwriting either side. The migration provenance and
P-673 uncommitted-document handoff are in `notes/scriptmark/MIGRATION.md`.
resolve divergence without overwriting either side.
- Linear is the execution home. Keep runnable examples, code, measurements and raw
artifacts here; maintain narrative/design documentation on the notes branch.
Historical code paths inside imported plans describe their original revision.
Expand Down Expand Up @@ -63,15 +62,17 @@ default Python is newer than the PyO3 version supports; see the P-676 validation

Single `scriptmark` crate (lib + bin) with `scriptmark-py` as separate cdylib for PyO3.

Data flows: TOML specs + student files → Runner → Results → Display/DB/HTML.
Data flows: TOML specs + student files → Runner → grading record (evidence + score
revisions) → Display/DB/HTML/Canvas. `rescore` adds a revision from saved evidence.

```
models/ Data models, TOML spec parsing, grading policies
discovery Student file discovery + ZIP extraction
runner/ PythonExecutor (subprocess), orchestrator, sandbox (setrlimit),
parametrize, expander, oracle, linter
checker/ Checker trait (8 impls) + Rhai + Python checkers
db/ SQLite (rusqlite bundled): students, sessions, results, similarity
record Grading record: versioned evidence, score revisions, rescore reuse checks
db/ SQLite (rusqlite bundled): students, sessions (= revisions), results, similarity
canvas/ Canvas LMS API client (reqwest + rustls): roster pull, grades push
tui/ ratatui terminal UI: students/sessions/similarity tabs
scriptmark-py PyO3 bindings: grade, run, discover, load_spec (maturin, separate crate)
Expand Down Expand Up @@ -100,6 +101,7 @@ Zero and withheld grades stay distinct. See the notes' teacher contract for poli
- `crates/scriptmark/src/runner/orchestrator.rs` — Runs vars→setup→expand→oracle→execute pipeline per student, tokio parallel.
- `crates/scriptmark/src/models/spec.rs` — All TOML spec structs (TestSpec, TestCase, SetupStep, Parametrize, Oracle, LintConfig, CheckMethod).
- `crates/scriptmark/src/discovery.rs` — File discovery + ZIP archive extraction with size/count limits.
- `crates/scriptmark/src/record.rs` — The grading record every consumer reads (`--output`): evidence with submission/spec fingerprints, append-only score revisions, `check()` refusing evidence whose tests, submissions or matching changed.
- `crates/scriptmark/src/grading.rs` — GradingPolicy dispatch (templates + Rhai formulas).
- `crates/scriptmark/src/main.rs` — All CLI command handlers.
- `crates/scriptmark-py/src/lib.rs` — PyO3 bindings: grade(), run(), discover(), load_spec(), StudentResult, TestSpec classes.
Expand Down
37 changes: 29 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,7 @@ and git push
Push the notes successfully **before** updating the code repository's gitlink. If a
fast-forward fails, resolve the divergence while preserving both versions. Do not
force-push or discard notes. Runtime code, examples and artifacts stay in this repository;
documentation edits belong on the notes branch. Original documentation history and
the P-673 worktree handoff are recorded in
[the migration record](https://github.com/Acture/obsidian-vault/blob/project/scriptmark/scriptmark/MIGRATION.md).
documentation edits belong on the notes branch.

Only when explicitly reproducing an older code revision, restore its recorded notes
with `git submodule update --init --checkout -- notes` from a clean notes checkout.
Expand Down Expand Up @@ -112,8 +110,22 @@ never pushed — instead of scoring it as 0.
# Grade with roster and database storage; points and any curve come from assignment.toml
scriptmark grade submissions/ -t tests/ -r roster.csv --db grades.db -a archive/

# Run tests only (raw JSON output)
scriptmark run submissions/ -t tests/ -o results.json
# Run tests only: the grading record's evidence, with no score revision yet
scriptmark run submissions/ -t tests/ -o output/results.json

# Score saved evidence under assignment.toml as it is now, without running anything:
# a `run` record gets revision 1, a graded one its next. Edit points or the curve and
# rescore again; earlier revisions are kept, and whose grade changed is shown. Changed
# specs, teacher files, submissions or [matching] rules are refused: grade again.
scriptmark rescore output/results.json

# Grade afresh over a record that holds rescored revisions, discarding them
scriptmark grade submissions/ -t tests/ --force

# Read the latest revision, or any other with --revision N
scriptmark summarize output/results.json --revision 1
scriptmark export output/results.json -o grades.csv
scriptmark db save output/results.json --revision 1 --db grades.db

# Preview student/file/function matching; edit assignment.toml to resolve candidates
scriptmark match submissions/ -t tests/ -o output/matches.json
Expand All @@ -126,14 +138,15 @@ scriptmark grade late/ -t tests/ --replay output/results.cases.json -o output/la
scriptmark similarity submissions/ --threshold 0.8

# Generate HTML report
scriptmark report results.json -o report.html
scriptmark report output/results.json -o report.html

# Canvas LMS: find a course, fetch an assignment, grade it offline, push grades back
export CANVAS_TOKEN=... CANVAS_URL=https://canvas.university.edu
scriptmark canvas courses
scriptmark canvas assignments --course-id 12345
scriptmark canvas fetch --course-id 12345 --assignment-id 67890 -o canvas/hw1
scriptmark grade --canvas canvas/hw1 -t tests/
# pushes the record's only revision; name one with --revision N once it holds several
scriptmark grades-push --course-id 12345 --assignment-id 67890 output/results.json

# Or just pull the roster
Expand All @@ -149,14 +162,22 @@ scriptmark tui grades.db
import scriptmark

# One-shot grading, under the assignment.toml beside tests/ (or pass assignment=...).
# freeze= keeps the generated inputs; replay= grades on ones kept earlier.
results = scriptmark.grade(["submissions/"], "tests/", freeze="output/cases.json")
# freeze= keeps the generated inputs; replay= grades on ones kept earlier; output= writes
# the grading record.
results = scriptmark.grade(
["submissions/"], "tests/", freeze="output/cases.json", output="output/results.json"
)
for r in results:
if r.grade is None:
print(f"{r.student_id}: withheld ({r.reason})")
else:
print(f"{r.student_id}: {r.grade} ({r.score}/{r.max} points)")

# After editing the policy, score the record again without running anything
# (records graded from a Canvas bundle are rescored with the CLI)
change = scriptmark.rescore("output/results.json") # {"revision": 2, "changes": [...]}
first = scriptmark.load_record("output/results.json", revision=1)

# Discover student files (convenience view — drops non-submitters and orphan files)
# Keys are rendered student keys: a bare 学号 once a roster confirms it, otherwise
# `local:<token>` — the prefix means nothing has vouched for that filename token yet.
Expand Down
Loading
Loading