Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ jobs:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- run: cargo fmt --check
- run: cargo clippy --all-targets -- -D warnings
- run: cargo test
- run: cargo fmt --all --check
- run: cargo clippy --workspace --all-targets -- -D warnings
- run: cargo test --workspace
55 changes: 1 addition & 54 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,37 +57,6 @@ jobs:
name: scriptmark-${{ matrix.target }}
path: target/${{ matrix.target }}/release/${{ matrix.artifact }}

wheels:
name: Wheel ${{ matrix.target }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- target: x86_64-unknown-linux-gnu
os: ubuntu-latest
- target: x86_64-apple-darwin
os: macos-latest
- target: aarch64-apple-darwin
os: macos-latest
- target: x86_64-pc-windows-msvc
os: windows-latest

steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist
manylinux: auto
- uses: actions/upload-artifact@v4
with:
name: wheel-${{ matrix.target }}
path: dist/*.whl

publish-github:
name: GitHub Release
needs: [build]
Expand Down Expand Up @@ -134,26 +103,4 @@ jobs:
- uses: dtolnay/rust-toolchain@stable

- name: Publish crate
run: cargo publish -p scriptmark --no-verify

publish-pypi:
name: Publish to PyPI
needs: [wheels]
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/v')
environment: release
permissions:
id-token: write

steps:
- name: Download wheels
uses: actions/download-artifact@v4
with:
path: dist
pattern: wheel-*
merge-multiple: true

- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
packages-dir: dist/
run: cargo publish --workspace
12 changes: 8 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -182,11 +182,15 @@ tests_submission/

# P-669 test fixtures — the broad rules above (*.csv, submissions, templates, config.toml)
# must not swallow committed test data.
!src/crates/scriptmark/tests/fixtures/**
!src/core/tests/fixtures/**
!src/cli/tests/fixtures/**
# ...but not what running the suite over them produces.
src/crates/scriptmark/tests/fixtures/**/__pycache__/
src/crates/scriptmark/tests/fixtures/**/.scriptmark_extracted/
src/crates/scriptmark/tests/fixtures/**/.DS_Store
src/core/tests/fixtures/**/__pycache__/
src/cli/tests/fixtures/**/__pycache__/
src/core/tests/fixtures/**/.scriptmark_extracted/
src/cli/tests/fixtures/**/.scriptmark_extracted/
src/core/tests/fixtures/**/.DS_Store
src/cli/tests/fixtures/**/.DS_Store

# P-674 example bundles ship their students; `submissions` above would drop them.
!examples/bundles/*/submissions/
Expand Down
64 changes: 32 additions & 32 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,41 +55,40 @@ Automated grading CLI for student programming assignments. Rust core with custom
```bash
cargo check # Fast type check
cargo build -p scriptmark # Build CLI binary
cargo test -p scriptmark # Core and integration tests; requires Python
cargo test --workspace # Includes the PyO3 binding crate
cargo clippy --all-targets # Lint — must be 0 warnings
cargo test -p scriptmark-core # Core tests; requires Python
cargo test -p scriptmark # CLI and adapter integration tests
cargo test --workspace # Both crates
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt # Format all code
cargo fmt --check # Verify formatting
maturin develop # Build + install PyO3 Python bindings locally
```

Single test: `cargo test -p scriptmark test_name -- --nocapture`.
For the PyO3 crate, select a supported interpreter via `PYO3_PYTHON` when the machine's
default Python is newer than the PyO3 version supports; see the P-676 validation record.
Single test: `cargo test -p scriptmark-core test_name -- --nocapture` (or `-p scriptmark`
for CLI tests). Python 3 is required by the subprocess executor, not by a Python SDK.

## Architecture

Single `scriptmark` crate (lib + bin) with `scriptmark-py` as separate cdylib for PyO3.
Two Rust crates live directly under `src/`: `src/core/` contains `scriptmark-core`,
and `src/cli/` contains the `scriptmark` CLI and its adapters. The CLI depends on the
core; core must not depend on CLI, HTTP, SQLite or terminal UI libraries.

All product source is under `src/`: Rust packages live in `src/crates/` and the
Python package entry point in `src/python/scriptmark/`. Workspace/build configuration
stays at the repository root, where Cargo and Maturin commands are run. The module
paths below are relative to `src/crates/scriptmark/src/`.
Cargo workspace/build configuration stays at the repository root. There is no Python
package, PyO3 binding or Python wheel release. Embedded Python executor/checker code
and runnable Python assignment examples remain part of the Rust product.

Data flows: TOML specs + student files → Runner → grading record (evidence + score
revisions) → Display/DB/HTML/Canvas. `rescore` adds a revision from saved evidence.

```
models/ Data models, TOML spec parsing, grading policies
discovery Student file discovery + ZIP extraction
runner/ PythonExecutor (subprocess), orchestrator, sandbox (setrlimit),
core/src/models/ Data models, TOML spec parsing, grading policies
core/src/discovery Student file discovery + archive extraction
core/src/runner/ PythonExecutor (subprocess), orchestrator, sandbox (setrlimit),
parametrize, expander, oracle, linter
checker/ Checker trait (8 impls) + Rhai + Python checkers
record Grading record: versioned evidence, score revisions, rescore reuse checks
db/ SQLite (rusqlite bundled): students, sessions (= revisions), results, similarity
canvas/ Canvas LMS API client (reqwest + rustls): roster pull, grades push
tui/ ratatui terminal UI: students/sessions/similarity tabs
scriptmark-py PyO3 bindings: grade, run, discover, load_spec (maturin, separate crate)
core/src/checker/ Checker trait + Rhai + Python checkers
core/src/record Versioned evidence, score revisions, rescore reuse checks
cli/src/db/ SQLite: students, sessions (= revisions), results, similarity
cli/src/canvas/ Canvas HTTP client and offline bundle adapter
cli/src/tui/ ratatui terminal UI: students/sessions/similarity tabs
```

## Key Design Decisions
Expand All @@ -110,23 +109,24 @@ Zero and withheld grades stay distinct. See the notes' teacher contract for poli

## Critical Files

- `src/crates/scriptmark/src/runner/python.rs` — PythonExecutor + embedded helper scripts (HELPER_SCRIPT, CHAIN_HELPER_SCRIPT). Contains import allowlist, env sanitization, `sandboxed_cmd()`, `spawn_with_timeout()`.
- `src/crates/scriptmark/src/runner/sandbox.rs` — SandboxConfig + setrlimit application. Platform-conditional: RLIMIT_AS skipped on macOS, RLIMIT_* constants differ between Linux (c_uint) and macOS (c_int).
- `src/crates/scriptmark/src/runner/orchestrator.rs` — Runs vars→setup→expand→oracle→execute pipeline per student, tokio parallel.
- `src/crates/scriptmark/src/models/spec.rs` — All TOML spec structs (TestSpec, TestCase, SetupStep, Parametrize, Oracle, LintConfig, CheckMethod).
- `src/crates/scriptmark/src/discovery.rs` — File discovery + ZIP archive extraction with size/count limits.
- `src/crates/scriptmark/src/record.rs` — The grading record every consumer reads (`--output`): evidence with submission/spec fingerprints, append-only score revisions, `check()` refusing evidence whose tests, submissions or matching changed.
- `src/crates/scriptmark/src/grading.rs` — GradingPolicy dispatch (templates + Rhai formulas).
- `src/crates/scriptmark/src/main.rs` — All CLI command handlers.
- `src/crates/scriptmark-py/src/lib.rs` — PyO3 bindings: grade(), run(), discover(), load_spec(), StudentResult, TestSpec classes.
- `src/core/src/runner/python.rs` — PythonExecutor and embedded harness. Starts the isolated Python subprocess and checks its output.
- `src/core/src/runner/sandbox.rs` — SandboxConfig + setrlimit application. Platform-conditional: RLIMIT_AS skipped on macOS, RLIMIT_* constants differ between Linux (c_uint) and macOS (c_int).
- `src/core/src/runner/orchestrator.rs` — Runs prepared bundles per student, tokio parallel.
- `src/core/src/models/spec.rs` — TOML test specification types.
- `src/core/src/discovery.rs` — File discovery and archive extraction with size/count limits.
- `src/core/src/record.rs` — Versioned grading evidence and score revisions, including reuse checks.
- `src/core/src/grading.rs` — GradingPolicy dispatch (templates + Rhai formulas).
- `src/cli/src/main.rs` — CLI command handlers and application orchestration.

## Conventions

- Workspace dependencies in root `Cargo.toml`, crates reference with `{ workspace = true }`
- `thiserror` for library error types, `anyhow` for CLI/binary error handling
- All checkers implement `Checker` trait in `src/crates/scriptmark/src/checker/mod.rs`
- All checkers implement `Checker` trait in `src/core/src/checker/mod.rs`
- Integration tests spawn real Python processes — need `python3` available
- `scriptmark-py` has `publish = false` (cdylib, distributed via PyPI/maturin, not crates.io)
- `scriptmark-core` exposes grading models and operations; CLI adapters import it directly, without compatibility re-exports.
- Core's `test-support` feature exposes shared graded/withheld report fixtures for adapter tests; it is enabled only by the CLI's dev-dependency.
- Publish both Rust packages with `cargo publish --workspace`, which orders workspace dependencies before their consumers.
- Platform-specific code uses `#[cfg(target_os = "macos")]` / `#[cfg(target_os = "linux")]` for rlimit types

## TOML Spec Example
Expand Down
9 changes: 3 additions & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[workspace]
resolver = "2"
members = [
"src/crates/scriptmark",
"src/crates/scriptmark-py",
"src/core",
"src/cli",
]

[workspace.package]
Expand All @@ -17,12 +17,12 @@ categories = ["command-line-utilities"]
readme = "README.md"

[workspace.dependencies]
scriptmark-core = { path = "src/core", version = "0.3.0" }
# Serialization
serde = { version = "1", features = ["derive"] }
serde_json = { version = "1", features = ["float_roundtrip"] }
toml = "0.8"
csv = "1"
quick-xml = { version = "0.37", features = ["serialize"] }

# Async
tokio = { version = "1", features = ["full"] }
Expand Down Expand Up @@ -58,9 +58,6 @@ libc = "0.2"
rand = "0.9"
rand_chacha = "0.9"

# Python bindings
pyo3 = { version = "0.24", features = ["extension-module"] }

ratatui = "0.29"
crossterm = "0.29"
rusqlite = { version = "0.34", features = ["bundled"] }
58 changes: 8 additions & 50 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,24 @@

[![CI](https://img.shields.io/github/actions/workflow/status/Acture/scriptmark/ci.yml?label=CI)](https://github.com/Acture/scriptmark/actions)
[![Crates.io](https://img.shields.io/crates/v/scriptmark)](https://crates.io/crates/scriptmark)
[![PyPI](https://img.shields.io/pypi/v/scriptmark)](https://pypi.org/project/scriptmark/)
[![License](https://img.shields.io/crates/l/scriptmark)](https://spdx.org/licenses/GPL-3.0-or-later.html)

Automated grading CLI for student programming assignments. Rust core, TOML test specifications, Python bindings via PyO3.
Automated grading CLI for student programming assignments. Rust core and CLI with TOML test specifications.

## Repository layout

```text
src/
crates/
scriptmark/ Rust library, CLI and integration tests
scriptmark-py/ Rust bindings for the Python extension
python/
scriptmark/ Python package entry point
core/ scriptmark-core: grading engine and input/record models
cli/ scriptmark: CLI, Canvas, SQLite, TUI and presentation
examples/ Runnable teacher test bundles
notes/ Private project documentation submodule
```

Run Cargo and Maturin commands from the repository root. `Cargo.toml` and
`pyproject.toml` configure both builds; the compiled Python extension is packaged
with `src/python/scriptmark/`.
Run Cargo commands from the repository root. The CLI depends on `scriptmark-core`;
the core has no CLI, network, database or terminal UI dependencies. Python 3 is still
required to execute student submissions, teacher checkers and reference solutions.
ScriptMark is distributed as a Rust CLI; it does not expose a Python package/API.

## Documentation and checkout

Expand Down Expand Up @@ -120,8 +117,7 @@ repository; ScriptMark does not carry its own synchronization script.
## Installation

```bash
cargo install scriptmark # Rust
pip install scriptmark # Python
cargo install scriptmark
```

## Quick Start
Expand Down Expand Up @@ -203,44 +199,6 @@ scriptmark roster-pull --course-id 12345
scriptmark tui grades.db
```

## Python API

```python
import scriptmark

# One-shot grading, under the assignment.toml beside tests/ (or pass assignment=...).
# freeze= keeps the generated inputs; replay= grades on ones kept earlier; output= writes
# the grading record.
results = scriptmark.grade(
["submissions/"], "tests/", freeze="output/cases.json", output="output/results.json"
)
for r in results:
if r.grade is None:
print(f"{r.student_id}: withheld ({r.reason})")
else:
print(f"{r.student_id}: {r.grade} ({r.score}/{r.max} points)")

# After editing the policy, score the record again without running anything
# (records graded from a Canvas bundle are rescored with the CLI)
change = scriptmark.rescore("output/results.json") # {"revision": 2, "changes": [...]}
first = scriptmark.load_record("output/results.json", revision=1)

# Discover student files (convenience view — drops non-submitters and orphan files)
# Keys are rendered student keys: a bare 学号 once a roster confirms it, otherwise
# `local:<token>` — the prefix means nothing has vouched for that filename token yet.
subs = scriptmark.discover(["submissions/"]) # {'local:alice': ['path/to/alice_lab5.py'], ...}

# The full input model: every student keeps an outcome, nothing is dropped
inp = scriptmark.load_input(["submissions/"], roster="roster.csv")
for s in inp["students"]:
print(s["identity"]["key"], s["state"]) # not_submitted | submitted_empty | executable
print(inp["unmatched"], inp["diagnostics"])

# Load and inspect a spec
spec = scriptmark.load_spec("tests/test_lab5.toml")
print(spec.name, spec.function, spec.num_cases)
```

## TOML Test Specs

Teachers can configure student ownership, item file patterns, function aliases and
Expand Down
26 changes: 0 additions & 26 deletions pyproject.toml

This file was deleted.

24 changes: 6 additions & 18 deletions src/crates/scriptmark/Cargo.toml → src/cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,44 +9,32 @@ repository.workspace = true
homepage.workspace = true
keywords.workspace = true
categories.workspace = true
readme = "../../../README.md"
readme = "../../README.md"

[[bin]]
name = "scriptmark"
path = "src/main.rs"

[dependencies]
scriptmark-core = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
toml = { workspace = true }
csv = { workspace = true }
quick-xml = { workspace = true }
tokio = { workspace = true }
clap = { workspace = true }
rhai = { workspace = true }
comfy-table = { workspace = true }
owo-colors = { workspace = true }
thiserror = { workspace = true }
anyhow = { workspace = true }
regex = { workspace = true }
zip = { workspace = true }
rust_xlsxwriter = { workspace = true }
libc = { workspace = true }
rand = { workspace = true }
rand_chacha = { workspace = true }
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
rusqlite = { workspace = true }
ratatui = { workspace = true }
crossterm = { workspace = true }
tar = "0.4.46"
flate2 = "1.1.10"
sevenz-rust2 = "0.23.0"
rars = "0.9.4"
bzip2-rs = "0.1.2"
lzma-rs = "0.3.0"
tempfile = "3"
sha2 = "0.10"

[dev-dependencies]
scriptmark-core = { workspace = true, features = ["test-support"] }
tempfile = "3"
zip = { workspace = true }
toml = { workspace = true }
calamine = { workspace = true }
wiremock = "0.6.5"
Loading
Loading