Skip to content

Add C training driver plus worked MNIST example (closes #44) #56

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
w4ffl35 merged 1 commit into from
Oct 14, 2026
Merged

Add C training driver plus worked MNIST example (closes #44) #56

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ jobs:
run: cmake --build build --parallel
- name: Test
run: ctest --test-dir build --output-on-failure
- name: Train with no Python on PATH
run: >-
PATH=/nonexistent ./build/examples/mnist_train
examples/mnist_tiny_graph.json
tests/fixtures/mnist_tiny_images.idx
tests/fixtures/mnist_tiny_labels.idx
./build/mnist_nopython_checkpoint.json 2

sanitizers:
runs-on: ubuntu-latest
Expand Down
3 changes: 3 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,12 @@ add_library(nir_c_runtime
src/checkpoint.c
src/minibatch.c
src/tape.c
src/driver.c
src/driver_loop.c
)
target_include_directories(nir_c_runtime PUBLIC include)
target_link_libraries(nir_c_runtime PUBLIC m)

enable_testing()
add_subdirectory(tests)
add_subdirectory(examples)
81 changes: 61 additions & 20 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,29 +84,70 @@ target-specific kernels remain future optimization work.
spike encoders (`encode.h`), IDX MNIST dataset loading (`data.h`),
and versioned checkpoint save/load with weight accessors and
summary export (`checkpoint.h`), all covered by C tests.
- [x] Backward ops for spatial/linear nodes (Conv2d, Conv1d, Flatten,
AvgPool2d, SumPool2d, Scale) and spiking neurons (LIF/IF/CubaLIF
with surrogate through the spike output, zero through the
hard-reset branch).
- [x] Graph-level BPTT tape (`tape.h`): retains per-step activations
and neuron state, backpropagates through Delay/LI recurrence,
matches `nir_runtime_run` bit for bit, finite-difference
verified on smooth topologies.
- [x] Mini-batch training (`minibatch.h`): mean loss/gradient
reduction over a batch-1 loop (no batch axis on the tensor
ABI) plus seeded Fisher-Yates epoch shuffling.
- [x] Optimizer-state checkpoints: v2 `optimizers` section persists
SGD velocity and Adam m/v/step per node for bit-identical
resume; v1 files still load.
- [x] Weight initializers (`init.h`: Xavier/He/uniform from
`nir_rng_t`), LR schedules (`schedule.h`: step/cosine/
exponential mirroring torch), and extra losses (`loss.h`:
MSE plus spike-count cross-entropy).
- [x] End-to-end C training driver (`driver.h`) plus a worked MNIST
example (`examples/mnist_train`): loads IDX, encodes, runs
the tape, steps SGD/Adam per batch, saves optimizer
checkpoints — zero Python at train time (see Training).

## Training

The driver wires the tested units into one loop; no Python or
torch is involved after the fixtures are committed:

```text
IDX MNIST (data.h) → encode (encode.h: repeat/rate)
→ tape forward (tape.h, unprimed per sample)
→ cross-entropy (train.h) → tape backward
→ minibatch mean (minibatch.h) → SGD/Adam step
→ checkpoint with optimizer state (checkpoint.h)
```

Build and run the worked example (tiny 2×2, 3-sample fixture,
4-step LI network, 20 Adam epochs):

```sh
cmake -S . -B build
cmake --build build --parallel
./build/examples/mnist_train \
examples/mnist_tiny_graph.json \
tests/fixtures/mnist_tiny_images.idx \
tests/fixtures/mnist_tiny_labels.idx \
/tmp/mnist_tiny_checkpoint.json 20
```

Expected output (deterministic from seed 7): initial loss
~2.39 at 0.000 accuracy, falling to ~1.16 at 0.667 accuracy
by epoch 20, saving a v2 checkpoint with `step 40` (2 batches
× 20 epochs). `tests/test_driver.c` asserts the same run beats
its initial loss by 0.5 and resumes bit-identically from the
saved checkpoint.

Zero-Python proof: the example is a pure C binary — CI runs it
via ctest and again with `PATH=/nonexistent` (no `python` or
`python3` resolvable), and it still trains and saves. The only
Python in the repo (`tools/`) regenerates fixtures; training
and inference never invoke it.

## Out of scope (for now)

- Full training pipelines. The training math and training IO above are
supported; still missing are the graph-level backprop-through-time tape
([#39](https://github.com/Capsize-Games/nir-c-runtime/issues/39)),
backward ops for spatial/linear
([#40](https://github.com/Capsize-Games/nir-c-runtime/issues/40)) and
spiking-neuron
([#41](https://github.com/Capsize-Games/nir-c-runtime/issues/41)) nodes,
mini-batch training
([#42](https://github.com/Capsize-Games/nir-c-runtime/issues/42)),
optimizer-state checkpoint resume
([#43](https://github.com/Capsize-Games/nir-c-runtime/issues/43)), and an
end-to-end C training driver with a worked example
([#44](https://github.com/Capsize-Games/nir-c-runtime/issues/44)).
Weight initializers
([#45](https://github.com/Capsize-Games/nir-c-runtime/issues/45)),
learning-rate schedules
([#46](https://github.com/Capsize-Games/nir-c-runtime/issues/46)), and
additional loss functions
([#47](https://github.com/Capsize-Games/nir-c-runtime/issues/47)) are filed
as follow-ups.
- General Curlee/JOE graph execution. The narrow, allocation-free event ABI is
integrated as the reference path for
[curlee#316](https://github.com/w4ffl35/curlee/issues/316); loading arbitrary
Expand Down
12 changes: 12 additions & 0 deletions examples/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
add_executable(mnist_train mnist_train.c)
target_link_libraries(mnist_train PRIVATE nir_c_runtime)

add_test(
NAME mnist_train_example
COMMAND mnist_train
${CMAKE_SOURCE_DIR}/examples/mnist_tiny_graph.json
${CMAKE_SOURCE_DIR}/tests/fixtures/mnist_tiny_images.idx
${CMAKE_SOURCE_DIR}/tests/fixtures/mnist_tiny_labels.idx
${CMAKE_BINARY_DIR}/mnist_tiny_checkpoint.json
20
)
241 changes: 241 additions & 0 deletions examples/mnist_tiny_graph.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
{
"nodes": [
{
"name": "input",
"kind": "Input",
"params": {}
},
{
"name": "flatten",
"kind": "Flatten",
"params": {}
},
{
"name": "fc1",
"kind": "Affine",
"params": {
"weight": [
[
-0.24913809321871228,
-0.4937738335497391,
0.2134535788025974,
-0.6046664021733632
],
[
0.05074481713564971,
-0.18994455527924115,
-0.6250839151671003,
0.010515714722665881
],
[
-0.6540799124877833,
-0.09383917408664855,
-0.6083162937620042,
-0.5788192074319221
],
[
-0.10674598641357513,
0.46223870760165797,
-0.5320243686803318,
-0.39139920978918025
],
[
0.1802177914228914,
0.6331560584184129,
0.10904003563382247,
-0.14611587400681747
],
[
0.6735264294789381,
-0.6412289224852223,
0.506950956469627,
-0.29753740066709794
],
[
-0.503099285861198,
-0.5405234005538385,
-0.2708476017960004,
0.4470701844911761
],
[
-0.45152108361952253,
0.11540005814332399,
0.196453311751703,
-0.1804571256694043
]
],
"bias": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
]
}
},
{
"name": "li1",
"kind": "LI",
"params": {
"tau": 2.0,
"r": 1.0,
"v_leak": 0.0
}
},
{
"name": "fc2",
"kind": "Affine",
"params": {
"weight": [
[
0.055130560259456174,
-0.5048478059836482,
-0.5085287661415836,
-0.33952963261325175,
0.20830794615661075,
-0.08360920362634078,
-0.21460436242522613,
0.09879832985700243
],
[
-0.05405802580922614,
-0.23120915652280033,
0.3399201458018255,
0.22977897976201023,
-0.29549189684225907,
0.08593709830386886,
0.0290944165163578,
0.4331714681048824
],
[
0.2649405992440478,
-0.244868377051209,
0.5544581549158596,
-0.44101965147445565,
-0.09454362176557207,
0.2969207698083346,
-0.4018536451918209,
-0.012744313822624975
],
[
-0.5320776283685758,
0.19423894010419307,
0.3055001216554002,
0.08432309255394632,
0.43356443147059,
-0.2150658471887773,
0.2255076645791675,
0.1089689478999486
],
[
0.092255035398914,
-0.050569727524399255,
0.3925609791894308,
0.5134734999282105,
-0.029908663726454288,
0.18954664003774813,
-0.5072952484799551,
0.23266294547934185
],
[
0.1698897675342217,
0.5693781467747452,
0.3717267244158634,
-0.24872765506018207,
-0.13187668289430077,
0.19474338183059503,
-0.5512968440164253,
-0.044230473532123504
],
[
-0.38330471559247825,
-0.4421396922596237,
-0.5092755694479254,
0.30972877620029526,
-0.42800104519055915,
-0.29142928740911,
-0.12592043650247992,
0.42888115348952227
],
[
-0.48430299731052506,
-0.05867333548025655,
0.057088289706041295,
0.44269351079791197,
0.36867260064257057,
0.42029306312256176,
-0.2558573160993044,
-0.09780715717824934
],
[
-0.16307701142624714,
0.4436276644072279,
0.5285424676502037,
-0.4030818180199479,
-0.3738715632298264,
-0.30950955019234344,
-0.3079169677400293,
-0.01736354337058954
],
[
0.10291095774188086,
-0.27395660642827574,
-0.5726233831569812,
-0.09359251878810587,
-0.15097296976962415,
0.07660424673050914,
0.5231924185423495,
0.21996312845274624
]
],
"bias": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
]
}
},
{
"name": "output",
"kind": "Output",
"params": {}
}
],
"edges": [
{
"source": "input",
"target": "flatten",
"delayed": false
},
{
"source": "flatten",
"target": "fc1",
"delayed": false
},
{
"source": "fc1",
"target": "li1",
"delayed": false
},
{
"source": "li1",
"target": "fc2",
"delayed": false
},
{
"source": "fc2",
"target": "output",
"delayed": false
}
]
}
Loading
Loading