Hand the trained model to the prediction in memory, keeping a file only when asked

[Flix6x]

Description

Closes #2111 and #2683.

Every forecast cycle pickled its trained model to a folder, only for the prediction to read it back in the same job a moment later. Nothing in production deleted it (delete_model was only ever set by tests), so the folder kept one model per sensor and cycle number. A single model can be hundreds of MB: one development checkout held 589 MB in 12 files, the largest 330 MB. The default folder, flexmeasures/data/models/forecasting/artifacts/models, was also relative, so a worker created it under whichever directory it was started from. And since files were named by sensor and cycle number only, two runs for the same sensor at the same time shared file names.

The file had no other reader: no documentation mentions it, the API cannot set the folder, and #2479's parallel per-horizon training fits its models in threads, so no model crosses a process boundary through the file.

• TrainPipeline.run returns the trained model, and PredictPipeline accepts it (model=), reading a model file only when none is handed over. A pipeline given neither says so, rather than failing to open None.
• model-save-dir defaults to None. A model is kept as a file only when the option is given, so flexmeasures add forecasts --model-save-dir DIR keeps one model per cycle in DIR, as before.
• Deleting a kept model after prediction (delete_model) still works, and is skipped when no file was kept.
• Added changelog items in documentation/changelog.rst and documentation/cli/change_log.rst, with a note that hosts can delete the old artifacts folder.

How to test

pytest flexmeasures/data/tests/test_forecasting_jobs_fresh_db.py -k keeps_its_model

A forecast runs in an empty working directory, since the old default folder was relative to it.

• [False]: without model-save-dir, the forecasts are made and no model file is written anywhere. On main, this fails: two model files appear under the working directory.
• [True]: with model-save-dir, one model per cycle is kept there, and it can be read back.

Full suite: 2973 passed, 4 xfailed; the only failures are test_closest_sensor[1] and [3], which fail the same way on main in my sandbox, because a bare postgres:16 lacks the distance functions they call.

Related Items

Closes #2111 and #2683. #2683 asked why the model was written at all; this PR answers both from the same change.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 flexmeasures | 🛠️ Build #35065614 | 📁 Comparing d75dfa5 against latest (b8aa005)

  🔍 Preview build  

7 files changed · ± 7 modified

± Modified

• changelog.html
• _autosummary/flexmeasures.data.models.forecasting.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.predict.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.train.html
• _autosummary/flexmeasures.data.models.forecasting.pipelines.train_predict.html
• api/v3_0.html
• cli/change_log.html

[Copilot]

🔵 Needs a closer look

It changes a default behavior across the core forecasting train/predict/service flow, so a human should confirm the operational impact of no longer persisting models by default even though the diff itself is clean and well-tested.

0 open findings

What changed in this PR

This PR changes the forecasting pipeline so a trained model is handed from training to prediction in memory within a single cycle, rather than being pickled to a directory and read back by the same job. Writing a model to disk becomes opt-in via the existing --model-save-dir CLI option, which now defaults to None. This closes #2111 (model .pkl files piling up in a relative, un-cleaned default folder) and #2683 (the file was merely the transport between the two halves of a cycle, not an artifact anyone asked for).

Changes:

• TrainPipeline.run now returns the trained model and writes a file only when model_save_dir is set; PredictPipeline accepts model= and reads a file only when no model is handed over, raising a clear error when given neither.
• model-save-dir defaults to None (schema + removal of the dead default-read in resolve_config); ForecastCycleResult.model_path is now optional, and the delete_model cleanup in the service is guarded by model_path is not None.
• Added a parametrized fresh_db test proving no model file is written by default and two are kept (and re-readable) when model-save-dir is given, plus changelog entries in both changelogs with an upgrade note that hosts can delete the old folder.

File	Description
flexmeasures/​data/​models/​forecasting/​pipelines/​train.py	run returns the model; file save gated on model_save_dir is not None; docstrings updated.
flexmeasures/​data/​models/​forecasting/​pipelines/​predict.py	New model= arg; load_model returns the handed-over model, else loads the file, else raises a clear error.
flexmeasures/​data/​models/​forecasting/​pipelines/​train_predict.py	compute_cycle passes model= and a conditional model_path; ForecastCycleResult.model_path made optional.
flexmeasures/​data/​services/​forecasting.py	delete_model cleanup skipped when no file was kept (model_path is not None).
flexmeasures/​data/​schemas/​forecasting/​pipeline.py	model-save-dir default → None; removed dead schema-default fallback; updated description.
flexmeasures/​data/​models/​forecasting/​__init__.py	Updated _clean_parameters comment for model-save-dir.
flexmeasures/​data/​tests/​test_forecasting_jobs_fresh_db.py	New parametrized test covering keep/no-keep behavior; fails on main.
documentation/​changelog.rst, documentation/​cli/​change_log.rst	Changelog entries (PR #2701) with host upgrade note.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[BelhsanHmida]

Nice, clean change.

I tried it locally with the CLI: without --model-save-dir, no model file is written, and with it, one model per cycle is kept in the given folder.

I pushed two small fixes on top:

• typed the model handed from training to prediction (BaseModel)
• dropped the old model-save-dir path from the forecasting tests, which still left .pkl files in the working tree

Approving.