fix: huggingface_hub 1.30.0, bookworm base images, Reolink legacy TLS - #412
fix: huggingface_hub 1.30.0, bookworm base images, Reolink legacy TLS#412MateoLostanlen wants to merge 9 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## fix/reolink-legacy-tls #412 +/- ##
=======================================================
Coverage 79.46% 79.46%
=======================================================
Files 6 6
Lines 852 852
=======================================================
Hits 677 677
Misses 175 175
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…is not pyro_predictor imports huggingface_hub (hf_hub_download, in vision.py) but never declared it. It resolves today only because pyroengine drags it in — an undeclared dependency that breaks the moment pyro_predictor is installed on its own, which is the point of having extracted it. pyroengine pins huggingface_hub==0.23.1 and imports it nowhere: zero occurrences across __init__, core, engine, sensors and utils. The pin is a leftover from before the split, and being exact it propagates to every consumer. It makes pyroengine uninstallable alongside anything needing a modern huggingface_hub — pyro-train requires >=1.8.0, and uv reports the requirements unsatisfiable. Declared as >=0.23.1 rather than pinned: the single call site uses hf_hub_download(repo_id=..., filename=..., local_dir=...), whose signature is unchanged from 0.23 through 1.x — verified against 1.8.0.
CI runs `uv sync --locked`, so changing pyproject.toml without relocking fails the build before it installs anything. The resolution is unchanged — huggingface-hub stays at 0.23.1, since `>=0.23.1` is satisfied by what was already locked. Only the declaration moves, from pyroengine to pyro-predictor. Consumers needing a newer version are now free to resolve one; this repo does not force it on them.
hf_hub 1.x logs its HTTP requests through httpx at INFO level, so the verbose=False test now filters caplog to the pyro_predictor logger it actually targets.
The runtime image stage has no git; pyroclient installs from requirements-git.txt in the git-deps stage.
b554d0b to
2fd7768
Compare
fe51
left a comment
There was a problem hiding this comment.
Hi @MateoLostanlen , thanks for the PR, it looks good overall — mypy clean, sensors tests pass, and dropping the huggingface_hub pin is safe (I grepped: pyroengine never imports it, and hf_hub_download(repo_id=, filename=, local_dir=) is unchanged in 1.x).
Main question: was requirements.txt regenerated by hand rather than with make lock? The header comment records only the uv export half of the recipe — Makefile:27 also greps:
grep -v 'git+' ... | grep -v '^opencv-python==' > requirements.txtTwo consequences, unless you hit something that made make lock unusable:
opencv-python==4.13.0.92is back — a 46.7 MB aarch64 wheel that #338 deliberately stripped for 4G deploys. It now also sits next toopencv-python-headless; both installcv2/, so whicheveruvwrites last wins. Was that reintroduction intentional?--no-emit-package pyroclientlooks redundant —grep -v 'git+'already covers it, and since the Makefile wasn't touched the nextmake lockreverts this commit. Did you see a case where it still leaked through?
If not, make lock should fix both. (Heads-up: the orphan # via ncnn comments on develop are correct Makefile output, not a bug.)
Non-blocking:
-
The new TLS test never runs. CI and
make testonly runtests/; from the repo root it fails to import (No module named 'pyro_camera_api.camera') and needsPYTHONPATH=pyro_camera_api. Pre-existing for all 7 files in that dir — do you want to wire the directory into CI here, or open a separate issue? -
Was the caplog filter meant as the fix for
verbose=False, or just to unblock CI?vision.py:28sets a root handler at INFO andverbose=Falseonly lowers thepyro_predictorlogger, so hf_hub's httpx lines still print at runtime. If suppressing them is in scope, one line in__init__does it:logging.getLogger("httpx").setLevel(logging.WARNING)
-
Was the module-level
requests.Sessiondeliberate? No credential bleed (auth is in the URL, not session state), but it means one connection pool for every camera, shared across patrol/stuck-check threads and FastAPI's threadpool, and keep-alive now persists where each call used to get a fresh socket. Withmax_retries=0in requests, a socket the camera drops surfaces asConnectionError— old Reolink firmware is exactly the population that does that. Patrol catches it, so worst case is intermittent skipped poses.self._sessionin__init__would remove all three, if there's no reason to share it. -
Was the
hf-xetdependency considered? hf_hub 1.x pulls it on aarch64 and downloads viatransfer.xethub.hf.coinstead ofcdn-lfs.huggingface.co— a new host for egress-restricted sites. If it wasn't planned,HF_HUB_DISABLE_XET=1is the escape hatch.
…/huggingface-hub-dependency
Drops the hand-edited export: opencv-python (stripped on purpose since #338) is filtered out again and the redundant --no-emit-package pyroclient flag is gone from the header, as requested in review.
331507a to
5ae7a57
Compare
|
Thanks @fe51, good catches all around. 1 and 2: you were right, the export was done by hand. Regenerated with 3 and 5: the TLS test and the Reolink session moved to #414, which we will merge first (this branch now includes it, so the diff will collapse to the dependency changes once it lands). On the session, #414 sends 4: the caplog filter was just to unblock CI. Runtime fix added: 6: hf-xet was not planned, good catch. Not blocking for us though: our sites do not filter this egress, except potentially the Ardèche one, and that pipe needs fixing anyway so we will handle it during the redeploy. We will just take the new default, and |
…/huggingface-hub-dependency
Dependency cleanup around
huggingface_hub.pyro_predictornow declareshuggingface_hub(it imports it invision.pybut relied onpyroenginepulling it in), andpyroenginedrops its unused==0.23.1pin that blocked every consumer needing a recent version (pyro-train for instance).huggingface_hubupgraded to 1.30.0 (latest), floor set to>=1.30.0. The only call site,hf_hub_download(repo_id=, filename=, local_dir=), is unchanged in 1.x.verbose=Falsenow also silences hf_hub's httpx request logs at runtime, and the test filters caplog to thepyro_predictorlogger.requirements.txtregenerated withmake lock(review feedback):opencv-pythonstays stripped as since Optimize Docker images for size and 4G deploy bandwidth #338, and the redundant--no-emit-package pyroclientflag is gone.Stacked on #414 (bookworm base images, Reolink legacy TLS): merge that one first, this diff then collapses to the dependency changes only.