Repository navigation
deflate compression method (dmcomp, a submodule) and the firmware SAL: FirmwareReady, Write/ReadPersistency - #341
Merged
Conversation
Dmod_Compression_* has a second method, "deflate" (raw DEFLATE, RFC 1951), by dmcomp (choco-technologies/dmcomp, vendored in lib/third-party/dmcomp like FastLZ is). The modules of the dmod-boot firmware are 172 kB with it, and 215 kB with FastLZ. * The decoder is 1.7 kB of code. Its tables (~3 kB) come from the heap (dmcomp_inflate_ws), not from the stack: a module is loaded from whatever thread asks for it, some of them have 4 kB of the stack. * The encoder (Dmod_Compression_Pack) allocates ~330 kB with malloc and is for the host tools. DMOD_USE_DMCOMP_ENCODER is ON when DMOD_USE_STDLIB is, so a firmware (dmod-boot has no stdlib) gets the decoder only, with no change in its configuration: +2.6 kB. * DMOD_USE_DMCOMP (ON) switches the method. * fastlz stays the first method (the default of todmfc and of DMOD_COMPRESSION_METHOD): a loader built before this cannot read deflate. todmfc's default level depends on the method: 2 for fastlz, 9 for deflate. Tests: Dmod_Compression_* with deflate on every level, against fastlz, damaged and cut data, no memory for the decoder (the tables are given back); DMFC -> DMF with both methods. The SDK installs the library with its license. Checked in the build of the Docker image (two passes, examples, tools install), a real module built as DMOD_MODE=DMOD_MODULE with DMOD_COMPRESSION_METHOD=deflate, and the dmod-boot firmware. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…sistency A platform with more than one firmware image (banks, with a boot loader that picks one) needs two things from the firmware, and the firmware needs them from the platform: * Dmod_FirmwareReady(): the firmware says that it has started properly. A firmware that is on trial stays only if it does; otherwise the platform falls back to the previous one after a reset. What "properly" means is the firmware's own decision (a console, the network, a service ...), so it is the firmware that asks. * Dmod_WritePersistency(Id, Data, Size) / Dmod_ReadPersistency(Id, Buffer, BufferSize): data that survives a reset and an update of the firmware (the settings, the secrets), kept apart from the images. An item is found by its Id; writing 0 bytes removes it; the read returns the size of the whole item (ask with a NULL buffer for the size). Like Dmod_Reboot, these are implemented by the platform; the default (weak) implementations return -ENOSYS and do nothing. Added now, together with the deflate method, so that the tools image has to be built once. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
lib/third-party/dmcomp is the submodule choco-technologies/dmcomp (7314bf2) instead of copied files, so the library has one place to be changed and tested. What builds it is lib/third-party/dmod_dmcomp (the CMakeLists.txt, and the Makefile that it writes; there are no sources in it): the same library dmod_dmcomp as before, for the CMake and the Make based builds. The checkouts of the CI workflows take the submodules, the script that builds the Docker image initializes them (the Dockerfile copies the directory), and the SDK installs the sources of the submodule where the build glue finds them. Without the submodule the configuration says what to do (git submodule update --init --recursive). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
dmcomp#2 (dmcomp_inflate_ws) is merged; the submodule was on its commit 7314bf2. The merge commit has the same tree, so nothing changes in the build. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
JohnAmadis
approved these changes
Oct 28, 2026
This was referenced Oct 28, 2026
Merged
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two additions, in two commits, made together so that the tools image (
chocotechnologies/dmod) has to be built once.1.
deflate, a compression method of the DMFC files (dmcomp)Dmod_Compression_*gets a second method, raw DEFLATE (RFC 1951) from choco-technologies/dmcomp (vendored inlib/third-party/dmcomplike FastLZ, with its license; the SDK installs it). The 35 modules of the dmod-boot firmware are 172 kB with it, 215 kB with FastLZ (zlib -9 gives the same as dmcomp).Dmod_Malloc(dmcomp_inflate_ws), not from the stack: a module is loaded from whatever thread asks for it, some of them have 4 kB of stack.Dmod_Compression_Pack): ~330 kB ofmalloc, for the host tools.DMOD_USE_DMCOMP_ENCODERis ON whenDMOD_USE_STDLIBis, so a firmware (dmod-boot has no stdlib) gets the decoder only without any change in its configuration (+2.6 kB of code; checked by building the firmware).DMOD_USE_DMCOMP(ON) switches the method.fastlzstays the first method, the default oftodmfcand ofDMOD_COMPRESSION_METHOD– a loader built before this cannot readdeflate. Use it withtodmfc in.dmf out.dmfc deflate(level 9) orDMOD_COMPRESSION_METHOD=deflate.2. SAL:
Dmod_FirmwareReady,Dmod_WritePersistency,Dmod_ReadPersistencyFor a platform with more than one firmware image (banks) and a boot loader that picks one:
int Dmod_FirmwareReady(void)– the firmware says that it has started properly; one that is on trial stays only if it does, otherwise the platform falls back to the previous one after a reset. What "properly" means is the firmware's decision (a console, the network, a service ...).int Dmod_WritePersistency(uint32_t Id, const void* Data, size_t Size)– data that survives a reset and a firmware update, found byId; writing 0 bytes (DataNULL) removes the item.int Dmod_ReadPersistency(uint32_t Id, void* Buffer, size_t BufferSize)– returns the size of the whole item (aNULLbuffer asks for the size only),-ENOENTwhen there is none.Like
Dmod_Reboot, they are implemented by the platform; the default (weak) implementations return-ENOSYSand do nothing (src/system/if/dmod_if_firmware.c).Tested
Dmod_Compression_*withdeflateon every level, againstfastlz, damaged and cut data, no memory for the decoder (and the tables are given back); DMFC → DMF with both methods; the-ENOSYSdefaults. The four tests that fail locally (tests_dmod_ctx,_ldr,_mgr,_dmfc_api: no test module built) fail the same on a cleandevelop.cmake --install --component tools), the Make based build of the library in both modes, the CI flow (MODULE then SYSTEM mode,dmod_loaderon the example modules), a real module (dmffs) built asDMOD_MODE=DMOD_MODULEwithDMOD_COMPRESSION_METHOD=deflate,dmod_loaderloading adeflate.dmfc, the dmod-boot firmware for the STM32F746G-DISCO.Notes
0126ec1, the merge commit of dmcomp#2 on dmcomp'smain(dmcomp_inflate_ws, which the decoder needs).--recursive(orgit submodule update --init --recursive; the configuration says so when the directory is empty). The CI checkouts take the submodules,Docker/build.shinitializes them before it builds the image (the Dockerfile copies the directory), and a project that has dmod as a submodule (dmod-boot) has to initialize the nested one.Dmod_FromDMFConly warns when the decompressed size differs from the header's, so a cutfastlzfile loads; a cutdeflateone is refused.🤖 Generated with Claude Code