Skip to content

hal: Add halcompupdate to migrate .comp files to the new HAL API#4256

Draft
grandixximo wants to merge 1 commit into
LinuxCNC:masterfrom
grandixximo:comp-update-master
Draft

hal: Add halcompupdate to migrate .comp files to the new HAL API#4256
grandixximo wants to merge 1 commit into
LinuxCNC:masterfrom
grandixximo:comp-update-master

Conversation

@grandixximo

@grandixximo grandixximo commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Problem

With the new HAL types and getter/setter API (#4099, with the in-tree conversion in #4247), out-of-tree .comp components that use the legacy declaration types (float, bit, s32, u32, s64, u64, signed, unsigned) and direct pin/param assignment will stop building when the API break is performed. The HAL shared memory version has already changed, so out-of-tree components must be recompiled regardless. This is the right moment to offer a painless, automatic migration path.

Solution

A new tool, halcompupdate, rewrites legacy .comp files to the new API automatically:

Area Conversion
Declaration types float->real, bit->bool, s32->si32, u32->ui32, s64->sint, u64->uint, signed->si32, unsigned->ui32 (port is left alone, no new-style replacement yet)
Writes to out/io pins and params x = e -> x_set(e), x += e -> x_set(x + (e)), x++ -> x_set(x + 1), arrays x(i) = e -> x_set(i, e), chained a = b = e -> a_set(b_set(e)), *x_ptr = e -> x_set(e), *x_ptr reads -> (x)
#define macros bodies are rewritten too, with macro parameters correctly shadowing same-named pins
Legacy C types double/real_t -> rtapi_real, hal_bit_t -> rtapi_bool, etc. (the volatile qualifier of the legacy types is dropped on purpose, it existed for direct HAL memory access and was never right for variables; --no-c-types to skip)
Reads unchanged, the bare name (or name(idx)) works in both APIs

Pin and parameter names are never renamed, because they are part of the HAL configuration interface. Existing .hal files keep working.

Usage:

halcompupdate mycomponent.comp        # show a diff of the required changes
halcompupdate -i mycomponent.comp     # rewrite in place (keeps a .bak)
halcompupdate --check *.comp          # CI mode: exit 1 if migration is needed

Where it hooks in

halcompile now emits a deprecation warning once per legacy type used, pointing at halcompupdate(1). It is placed exactly at the spot halcompile.g reserved for it ("when we start warning ... this is where the warning goes"). Users hit the warning at compile time and get the remedy in the same message. The now-superseded deprmap/deprecated placeholders are removed. Until the in-tree conversion lands, a build emits one warning per legacy type per component, which is the intended transition signal.

Safety

  • In-place rewriting is atomic (temp file + os.replace(), original mode preserved). Backups are created with O_EXCL|O_NOFOLLOW and a unique-name fallback, so a pre-existing .bak (or a planted symlink) is never clobbered.
  • Constructs that cannot be converted safely are left unchanged with a warning instead of producing broken or silently different code: taking the address of a pin/param (&x), direct use of the legacy hal_pin_*_new/hal_param_*_new creation API, postfix ++/-- whose value is used (the setter yields the new value, postfix must yield the old), and array indices with side effects.

Validation

  • Converted all 124 in-tree components from master and compiled them with this tree's halcompile: 119 compile cleanly. The other 5 (spindle, raster, mesa_pktgyro_test, homecomp, tpcomp) reference in-tree-only headers/skeletons and fail identically for the already-converted versions when built out-of-tree.
  • Output matches the hand conversions in Draft - HAL types update and HAL isolation #4247 functionally. Remaining diffs are column alignment, deliberate pin renames done there, and two cases where the manual conversion mangled doc text (e.g. "32 bool integer" in reset.comp) which the tool preserves correctly.
  • New regression test tests/halcompile/update-api. scripts/runtests tests/halcompile gives 10/10 pass.

Docs

Migration section in docs/src/hal/comp.adoc (new-type table and _set() accessor documentation), new halcompupdate(1) manpage, and a SEE ALSO link in halcompile(1).

Dependencies

None beyond #4099 (already merged). Fully parallel to #4247: this PR provides the migration tool, the deprecation warning, and the docs, while #4247 converts the in-tree components. The two branches merge cleanly in either order (verified with git merge-tree --write-tree).

@grandixximo
grandixximo marked this pull request as draft July 20, 2026 11:21
@grandixximo

Copy link
Copy Markdown
Contributor Author

@BsAtHome If this is too ugly I'll drop it, this does most of your careful hand conversion, to alleviate the porting churn for comps out of tree, let me know how much you hate it ;-)
Keeping as draft for now....

@BsAtHome

Copy link
Copy Markdown
Contributor

The old hal types should never be used anywhere anymore. All places I've see they were used wrongly as variable types and that fails the smell-test miserably because they are volatile.
For example, hal_bit_t must be rtapi_bool when used in as a variable. hal_bit_t * becomes hal_bool_t when dealing with pins. And hal_bit_t becomes hal_bool_t when dealing with parameters, BUT, when you deal with parameters, you need to adapt the code because parameters were used as variables.

For components (.comp files), that only declare their pins and params through halcompile means, will not see a difference (except for the _set() suffix for writing). However, those constructs that add their own pins/params need more care in the conversion.

Out-of-tree .comp components using the legacy HAL types (float, bit,
s32, u32, s64, u64, signed, unsigned) and direct pin/param assignment
stop working when the HAL API break is performed (LinuxCNC#4099, LinuxCNC#4247).
halcompupdate rewrites them to the new API automatically:

* declaration types are converted: float->real, bit->bool, s32->si32,
  u32->ui32, s64->sint, u64->uint, signed->si32, unsigned->ui32
  ('port' is left alone, it has no new-style replacement yet)
* writes to out/io pins and to params become <name>_set(...) calls,
  including compound assignments, ++/--, array pins, chained
  assignments, *<name>_ptr dereferences and writes inside #define
  macros (macro parameters shadow same-named pins)
* legacy C types are modernized (double/real_t -> rtapi_real,
  hal_bit_t -> volatile rtapi_bool, ...)
* reads are unchanged and pins are never renamed, so existing HAL
  configurations keep working

Constructs that cannot be converted safely are left unchanged with a
warning for manual conversion: taking the address of a pin/param,
direct use of the legacy hal_pin_*_new/hal_param_*_new creation API,
postfix ++/-- whose value is used, and array indices with side
effects.

In-place rewriting is atomic (temp file + rename) and keeps a .bak
backup created with O_EXCL|O_NOFOLLOW.

halcompile now warns once per deprecated type, pointing to
halcompupdate(1), at the spot previously marked for this warning.
Docs: migration section in comp.adoc, new halcompupdate(1) manpage,
SEE ALSO in halcompile(1).  Regression test in
tests/halcompile/update-api.

Validated by converting all in-tree components from master: 119/124
compile (the other 5 need in-tree headers and fail identically for
the already-converted versions), and the output matches the hand
conversions in LinuxCNC#4247 functionally.
@grandixximo

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Aligned the C-type conversion with that:

  • hal_bit_t (and the other hal_t) used as variable types now convert to plain rtapi_bool/rtapi. The volatile is dropped, it was only there for direct HAL memory access.
  • hal_bit_t * and friends are left unchanged with a warning. Pointers into HAL memory become hal_bool_t refs and need the manual hal_get/hal_set treatment, the tool does not pretend to do that part.
  • Comps that only declare pins/params through halcompile get exactly the type rename plus set() treatment automatically. Comps using hal_pinnew/hal_paramnew directly are detected, their hal*_t types are kept intact, and a warning points at the manual part.

@BsAtHome

Copy link
Copy Markdown
Contributor

FWIW, I don't think we should even attempt to fix the component files automatically.

The only risk-free change is to fixup pins/params to use the new types. Everything else is intrinsically a huge risk in an auto conversion that needs to consider the context it is used in. That is not something you can do automatically. You could do considerable damage.

IMO, the risks outweigh the benefits.

@grandixximo

Copy link
Copy Markdown
Contributor Author

FWIW, I don't think we should even attempt to fix the component files automatically.

Partially I agree, but I don't like what the lack of an attempt says...

IMO, the risks outweigh the benefits.

The tool is strictly opt-in. It does not pop up and ask you to convert like the INI converter does, and doesn't convert automatically on build, it sits there until someone runs it, and its default mode only prints a diff for review.

The only risk-free change is to fixup pins/params to use the new types

Types and writes are inseparable. Converting pin out float x -> real while leaving x = 1; in the body breaks the comp immediately (assignment to a getter macro), while doing nothing keeps it compiling until the API break. So the only coherent options are mechanical conversion of both, or nothing but warnings, and IMO only the first one helps out-of-tree comps mentioned by @rene-dev at the meeting.

The tool already stays inside the risk-free zone you describe, it converts only the mechanical subset (type renames and direct writes.

The writable namespace is exactly the declared pins/params, no aliasing, no pointers beyond _ptr) and defers everything that needs context with a warning, unchanged: &pin, raw hal_pin_*_new, postfix ++/-- whose value is used, side-effect indices. Validated by converting all 124 in-tree comps and building each from a plain directory, which is the out-of-tree scenario: 119 compile cleanly. The other 5 (spindle, raster, mesa_pktgyro_test, homecomp, tpcomp) depend on in-tree-only headers or the TOPDIR skeleton and fail the same way before and after conversion, so those failures are unrelated to the tool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants