Skip to content

Add Optional 128-bit Integer Support to C++ Runtime#239

Open
AaronWebster wants to merge 9 commits into
masterfrom
int128-runtime-support
Open

Add Optional 128-bit Integer Support to C++ Runtime#239
AaronWebster wants to merge 9 commits into
masterfrom
int128-runtime-support

Conversation

@AaronWebster

@AaronWebster AaronWebster commented Dec 3, 2025

Copy link
Copy Markdown
Collaborator

Adds optional support for 128-bit UInt and Int fields (65–128 bits) to the C++ backend and runtime, using the compiler's native __int128_t / __uint128_t where available.

What this does

  • prelude.emb: UInt and Int now permit static sizes up to 128 bits (previously 64).
  • The runtime detects 128-bit support via EMBOSS_HAS_INT128 (derived from __SIZEOF_INT128__). When it is unavailable, fields wider than 64 bits fail to compile, exactly as before.
  • The concrete 128-bit type is supplied through the EMBOSS_INT128_T / EMBOSS_UINT128_T macros, which a client may override (e.g. to a library 128-bit type) before including the runtime.
  • LeastWidthInteger, BitBlock, MemoryAccessor, and ByteSwap are extended to 128 bits. Reads and writes work for full-width, non-full-width (e.g. 72/96-bit), little- and big-endian, arrays, and the text format.

Scope / limitations

  • Anonymous bits types remain limited to 64 bits (widening them would require further work in OffsetBitBlock and the constraints checker).
  • 128-bit support requires a platform/compiler with __int128 (64-bit GCC/Clang). 32-bit targets and MSVC are unaffected and continue to reject >64-bit fields at compile time.

Testing

  • Runtime unit tests for ByteSwap, MaskToNBits, IsPowerOfTwo, LeastWidthInteger, MemoryAccessor (LE/BE, non-full-width), and UIntView / IntView read/write/sign-extension at 128 bits.
  • Generated-code integration test (int128_sizes.emb) covering little-/big-endian structs, arrays, min/max, CopyFrom, out-of-range rejection, and text round-trips.

Changes since the previous review

This revision rebases onto current master and fixes correctness issues in the write/validation path that the earlier version did not handle:

  • Range-check truncation. UIntView / IntView::CouldWriteValue compared the candidate value against the field bounds after casting it to uint64_t / int64_t. With a 128-bit ValueType this truncated the value, so an out-of-range value for a 65–127-bit field was accepted, and TryToWrite then aborted (or silently truncated) instead of returning false. The comparison is now done in the widest supported integer type.
  • is_signed vs __int128. std::is_signed<__int128_t> is false under a strict -std=c++NN, so the signed lower-bound check (and negative-value text parsing) was compiled out for 128-bit signed fields. Signedness is now determined via std::numeric_limits<>::is_signed, consistent with how these methods are enabled.
  • Reverted two unrelated whitespace-only changes flagged in the earlier review, defined the 128-bit ByteSwap in terms of EMBOSS_UINT128_T for override consistency, and sorted the testdata/BUILD entry.

@AaronWebster AaronWebster self-assigned this Jan 29, 2026
@AaronWebster AaronWebster added the enhancement New feature or request label Jan 29, 2026
Comment thread runtime/cpp/emboss_defines.h
Comment thread runtime/cpp/emboss_defines.h Outdated
Comment thread runtime/cpp/emboss_cpp_types.h
Comment thread compiler/back_end/cpp/testcode/int128_sizes_test.cc Outdated
Comment thread testdata/__init__.py
Comment thread testdata/int128_sizes.emb Outdated
@AaronWebster

Copy link
Copy Markdown
Collaborator Author

@jasongraffius Ready for another look

AaronWebster and others added 7 commits July 19, 2026 15:09
- Update prelude.emb to allow UInt and Int sizes up to 128 bits
  (previously limited to 64 bits). Added documentation noting that
  128-bit support requires platform support (__uint128_t/__int128_t).

- Update constraints_test.py to test with sizes > 128 bits now that
  128-bit fields are valid.

- Add comprehensive unit tests for 128-bit integer support:
  - emboss_bit_util_test.cc: ByteSwap, MaskToNBits, IsPowerOfTwo tests
  - emboss_memory_util_test.cc: MemoryAccessor, BitBlock tests for
    128-bit values including non-full-width (72, 96 bits) and
    cross-64-bit-boundary operations
  - emboss_prelude_test.cc: UIntView and IntView tests for 128-bit
    read/write, sign extension, and CouldWriteValue

- Add int128_sizes_test.cc integration test for generated code with
  128-bit integer fields (UInt128Sizes, Int128Sizes, big-endian
  variants, and arrays).

- Update int128_sizes.emb documentation and remove AnonymousBits128
  struct (128-bit bits types require separate implementation work).
- Add EMBOSS_INT128_T and EMBOSS_UINT128_T macros for extensibility.
- Change include path in emboss_cpp_types.h to relative.
- Update copyright years to 2026 in testcode and .emb files.
- Restore testdata/__init__.py copyright header.
- Revert unrelated whitespace change.
UIntView::CouldWriteValue and IntView::CouldWriteValue range-checked the
candidate value after casting it to uint64_t / int64_t.  Once fields wider
than 64 bits are allowed, the view's ValueType is __int128 and those casts
truncate the value before it is compared against the field bounds, so an
out-of-range value whose low 64 bits happen to fall in range is accepted.
TryToWrite then proceeds into WriteUInt, which either aborts (EMBOSS_CHECK)
or silently truncates -- so the "safe" write path crashes on out-of-range
input instead of returning false.

Additionally, IntView::CouldWriteValue gated its lower-bound check on
::std::is_signed<IntT>, which is false for __int128_t under a strict
-std=c++NN (extended integer types are not "integral types" in the standard
sense).  The method is *enabled* via ::std::numeric_limits<>::is_integer, so
the signedness test is now made the same way, via
::std::numeric_limits<>::is_signed, keeping the two consistent.

Compare against the field bounds in the widest supported integer type
(MaxWidthUnsigned / MaxWidthSigned) so no truncation occurs.  Add regression
tests at a non-power-of-two width above 64 bits (72) covering both the
CouldWriteValue result and that TryToWrite fails gracefully rather than
aborting.
…pport

- emboss_defines.h: revert the incidental EMBOSS_ALIAS_SAFE_POINTER_CAST
  whitespace change that was flagged in review as unrelated to this change.
- parser_types.py: revert an unrelated trailing-whitespace-only edit.
- emboss_bit_util.h: define the 128-bit ByteSwap overload in terms of
  EMBOSS_UINT128_T rather than the raw __uint128_t, so that a client that
  overrides the type macro (e.g. to a library 128-bit type) also gets a
  consistent ByteSwap, matching how the type is used elsewhere.
- testdata/BUILD: sort int128_sizes.emb before int_sizes.emb (buildifier).
@AaronWebster
AaronWebster force-pushed the int128-runtime-support branch from 3a9942c to 357dad7 Compare July 19, 2026 22:30
DecodeInteger gated its handling of a leading '-' on ::std::is_signed<IntType>,
which is false for __int128_t under a strict -std=c++NN.  As a result a
negative value could not be parsed into a signed field wider than 64 bits (the
'-' was treated as an invalid character and parsing failed), even though the
same value encodes fine on the way out.  Use ::std::numeric_limits<>::is_signed
instead, matching the fix applied to IntView::CouldWriteValue.

Add text-format round-trip tests for values that do not fit in 64 bits,
including a negative value into a >64-bit signed field and rejection of an
out-of-range text value.

@robrussell robrussell left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs at https://github.com/google/emboss/blob/master/doc/language-reference.md#uint still say max size of Int and UInt are both 64 bits.

The different limit for the size of anonymous bit fields should be mentioned at https://github.com/google/emboss/blob/master/doc/language-reference.md#anonymous-bits .

Comment thread compiler/front_end/constraints_test.py Outdated
Comment thread testdata/int128_sizes.emb Outdated
Comment thread compiler/back_end/cpp/testcode/int128_sizes_test.cc Outdated
Comment thread compiler/back_end/cpp/testcode/int128_sizes_test.cc Outdated
-- 128-bit integers (__uint128_t). The C++ backend will use EMBOSS_HAS_INT128
-- to detect availability. On platforms without 128-bit integer support,
-- fields larger than 64 bits will fail to compile.
[static_requirements: $is_statically_sized && 1 <= $static_size_in_bits <= 128]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there's a place where a check for this this could be included at

if in_attribute and in_attribute.name.text == attributes.STATIC_REQUIREMENTS:
but I'm not sure constraints.py would have enough information to know whether the platform supports 128-bit integers.

template <int kBits>
struct LeastWidthInteger final {
static_assert(kBits <= 64, "Only bit sizes up to 64 are supported.");
using Unsigned = typename LeastWidthInteger<kBits + 1>::Unsigned;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a little confused about this (including the previously existing implementation)
If kBits is 64 then this is

using Unsigned = typename LeastWidthInteger<65>::Unsigned;

Is this using just never applicable because the specializations for 64, 32, 16, and 8 always apply? Or is there something else I'm missing?

… support

- Document the new 128-bit UInt/Int limits (and the platform-support
  caveat) in the language reference, and note the unchanged 64-bit limit
  on bits types, including anonymous bits.
- Move the bits-size note out of testdata/int128_sizes.emb into the docs.
- Reword test comments (drop "now"; -(2**120) instead of ambiguous
  -2**120) and use a hex literal in the text-format test so the value is
  easy to relate to the expected shifted constant.
- Explain the LeastWidthInteger rounding-up recursion in a comment and
  make the no-int128 static_assert message say why the width is
  unsupported.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants