Support for JPEG XL (JXL) images by winscripter · Pull Request #3153 · SixLabors/ImageSharp

winscripter · 2026-07-15T09:41:42Z

Prerequisites

I have written a descriptive pull-request title
I have verified that there are no overlapping pull-requests open
I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
I have provided test coverage for my change (where applicable)

Description

This is a work-in-progress PR whose goal is to introduce decoding and encoding of JPEG XL (*.jxl) images.

Reference software
I use libjxl as reference. See https://github.com/libjxl/libjxl.

Performance
I will begin by applying light optimizations as I implement parts of the JPEG XL codec. Once the codec seems complete enough to handle decoding and encoding of JPEG XL images, I will apply heavier optimizations. Examples include but are not limited to stack allocation, array pooling, and SIMD.

Other components
The JPEG XL codec, additionally, uses the LZ77 and Brotli compression codec. I will also have to implement those eventually.

Implementations
The JPEG XL codec lives under src/ImageSharp/Formats/Jxl.

Brotli and LZ77 implementations will live under src/ImageSharp/Compression.

Testing
I will start adding tests whenever the codec is complete enough to handle decoding of JPEG XL images.

Additionally, JPEG XL reference software, libjxl, contains its own tests too, which I might also implement without modification.

CLAassistant · 2026-07-15T09:41:49Z

All committers have signed the CLA.

Implementation of ac_strategy.h and ac_strategy.c

For now JxlMemoryManager will be a wrapper around MemoryPool<T>.

Implementation of image.h and image.c; AC strategy implementation was slightly adjusted to reduce errors.

This is an implementation of field_encodings.h. Note that I avoided implementing EnumValid() and Values() functions, as we have dedicated methods in .NET to do exactly that (Enum.IsDefined, Enum.GetValues)

Implementation of spline.h

Implemented ANS constants

winscripter · 2026-07-16T07:37:16Z

While I'm working on this, I'd like to note something important.

Libjxl is licensed under the BSD 3-Clause license, and since I'm using libjxl code as reference, that means the license must be included.

I'm not really sure what would be the proper way to include the license. I might place the LICENSE.txt file in the Jxl folder or add a README linking to the libjxl repo.

See ans_common.h

It is too large for a struct.

See ans_common.h

Add JxlAnsEntry and JxlAnsSymbol. See ans_common.h. These correspond to the Entry and Symbol structures within AliasTable.

Currently, there's a VarLenUint8/VarLenUint16 as well as histogram parsing implementation. I will additionally have to implement parsing of ANS codes, uint config and LZ77 parameters.

gfoidl · 2026-08-01T21:05:54Z

+
+    public static bool ReadPermutation(int skip, int size, Span<int> order, JxlBitReader bitReader, JxlAnsSymbolReader reader, Span<byte> contextMap)
+    {
+        Span<uint> lehmer = stackalloc uint[size];


Is size limited to some maximum value?
If so, there should be an assert here to make it clear.

For the stackalloc it's better to use a constant value, then slice it if needed. Thus produces most of the time better code. E.g. stackalloc uint[128].Slice(0, size).

Although neither the reference implementation nor this implementation explicitly bounds size here, it is constrained by the JPEG XL bitstream syntax. size is derived from the transform strategy and can only take one of the specification-defined coefficient counts (64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768, or 65536). The maximum possible value is therefore 65536.

I'll add an assert.

See inverse_mtf-inl.h

- Add decoding of Huffman Codes (see dec_huffman.cc and dec_huffman.h) - Add constructors to JxlImage3* classes - Make JxlColorCorrelationMap.Create 'xyb' parameter use true as a default value - Prototype of DCT quant weight parameters - Add passes shared state (see passes_state.cc and passes_state.h) - Add prototype for image operations (see image_ops.cc and image_ops.h) - Simplify inverse MTF (Move to Front) transform - Add patch context (see patch_dictionary_internal.h) - Add prototype of quantizer weights - Add 2nd prototype of ANS entropy decoding (see dec_ans.cc and dec_ans.h) - Add prototype of patch dictionary decoding (see dec_patch_dictionary.cc and dec_patch_dictionary.h)

This massively reduces number of syntax errors

- Add a slice & assert to JxlHuffmanDecoder alphabetSize to allocate at most 256 items - Use [0, 0] instead of stackalloc[2] followed by Clear() in JxlAnsReader - Add assert to Butteraugli ComputeKernel method & use float for Butteraugli Wmul & use InlineArray - Add transpose. - Note: transpose is scalar, it doesn't support SIMD yet - Add shared constants & file signature - Improve while loop in JxlImageOperations.Mirror - Floating-point Discrete Cosine Transform (1D and 2D) - Add an inline array of 2 items Source files implemented from libjxl with this commit: - dct-inl.h - dct_block-inl.h - transpose-inl.h

gfoidl · 2026-08-12T13:47:58Z

+                Vector<float> in1 = new(aIn1[(i * sz)..]);
+                Vector<float> in2 = new(aIn2[((n - i - 1) * sz)..]);
+                (in1 + in2).CopyTo(aOut[(i * sz)..]);


Is this safe?
Vector<T>'s size is dependent on the hardware, so not (really) fixed size.
Should it rather be Vector128<float>?

Yup, it should be fixed-size vector. Libjxl reference configures vector sizes to be equal to the sz parameter which I didn't spot.

The only issue is, what would be the right way to do this? The sz parameter can be equal to 1, 2, 4, 8, 16, 32, 64, or 128. In order to achieve good performance using SIMD for discrete cosine transform, I'd have to duplicate the code multiple times: scalar (1), Vector64 (2), Vector128 (4), Vector256/128 (8), Vector512/256/128 (16), and the rest with nested loops, which is what libjxl seems to do. It would result in a large source file. Is it fine?

Unfortunately duplication is at the moment the only way, as something like dotnet/runtime#76244 hasn't landed so far.

Or something like TensorPrimitives is used (I think all needed APIs are there).
But I'll let @JimBobSquarePants say a word about having such a dependency.

I’ve got an inline copy of the tensor primitives code for several methods in main already. (Tweaked with custom inlining rules for perf) and we should use them where applicable.

We should be trying to use existing code as a guide where possible. For example DCT is a function of JPEG. Can it be solved in the same way?

I strongly advise ensuring this branch is updated from main.

I strongly advise ensuring this branch is updated from main.

Done.

We should be trying to use existing code as a guide where possible. For example DCT is a function of JPEG. Can it be solved in the same way?

Technically yes, but there's a problem. JPEG only supports 8x8 blocks, while JPEG XL supports 2x2 all the way up to 256x256. So using JPEG's DCT wouldn't be possible for JPEG XL's. The JPEG DCT implementation also seems to use the Arai, Agui and Nakajima variant of the DCT which is incredibly fast and relies on approximations, but there's only a 4x4 and 8x8 variant of these.

I’ve got an inline copy of the tensor primitives code for several methods in main already. (Tweaked with custom inlining rules for perf) and we should use them where applicable.

That's awesome, I've worked with the TensorPrimitives class before and I think I understand how it works. However, the copy unfortunately doesn't have a Subtract method, which is important for the discrete cosine transform.

Technically yes, but there's a problem. JPEG only supports 8x8 blocks, while JPEG XL supports 2x2 all the way up to 256x256. So using JPEG's DCT wouldn't be possible for JPEG XL's. The JPEG DCT implementation also seems to use the Arai, Agui and Nakajima variant of the DCT which is incredibly fast and relies on approximations, but there's only a 4x4 and 8x8 variant of these.

Very good to know!

TensorPrimitives class before and I think I understand how it works. However, the copy unfortunately doesn't have a Subtract method, which is important for the discrete cosine transform.

We can copy that easily enough... However, it's very likely i'll be introducing the real thing soon enough. I expect us to start moving to .NET 10 and starting on v5 in the next few weeks.

- Implemented decode.cc - Added prototype of JXL image info - Made JxlBitReader use ReadOnlyMemory - Made changes to ICC codecs, changing accessibility of a few methods from private to internal and exposing IccDataReader's index - Added JxlMemoryWriter as a MemoryAllocator alternative to MemoryStream

winscripter · 2026-08-15T12:02:25Z

I've implemented the JxlDecoderCore prototype, but in doing so I had to change a few things.

JPEG XL reference, libjxl, as its own ICC parsing code (icc_codec.cc, icc_codec.h, icc_codec_common.cc and icc_codec_common.h). Of course, it's best to reuse existing components rather than reinventing the wheel, and I noticed that ImageSharp contains an ICC implementation too.

However, after parsing the ICC profile, the codec needs to know how many bytes of ICC were actually read, which by default the ImageSharp ICC codec did not expose.

So I had to make the following changes.

https://github.com/winscripter/ImageSharp/blob/jxl-support/src/ImageSharp/Metadata/Profiles/ICC/DataReader/IccDataReader.cs#L30-L33

I had to add a public Index property here so we can know how many bytes were actually read.

Also:

https://github.com/winscripter/ImageSharp/blob/jxl-support/src/ImageSharp/Metadata/Profiles/ICC/IccReader.cs#L56

https://github.com/winscripter/ImageSharp/blob/jxl-support/src/ImageSharp/Metadata/Profiles/ICC/IccReader.cs#L82

I had to change the accessibility of these methods from private to internal.

Let me know if that's fine.

gfoidl · 2026-08-15T14:08:00Z

+            // TODO: optimize this? ImageSharp ICC doesn't support spans
+            // so we need to allocate an array.


👍🏻

IMO this could also be done post this PR (but then an issue should be filed for this).

I'm currently working on changing the JPEG XL input access so we don't have to copy the entire stream into an array to continue decoding from and just use the stream directly (the copy approach can use a lot of memory for larger files, especially animated JXLs), so later the ICC code here won't use spans and will just pass the stream directly.

Still, ICC only operates on byte arrays. However, the ICC decoding methods are quite abstract, so using Stream for ICC won't be as difficult as with spans (because a class needs to be a ref struct so we can store a Span inside).

JimBobSquarePants · 2026-08-17T02:37:18Z

Hey @winscripter

Could you please do me a favour and grant me full write access to your fork? I want to help with this, but Git LFS can cause problems on GitHub which block me being able to push.

I'll spend some time reading and documenting methods to start with.

winscripter · 2026-08-17T11:16:15Z

Hello @JimBobSquarePants I've added you as a collaborator to the repo.

I want to help with this

I appreciate that!

JxlBoxHeader contains a header for boxes in the JPEG XL container. JxlFileTypeBox represents the ftyp box. BinaryUtils contains helper methods to read/write primitives from/to Stream in custom endianness

gfoidl · 2026-08-18T08:05:44Z

+    /// </summary>
+    /// <param name="typeString">Input type string to convert</param>
+    /// <returns>Unsigned integer representation of the type string</returns>
+    public static uint TypeFromString(string typeString)


Is the possible set of typeStrings fixed, i.e. known in advance?
If so the uint type code could be returned via a switch instead of going the route through UTF-8. Or any other kind of lookup.

gfoidl · 2026-08-18T08:06:16Z

+        Span<byte> buffer = stackalloc byte[4];
+        BinaryPrimitives.WriteUInt32BigEndian(buffer, typeCode);
+
+        return Encoding.ASCII.GetString(buffer);


gfoidl · 2026-08-18T08:08:24Z

+namespace SixLabors.ImageSharp.Formats.Jxl.IO;
+
+
+/// <summary>


Nit:

Suggested change

namespace SixLabors.ImageSharp.Formats.Jxl.IO;

/// <summary>

namespace SixLabors.ImageSharp.Formats.Jxl.IO;

/// <summary>

gfoidl · 2026-08-18T08:09:01Z

+/// Reads primitives from streams with correct endianness.
+/// </summary>
+// TODO: move this class into the IO or Common folder?
+internal static class BinaryUtils


The methods here are quite repetitive. Can generic math be used to reduce the code size?

OK, in the next file I see it's generated 😉 (I didn't check the file name first 🙈).

Anyway w/ generic math -- if possible -- the tt-file could be removed altogether.

winscripter added 3 commits July 14, 2026 22:51

Add JPEG XL format

85c851b

Implement ac_context.h

996e93d

Implemented frame_dimensions.h plus part of ac_strategy.h

d87178f

winscripter added 12 commits July 15, 2026 14:44

Implement AC strategy

1ac6140

Implementation of ac_strategy.h and ac_strategy.c

Implement JxlMemoryManager

31b5505

For now JxlMemoryManager will be a wrapper around MemoryPool<T>.

Implement image and memory buffers

21ff33f

Implementation of image.h and image.c; AC strategy implementation was slightly adjusted to reduce errors.

Add metadata models

6fea920

Add headers & aspect ratio helpers

762edd1

Implement field encodings, move IJxlFields

487410e

This is an implementation of field_encodings.h. Note that I avoided implementing EnumValid() and Values() functions, as we have dedicated methods in .NET to do exactly that (Enum.IsDefined, Enum.GetValues)

Add xorshift implemetation

ac1c866

It's Jxl not Jpeg

8581222

Implement spline abstractions

9b58f4b

Implementation of spline.h

Add quantized spline implementations

064376a

Prefer the term "coefficient"

90680b7

Start work on ANS entropy

1e40482

Implemented ANS constants

gfoidl reviewed Jul 16, 2026

View reviewed changes

winscripter added 6 commits July 16, 2026 13:53

Implement JxlAnsHelper.GetPopulationCountPrecision

396cd8d

See ans_common.h

Turn JxlFrameDimensions into a sealed class

cb364fa

It is too large for a struct.

Implement CreateFlatHistogram

528149f

See ans_common.h

Add ANS entropy structs

490f02f

Add JxlAnsEntry and JxlAnsSymbol. See ans_common.h. These correspond to the Entry and Symbol structures within AliasTable.

Prefer byte for enums

73941c1

Implement ANS entropy helpers

20c2dd3

gfoidl reviewed Jul 16, 2026

View reviewed changes

Comment thread src/ImageSharp/Formats/Jxl/IO/JxlAnsHelper.cs Outdated

Comment thread src/ImageSharp/Formats/Jxl/IO/JxlAnsHelper.cs Outdated

winscripter added 3 commits July 16, 2026 22:15

Add bit-stream reader

fed0af3

Avoid 'using static'

25e41da

Simplify

4ec9b95

gfoidl reviewed Jul 16, 2026

View reviewed changes

Comment thread src/ImageSharp/Formats/Jxl/JxlThrowHelper.cs Outdated

Add ANS VarLen & histogram parser

b716d4e

Currently, there's a VarLenUint8/VarLenUint16 as well as histogram parsing implementation. I will additionally have to implement parsing of ANS codes, uint config and LZ77 parameters.

gfoidl reviewed Aug 1, 2026

View reviewed changes

winscripter added 5 commits August 6, 2026 13:15

Add Move to Front transform

79e8ce7

See inverse_mtf-inl.h

Prefer inline arrays

d1f6646

Prevent potential overflow

fe41799

Refactor

8209aa1

Validate the size parameter in JxlCoefficientOrder.ReadPermutation

074ff1b

JimBobSquarePants reviewed Aug 6, 2026

View reviewed changes

Comment thread src/ImageSharp/Formats/Jxl/Processing/JxlInverseMtf.cs Outdated

JimBobSquarePants reviewed Aug 6, 2026

View reviewed changes

Comment thread src/ImageSharp/Formats/Jxl/Processing/JxlInverseMtf.cs Outdated

winscripter added 8 commits August 8, 2026 18:17

Add image copy operations

ff98ffd

Implement image operations

e718a4a

Reduce number of errors

048945c

Implement JPEG XL numerics functions & reduce errors

4f71c52

Add rectangle support to images

4d0a468

This massively reduces number of syntax errors

Document JxlPlane<T> and JxlPlaneBase

2c517a2

Add passes decoder state prototype & reduce errors in JxlQuantizedSpline

50813c1

gfoidl reviewed Aug 9, 2026

View reviewed changes

winscripter added 4 commits August 12, 2026 16:06

Remove GC.SuppressFinalize(this)

0d9aea4

Add image bundle

20e3a34

Use inline arrays in Butteraugli

6e432ca

gfoidl reviewed Aug 12, 2026

View reviewed changes

winscripter added 2 commits August 13, 2026 12:07

Merge branch 'SixLabors:main' into jxl-support

4308af8

gfoidl reviewed Aug 15, 2026

View reviewed changes

Add JPEG XL container prototype & file type box

e03a581

JxlBoxHeader contains a header for boxes in the JPEG XL container. JxlFileTypeBox represents the ftyp box. BinaryUtils contains helper methods to read/write primitives from/to Stream in custom endianness

gfoidl reviewed Aug 18, 2026

View reviewed changes

		// TODO: optimize this? ImageSharp ICC doesn't support spans
		// so we need to allocate an array.

Uh oh!

Uh oh!

Conversation

winscripter commented Jul 15, 2026

Prerequisites

Description

Uh oh!

CLAassistant commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

winscripter commented Jul 16, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

winscripter commented Aug 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JimBobSquarePants commented Aug 17, 2026

Uh oh!

winscripter commented Aug 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

CLAassistant commented Jul 15, 2026 •

edited

Loading