Accelerate System.Half arithmetic and conversions with FP16#130512
Open
tannergooding wants to merge 23 commits into
Open
Accelerate System.Half arithmetic and conversions with FP16#130512tannergooding wants to merge 23 commits into
tannergooding wants to merge 23 commits into
Conversation
Introduce TYP_HALF as an internal JIT SIMD base type only, used to drive FP16 scalar instruction selection (vaddsh, etc.). It is never the type of a local, argument, or return, so the ABI is unchanged. Update the WASM var_types mapping tables for the shifted enum. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Extend the HARDWARE_INTRINSIC ins array with an 11th slot for TYP_HALF across all targets, add the internal NI_AVX10v1_* FP16 scalar intrinsic entries used to drive vaddsh/vsubsh/etc. selection, and declare the NI_System_Half_* named intrinsics. No behavior change yet; these tables are wired up in later commits. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add INS_Flags_IsDstSrcSrcAVXInstruction to the FP16 scalar arithmetic and conversion instructions and INS_Flags_IsDstDstSrcAVXInstruction to vfmadd213sh, so the emitter encodes their EVEX vvvv src register. Fix the MAP5 escape on vminsh/vminph/vmovsh (were 0x00, must be 0x05). Add FIRST/LAST_AVX10V1_FMA_INSTR markers and recognize that range in Is3OpRmwInstruction. Relax the EVEX MAP5/MAP6 debug asserts to also accept AVX10v1, and add perfscore characteristics plus the missing PERFSCORE_THROUGHPUT_1P5X/4P5C macros. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Group the FP16 FMA instruction families so genFmaIntrinsic's 132/231 form derivation stays valid, dispatch AVX10v1 through genAvxFamilyIntrinsic and the FMA jumptable fallback, lower the Half compare-scalar intrinsics to VCOMISH/ VUCOMISH, contain RoundScaleScalar, and build FusedMultiplyAddScalar as an FMA. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Recognize the Half operators, Min/Max, Sqrt, reciprocal estimates, rounding, comparisons, and FusedMultiplyAdd as named intrinsics and expand them by moving the Half struct bits into/out of SIMD scalars (impSimdCreateScalarHalf/ impSimdToScalarHalf) and selecting the AVX10v1 *sh scalar instructions via a TYP_HALF SIMD base type. Half is never a GenTree/arg/return type, so the ABI is unchanged. Add lookupHalfNamedIntrinsic/lookupHalfIntrinsic/lookupHalfRoundingMode and mark the corresponding Half.cs members [Intrinsic]. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The uniform 11-slot HARDWARE_INTRINSIC macro requires every list file to provide the TYP_HALF instruction column. The SVE list was missed, causing a C4003 not-enough-arguments error when building the arm64 altjit variant. Add the INS_invalid TYP_HALF column to each SVE entry. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…dded rounding The DivideScalar/SqrtScalar entries route through the special HWIntrinsic costing switch (FltCost == -1), and AddScalar/FusedMultiplyAddScalar carry HW_Flag_EmbRoundingCompatible, so all four must be handled explicitly to avoid hitting the asserting default paths when the FP16 scalar path is used. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
vfmadd132sh/213sh/231sh were spelled with a leading `vv` in their display string, so JitDisasm printed e.g. `vvfmadd213sh`. These scalar forms are the first FP16 instructions actually emitted by the Half acceleration path, which surfaces the typo. Match the vfmadd*ss naming. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
lookupHalfRoundingMode mapped Half.Ceiling to imm 1 and Half.Floor to imm 2, but the RNDSCALE rounding-control encoding is ToNegativeInfinity==1 (floor) and ToPositiveInfinity==2 (ceiling), so the two were swapped. Use the canonical FloatRoundingMode enum to make the mapping self-documenting and correct. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The raw vminsh/vmaxsh instructions return src2 on unordered inputs and don't order +0/-0, which matches MaxNative/MinNative but not the IEEE maximum/minimum semantics of Half.Max/Min. Drop them from the FP16 fast path; the managed body `(Half)float.Max((float)x, (float)y)` remains correct and still F16C-accelerates the conversions while intrinsifying float.Max/Min via gtNewSimdMinMaxNode with the proper NaN/+-0 fixup. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
InstructionSet_AVX10v1 already implies InstructionSet_AVX10v2 in the JIT ISA model, so compIsaSupportedDebugOnly(InstructionSet_AVX10v1) covers the AVX10v2 case. Drop the redundant AVX10v2 checks from the EVEX map asserts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Follows the non-ABI-breaking premise from the F16C work (dotnet#127094): Half is never a GenTree value type, so no TYP_HALF exists and the ABI is unchanged (this is what caused the revert of dotnet#122649). Half is bound internally as a TYP_USHORT SIMD base type on TYP_SIMD16 nodes purely for instruction selection, with the ushort bits moved into and out of SIMD scalars at the import sites via impSimdCreateScalarHalf/impSimdToScalarHalf. Removes TYP_HALF and its per-platform HARDWARE_INTRINSIC instruction column, restoring the arm64/arm64sve/wasm/xarch tables to their original layout. Adds the FP16 conversions to/from float, double, int, uint, long, and ulong plus op_Increment/op_Decrement. Because Half binds as an integer-register base type, the affected AVX10v1 scalar entries carry an IntCost matching FltCost so the HWIntrinsic cost model no longer hits the unhandled-costing assert, and the general-purpose integer conversions produce their result with the actual machine type (TYP_INT/TYP_LONG) so the signedness is encoded solely by the selected vcvttsh2si/vcvttsh2usi instruction. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The scalar Half<->integer converts (vcvttsh2si/vcvttsh2usi and vcvtsi2sh/vcvtusi2sh) were table-driven and emitted with the hardcoded EA_16BYTE SIMD size, so their general purpose operand rendered the 64-bit register name for the 32-bit forms (e.g. `rax`/`rcx` instead of `eax`/`ecx`). The instruction encoding was already correct since the width is baked into the selected instruction, only the disassembly display was wrong. Mark the four ConvertTo*WithTruncation and ConvertScalarToVector128Half intrinsics as HW_Flag_SpecialCodeGen and emit them with the appropriate operand size: emitTypeSize(targetType) for the integer result and emitActualTypeSize(baseType) for an integer source, leaving floating-point sources at the full 128-bit size. Also handle IsXMMReg in emitRegName's EA_2BYTE case for parity with the EA_4BYTE/EA_8BYTE/EA_16BYTE cases. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Declare an internal-only Fp16 ISA (no public managed API) mirroring the x86 F16C/Evex internal-ISA pattern, giving the JIT an opportunistic FEAT_FP16 gate for accelerating System.Half. Regenerated the JIT/EE thunk files and bumped the interface GUID. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add ARM64IntrinsicConstants_Fp16 with per-OS detection (Linux HWCAP_FPHP+ASIMDHP, macOS sysctl hw.optional.arm.FEAT_FP16; Windows has no IsProcessorFeaturePresent flag yet, left as a TODO), an EnableArm64Fp16 config knob, and wire it through the VM (codeman), the JIT default flags, and the shared ILC/crossgen2 mapping. Insert the Fp16 placeholder rows in hwintrinsicIsaRangeArray to keep the positional array aligned with the shifted InstructionSet enum. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Teach the Arm64 emitter to encode scalar half-precision (FEAT_FP16) FP instructions used to accelerate System.Half: ftype=0b11 (EA_2BYTE) for DV_3D/DV_2G/DV_2K forms, plus four new INS_OPTS values for the half<->int conversions (DV_2H fcvtzs/fcvtzu and DV_2I scvtf/ucvtf). Relax the relevant sanity, size, and perfscore paths to accept EA_2BYTE, and add a genArm64EmitterUnitTestsFp16 section covering the new forms. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Define the JIT-internal NI_Fp16_* hardware intrinsics used to accelerate System.Half on Arm64 (FEAT_FP16). These mirror the scalar half-precision operations, binding Half internally as TYP_USHORT on TYP_SIMD nodes, and are created directly by the importer rather than via a managed API. Wire up the Fp16 ISA range so the debug range validation recognizes the new contiguous block. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Import each System.Half operator/method into the internal NI_Fp16_* scalar hardware intrinsics when FEAT_FP16 is opportunistically available, mirroring the xarch AVX10v1 path. Half continues to bind as TYP_USHORT on TYP_SIMD nodes, so there is no ABI impact. Notable Arm64 deviations from xarch: Sqrt/ReciprocalEstimate/RSqrtEstimate and the rounding ops are unary (no dummy upper-bits source, no rounding immediate); Half->arithmetic converts return their scalar result directly in the target register (no vector extraction); and all six integer/floating conversions are available regardless of pointer size. Extend lookupHalfIntrinsic to compile for both xarch and Arm64. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Emit the special-codegen sequences for the internal NI_Fp16_* compare and conversion intrinsics used to accelerate System.Half: - Compares lower to fcmp + cset. Less-than and less-than-or-equal use the 'mi'/'ls' conditions rather than 'lt'/'le' so an unordered (NaN) operand yields false, matching C# ordered comparison semantics. - Half<->arithmetic conversions lower to fcvt/fcvtzs/fcvtzu/scvtf/ucvtf with the appropriate conversion option and destination size. Arithmetic, unary and fmadd intrinsics need no additions here since they are scalar and handled by the table-driven path. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
FCVT between half and single/double is part of the Armv8.0 FP baseline and does not require FEAT_FP16, so these precision conversions can always be accelerated -- mirroring how F16C provides float<->half below the AVX10.1 arithmetic bar on x64. Arm64 is even better here since baseline FCVT covers both float<->half and double<->half. Represent them as internal NI_ArmBase_ConvertToSingle/Double/Half intrinsics (ArmBase is guaranteed on Arm64 and scalar FCVT is not strictly SIMD), so the codegen ISA assert on NI_Fp16_* does not block reuse at the baseline. Integer <->half conversions (SCVTF/UCVTF/FCVTZS/FCVTZU with a half operand) still require FEAT_FP16 and remain gated on NI_Fp16_*. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch |
…ate-half-fp16 # Conflicts: # src/coreclr/inc/jiteeversionguid.h
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the JIT’s ability to recognize and lower System.Half operators/conversions to scalar FP16 instruction sequences (x64 via AVX10v1/F16C paths; Arm64 via FEAT_FP16 plus baseline FCVT conversions), while keeping the ABI boundary representation as ushort and avoiding any new managed API surface.
Changes:
- Adds an Arm64
Fp16instruction set (feature detection + config gating + R2R plumbing) and uses it to enable scalar half arithmetic and int↔half conversions. - Marks additional
System.Halfmembers as[Intrinsic]and teaches the importer/lowering/codegen to emit FP16 instructions for arithmetic, comparisons, rounding, and conversions. - Extends xarch AVX10v1 intrinsic coverage (instruction defs, lowering, emitter characteristics) needed for scalar FP16 operations.
Show a summary per file
| File | Description |
|---|---|
| src/native/minipal/cpufeatures.h | Adds ARM64IntrinsicConstants_Fp16 feature bit. |
| src/native/minipal/cpufeatures.c | Detects FEAT_FP16 on Arm64 (Linux/macOS) and plumbs into feature flags. |
| src/libraries/System.Private.CoreLib/src/System/Half.cs | Adds [Intrinsic] to selected Half operators and conversion/rounding APIs to enable importer recognition. |
| src/coreclr/vm/codeman.cpp | Enables InstructionSet_Fp16 based on runtime feature detection + EnableArm64Fp16 config. |
| src/coreclr/tools/Common/JitInterface/ThunkGenerator/InstructionSetDesc.txt | Adds Arm64 Fp16 instruction set entry and updates next available R2R bit. |
| src/coreclr/tools/Common/JitInterface/CorInfoInstructionSet.cs | Adds ARM64_Fp16 / ARM64_Fp16_Arm64 enums and implication expansion logic. |
| src/coreclr/tools/Common/Internal/Runtime/ReadyToRunInstructionSetHelper.cs | Maps Arm64 Fp16 instruction set to R2R encoding. |
| src/coreclr/tools/Common/Internal/Runtime/ReadyToRunInstructionSet.cs | Adds Fp16 = 93 to the R2R instruction set enum. |
| src/coreclr/tools/Common/Compiler/HardwareIntrinsicHelpers.cs | Adds Arm64 fp16 flag mapping/printing support. |
| src/coreclr/jit/valuenumfuncs.h | Adjusts comment for hardware intrinsic VN function defs (now includes non-xarch). |
| src/coreclr/jit/namedintrinsiclist.h | Adds named intrinsics for System.Half operators/operations (some currently unused). |
| src/coreclr/jit/lsraxarch.cpp | Treats NI_AVX10v1_FusedMultiplyAddScalar as delay-free eligible like other FMA variants. |
| src/coreclr/jit/lowerxarch.cpp | Lowers AVX10v1 FP16 compare intrinsics via CC-setting forms (VCOMISH/VUCOMISH). |
| src/coreclr/jit/jitconfigvalues.h | Adds EnableArm64Fp16 JIT config gate. |
| src/coreclr/jit/instrsxarch.h | Adds/adjusts scalar FP16 instruction flags and defines AVX10v1 FMA instruction range. |
| src/coreclr/jit/instr.h | Adds Arm64 half↔int conversion insOpts variants. |
| src/coreclr/jit/importercalls.cpp | Adds importer expansions for Half conversions/arithmetic/comparisons/rounding; introduces lookupHalf* helpers. |
| src/coreclr/jit/hwintrinsiclistxarch.h | Adds AVX10v1 scalar FP16 intrinsics + VCOMISH/VUCOMISH entries. |
| src/coreclr/jit/hwintrinsiclistarm64.h | Adds Arm64 Fp16 intrinsic group and internal ArmBase half↔float/double conversions (file currently has a BOM). |
| src/coreclr/jit/hwintrinsiccodegenxarch.cpp | Adds codegen handling for AVX10v1 scalar half conversions to/from integers and half creation. |
| src/coreclr/jit/hwintrinsiccodegenarm64.cpp | Adds Arm64 codegen for half conversion/conversion-to-int and half comparisons. |
| src/coreclr/jit/hwintrinsic.cpp | Plumbs AVX10v1 and Arm64 Fp16 intrinsic ranges into ISA-range table. |
| src/coreclr/jit/gentree.cpp | Adds cost/rounding support for new AVX10v1 scalar FP16 intrinsics. |
| src/coreclr/jit/emitxarch.cpp | Updates EVEX prefix asserts, size adjustments, mov semantics, reg naming, and perf characteristics for new FP16 ops. |
| src/coreclr/jit/emitarm64.h | Allows EA_2BYTE as valid scalar size; extends convert option range checks. |
| src/coreclr/jit/emitarm64.cpp | Extends sanity checks and encoding emission for half-sized FP ops/conversions. |
| src/coreclr/jit/emit.h | Adds new perf throughput constants used by FP16 instruction characteristics. |
| src/coreclr/jit/compiler.h | Declares lookupHalfNamedIntrinsic, lookupHalfIntrinsic, and rounding mode helper. |
| src/coreclr/jit/compiler.cpp | Adds EnableArm64Fp16 handling to Arm64 instruction set flags. |
| src/coreclr/jit/codegenlinear.cpp | Wires new Arm64 FP16 emitter unit test section. |
| src/coreclr/jit/codegenarm64test.cpp | Adds DEBUG-only emitter unit tests for scalar FP16 instruction encodings. |
| src/coreclr/jit/codegen.h | Declares genArm64EmitterUnitTestsFp16. |
| src/coreclr/inc/readytoruninstructionset.h | Adds READYTORUN_INSTRUCTION_Fp16 = 93. |
| src/coreclr/inc/jiteeversionguid.h | Updates JIT-EE version GUID due to instruction set enum shape change. |
| src/coreclr/inc/corinfoinstructionset.h | Adds Arm64 Fp16 / Fp16_Arm64 to CORINFO enums and validation/mapping helpers. |
| src/coreclr/inc/clrconfigvalues.h | Adds EXTERNAL_EnableArm64Fp16 runtime config switch. |
Copilot's findings
- Files reviewed: 36/36 changed files
- Comments generated: 2
…ic dispatch The Half constant getters (One, Zero, Epsilon, NaN, MinValue, MaxValue, PositiveInfinity, NegativeInfinity) now intercept as intrinsics and fold to scalar constants, and op_Increment/op_Decrement create the 0x3C00 constant directly rather than converting 1.0f at runtime. More importantly, all Half-returning cases now use sig->retTypeSigClass rather than clsHnd when calling impSimdToScalarHalf. clsHnd is the constrained. token's class handle (the interface/canonical type) under generic constrained dispatch, not System.Half, so it tripped the isSystemHalfClass assert whenever a Half operator or getter was invoked through a generic constraint (e.g. T.PositiveInfinity or x++ on T : INumber<T>). Also drops the stray UTF-8 BOM from hwintrinsiclistarm64.h. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Member
Author
|
CC. @dotnet/jit-contrib, @EgorBo, @dhartglassMSFT for review FYI @dotnet/intel, @dotnet/arm64-contrib as an FYI that this finishes the scalar acceleration for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This accelerates
System.Halfarithmetic and conversions in the JIT by following the non-ABI-breaking premise from #127094, rather than theTYP_HALFapproach from #122649 which had to be reverted due to ABI issues.The key idea, as with the F16C change, is that
Halfcontinues to be passed and returned as aushortat the ABI boundary. Internally the JIT binds it ontoTYP_SIMDnodes (asTYP_USHORT) and lights up scalar FP16 instructions via internalNI_Fp16_*hardware intrinsics that are created directly by the importer -- there is no newTYP_HALF, no VM-side work, and no managed API surface.x64 (AVX10.1 / F16C)
Add/Subtract/Multiply/Divide/Sqrt/FusedMultiplyAdd/Min/Max/comparisons/rounding, etc.) lights up when AVX10.1 is available, using the scalar FP16 (.sh) forms.Half<->floatconversions light up at the lower F16C bar (vcvtph2ps/vcvtps2ph), matching Add support for utilizing F16C instructions on xarch #127094, so conversions accelerate even without AVX10.1.AVX10v1is sufficient in the emitter asserts since it impliesAVX10v2; the redundantAVX10v2checks were dropped.Arm64 (FEAT_FP16, with a baseline conversion split)
The original PR did not touch Arm64 at all; this adds it.
SCVTF/UCVTF/FCVTZS/FCVTZUwith a half operand) requireFEAT_FP16and are gated on the internalFp16ISA, detected at runtime viaID_AA64PFR0_EL1.FP.Half<->float/doubleconversions do not requireFEAT_FP16--FCVTbetween half and single/double is part of the Armv8.0 FP baseline and is always present. These accelerate unconditionally, mirroring how F16C providesfloat<->halfbelow the AVX10.1 arithmetic bar on x64. Arm64 is actually broader here since baselineFCVTcovers bothfloat<->halfanddouble<->half.NI_Fp16_*codegen path asserts theFp16ISA, the baseline precision conversions are represented as internalNI_ArmBase_ConvertTo{Single,Double,Half}intrinsics instead (ArmBaseis guaranteed on Arm64 and scalarFCVTis not strictly SIMD).There is no new public API and no ABI change; all of the
NI_Fp16_*/NI_ArmBase_Convert*intrinsics are JIT-internal and only ever created by the importer, never bound by managed method name.CC. @dotnet/jit-contrib
Note
This PR body and the accompanying implementation were produced with the assistance of GitHub Copilot.