Cryptocb hooks rsapss ed448 cmac by night1rider · Pull Request #10886 · wolfSSL/wolfssl

night1rider · 2026-07-12T03:51:48Z

Adds WOLF_CRYPTO_CB crypto callback hooks so a device can service three more operations:
Ed448 sign and verify, mirroring the existing Ed25519 hooks.
CMAC context free on wc_CmacFree (WOLF_CRYPTO_CB_FREE), letting a device release offload state.
RSA-PSS verify with the digest (WOLF_CRYPTO_CB_RSA_PAD) so the device performs the full signature and padding check.
Includes testwolfcrypt and API unit test coverage for each hook.

github-actions · 2026-07-12T03:52:18Z

retest this please

github-actions · 2026-07-12T03:59:13Z

MemBrowse Memory Report

linuxkm-pie

Data: __patchable_function_entries +16 B (+0.1%, 26,456 B)

linuxkm-standard

Data: __patchable_function_entries +16 B (+0.0%, 49,296 B)
No memory changes detected for:
gcc-arm-cortex-m0plus
gcc-arm-cortex-m3
gcc-arm-cortex-m4
gcc-arm-cortex-m4-baremetal
gcc-arm-cortex-m4-crypto-only
gcc-arm-cortex-m4-dtls13
gcc-arm-cortex-m4-min-ecc
gcc-arm-cortex-m4-openssl-compat
gcc-arm-cortex-m4-pkcs7
gcc-arm-cortex-m4-pq
gcc-arm-cortex-m4-rsa-only
gcc-arm-cortex-m4-sp-math
gcc-arm-cortex-m4-tls12
gcc-arm-cortex-m4-tls13
gcc-arm-cortex-m7
gcc-arm-cortex-m7-pq
gcc-arm-cortex-m7-tls13
stm32-sim-stm32h753

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #10886

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 4
4 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #10886

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 3
3 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #10886

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 2
2 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

wolfSSL-Fenrir-bot

Fenrir Automated Review — PR #10886

Scan targets checked: wolfcrypt-bugs, wolfcrypt-port-bugs, wolfcrypt-rs-bugs, wolfcrypt-src, wolfssl-bugs, wolfssl-src

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Findings are non-blocking.

night1rider · 2026-07-23T03:30:01Z

jenkins retest this please

dgarske

Please resolve merge conflicts. Thanks

dgarske · 2026-08-07T00:35:51Z

Recommend we do this check for crypto callback too right? Maybe move it up higher? Also bring in the brace to same line if under 80 chars.

dgarske · 2026-08-07T00:36:16Z

+    if (key->devId != INVALID_DEVID)
+    #endif
+    {
+        ret = wc_CryptoCb_Ed448Verify(sig, sigLen, msg, msgLen, res, key, type,


Set *res = 0; before, just in case.

dgarske · 2026-08-07T20:51:07Z

 #endif

    /* sanity check on arguments */
    if ((in == NULL) || (out == NULL) || (outLen == NULL) || (key == NULL) ||


Recommendation: Relocate the WOLFSSL_CHECK_MEM_ZERO registration block to after the crypto-callback hook (mirroring wc_ed25519_sign_msg_ex), or alternatively goto a common exit label instead of returning directly. Add a --enable-cryptocb + WOLFSSL_CHECK_MEM_ZERO build to the validation matrix for this PR.

dgarske · 2026-08-07T20:52:55Z

+                 * so report no inline output rather than a misleading pointer. */
+                if (out != NULL)
+                    *out = NULL;
+                ret = (res != 0) ? (int)inLen : SIG_VERIFY_E;


wc_RsaPSS_VerifyCheck device path returns a different length and leaves out untouched, with no way for the caller to tell.

The software path returns the value of wc_RsaPSS_Verify_ex(), which ultimately comes from RsaUnPad_PSS (rsa.c:2024): return saltLen + hLen; — 64 for SHA-256 with a 32-byte salt — and it fills the caller's out buffer with that recovered PSS block. The new crypto-callback path returns (int)inLen — 256 for a 2048-bit key — and never writes out at all.

Recommendation: Do not return inLen as if it were the recovered-PSS-block length. Either have the device report the actual output length and fill out, or define an unambiguous "device verified, no recovered data" signal. Whichever is chosen, document it in doc/dox_comments/header_files/rsa.h for wc_RsaPSS_VerifyCheck as well as wc_RsaPSS_VerifyCheckInline, and add a test that asserts the device return value equals the software return value (or the documented sentinel).

dgarske · 2026-08-07T20:53:28Z

+    ExpectIntEQ(wc_RsaSetRNG(key, &rng), 0);
+
+    /* PSS sign runs in software (device declines). */
+    ExpectIntGT(sigLen = (word32)wc_RsaPSS_Sign(digest,


Buffer overflow in new RSA-PSS test when wc_RsaPSS_Sign fails: word32 sigLen holds a cast negative error code

Recommendation: Capture the signature length in a signed int and only widen to word32 after confirming it is positive, and additionally gate the plain-if inline sub-test on EXPECT_SUCCESS() so it does not run after a failure.

dgarske · 2026-08-07T20:54:14Z

+                    *out = NULL;
+                ret = (res != 0) ? (int)inLen : SIG_VERIFY_E;
+            }
+            return ret;


Recommendation: Explicitly map any positive device return to SIG_VERIFY_E (fail closed), and document in cryptocb.h that a WC_PK_TYPE_RSA_PSS_VERIFY handler must return exactly 0 (with *res set) or a negative error.

dgarske · 2026-08-07T20:54:56Z

+    if (cmac->devId != INVALID_DEVID)
+    #endif
+    {
+        (void)wc_CryptoCb_Free(cmac->devId, WC_ALGO_TYPE_CMAC, (int)cmac->type,


Reset cmac->devId = INVALID_DEVID after the ForceZero so a repeat wc_CmacFree() is inert

Writing devId back after the ForceZero breaks WOLFSSL_CHECK_MEM_ZERO, which requires the freed struct to be all zero. On the repeat-free concern: with nothing registered at devId 0 the dispatch finds no device and is a no-op, and if something is registered there it receives a fully zeroed context with a NULL devCtx and nothing to release. wc_AesFree and wc_Sha256Free behave the same way, so I left CMAC consistent with them.

dgarske · 2026-08-07T20:55:52Z

        WOLFSSL_MSG_EX("CryptoDevCb: Pk Type %d\n", info->pk.type);
    #endif

+    #if defined(WC_RSA_PSS) && defined(WOLF_CRYPTO_CB_RSA_PAD)


Add the same WOLF_CRYPTO_CB_ONLY_RSA escape as the sibling branches, or add && !defined(WOLF_CRYPTO_CB_ONLY_RSA) to the block guard.

Add WOLF_CRYPTO_CB dispatch hooks so a device can service: * Ed448 sign and verify, mirroring the existing Ed25519 hooks. * CMAC context free on wc_CmacFree (WOLF_CRYPTO_CB_FREE), letting a device release offload state. * RSA-PSS verify with the digest (WOLF_CRYPTO_CB_RSA_PAD) so the device does the full signature and padding check. On that path *out is set to NULL with a positive return, documented in rsa.h. Includes testwolfcrypt and API unit test coverage for each hook.

The PSS hook can hand back the recovered block through out/outSz/outLen. A device that reports only a verdict leaves outLen at 0; wolfSSL then zeroes the buffer and returns saltLen + hLen, and rejects a buffer smaller than that with RSA_BUFFER_E. A reported length is clamped to the buffer size, and any positive handler return maps to SIG_VERIFY_E. Move the Ed448 sign WOLFSSL_CHECK_MEM_ZERO registration below the crypto callback hook so the device path no longer returns past it, and guard the RSA-PSS test callback against WOLF_CRYPTO_CB_ONLY_RSA. Adds tests for the recovered-data, over-claimed-length and undersized buffer paths, and an os-check config that builds the hooks under WOLFSSL_CHECK_MEM_ZERO.

night1rider · 2026-08-08T23:24:32Z

Jenkins retest this please

night1rider self-assigned this Jul 12, 2026

night1rider marked this pull request as ready for review July 12, 2026 03:51

night1rider requested a review from wolfSSL-Fenrir-bot July 12, 2026 03:52

wolfSSL-Fenrir-bot reviewed Jul 12, 2026

View reviewed changes

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from 0108bd2 to 10370ac Compare July 12, 2026 04:45

night1rider requested a review from wolfSSL-Fenrir-bot July 12, 2026 04:45

wolfSSL-Fenrir-bot reviewed Jul 12, 2026

View reviewed changes

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from 10370ac to 513619f Compare July 12, 2026 06:08

night1rider requested a review from wolfSSL-Fenrir-bot July 12, 2026 06:08

wolfSSL-Fenrir-bot reviewed Jul 12, 2026

View reviewed changes

night1rider requested a review from wolfSSL-Fenrir-bot July 13, 2026 16:29

wolfSSL-Fenrir-bot reviewed Jul 13, 2026

View reviewed changes

Comment thread wolfcrypt/src/rsa.c Outdated

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch 2 times, most recently from b20a6df to dc84530 Compare July 20, 2026 22:42

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from dc84530 to 54ed373 Compare July 23, 2026 06:07

night1rider assigned wolfSSL-Bot and JacobBarthelmeh and unassigned night1rider Jul 23, 2026

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from 54ed373 to c8482b1 Compare July 27, 2026 22:32

night1rider mentioned this pull request Jul 28, 2026

Xilinx Versal Gen2 ASU port: AES ciphers, CMAC, GMAC offload #10994

Open

JacobBarthelmeh requested changes Jul 28, 2026

View reviewed changes

Comment thread wolfcrypt/src/ed448.c Outdated

Comment thread wolfcrypt/src/ed448.c Outdated

Comment thread wolfcrypt/src/rsa.c Outdated

Comment thread wolfcrypt/test/test.c Outdated

Comment thread wolfssl/wolfcrypt/rsa.h Outdated

JacobBarthelmeh assigned night1rider and unassigned JacobBarthelmeh and wolfSSL-Bot Jul 28, 2026

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from c8482b1 to 8d3993d Compare July 29, 2026 00:01

night1rider assigned JacobBarthelmeh and unassigned night1rider Jul 29, 2026

night1rider assigned night1rider and unassigned JacobBarthelmeh Jul 29, 2026

night1rider requested a review from wolfSSL-Fenrir-bot July 29, 2026 17:36

JacobBarthelmeh previously approved these changes Jul 29, 2026

View reviewed changes

night1rider assigned wolfSSL-Bot and unassigned night1rider Jul 30, 2026

This was referenced Aug 5, 2026

Xilinx asu ecc standalone #11053

Open

Xilinx asu rsa standalone #11052

Open

dgarske requested changes Aug 5, 2026

View reviewed changes

dgarske assigned night1rider Aug 5, 2026

night1rider dismissed JacobBarthelmeh’s stale review via 8db30e3 August 6, 2026 21:25

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from 8d3993d to 8db30e3 Compare August 6, 2026 21:25

dgarske unassigned night1rider Aug 6, 2026

dgarske self-requested a review August 6, 2026 22:40

dgarske requested changes Aug 7, 2026

View reviewed changes

dgarske assigned night1rider Aug 7, 2026

dgarske requested changes Aug 7, 2026

View reviewed changes

night1rider added 7 commits August 8, 2026 15:14

Set cmac->type before cryptocb init so wc_CmacFree can clean up

b35f60e

Validate Ed448ph prehash length before the sign crypto callback

7952b07

Clear *res before invoking the Ed448 verify crypto callback

3ade5f6

Cast type to CmacType so cmac.c builds as C++

4d19228

Release Ed448 sign MemZero registrations on the crypto callback exit

33e03a5

night1rider force-pushed the cryptocb-hooks-rsapss-ed448-cmac branch from da24c28 to 375febc Compare August 8, 2026 21:18

night1rider assigned night1rider and unassigned night1rider Aug 9, 2026

Uh oh!

Conversation

night1rider commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 12, 2026

Uh oh!

github-actions Bot commented Jul 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #10886

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #10886

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #10886

Uh oh!

wolfSSL-Fenrir-bot left a comment

Choose a reason for hiding this comment

Fenrir Automated Review — PR #10886

Uh oh!

Uh oh!

night1rider commented Jul 23, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dgarske left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

night1rider commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

night1rider commented Jul 12, 2026 •

edited

Loading

github-actions Bot commented Jul 12, 2026 •

edited

Loading