NULL pointer dereference in nvidia-modeset via nv_drm_framebuffer_create (DRM_IOCTL_MODE_ADDFB2)

[LeChatOTapas]

Posting here rather than as an Issue: the functional-bug template requires confirming the problem does not occur with the proprietary driver, which would be untrue here. The defective code is shared verbatim between kernel/ and kernel-open/, so both flavours are affected. Happy to move this wherever you prefer.

nv_drm_framebuffer_init() calls nvKms->isVidmem(nv_gem->pMemory) on GEM objects of arbitrary type, without checking pMemory. Several GEM types legitimately have pMemory == NULL, and isVidmem() dereferences it unconditionally. Any Wayland compositor that hands such a buffer to DRM_IOCTL_MODE_ADDFB2 oopses the kernel.

Reproducible 100% of the time on the affected setup, and fixed by a two-line null check (patch below, validated).

Still present in main as of 615.71.09. Verified on kernel-open/nvidia-drm/nvidia-drm-fb.c in this repository: the offending line is unchanged from 595.99.02, and the same file ships byte-identically in the proprietary driver's kernel/ tree (verified by sha256 against the 595.99.02 .run).

Environment

Driver	595.99.02, proprietary kernel modules (--kernel-module-type=proprietary). Code path unchanged in main at 615.71.09.
Kernel	6.12.107+deb13-amd64 (Debian 13)
GPU	RTX 4050 Laptop (AD107M), Optimus, alongside an Intel Raptor Lake-P iGPU
Compositor	KWin / KDE Plasma on Wayland
Topology	internal panel (eDP) on the iGPU, HDMI connector wired to the dGPU

Not reproducible under X11: the NVIDIA DDX does not use the DRM ADDFB2 path.

Reproduction

1. Boot with the internal panel on the iGPU only, start a Plasma Wayland session.
2. Hot-plug a display into the HDMI port wired to the dGPU.
3. KWin brings up the second GPU as a scanout device mid-session, imports a buffer cross-GPU, and calls ADDFB2. The kernel oopses and kwin_wayland is killed.

The following OpenGL errors from the compositor immediately precede the oops, and are the signature of the cross-GPU buffer import that produces the offending framebuffer:

kwin_scene_opengl: 0x500: GL_INVALID_ENUM error generated. Invalid <face>.
kwin_scene_opengl: Invalid framebuffer status: "GL_FRAMEBUFFER_INCOMPLETE_MISSING_ATTACHMENT"
kwin_scene_opengl: 0x502: GL_INVALID_OPERATION error generated. <image> and <target> are incompatible
kwin_scene_opengl: Invalid framebuffer status: "GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT"

Oops

BUG: kernel NULL pointer dereference, address: 0000000000000028
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
CPU: 1 UID: 1000 PID: 1033 Comm: kwin_wayland Tainted: P OE 6.12.107+deb13-amd64
RIP: 0010:_nv000556kms+0x4/0x10 [nvidia_modeset]
Code: ... f3 0f 1e fa <0f> b6 47 28 c3 0f 1f 80 00 00 00 00 ...
RDI: 0000000000000000
CR2: 0000000000000028
Call Trace:
 <TASK>
 nv_drm_framebuffer_create+0x218/0x460 [nvidia_drm]
 drm_internal_framebuffer_create+0x3e2/0x570 [drm]
 drm_mode_addfb2+0x4a/0xf0 [drm]
 drm_ioctl_kernel+0xad/0x100 [drm]
 drm_ioctl+0x277/0x4d0 [drm]
 __x64_sys_ioctl+0x94/0xc0
 do_syscall_64+0x87/0x1b0

_nv000556kms is the isVidmem entry of struct NvKmsKapiFunctionsTable. This is established without guessing: in nv-modeset-kernel.o_binary the symbol has exactly one relocation, inside nvKmsKapiGetFunctionsTableInternal, storing it at table offset +0xc0, which is offsetof(struct NvKmsKapiFunctionsTable, isVidmem) as computed from the shipped nvkms-kapi.h. The five neighbouring slots line up with mapMemory, unmapMemory, gc6BlockerRefCntInc, gc6BlockerRefCntDec and createSurface.

The function body is a 9-byte accessor, and the faulting instruction in the Code: field above matches it exactly:

endbr64
movzbl 0x28(%rdi),%eax    ; <-- faults, %rdi == NULL
ret

Root cause

nvidia-drm/nvidia-drm-fb.c, nv_drm_framebuffer_init() (line 134), loop at line 158, offending call at line 167:

for (i = 0; i < NVKMS_MAX_PLANES_PER_SURFACE; i++) {
    struct drm_gem_object *gem = fb->obj[i];
    if (gem != NULL) {                                  /* checks the GEM... */
        nv_gem = to_nv_gem_object(gem);
        params.planes[i].memory = nv_gem->pMemory;
        ...
        if (!nvKms->isVidmem(nv_gem->pMemory) && ...)   /* ...but not pMemory */

isVidmem() is documented in nvkms-kapi.h as taking "Memory allocated using allocateMemory()", so passing NULL is a caller-side bug. Two paths legitimately produce pMemory == NULL:

• nv_drm_gem_prime_import_sg_table() (nvidia-drm-gem-dma-buf.c:153-156) sets pMemory = NULL and only fills it from getSystemMemoryHandleFromDmaBuf(), which can fail. The rest of the driver handles that explicitly, e.g. nv_drm_gem_export_dmabuf_memory_ioctl() at line 199 tests if (!nv_dma_buf->base.pMemory).
• nv_drm_gem_import_userspace_memory() (nvidia-drm-gem-user-memory.c:232) always passes NULL /* pMemory */.

nvidia-drm-fb.c:167 is the only site in the driver that calls isVidmem() on a GEM of arbitrary type. The five other call sites are all in nvidia-drm-gem-nvkms-memory.c and operate on objects created through __nv_drm_nvkms_gem_obj_init(), whose four callers all null-check pMemory beforehand (lines 414, 477, 596, 643).

Proposed fix

--- a/kernel-open/nvidia-drm/nvidia-drm-fb.c
+++ b/kernel-open/nvidia-drm/nvidia-drm-fb.c
@@ -164,7 +164,13 @@
             params.planes[i].offset = fb->offsets[i];
             params.planes[i].pitch = fb->pitches[i];
 
-            if (!nvKms->isVidmem(nv_gem->pMemory) && nv_dev->hasVideoMemory) {
+            if (nv_gem->pMemory == NULL) {
+                /*
+                 * No backing NvKmsKapiMemory (user memory, or a PRIME-imported
+                 * dma-buf for which getSystemMemoryHandleFromDmaBuf() failed).
+                 * It cannot be registered as a scanout surface with NVKMS, and
+                 * nvKms->isVidmem() must not be called on it.
+                 */
+                non_scanout_mem_backed = true;
+            } else if (!nvKms->isVidmem(nv_gem->pMemory) &&
+                       nv_dev->hasVideoMemory) {
                 non_scanout_mem_backed = true;
             }

This reuses the existing non_scanout_mem_backed path, so it introduces no new state: nv_fb->pSurface stays NULL, and the two consumers of pSurface (cursor_plane_req_config_update() and plane_req_config_update() in nvidia-drm-crtc.c) already null-check it and return -EINVAL at atomic check time. ADDFB2 therefore succeeds and any scanout attempt fails cleanly instead of panicking. Behaviour is unchanged whenever pMemory is valid.

Validation

Reproduced in a VM (Debian 13, KDE Wayland, same GPU via VFIO passthrough) with an identical call trace down to the offsets, then fixed. With the patch applied and an added one-shot log line, the kernel confirms the path is taken and handled:

[drm] [nvidia-drm] [GPU ID 0x00000002] plane 0 has no NVKMS memory; framebuffer not registered with NVKMS

On the original hardware, the patched driver no longer oopses and the external display now works, which it never did before, since the hot-plug always panicked first.

Unrelated second issue in the same file

nv_drm_gem_prime_import_sg_table() (nvidia-drm-gem-dma-buf.c:146) returns a bare NULL on allocation failure. The DRM core checks this hook with IS_ERR(), which is false for NULL, so the returned NULL would be treated as a valid GEM object and dereferenced. The sibling hook nv_drm_gem_prime_import() (nvidia-drm-gem.c:168), registered next to it in the same drm_driver, correctly returns ERR_PTR(-ENOTSUPP).

-        return NULL;
+        return ERR_PTR(-ENOMEM);

Both are still present verbatim in main at 615.71.09.