[gvozdetsky]

seccomp can't filter io_uring operations: they are submitted through the ring, not as syscalls, so runtimes currently block io_uring entirely. Linux 7.0 added task-level io_uring BPF filters, which are inherited across fork and exec and can only be tightened, like seccomp.

This adds experimental support behind the run.oci.io_uring annotation, in the shape proposed for linux.ioUring in opencontainers/runtime-spec#1328 (proposal: opencontainers/runtime-spec#1323):

{
  "ops": ["IORING_OP_READ", "IORING_OP_WRITE", "IORING_OP_SOCKET"],
  "socketFamilies": ["AF_UNIX", "AF_INET", "AF_INET6"]
}

• ops is an allowlist, every other operation is denied. It doesn't depend on the SQE flags.
• Unknown names are ignored with a warning: what they would allow stays denied.
• socketFamilies, when present, limits IORING_OP_SOCKET to these families. It requires IORING_OP_SOCKET in ops, and an empty list is refused; to deny all sockets, leave IORING_OP_SOCKET out of ops.
• The policy is validated, and kernel support checked, before the container is created or restored, and before crun exec. crun also checks that the kernel keeps the restrictions on exec: Linux 7.0 and 7.1 without bc0e8faf90e7 drop them for a task that has used io_uring. This check forks and execs crun, so it is not done for crun exec nor on 7.2+, and otherwise once per kernel: its result is cached under the state root.
• Every opcode the kernel knows gets an allow or a deny filter, so the policy also holds when crun itself runs under io_uring filters.
• It is applied to the container init and to every crun exec process (they don't inherit it from init). Like seccomp, registering a filter needs no_new_privs or CAP_SYS_ADMIN, so it is applied before the capabilities are dropped, or, with noNewPrivileges, once no_new_privs is set; in both cases before seccomp.
• CRIU doesn't restore the restrictions, so on crun restore crun applies them to itself before running CRIU, and the restored processes inherit them.
• Only BPF filters are used, never IORING_REGISTER_RESTRICTIONS: the workload can add its own BPF filters and restrict its own rings, but not register task restrictions.
• crun features reports run.oci.crun.io_uring.

youki has the same semantics in youki-dev/youki#3805, so the spec shape is now implemented by two runtimes. Once the spec has the field, the annotation will be replaced by linux.ioUring.

Tests: tests/test_io_uring.py checks the policy in init and in exec (with and without noNewPrivileges), SQE-flag independence, deny-all, unknown names, crun running under an outer policy (fails without a filter on every opcode), invalid policies, and the cache of the exec check; checkpoint-restore-io-uring checks the init of a restored container. They are skipped on kernels without io_uring BPF filters. On Ubuntu's 7.0.0-42, which has the exec fix, they pass, and each of them fails with the corresponding apply call removed; on 7.0 without the exec fix (mainline, or Ubuntu's 7.0.0-38), crun refuses the annotation and the tests are skipped. The exec, seccomp, capabilities and checkpoint/restore tests pass too.

[giuseppe]

left some comments, we also need the check and libcrun_apply_io_uring_restrictions for the libcrun_container_restore case

[giuseppe]

this looks painful to maintain, but I can't see any alternative given what the uapi offers

[gvozdetsky]

Agreed. enum io_uring_op has no macros, so there is no way to get the names, or even to check the table against the headers. New opcodes are appended, so it is one line per release, and an opcode the running kernel doesn't know is skipped (EINVAL), so the table may be ahead of the kernel.

[giuseppe]

Opus 5.5 suggests to not accept socketFamilies without IORING_OP_SOCKET_VALUE:

  +      if (UNLIKELY (! policy->ops[IORING_OP_SOCKET_VALUE]))
  +        {
  +          ret = crun_make_error (err, 0, "%s: `socketFamilies` requires IORING_OP_SOCKET in `ops`", IO_URING_ANNOTATION);
  +          goto exit;
  +        }
  +

[gvozdetsky]

Done, a policy with socketFamilies but without IORING_OP_SOCKET in ops is now refused. I changed the spec proposal (opencontainers/runtime-spec#1328) and youki (youki-dev/youki#3805) the same way, so they stay in sync.

[giuseppe]

Suggested change

	/* Before seccomp, which may deny io_uring_register, and before the
	capabilities are dropped: like seccomp, the restrictions need
	CAP_SYS_ADMIN or no_new_privs. */
	/* Before seccomp, which may deny io_uring_register, and before the
	capabilities are dropped: registering a filter needs CAP_SYS_ADMIN
	in the user namespace. */

no_new_privs is set later as part of libcrun_set_caps.

Should we do like seccomp that we try to apply it in two different places?

[gvozdetsky]

Good catch, no_new_privs isn't set at that point, so it only worked thanks to CAP_SYS_ADMIN. It is now done like seccomp: without noNewPrivileges before the capabilities are dropped, otherwise once libcrun_set_caps has set no_new_privs, both for the init and for exec, and in both cases before seccomp. The tests now run with both values of noNewPrivileges.

[giuseppe]

do we need to add here a check for exec'ed processes?

    ret = libcrun_check_io_uring_restrictions (container, err);
    if (UNLIKELY (ret < 0))
      return ret;

[gvozdetsky]

Done.

[gvozdetsky]

@giuseppe thanks for the review. About restore: added both. The check is done before anything is created, and crun applies the restrictions to itself before running CRIU, like the keyring: CRIU doesn't restore them, but the restored processes inherit them from CRIU. crun itself stays restricted for the rest of the restore, which is fine as it doesn't use io_uring. The new checkpoint-restore-io-uring test checks the init of the restored container, not an exec.

The fixes are fixup commits for now, I'll squash them once you are happy with them.

[giuseppe]

thanks, LGTM

@kolyshkin PTAL

[gvozdetsky]

Rebased onto main, no changes to the patches; CI is green. @kolyshkin PTAL when you have time.

[kolyshkin]

Thanks for working on this, the overall structure looks good, and the ordering with respect to caps / no_new_privs / seccomp mirrors seccomp nicely. I checked the semantics against the kernel source (io_uring/bpf_filter.c, io_uring/register.c, 7.3-rc7). Inline comments below; the first one (DENY_REST only fills empty slots) is the one I consider blocking, as it makes the policy fail open when the task already has filters.

A few things that don't map to a specific line:

• The commit message and the PR description say that, since only BPF filters are used, "the workload can still add its own filters". That's true for BPF filters, but IORING_REGISTER_RESTRICTIONS for the task (io_register_restrictions_task) returns -EPERM once current->io_uring_restrict exists, and registering a BPF filter creates it. So the workload can no longer register task-level classic restrictions. Please reword.
• Old crun versions (and runc) silently ignore the annotation, i.e. no restrictions at all, and there's no way to detect support via crun features (only the run.oci. prefix is listed). That's inherent to experimental annotations, but maybe worth a line in the man page.
• Nit: the magic 37 is duplicated in tests/init.c; the policy is parsed twice (check + apply), which is fine but could be shared.

_{This comment was written by an AI (Claude Code).}

[kolyshkin]

This relies on DENY_REST alone to deny every opcode not in ops, but DENY_REST only fills empty slots (io_register_bpf_filter: if (old_filter) continue;). If the task already has filters when crun runs, the policy fails open.

Example: crun runs inside a container whose (outer) policy allows {READ, WRITE}, so both slots hold an allow filter and the rest the dummy deny. The inner policy is {READ}: crun registers allow on READ with DENY_REST, but the WRITE slot is not empty, so it stays allowed, and WRITE is permitted inside the inner container contrary to its policy.

Suggestion: explicitly register a deny program (BPF_RET|BPF_K, 0) for every opcode not in ops, iterating opcodes from 0 until the kernel returns EINVAL (that covers also opcodes newer than crun's table), and keep DENY_REST on the last one as a safety net. Appending a filter to a slot holding the dummy filter is fine, the dummy stays last and still denies. A test with nested policies (apply a wider policy to the test runner/parent first, or run crun under an already restricted crun) would catch this.

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Confirmed: under an outer policy that allows WRITE, the container could submit WRITE. Fixed in 7594442: every opcode the kernel knows gets its own allow or deny filter. I dropped DENY_REST, as it has nothing left to fill. The new io-uring-nested test runs crun under tests/init io_uring-restrict nop,read,write; it fails on the old code (WRITE allowed). youki had the same bug, fixed in youki-dev/youki#3805.

[kolyshkin]

EINVAL is treated as "the kernel doesn't know this opcode", but io_bpf_filter_import / io_uring_check_cbpf_filter also return EINVAL for a rejected BPF program, non-zero reserved fields, bad flags, etc. With the current fixed programs it's fail-closed (e.g. IORING_OP_SOCKET would just be silently denied), but it would be impossible to diagnose. Maybe only skip EINVAL for i >= <number of opcodes the kernel supports> (determined once, e.g. by probing), and report other EINVALs as errors.

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Fixed by the same change (7594442): the opcodes are registered in order with fixed one-instruction programs, so EINVAL can only mean that the kernel doesn't know the opcode. The socket family program is registered after the loop, where any error is reported.

[kolyshkin]

Re the earlier discussion that there's no way to check the table against the headers: the enum values can be used both in designated initializers and in static asserts, so when HAVE_LINUX_IO_URING_H (already checked in configure.ac) is defined, something like

#ifdef HAVE_LINUX_IO_URING_H
#  include <linux/io_uring.h>
_Static_assert (IORING_OP_SOCKET == IORING_OP_SOCKET_VALUE, "...");
_Static_assert (IORING_OP_LAST <= IO_URING_OPS_LEN, "update io_uring_ops");  /* or == */
#endif

(or the table itself as [IORING_OP_READ] = "IORING_OP_READ", ...) catches drift at build time. Same for these constants and the struct layout: <linux/io_uring/bpf_filter.h> is in the uapi headers, so the hand-rolled structs could be checked with sizeof / offsetof asserts, and CRUN_IO_URING_BPF_CTX_PDU_OFFSET could be offsetof (struct io_uring_bpf_ctx, socket.family).

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Added in 98075ad for the constants and the struct layouts, when the headers have <linux/io_uring/bpf_filter.h> (new configure check). Not for the table: designated initializers break the build with older headers, and IORING_OP_LAST <= IO_URING_OPS_LEN breaks building a released crun against newer ones. Since every opcode now gets a filter by number, a stale table only means that a newer name is ignored, and stays denied.

[kolyshkin]

Since ops is an allowlist, ignoring an unknown name (with a warning) is still fail-closed, and it's forward compatible: a policy generated by a newer podman/CRI listing a newly added opcode would otherwise make the container fail to start on an older crun. This is how unknown syscall names in seccomp profiles are commonly handled. (Rejecting unknown keys in the object is right, though, as those may add restrictions.) Same applies to unknown socket families.

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Agreed, done in 81f20a5: unknown names are ignored with a warning, like unknown seccomp syscalls. If none of the families is known, IORING_OP_SOCKET is denied. Unknown keys are still an error. Same in the spec PR and in youki.

[kolyshkin]

Only 9 families can be named, so e.g. AF_CAN, AF_TIPC, AF_BLUETOOTH, AF_RDS, AF_KCM can't be allowed at all. Maybe either list all AF_* from <sys/socket.h>, or accept numeric values too (this is probably a question for the spec proposal as well).

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

81f20a5 names all the families of <linux/socket.h>, by number, as libc may lack the newer ones. I kept names only, like ops.

[kolyshkin]

Nit: this message is misleading when crun itself runs under a seccomp profile denying io_uring_register (e.g. crun in a container with the default docker/podman profile): the probe gets EPERM/ENOSYS and the user is told the kernel is too old. Maybe include the errno, or say "not supported or not permitted".

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Done in 81f20a5: "not supported or not permitted", with the errno.

[kolyshkin]

Note that the restrictions are per task (thread) and permanent, so crun stays restricted for the rest of the restore. Fine for the CLI, but for programs using libcrun as a library this is an irreversible side effect on the calling thread; maybe worth a comment. Also, filters the workload added itself are lost on restore (CRIU doesn't dump them), probably worth documenting.

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Added the comment in a1821b8; the man page now says that filters the processes added themselves are lost on restore.

[kolyshkin]

Worth documenting: the kernel copies the task restrictions into a ring when it is created (io_ctx_restriction_clone at io_uring_setup), and checks submissions against the ring's filters. So a ring created outside the container and passed in (SCM_RIGHTS, --preserve-fds, LISTEN_FDS) is not restricted by this policy.

Also, socketFamilies only looks at the family, not type/protocol (so e.g. SOCK_RAW is allowed for AF_INET if the caps permit), and sockets created via socket(2) (subject to seccomp only) can still be used with any allowed io_uring op.

_{This comment was written by an AI (Claude Code).}

[gvozdetsky]

Documented in a1821b8.

[gvozdetsky]

Thanks for the review! The fixes are fixup commits, I'll squash them once you're happy.

• The commit message now says what the workload can still do: add BPF filters and restrict its own rings, but not register task restrictions.
• crun features reports run.oci.crun.io_uring (a1821b8), so callers can tell whether crun knows the annotation.
• tests/init.c uses <linux/io_uring/bpf_filter.h> instead of 37. The policy is still parsed twice, but only the check prints the warnings.
• Following your comment on the spec PR, crun now refuses the annotation if the kernel drops the restrictions on exec (d407ed3): a child restricts itself, creates a ring and re-executes itself. The check is in a libcrun constructor, like libcrun_rexec, so it works for any program that uses libcrun. On mainline 7.0 the annotation is refused; on 7.2.9 and on Ubuntu's 7.0.0-42, which has the backport, it is accepted.

The io_uring, exec, seccomp, capabilities, features and checkpoint/restore tests pass on 7.0.0-42.

[packit-as-a-service]

Ephemeral COPR build failed. @containers/packit-build please check.

[gvozdetsky]

Rebased onto main: Packit couldn't merge the PR because of a conflict in configure.ac. I also regenerated crun.1 with go-md2man, which CI now checks (78394eb). The other fixups are unchanged, but their hashes in my replies changed: 7594442 → f13d4df, 81f20a5 → e5f4833, d407ed3 → 76de313, 98075ad → 4302745, a1821b8 → 5b62b4f.

[kolyshkin]

This runs on every run/create/exec/restore and it is really crazy (especially for exec).

I think this should be done once per kernel version (uname.release) and stored into cache dir. See code using SYSTEMD_MISSING_PROPERTIES_DIR).

Maybe if the uname.release shows 7.2+, don't do anything at all.

OR, we can change the spec requirement that runtime must check that the properties are applied. I mean, this is definitely a kernel bug -- maybe it's better to not report io_uring limiting in crun features if the bug is present.

[gvozdetsky]

Agreed, done in 691bb28:

• crun exec no longer runs it: the container passed it when it was created, on the same kernel. exec still validates the policy and probes for support, which is one syscall.
• On 7.2 and newer it isn't run at all.
• Otherwise it runs once per kernel: a kernel that passed it is recorded in <state root>/.cache/io-uring-exec-check/<release>, like the missing systemd properties. The file holds uname -v, so a kernel rebuilt with the same release is checked again. A failure isn't cached, as it refuses the container anyway.

Counting the processes created per command on 7.0.0-42: with the annotation, only the first crun run creates the extra child; the following runs and every crun exec create as many processes as without the annotation. 78deddb tests the cache.

About moving it to crun features instead: I'd rather keep the check in the runtime. Reporting the annotation only on fixed kernels leaves it to every caller to check the features first, and a caller that doesn't would silently get restrictions that a process loses when it execs. Refusing fails closed, and with the cache it costs one fork per kernel.

[kolyshkin]

LGTM, but I'm not sure we should merge it before runtime-spec.

[kolyshkin]

@gvozdetsky please autosquash your commits

[gvozdetsky]

Autosquashed and rebased onto main. One more fixup went in before the squash: make syntax-check rejects sscanf, so the kernel version is now parsed with strtol. The commit message now mentions that the exec check is cached and skipped on 7.2+ and by exec.

The remaining runc integration failure, readonly rootfs keeps nosuid,nodev, comes from opencontainers/runc#5588 (merged Nov 1) and fails on other PRs too, e.g. #2441.