No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers
Abstract
Conventional vulnerability analysis relies on source-level or binary-level access, or dynamic interaction, all of which may be unavailable to third-party analysts auditing closed-source, remotely hosted, or commercially gated software. Therefore, we propose a new paradigm of no-box vulnerability analysis in which neither source code nor runtime interaction is accessible, and only functionality metadata is available. Such metadata defines the intended behavior of the system, including its inputs, outputs, and side effects, while constraining the space of implementations consistent with that behavior. The intended behavior implies irreducible data flows, the minimal source-to-sink data flow skeletons shared by all conforming implementations. By reasoning over irreducible data flows, an analyst can formulate vulnerability hypotheses without observing or interacting with the target system. The analyst can later validate these hypotheses when additional access becomes available. We showcase the feasibility of no-box vulnerability analysis through implementing a pipeline called MCPSEC, which audits Model Context Protocol (MCP) servers for indirect prompt injection vulnerabilities using only the tool metadata exposed at server registration time. We evaluate MCPSEC on 20 widely deployed MCP servers comprising 177 tools, among which human evaluators confirm 95 vulnerable tools. MCPSEC identified 143 tools as vulnerable, and for each vulnerable tool, it produces Theory of Concepts (ToCs), a hypothesized attack scenario for later analyst validation. Using metadata alone, MCPSEC recovers 94 (98.9% recall) real verified vulnerabilities, compared with 80 (84.2% recall) for the LLM baseline. Overall, our results introduce no-box vulnerability analysis as a new analysis paradigm and demonstrate its practical feasibility in realistic systems.
When you're not allowed to touch the target
You're red-teaming a network and find a promising server. Your contact winces: that is the company's only card-processing box, it loses thousands of dollars a second if it goes down, and you may not send it traffic. Every established paradigm of vulnerability analysis assumes you can do something to the target: white-box needs its source or binaries, gray-box needs execution feedback, black-box at least needs to send inputs and read outputs. Closed-source, remotely hosted or commercially gated systems, or a hospital's MRI machine with no digital twin, offer none of that. The paper's proposal is a fourth paradigm, no-box vulnerability analysis: reason from the one thing an analyst can always get, the metadata describing what the system is supposed to do.
Irreducible data flows and Theories of Concept
Metadata pins down a system's intended behavior, its inputs, outputs and side effects, without saying how that behavior is implemented. It therefore defines a whole space of conforming implementations, and the trick is to reason about what all of them must share. If a tool's documentation says it takes a user-supplied filter and returns matching database records, then every implementation contains a flow from user input to a database query. That minimal source-to-sink skeleton is an irreducible data flow, and it immediately suggests a hypothesis: an SQL-injection vulnerability, contingent on how the query is built. Speculation fills in plausible implementation details around the skeleton (parsers, transformations, validation, sanitization); risk analysis checks whether at least one metadata-consistent path satisfies every precondition for exploitation. The output is a Theory of Concept, or ToC: an explicit attack scenario naming the affected functionality, the attacker's role and capabilities, the assumptions the exploit rests on, and the impact. Unlike a proof of concept it is never executed. When source code or runtime access arrives later, an analyst turns it into one.
Why MCP servers are the natural first target
The Model Context Protocol is how LLM agents reach tools and external data, and at registration
every server publishes exactly the metadata no-box analysis needs: a tool name, a natural-language
description and a JSON schema for its inputs. The ecosystem is also hard to audit any other way. Of
18,770 servers in the official registry, 3,007 (16.0%) have no public source and 1,210 (6.4%) demand
secrets before they will even connect, and probing a live server for indirect prompt
injection (IPI) means planting payloads that other users of a shared service may stumble
into. IPI is the vulnerability class studied here: attacker-controlled content (a web page, an
issue, a document, a third-party API response) is retrieved by a tool, lands in the LLM's context and
steers the agent against the user's intent. The threat model is deliberately narrow. The user is
benign, the agent follows the user but is influenced by tool responses, the server is honest but
possibly vulnerable, and only external data sources are hostile. The irreducible flow is then
external source → MCP tool → LLM context, and the question for each tool is whether an
attacker can ride it.
Four preconditions for an injection
A source-to-sink flow alone doesn't make a vulnerability; a payload also has to survive the trip. Every speculated flow is graded on four independent axes, and a flow is flagged only when all four line up:
| Axis | High risk | Medium | Low risk |
|---|---|---|---|
| Payload fidelity | content passed verbatim | — | reduced to summaries or metadata |
| Attacker controllability | public or supply-chain, no auth | authenticated remote account | requires local access |
| Semantic executability | free-form text or code | — | structured data, IDs, enums |
| Sanitization | none | truncation or length limits | escaping, filtering, redaction |
MCPSec, in two stages
Stage 1 · speculate the data flows
An LLM first discovers the entities a server plainly implies (external APIs, storage, pipelines) from the full tool set, then, per tool, proposes additional entities that are merely plausible (intermediate caches, auth components, background services), admitting low-confidence ones on purpose so no candidate path is lost. After consolidation, a per-tool profile records direction, taint and transformations. A rule-based assembler then wires entities and profiles into concrete flows, marks an output edge as an injection surface when it carries untrusted data verbatim or has an untrusted inbound edge, and a structural validator throws away anything malformed. The speculation is deliberately over-approximate; the deterministic stage keeps it honest.
Stage 2 · assess the risk and write the ToC
Each validated flow goes to a single LLM call with the tool metadata, the four-axis rubric and the threat model, which returns the axis labels and the ToC in one reasoning trace so the narrative and the labels can be cross-checked against each other. Everything runs on GPT-5.4 at high reasoning effort for an average of $1.65 per server. The baseline it is compared with is the obvious alternative: one prompt per tool carrying the same threat model, rubric and ToC requirements but none of the entity discovery, flow assembly or graph-conditioned reasoning.
Four levels of evidence
MCPSec was run once on the raw metadata of 177 tools from 20 servers (search and retrieval, browser automation, authenticated SaaS collaboration, and infrastructure platforms) and its output frozen. Evaluators then judged it under progressively stronger evidence, ending with ethically controlled proof-of-concept injections that confirmed 95 tools as vulnerable:
| Evidence available | Question | MCPSec | LLM baseline |
|---|---|---|---|
| Metadata only | plausible to analysts? | 86.9% label agreement | 74.5% |
| + source code | survives white-box review? | 69.8% · 75.5% of ToCs hold | 58.4% |
| + benign execution | reaches the LLM at runtime? | 76.2% of flows reachable | — |
| Controlled PoCs | recovers the 95 confirmed? | 94 · 98.9% recall | 80 · 84.2% recall |
Agreement figures are on the 97 tools both methods flagged; MCPSec flagged 143 in total, the baseline 97.
The precision tells the other half of the story: 65.7% for MCPSec against 82.5% for the baseline. That trade is the design. No-box analysis reasons under implementation uncertainty, so it over-approximates to miss almost nothing and hands the analyst a ranked list to validate. The misses it avoids are telling: 11 of the baseline's 15 come from the SaaS category, where the attacker arrives through an authenticated co-tenant channel (a comment, an issue body, a shared document) that a direct prompt treats as trusted internal state and only a reconstructed data flow exposes. When MCPSec's own ToCs fail under source review, the cause is rarely a real defense: in 63.2% of failures the external content simply isn't preserved, in 57.9% it is transformed on the way out, and effective sanitization accounts for just 15.8%.
The ecosystem finding
Because the flows were validated against real code and real runs, the study doubles as an audit.
Of the 143 flagged tools, 132 (92.3%) apply no sanitization at all between external
data and the LLM context, 5 truncate or cap length, and only 6 escape, filter or redact. Those six
are the exceptions that prove the point: every one of them received an implausible ToC, so defenses
work when they exist; they just almost never exist. Servers treat a valid API response as safe model
input and put the trust boundary at the API rather than at the external source. The paper's
recommendations follow directly: minimize output (return identifiers, counts and status when a task
allows it), keep provenance and separate external content structurally from server-generated text,
and treat heuristic filters as partial at best. Two case studies bracket the method. In
chrome-devtools-mcp, a tool description that merely mentions running a JavaScript
function in the current page, with an example that fetches a remote URL, was enough to derive a ToC
for exfiltrating browser state; a proof of concept on an unmodified install succeeded in two of three
runs. In a Slack server, the ToC for a user-group update depends on a paid workspace and a
privileged OAuth scope, so black-box testing is blocked, yet the metadata-derived hypothesis stands
and tells an authorized tester exactly what to check.
Where no-box fits
No-box analysis is a front-end triage layer: cheap enough to run at listing or install time, and precise enough about its assumptions that the ToCs it emits prioritize the white-box, gray-box or runtime work that follows. It suits vulnerability classes whose exploitation conditions can be stated in terms of data flow (injection, SQLi, XSS, SSRF, path traversal) and less so memory corruption or race conditions, whose triggers hide in low-level detail. It also leans on MCP's unusually standardized descriptions; ecosystems with thinner specifications will yield more false positives. MCPSec reasons about one tool at a time, leaves multi-hop chains out of scope, and samples at most ten tools per server, so the study measures feasibility rather than ecosystem-wide prevalence.
Resources
Paper: arXiv:2609.10854 · preprint · led by Zehua Zhang, ASU
Vulnerable tools were reported to their vendors and all experiments ran in local, containerized settings. Analysis data, code and ToCs are to be released on acceptance, after the reported bugs are fixed.