Lessons · LLM03: Excessive Agency · error
A tool name from the model is used to look up and call a function, with no check.
Caught by tool-call-dispatch-without-allowlist · CWE-470, CWE-77
With tool calling, the model replies with the name of a tool and some arguments, and your code runs the matching function. A common way to wire that up is a lookup: handlers[call.toolName](call.args).
That lookup trusts the model's choice completely. If the handlers object holds anything besides the tools you meant to offer for this request, the model can call it. Arguments are passed through with no validation.
The model's choice can be steered by the user, or by text the model reads, like a web page or an email.
Your assistant can read and summarize a user's emails. The same handlers object also holds deleteAccount and sendEmail, used elsewhere in the app.
for (const call of result.toolCalls) {
await handlers[call.toolName](call.args);
}
The cost: The attacker gets to run any function in your handler map, with arguments they choose, as the signed-in user.
Paste this into Claude Code, Cursor, or whatever wrote the code. When llm-audit finds the mistake in your project, npx llm-audit prompt gives you the same prompt with every file and line filled in.
Fix this in my project: a function is chosen by the model's tool name and called without checking the name against an allowed list.
Please:
1. Define the allowed tool names for this request as a fixed list.
2. Before calling anything, reject any tool name not on that list and log the rejected name (not the arguments).
3. Validate each tool's arguments with its own schema before the handler runs.
4. For tools that change data or send messages, require a confirmation step from the user.
Show me the diff. Use a switch statement on literal tool names if that is clearer than a lookup.A prompt that looks for the same mistake everywhere else, without changing anything yet.
Check how this project runs AI tool calls, before I ship it.
Look for:
- Tool names from the model used to look up a function, like handlers[call.toolName] or tools[call.function.name].
- Loops over toolCalls or tool_calls that never check the name against a fixed list for that request.
- Tool arguments passed to a handler without schema validation.
- Tools that delete data, send messages, or spend money with no confirmation from the user.
For each one, give me the file and line and explain what a malicious web page or email could make the model do. Do not change any code yet.npx llm-audit scan
Runs on your machine with Semgrep. Nothing is uploaded.