Lessons · LLM10: Improper Output Handling · error
A string the model wrote is passed to eval, a shell, or innerHTML.
Caught by llm-output-insecure-handling · CWE-94, CWE-78, CWE-79
The code takes the model's reply and executes it: with eval, new Function, child_process.exec, or by writing it into the page with innerHTML or dangerouslySetInnerHTML.
Model output is untrusted input. A model can be talked into writing anything, and anyone who can influence its prompt can influence what it writes.
On the server, a string that reaches eval or a shell runs with your server's permissions. In the browser, a string that reaches innerHTML runs as your site, in your user's session.
Your app asks the model to write a small JavaScript function that formats a report, then runs it.
const reply = await generateText({ model, prompt });
const format = eval(reply.text);
The cost: Full control of your server process, or of your user's session in the browser. This is usually the worst finding on the list.
Paste this into Claude Code, Cursor, or whatever wrote the code. When llm-audit finds the mistake in your project, npx llm-audit prompt gives you the same prompt with every file and line filled in.
Fix this in my project: model output is run as code (eval, new Function, a shell command) or written into the page as HTML.
Please:
1. Remove the code execution. Do not replace it with a different way of running the string.
2. If the model needs to choose an action, have it return structured JSON with a fixed set of allowed values, validate it with a schema, and map each value to code I wrote.
3. If a shell command is truly needed, use execFile or spawn with a fixed command and an argument array, never a shell string.
4. If the output is shown in the page, render it as text, or sanitize it with DOMPurify first.
Show me the diff and list any other places in the file that run or render model output.A prompt that looks for the same mistake everywhere else, without changing anything yet.
Check this project for places that run AI model output as code, before I ship it.
Look for:
- eval, new Function, or setTimeout and setInterval called with a string.
- exec, execSync, or spawn with shell: true, where any part of the command is not a fixed string.
- innerHTML or dangerouslySetInnerHTML set from a model reply.
- Code that asks an AI model for code, a shell command, HTML, or a SQL query and then runs the result.
For each one, give me the file and line, say where the string comes from, and explain what an attacker could run through it. Do not change any code yet.npx llm-audit scan
Runs on your machine with Semgrep. Nothing is uploaded.