AppSec: find and prove a chain in an agent repo
Goal. Take a repository that gives a model a way to act, find the path from the model’s reply to a dangerous call, prove it well enough that a colleague can check it without trusting you, then break the path and show it is gone.
Time. 25 minutes. You are an application security engineer reviewing an agent before it ships.
You need. The Keryn CLI (install), git, and jq for step 6. Everything runs on your machine and nothing is uploaded.
1. The repository
Section titled “1. The repository”The example is a help-desk agent. A ticket goes to a model, the model answers with a shell command that should diagnose the problem, and the agent runs it. Make a directory and write the three files. They are written for this page and are not anyone’s real code.
import subprocessimport urllib.request
def run_command(command: str) -> str: """Run a shell command for the agent and return what it printed.""" done = subprocess.run(command, shell=True, capture_output=True, text=True, timeout=30) return done.stdout
def fetch_page(url: str) -> str: """Download a page so the agent can read it.""" with urllib.request.urlopen(url, timeout=10) as resp: return resp.read().decode("utf-8", "replace")import os
from anthropic import Anthropic
from helpdesk.tools import run_command
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
SYSTEM = "You are a help desk agent. Reply with one shell command that diagnoses the ticket."
def handle_ticket(ticket_text: str) -> str: reply = client.messages.create( model="claude-sonnet-4-5", max_tokens=200, system=SYSTEM, messages=[{"role": "user", "content": ticket_text}], ) command = reply.content[0].text return run_command(command)anthropic==0.40.0Commit it. The commit is the “before” the rest of the page compares against.
touch helpdesk/__init__.pygit add -Agit commit -q -m "help-desk agent"2. Scan it
Section titled “2. Scan it”keryn scan . --verboseKeryn — Policy blocked (Hold); dependency advisories not checked (offline) 1 violation(s)/finding(s) at or above 'error' dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks themAnalysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recordedbase not recorded → analyzed b678d0947e93e31a844f0ba149d7cab252490bcbComparison: not compared: no finding baseline suppliedCases: 2 current = 0 new + 0 existing + 2 unclassified; unknown cleared from previous assessmentNewly blocking: unknown (no complete base run to compare with)Cases: 2 of 2 shown (limit 20); 0 omitted by display limit [blocking; unclassified] Model-generated output is passed to a system shell command. This can lead to command injection. helpdesk/tools.py:7 · path-cross-file · 4 contributing rule(s) · sha256:308d65b90cda6717aea1e4504d08365de4b1996bcabbfab40407f2b3398b9150 Cross-file flow [proof 3], confirmed by cross-file analysis source helpdesk/agent.py:13 (client.messages.create) propagation helpdesk/agent.py:19 (command) propagation helpdesk/tools.py:5 (command) sink helpdesk/tools.py:7 [review; unclassified] A URL taken straight from a function parameter (typically a tool argument the model fills) is fetched from inside the service, so the model can point the service at internal endpoints or the instance metadata service. Resolve the host and check it against an allowlist before fetching; a well-formedness check is not a destination check. helpdesk/tools.py:13 · path-in-file · 3 contributing rule(s) · sha256:dbc01b85e4e108d17cb918da46fe9415c387d250f1a848ac0d8326f6930da9d1 Within-file flow [proof 2]Accounting: 12 recorded rule hits; 10 non-case observations; 0 suppressed rule hits; 1 failed checksExceptions: 0 expired suppression entries; changes since the base unknown (no base run with its exceptions)Coverage · source patterns: scanned 4 files; 1598/1598 requested rules; 0 skipped; 0 excludedCoverage · dataflow: executed; see the recorded cross-file analysis scopeCoverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)Coverage · known-fix matching: executed; see recorded known-fix scope Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation. Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules. Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/). Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning). Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only. Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only. Limitation: Maintainer/install-hook transition not checked without --base evidence; a lockfile alone does not record authenticated maintainer history. Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents. Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.Exit code: 1counts: 12 rule matches · 12 findings · 9 excluding AI-BOM inventory · 1 blocking before answersrules: 1,598 rules available · 1,598 supported · 728 applied to the 4 scanned fileserrors: 1 confirmed error · 0 unconfirmed (pattern match only)warnings: 1 confirmed warning · 0 unconfirmed (pattern match only)pattern findings: 4 confirmed by cross-file analysis3. Read the chain
Section titled “3. Read the chain”The --verbose flag lists each finding with its proof rung and, when there is one, the flow. The first carries proof 3path-cross-file from the start: one path, reported once at the call that runs it, tools.py:7, with every rule that fires on that path counted among its contributing rules. Read it as a sentence.
- source,
helpdesk/agent.py:13: the model call. What comes back is text an attacker can influence through the ticket. - propagation,
agent.py:19: the text is assigned tocommand. - propagation,
tools.py:5: it is passed intorun_commandin another file. - sink,
tools.py:7:subprocess.run(command, shell=True).
That is one edge: the agent calls a shell with text the model wrote. The text crossed from a model, which anyone can steer through its input, into code that runs with your privileges.
Some of those contributing rules are patterns on the same sink. A pattern on its own only shows that shell=True appears near a model call, which would be proof 1pattern. Here the cross-file data-flow pass followed the model’s output to that call, so the case carries the trace and proof 3path-cross-file, marked confirmed by cross-file analysis. The path is what makes a finding a claim about this repository: it ties the model’s output to the call across a file boundary. The proof ladder says what each rung means, and paths and chains says how paths combine.
Ask the CLI what the pattern rule claims, offline:
keryn explain seceng.llm04.python-model-output-to-system-shellseceng.llm04.python-model-output-to-system-shell
Model-generated output is passed to a system shell command. This can lead to command injection.
Severity ERRORConfidence P1FP risk mediumKind vulnEdge calls (boundary: model->code)Languages pythonTechnology llm04-insecure-output-handlingOWASP LLM LLM05 Improper Output HandlingOWASP AgenticASI05 Unexpected Code Execution; ASI02 Tool Misuse and ExploitationCWE CWE-78
Framework ids are references for readers who work in those frameworks, not a compliance claim.The Edge line says which edge kind the rule inspects and which boundary it crosses. The same lists are in the rule catalog and the taxonomies.
Zip the directory, open keryn.app, choose Zip / file and drop the zip. The scan runs in a worker in the tab and nothing is uploaded.






A zip keeps every path it carries, whether the files sit at its root or in one folder, so the cross-file import Deep needs resolves. If Deep cannot run, the Results view says “Deep did not run, so there is no cross-file proof here” and gives the reason; use the CLI for the proof in that case.
You just did this in steps 2 and 3. --json writes the same findings, flows included, for scripts.
4. Check the rule yourself
Section titled “4. Check the rule yourself”The cross-file path comes from the deep pass. The recorded single-file scan uses a small data-flow rule: the model’s reply is the source and subprocess.run is the sink. The code is the same function with the helper inlined. The result is proof 2path-in-file, a path inside one file.
import subprocess
from anthropic import Anthropic
client = Anthropic()
def handle_ticket(ticket_text):
reply = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=200,
messages=[{"role": "user", "content": ticket_text}],
)
command = reply.content[0].text
done = subprocess.run(command, shell=True, capture_output=True, text=True)
return done.stdoutRule
rules:
- id: review.model-reply-to-subprocess
languages: [python]
severity: ERROR
message: Text from a model call reaches a subprocess call.
mode: taint
pattern-sources:
- pattern: $CLIENT.messages.create(...)
pattern-sinks:
- pattern: subprocess.run($CMD, ...)1 finding · 1 of 1 rules ran on 1 file
- error
review.model-reply-to-subprocessline 14proof 2 path-in-fileText from a model call reaches a subprocess call.
Flow source (line 8) → propagation (line 8) → propagation (line 13)
done = subprocess.run(command, shell=True, capture_output=True, text=True)
Result computed when this page was built. Run an example with the CLI
Step 6 applies the same fix in the repository by keeping the model’s reply out of the subprocess call.
5. Keep the proof
Section titled “5. Keep the proof”Write the evidence bundle now, before you change any code. It holds the result, the SARIF and the AI-BOM in one file with a SHA-256 for every embedded document.
keryn bundle . --out keryn-out/beforekeryn verify-bundle keryn-out/before.bundle.jsonwrote evidence bundle -> keryn-out/before.bundle.jsonOK: 5 verified anchor(s), 1 unanchored, 0 digest mismatch(es)Verification recomputes every digest and exits 0 only when all of them match. Edit the recorded gate result to say the scan passed, which is what someone who wanted to hide the finding would do, and verify again:
jq '.documents.result.gate.exitCode = 0 | .documents.result.gate.reasons = []' keryn-out/before.bundle.json > keryn-out/edited.bundle.jsonkeryn verify-bundle keryn-out/edited.bundle.json | sed -n '1p'FAIL: 5 verified anchor(s), 1 unanchored, 1 digest mismatch(es)The command exits 1, and the lines after the first (cut off here) name the document whose digest no longer matches. A bundle does not prove the finding is exploitable. It proves that what you hand over is what the scan wrote, so a reviewer can argue about the finding rather than about whether it was changed. See evidence bundles.
6. Break the edge
Section titled “6. Break the edge”The model should choose which diagnostic to run, not write a command line. Make the change on a branch: the tool takes a name from a fixed table, and the agent passes the model’s one-word answer to it.
git switch -c allowlistimport subprocess
# The model can only pick a name from this table. It never supplies a command line.DIAGNOSTICS = { "disk": ["df", "-h"], "uptime": ["uptime"], "memory": ["vm_stat"],}
def run_diagnostic(name: str) -> str: """Run one of the fixed diagnostics and return what it printed.""" argv = DIAGNOSTICS[name] done = subprocess.run(argv, capture_output=True, text=True, timeout=30) return done.stdoutimport os
from anthropic import Anthropic
from helpdesk.tools import DIAGNOSTICS, run_diagnostic
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
SYSTEM = "You are a help desk agent. Reply with exactly one word from: " + ", ".join(DIAGNOSTICS)
def handle_ticket(ticket_text: str) -> str: reply = client.messages.create( model="claude-sonnet-4-5", max_tokens=20, system=SYSTEM, messages=[{"role": "user", "content": ticket_text}], ) name = reply.content[0].text.strip() return run_diagnostic(name)The new tools.py also drops fetch_page, which nothing used. Scan the branch against main and write the delta as JSON:
keryn scan . --base main --json keryn-out/after.jsonKeryn — Checks passed (Go); dependency advisories not checked (offline) dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks themAnalysis: complete · Dependency advisories: not checked (offline) · Policy: checks passed · Approval: not recordedbase b678d0947e93e31a844f0ba149d7cab252490bcb (main) → analyzed b678d0947e93e31a844f0ba149d7cab252490bcb-dirtyComparison: recorded finding delta; baseline compatibility not independently verifiedChange classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 2 cleared, 0 not comparableCases: 0 current = 0 new + 0 existing + 0 unclassified; 2 cleared from previous assessmentNewly blocking: 0 (blocking now, not blocking in the base run under this policy)Cases: 0 of 0 shown (limit 20); 0 omitted by display limitAccounting: 11 recorded rule hits; 11 non-case observations; 0 suppressed rule hits; 0 failed checksdiff: +0 / -0 components recorded rule-hit changes: +2 / -3 (includes non-case observations)Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entriesCoverage · source patterns: scanned 4 files; 1598/1598 requested rules; 0 skipped; 0 excludedCoverage · dataflow: executed; see the recorded cross-file analysis scopeCoverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)Coverage · known-fix matching: executed; see recorded known-fix scope Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation. Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules. Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/). Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning). Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only. Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only. Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents. Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.Exit code: 0counts: 11 rule matches · 11 findings · 8 excluding AI-BOM inventory · 0 blocking before answersrules: 1,598 rules available · 1,598 supported · 728 applied to the 4 scanned fileswrote Keryn JSON -> keryn-out/after.jsonThe gate passes. List the findings the change took away that were backed by a path, and anything it added:
jq -r '.diff.findings.removed[] | select(.proof.level >= 2) | "removed \(.ruleId) \(.path):\(.line) proof \(.proof.level)"' keryn-out/after.jsonjq -r '.diff.findings.added[] | "added \(.severity) \(.ruleId) \(.path):\(.line)"' keryn-out/after.jsonremoved seceng.llm04.python-model-output-to-system-shell helpdesk/tools.py:7 proof 3removed keryn.gaps.parameter-url-fetched helpdesk/tools.py:13 proof 2added INFO keryn.vendor.anthropic-messages-without-metadata-user-id-py helpdesk/agent.py:13added INFO keryn.general.python.subprocess-without-shell helpdesk/tools.py:14The shell case at proof 3 is gone, along with the URL-fetch finding that went with the deleted function. The change added two informational findings, which are not a verdict on the fix. Write a second bundle so the repository holds the before and the after:
keryn bundle . --out keryn-out/afterkeryn verify-bundle keryn-out/after.bundle.jsonwrote evidence bundle -> keryn-out/after.bundle.jsonOK: 3 verified anchor(s), 1 unanchored, 0 digest mismatch(es)What you just proved
Section titled “What you just proved”- The edge. The agent calls a shell with text a model wrote, across the model-to-code boundary, with a path of four steps in two files.
- How well. Rung 3 of the proof ladder, not a pattern match. The ladder stops there for this page: rungs 4 and 5 need an advisory match or a reproduction, which this code does not have.
- That you can show it. A bundle whose digests anyone can recompute, and a second one after the change.
- That the fix removed the path and not only the pattern. The model’s reply no longer reaches a command line, and the delta shows which findings left.
What this does not show: that the allowlist is the right set of commands, that df -h is harmless to expose, or that nothing else in the agent acts on the model’s text. A scan reports what it can read. What Keryn does not claim lists the rest.
- Put the same check in front of every pull request: Platform and CI: gate pull requests.
- Draw this agent’s reach, not only one path: Model your agent on the Trust Canvas.
- Browse every rule that inspects the model-to-code boundary in the rule catalog, or the classes of finding in the taxonomies.