Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    AppSec: find and prove a chain in an agent repo

    View as Markdown

    Goal. Take a repository that gives a model a way to act, find the path from the model’s reply to a dangerous call, prove it well enough that a colleague can check it without trusting you, then break the path and show it is gone.

    Time. 25 minutes. You are an application security engineer reviewing an agent before it ships.

    You need. The Keryn CLI (install), git, and jq for step 6. Everything runs on your machine and nothing is uploaded.

    The example is a help-desk agent. A ticket goes to a model, the model answers with a shell command that should diagnose the problem, and the agent runs it. Make a directory and write the three files. They are written for this page and are not anyone’s real code.

    helpdesk/tools.py
    import subprocess
    import urllib.request
    def run_command(command: str) -> str:
    """Run a shell command for the agent and return what it printed."""
    done = subprocess.run(command, shell=True, capture_output=True, text=True, timeout=30)
    return done.stdout
    def fetch_page(url: str) -> str:
    """Download a page so the agent can read it."""
    with urllib.request.urlopen(url, timeout=10) as resp:
    return resp.read().decode("utf-8", "replace")
    helpdesk/agent.py
    import os
    from anthropic import Anthropic
    from helpdesk.tools import run_command
    client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
    SYSTEM = "You are a help desk agent. Reply with one shell command that diagnoses the ticket."
    def handle_ticket(ticket_text: str) -> str:
    reply = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=200,
    system=SYSTEM,
    messages=[{"role": "user", "content": ticket_text}],
    )
    command = reply.content[0].text
    return run_command(command)
    anthropic==0.40.0

    Commit it. The commit is the “before” the rest of the page compares against.

    Terminal window
    touch helpdesk/__init__.py
    git add -A
    git commit -q -m "help-desk agent"
    Terminal window
    keryn scan . --verbose
    Keryn — Policy blocked (Hold); dependency advisories not checked (offline)
    1 violation(s)/finding(s) at or above 'error'
    dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
    Analysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recorded
    base not recorded → analyzed b678d0947e93e31a844f0ba149d7cab252490bcb
    Comparison: not compared: no finding baseline supplied
    Cases: 2 current = 0 new + 0 existing + 2 unclassified; unknown cleared from previous assessment
    Newly blocking: unknown (no complete base run to compare with)
    Cases: 2 of 2 shown (limit 20); 0 omitted by display limit
    [blocking; unclassified] Model-generated output is passed to a system shell command. This can lead to command injection.
    helpdesk/tools.py:7 · path-cross-file · 4 contributing rule(s) · sha256:308d65b90cda6717aea1e4504d08365de4b1996bcabbfab40407f2b3398b9150
    Cross-file flow [proof 3], confirmed by cross-file analysis
    source helpdesk/agent.py:13 (client.messages.create)
    propagation helpdesk/agent.py:19 (command)
    propagation helpdesk/tools.py:5 (command)
    sink helpdesk/tools.py:7
    [review; unclassified] A URL taken straight from a function parameter (typically a tool argument the model fills) is fetched from inside the service, so the model can point the service at internal endpoints or the instance metadata service. Resolve the host and check it against an allowlist before fetching; a well-formedness check is not a destination check.
    helpdesk/tools.py:13 · path-in-file · 3 contributing rule(s) · sha256:dbc01b85e4e108d17cb918da46fe9415c387d250f1a848ac0d8326f6930da9d1
    Within-file flow [proof 2]
    Accounting: 12 recorded rule hits; 10 non-case observations; 0 suppressed rule hits; 1 failed checks
    Exceptions: 0 expired suppression entries; changes since the base unknown (no base run with its exceptions)
    Coverage · source patterns: scanned 4 files; 1598/1598 requested rules; 0 skipped; 0 excluded
    Coverage · dataflow: executed; see the recorded cross-file analysis scope
    Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
    Coverage · known-fix matching: executed; see recorded known-fix scope
    Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
    Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
    Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
    Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
    Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
    Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
    Limitation: Maintainer/install-hook transition not checked without --base evidence; a lockfile alone does not record authenticated maintainer history.
    Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
    Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
    Exit code: 1
    counts: 12 rule matches · 12 findings · 9 excluding AI-BOM inventory · 1 blocking before answers
    rules: 1,598 rules available · 1,598 supported · 728 applied to the 4 scanned files
    errors: 1 confirmed error · 0 unconfirmed (pattern match only)
    warnings: 1 confirmed warning · 0 unconfirmed (pattern match only)
    pattern findings: 4 confirmed by cross-file analysis

    The --verbose flag lists each finding with its proof rung and, when there is one, the flow. The first carries proof 3path-cross-file from the start: one path, reported once at the call that runs it, tools.py:7, with every rule that fires on that path counted among its contributing rules. Read it as a sentence.

    1. source, helpdesk/agent.py:13: the model call. What comes back is text an attacker can influence through the ticket.
    2. propagation, agent.py:19: the text is assigned to command.
    3. propagation, tools.py:5: it is passed into run_command in another file.
    4. sink, tools.py:7: subprocess.run(command, shell=True).

    That is one edge: the agent calls a shell with text the model wrote. The text crossed from a model, which anyone can steer through its input, into code that runs with your privileges.

    model replyshellcallsmodel to code
    The boundary the path crosses. Keryn calls it model to code.

    Some of those contributing rules are patterns on the same sink. A pattern on its own only shows that shell=True appears near a model call, which would be proof 1pattern. Here the cross-file data-flow pass followed the model’s output to that call, so the case carries the trace and proof 3path-cross-file, marked confirmed by cross-file analysis. The path is what makes a finding a claim about this repository: it ties the model’s output to the call across a file boundary. The proof ladder says what each rung means, and paths and chains says how paths combine.

    Ask the CLI what the pattern rule claims, offline:

    Terminal window
    keryn explain seceng.llm04.python-model-output-to-system-shell
    seceng.llm04.python-model-output-to-system-shell
    Model-generated output is passed to a system shell command. This can lead to command injection.
    Severity ERROR
    Confidence P1
    FP risk medium
    Kind vuln
    Edge calls (boundary: model->code)
    Languages python
    Technology llm04-insecure-output-handling
    OWASP LLM LLM05 Improper Output Handling
    OWASP AgenticASI05 Unexpected Code Execution; ASI02 Tool Misuse and Exploitation
    CWE CWE-78
    Framework ids are references for readers who work in those frameworks, not a compliance claim.

    The Edge line says which edge kind the rule inspects and which boundary it crosses. The same lists are in the rule catalog and the taxonomies.

    Zip the directory, open keryn.app, choose Zip / file and drop the zip. The scan runs in a worker in the tab and nothing is uploaded.

    The release decision Keryn Web opens after a scan of the help-desk repository: Policy blocked (Hold), the gate and policy it used, two Holds, each with its cross-file flow into a shell, and the coverage line.The release decision Keryn Web opens after a scan of the help-desk repository: Policy blocked (Hold), the gate and policy it used, two Holds, each with its cross-file flow into a shell, and the coverage line.
    The release decision Keryn Web opens after the scan: Policy blocked (Hold), with the two Holds and their cross-file flows. Taken from keryn.app main-DIcUUbkq.js.
    The AI-BOM view of Keryn Web for the help-desk scan: the model and the Anthropic SDK the agent references, each with where it is used.The AI-BOM view of Keryn Web for the help-desk scan: the model and the Anthropic SDK the agent references, each with where it is used.
    The AI-BOM: the model and the SDK the agent references, and where. Taken from keryn.app main-DIcUUbkq.js.
    The findings view of Keryn Web for the help-desk scan: each finding with its severity, its evidence level such as Within-file flow, the rule, and the file and line.The findings view of Keryn Web for the help-desk scan: each finding with its severity, its evidence level such as Within-file flow, the rule, and the file and line.
    The findings list: each finding carries its severity, its evidence level and the rule that raised it. Taken from keryn.app main-DIcUUbkq.js.

    A zip keeps every path it carries, whether the files sit at its root or in one folder, so the cross-file import Deep needs resolves. If Deep cannot run, the Results view says “Deep did not run, so there is no cross-file proof here” and gives the reason; use the CLI for the proof in that case.

    The cross-file path comes from the deep pass. The recorded single-file scan uses a small data-flow rule: the model’s reply is the source and subprocess.run is the sink. The code is the same function with the helper inlined. The result is proof 2path-in-file, a path inside one file.

    One file: model reply to subprocessExample result
    import subprocess
    from anthropic import Anthropic
    
    client = Anthropic()
    
    
    def handle_ticket(ticket_text):
        reply = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=200,
            messages=[{"role": "user", "content": ticket_text}],
        )
        command = reply.content[0].text
        done = subprocess.run(command, shell=True, capture_output=True, text=True)
        return done.stdout

    Rule

    rules:
      - id: review.model-reply-to-subprocess
        languages: [python]
        severity: ERROR
        message: Text from a model call reaches a subprocess call.
        mode: taint
        pattern-sources:
          - pattern: $CLIENT.messages.create(...)
        pattern-sinks:
          - pattern: subprocess.run($CMD, ...)

    1 finding · 1 of 1 rules ran on 1 file

    • errorreview.model-reply-to-subprocessline 14proof 2 path-in-file

      Text from a model call reaches a subprocess call.

      Flow source (line 8) → propagation (line 8) → propagation (line 13)

      done = subprocess.run(command, shell=True, capture_output=True, text=True)

    Result computed when this page was built. Run an example with the CLI

    Step 6 applies the same fix in the repository by keeping the model’s reply out of the subprocess call.

    Write the evidence bundle now, before you change any code. It holds the result, the SARIF and the AI-BOM in one file with a SHA-256 for every embedded document.

    Terminal window
    keryn bundle . --out keryn-out/before
    keryn verify-bundle keryn-out/before.bundle.json
    wrote evidence bundle -> keryn-out/before.bundle.json
    OK: 5 verified anchor(s), 1 unanchored, 0 digest mismatch(es)

    Verification recomputes every digest and exits 0 only when all of them match. Edit the recorded gate result to say the scan passed, which is what someone who wanted to hide the finding would do, and verify again:

    Terminal window
    jq '.documents.result.gate.exitCode = 0 | .documents.result.gate.reasons = []' keryn-out/before.bundle.json > keryn-out/edited.bundle.json
    keryn verify-bundle keryn-out/edited.bundle.json | sed -n '1p'
    FAIL: 5 verified anchor(s), 1 unanchored, 1 digest mismatch(es)

    The command exits 1, and the lines after the first (cut off here) name the document whose digest no longer matches. A bundle does not prove the finding is exploitable. It proves that what you hand over is what the scan wrote, so a reviewer can argue about the finding rather than about whether it was changed. See evidence bundles.

    The model should choose which diagnostic to run, not write a command line. Make the change on a branch: the tool takes a name from a fixed table, and the agent passes the model’s one-word answer to it.

    Terminal window
    git switch -c allowlist
    helpdesk/tools.py
    import subprocess
    # The model can only pick a name from this table. It never supplies a command line.
    DIAGNOSTICS = {
    "disk": ["df", "-h"],
    "uptime": ["uptime"],
    "memory": ["vm_stat"],
    }
    def run_diagnostic(name: str) -> str:
    """Run one of the fixed diagnostics and return what it printed."""
    argv = DIAGNOSTICS[name]
    done = subprocess.run(argv, capture_output=True, text=True, timeout=30)
    return done.stdout
    helpdesk/agent.py
    import os
    from anthropic import Anthropic
    from helpdesk.tools import DIAGNOSTICS, run_diagnostic
    client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
    SYSTEM = "You are a help desk agent. Reply with exactly one word from: " + ", ".join(DIAGNOSTICS)
    def handle_ticket(ticket_text: str) -> str:
    reply = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=20,
    system=SYSTEM,
    messages=[{"role": "user", "content": ticket_text}],
    )
    name = reply.content[0].text.strip()
    return run_diagnostic(name)

    The new tools.py also drops fetch_page, which nothing used. Scan the branch against main and write the delta as JSON:

    Terminal window
    keryn scan . --base main --json keryn-out/after.json
    Keryn — Checks passed (Go); dependency advisories not checked (offline)
    dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
    Analysis: complete · Dependency advisories: not checked (offline) · Policy: checks passed · Approval: not recorded
    base b678d0947e93e31a844f0ba149d7cab252490bcb (main) → analyzed b678d0947e93e31a844f0ba149d7cab252490bcb-dirty
    Comparison: recorded finding delta; baseline compatibility not independently verified
    Change classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 2 cleared, 0 not comparable
    Cases: 0 current = 0 new + 0 existing + 0 unclassified; 2 cleared from previous assessment
    Newly blocking: 0 (blocking now, not blocking in the base run under this policy)
    Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
    Accounting: 11 recorded rule hits; 11 non-case observations; 0 suppressed rule hits; 0 failed checks
    diff: +0 / -0 components
    recorded rule-hit changes: +2 / -3 (includes non-case observations)
    Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entries
    Coverage · source patterns: scanned 4 files; 1598/1598 requested rules; 0 skipped; 0 excluded
    Coverage · dataflow: executed; see the recorded cross-file analysis scope
    Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
    Coverage · known-fix matching: executed; see recorded known-fix scope
    Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
    Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
    Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
    Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
    Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
    Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
    Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
    Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
    Exit code: 0
    counts: 11 rule matches · 11 findings · 8 excluding AI-BOM inventory · 0 blocking before answers
    rules: 1,598 rules available · 1,598 supported · 728 applied to the 4 scanned files
    wrote Keryn JSON -> keryn-out/after.json

    The gate passes. List the findings the change took away that were backed by a path, and anything it added:

    Terminal window
    jq -r '.diff.findings.removed[] | select(.proof.level >= 2) | "removed \(.ruleId) \(.path):\(.line) proof \(.proof.level)"' keryn-out/after.json
    jq -r '.diff.findings.added[] | "added \(.severity) \(.ruleId) \(.path):\(.line)"' keryn-out/after.json
    removed seceng.llm04.python-model-output-to-system-shell helpdesk/tools.py:7 proof 3
    removed keryn.gaps.parameter-url-fetched helpdesk/tools.py:13 proof 2
    added INFO keryn.vendor.anthropic-messages-without-metadata-user-id-py helpdesk/agent.py:13
    added INFO keryn.general.python.subprocess-without-shell helpdesk/tools.py:14

    The shell case at proof 3 is gone, along with the URL-fetch finding that went with the deleted function. The change added two informational findings, which are not a verdict on the fix. Write a second bundle so the repository holds the before and the after:

    Terminal window
    keryn bundle . --out keryn-out/after
    keryn verify-bundle keryn-out/after.bundle.json
    wrote evidence bundle -> keryn-out/after.bundle.json
    OK: 3 verified anchor(s), 1 unanchored, 0 digest mismatch(es)
    • The edge. The agent calls a shell with text a model wrote, across the model-to-code boundary, with a path of four steps in two files.
    • How well. Rung 3 of the proof ladder, not a pattern match. The ladder stops there for this page: rungs 4 and 5 need an advisory match or a reproduction, which this code does not have.
    • That you can show it. A bundle whose digests anyone can recompute, and a second one after the change.
    • That the fix removed the path and not only the pattern. The model’s reply no longer reaches a command line, and the delta shows which findings left.

    What this does not show: that the allowlist is the right set of commands, that df -h is harmless to expose, or that nothing else in the agent acts on the model’s text. A scan reports what it can read. What Keryn does not claim lists the rest.