Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    benchmark-gaps-v1

    View as Markdown

    14 rules · Keryn Software Licence · rules 0.8.0 · all packs

    How to read a rule
    Severity
    How bad the finding is if it is real: error fails a default gate, warning and info do not.
    Confidence
    P0 exploitable as written; P1 dangerous in context; P2 a heuristic or a posture check: confirm by reading the code.
    False-positive risk
    low: an exact API or flag, so hits are almost always real; medium: the pattern is legitimate in many programs, so check each hit; high: it finds a lead to confirm, so expect benign hits.
    Edge
    The relationship the finding sits on. reaches: content or traffic gets to a thing; calls: one thing invokes another; reads: one thing reads from another; writes: one thing writes to another; sends: data leaves for another party; grants: authority is given or left unchecked; depends_on: something relies on a component it does not verify; trusts: one party believes another without evidence. The boundary it crosses, when there is one: user->prompt (untrusted input reaching a prompt); model->code (model output reaching code, a shell, a query or a page); secret->outside (a secret leaving the trusted side); supply-chain (code, models or prompts pulled in from outside).
    Taxonomy chips
    References for readers who work in OWASP LLM 2025, OWASP Agentic, CWE or NIST AI RMF. They say where a rule belongs, not that a framework is satisfied.

    warning python data-flow rule openai, pinecone, chromadb, langchain

    Text that came back from a retrieval call (vector index query, retriever, similarity search, or a list of retrieved documents) is placed in the system role, so a document anyone can plant becomes the model’s instructions. Keep retrieved text in a user or tool message, fenced and labelled as data.

    Confidence P1: dangerous in context. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: reaches content or traffic gets to a thing; it crosses the user->prompt boundary (untrusted input reaching a prompt).

    CWE-77 LLM01 ASI01

    Fix: Put retrieved passages in a user-role message (or a tool result) wrapped in explicit delimiters, and keep the system message to instructions you wrote. If a system message must reference documents, reference them by id and have the model fetch them as data.

    References: https://developers.openai.com/api/docs/guides/text#message-roles-and-instruction-following · https://docs.pinecone.io/reference/api/2024-10/data-plane/query

    keryn.gaps.instructions-concatenated-with-input

    Section titled “keryn.gaps.instructions-concatenated-with-input”

    warning python openai, python

    A constant that holds the system instructions is concatenated (or formatted) with a variable into one prompt string, so the caller’s text sits in the same instruction channel as yours and can override it. Send instructions as the system message and the variable text as a separate user message.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: reaches content or traffic gets to a thing; it crosses the user->prompt boundary (untrusted input reaching a prompt).

    CWE-77 LLM01

    Fix: Keep the instruction constant in the system role and pass user-supplied text as its own user-role message; never build one string from both.

    References: https://developers.openai.com/api/docs/guides/text#message-roles-and-instruction-following

    warning python flask, fastapi, python

    A provider API key, token or secret read from the environment is placed in a response body returned to the caller. Keep provider credentials server-side; return a model name or an opaque session id instead.

    Confidence P1: dangerous in context. False-positive risk low: an exact API or flag, so hits are almost always real.

    Edge inspected: sends data leaves for another party; it crosses the secret->outside boundary (a secret leaving the trusted side).

    CWE-200 LLM02

    Fix: Never serialise provider credentials into an HTTP response; proxy provider calls server-side or mint short-lived, scoped tokens for clients.

    References: https://flask.palletsprojects.com/en/stable/api/#flask.json.jsonify · https://docs.python.org/3/library/os.html#os.environ

    warning python python, logging

    The prompt, the message list or the transcript is written to a log or stdout as-is, so whatever a user typed (PII, secrets) is persisted verbatim. Log a redacted copy, a hash or counts; never the raw content.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: sends data leaves for another party.

    CWE-532 LLM02

    Fix: Redact before logging (redact(messages)), or log structural facts (message count, token count, a content hash) and keep the raw transcript in a store with its own access control and retention.

    References: https://docs.python.org/3/library/logging.html

    keryn.gaps.bytes-from-network-deserialized

    Section titled “keryn.gaps.bytes-from-network-deserialized”

    warning python data-flow rule requests, pickle, torch, joblib

    Bytes fetched over HTTP are deserialised (pickle, torch.load, joblib, marshal) or written to the artifact store without a digest check, so whoever controls the URL or the network path controls what runs when the artifact is loaded. Verify a pinned digest or signature before deserialising or storing.

    Confidence P1: dangerous in context. False-positive risk low: an exact API or flag, so hits are almost always real.

    Edge inspected: depends_on something relies on a component it does not verify; it crosses the supply-chain boundary (code, models or prompts pulled in from outside).

    CWE-502 CWE-494 LLM03 LLM05

    Fix: Compare the downloaded bytes against a pinned SHA-256 (or verify a signature) before pickle.loads / torch.load / writing them into the model cache; prefer safetensors for weights.

    References: https://docs.python.org/3/library/pickle.html · https://docs.pytorch.org/docs/stable/generated/torch.load.html · https://requests.readthedocs.io/en/latest/api/#requests.get

    warning python data-flow rule mcp, python

    A path taken from the tool-call arguments is opened, read or deleted without confining it to a root directory, so the model (or whoever steers it) chooses which file the tool touches. Resolve the path and reject it unless it is inside an allowed root.

    Confidence P1: dangerous in context. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: reads one thing reads from another.

    CWE-22 LLM06 ASI02

    Fix: Build the path as (ROOT / arguments["path"]).resolve(), refuse it unless is_relative_to(ROOT), and prefer an allowlist of files or ids over free-form paths.

    References: https://modelcontextprotocol.io/specification/2025-06-18/server/tools · https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_relative_to

    keryn.gaps.vector-query-without-tenant-filter

    Section titled “keryn.gaps.vector-query-without-tenant-filter”

    warning python pinecone, chromadb, langchain

    A vector index shared by several users or tenants is queried on similarity alone, with no metadata filter, so one caller’s query can return another’s documents. Pass a tenant or owner filter on every query.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: grants authority is given or left unchecked.

    CWE-862 LLM08

    Fix: Add filter={"tenant": tenant_id} (Pinecone), where={"owner": user_id} (Chroma) or the store’s equivalent to every query, or give each tenant its own namespace or collection.

    References: https://docs.pinecone.io/guides/indexes/index-data-overview#metadata · https://docs.trychroma.com/docs/querying-collections/metadata-filtering

    warning python openai, anthropic, google-genai

    A while True loop calls the model on every iteration with no iteration, time or token budget, so the model (or an injected instruction) decides how long the loop runs and how much it spends. Bound the loop with a counter or a budget and stop when it is exhausted.

    Confidence P1: dangerous in context. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: grants authority is given or left unchecked.

    CWE-770 LLM10 ASI08

    Fix: Use for turn in range(MAX_TURNS): or a budget check such as if turns >= MAX_TURNS: break inside the loop, and cap total tokens; a model-emitted “DONE” is not a bound.

    References: https://developers.openai.com/api/docs/guides/function-calling#handling-function-calls

    warning python python

    The module to import is built from a variable, so whoever controls that value (a plan step, a tool argument, model output) chooses what gets imported and therefore what runs at import time. Map the value to a fixed table of handlers instead of importing by name.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: depends_on something relies on a component it does not verify; it crosses the supply-chain boundary (code, models or prompts pulled in from outside).

    CWE-470 LLM06 ASI02

    Fix: Keep a dict of allowed handler modules keyed by the step name (HANDLERS[name]) and raise on an unknown key; never pass a formatted or concatenated string to import_module / __import__.

    References: https://docs.python.org/3/library/importlib.html#importlib.import_module

    warning python python

    A file path is built by joining a base directory with a value and then written, with no check that the result stays under the base, so a value such as ../../.ssh/authorized_keys writes outside the directory. Resolve the joined path and refuse it unless it starts with the base.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: writes one thing writes to another.

    CWE-22 LLM06 ASI02

    Fix: After joining, real = os.path.realpath(path) and check real.startswith(BASE + os.sep) (or Path(...).resolve().is_relative_to(BASE)) before opening for write; reject or rename anything else.

    References: https://docs.python.org/3/library/os.path.html#os.path.realpath · https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.is_relative_to

    keryn.gaps.money-moving-call-without-approval

    Section titled “keryn.gaps.money-moving-call-without-approval”

    warning python requests, python

    A call that moves money or changes a customer account (refund, payment, charge, transfer, payout) is made from a tool with no approval, confirmation or authorisation step in the same function, so the model’s say-so is enough to execute it. Gate it behind an explicit human approval of the exact amount and target.

    Confidence P1: dangerous in context. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: grants authority is given or left unchecked.

    CWE-862 LLM06 ASI02 ASI05

    Fix: Call an approval helper with the exact ticket, amount and reason before the request and return early when it is refused; log the approver with the action.

    References: https://requests.readthedocs.io/en/latest/api/#requests.post

    warning python data-flow rule requests, httpx, python

    A URL taken straight from a function parameter (typically a tool argument the model fills) is fetched from inside the service, so the model can point the service at internal endpoints or the instance metadata service. Resolve the host and check it against an allowlist before fetching; a well-formedness check is not a destination check.

    Confidence P1: dangerous in context. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: reaches content or traffic gets to a thing; it crosses the user->prompt boundary (untrusted input reaching a prompt).

    CWE-918 LLM06 ASI02

    Fix: Parse the URL, resolve the host, refuse private, loopback and link-local ranges, and allow only hosts on an explicit list (if urlparse(url).hostname not in ALLOWED_HOSTS: return); then fetch with redirects disabled.

    References: https://requests.readthedocs.io/en/latest/api/#requests.get · https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse

    keryn.gaps.table-dispatch-from-model-decision

    Section titled “keryn.gaps.table-dispatch-from-model-decision”

    warning python python

    Both the tool name and its keyword arguments come from a decision function fed with untrusted content, and they are dispatched through a table (TOOLS[name](**kwargs)) with no mediation, so an injected instruction picks the tool and its arguments. Validate the chosen tool against an allowlist and its arguments against a schema before dispatching.

    Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.

    Edge inspected: calls one thing invokes another; it crosses the model->code boundary (model output reaching code, a shell, a query or a page).

    CWE-862 LLM06 ASI02 ASI05

    Fix: Check name in ALLOWED_TOOLS, validate kwargs with the tool’s schema, and require approval for tools that change state before calling TOOLS[name](**kwargs).

    References: https://developers.openai.com/api/docs/guides/function-calling#handling-function-calls

    keryn.gaps.text-written-to-tempfile-then-executed

    Section titled “keryn.gaps.text-written-to-tempfile-then-executed”

    warning python python, mcp

    Text received by the tool (or produced by the model) is written to a temporary file and the file is then run with the Python interpreter, so the “argv is a literal list” shape hides arbitrary code execution: the payload travels through the file, not the command line. Execute only vetted, committed scripts, or run untrusted code in a sandbox with no network and no credentials.

    Confidence P1: dangerous in context. False-positive risk low: an exact API or flag, so hits are almost always real.

    Edge inspected: calls one thing invokes another; it crosses the model->code boundary (model output reaching code, a shell, a query or a page).

    CWE-94 LLM06 ASI02 ASI05

    Fix: Do not execute text that arrived as a tool argument or model output. If code execution is the tool’s purpose, run it in an isolated sandbox (separate process namespace, no network, no secrets, strict time and memory limits) and never on the host interpreter with the service’s credentials.

    References: https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile · https://docs.python.org/3/library/subprocess.html#security-considerations