Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    What Keryn does not do

    View as Markdown
    • General static analysis recall is low. On a benchmark of 582 real vulnerability-fix pairs, the default keryn scan finds 62% of the AI-specific pairs and 21% of the general ones (40% overall, about 36% without the known-fix rung). A clean scan is not a statement that code is safe. In the held-out November 2026 round, detections rose from 13 to 15 of 83 scored pairs. JavaScript rose from 1 to 3 of 15, while TypeScript stayed at 3 of 29 and Python at 9 of 42. Detections that still fired on the fixed code rose from 10 of 13 to 12 of 15. See the rules repository receipts scoreboard-2026-11-18-engines-base-holdout-jstspy.json and scoreboard-2026-11-18-engines-r2-holdout-jstspy.json.
    • Pattern findings often survive the fix. See the proof ladder. Treat rung-1 findings as leads.
    • Cross-file reach is bounded, and differs by language. Deep follows flows across import boundaries. For Python and TypeScript/JavaScript it joins chains file after file (up to 12 edges per path), follows re-exports and aliases for up to 4 hops and return values for up to 4 rounds, types a receiver from the text (Foo(), x: Foo, this) and follows extends/implements to subclass overrides (at most 4). Java, C# and Go follow subtype dispatch and return values across files under the same caps, but resolve an import one level deep. C and C++ follow a quoted #include one level, with no inheritance. Other languages are analysed within one file. Deep does not infer return types, fields or generics, and an ambiguous call adds no edge. This is measured on a synthetic suite of 47 repositories (30 of 30 positives found, 17 of 17 negatives clean); the real-fix benchmark has changed files only, so real cross-file recall is not measured.
    • Coverage is stated, not assumed. Rules a language cannot run are counted in every output; exit code 3 flags it.
    • Keryn’s own earlier outputs are not scanned. A result, SARIF, AI-BOM or bundle file that Keryn wrote is recognised by its content and skipped, and the coverage line counts them.
    • .gitignore is respected by the CLI, the language server and the web page’s folder picker (Chromium). Ignored paths are counted on the coverage line, and --no-ignore scans them. A pattern that matches a file git tracks does not hide it.
    • A zip scanned in Keryn Web keeps every path it carries. Only a single wrapping folder (a GitHub zipball, or a zipped folder) is removed.
    • If Deep cannot run in Keryn Web (engine missing, out of memory, a blocked worker), the Results view says “Deep did not run, so there is no cross-file proof here” and gives the reason. A result at proof 1 and 2 without that line would be a failure to read, not a clean repository.
    • The web page and the CLI reach the same cross-file proof with different rule sets. The page scans with the risk and AI-BOM packs, the CLI with every bundled pack, so finding counts differ while the proof-3 findings agree.

    A match says that code resembles a block an advisory’s fix removed. It is not proof of exploitability. The benchmark recall of this rung is not independent evidence, because signatures may come from the same advisories as the benchmark pairs.

    Reproduction needs Docker. It runs only on code you point it at, inside a hardened container, within a budget you set. A result it cannot decide is reported as indeterminate. Today its hermetic driver handles Python only.

    The trust map and the routes-to-tools view (Keryn Web’s Agent surface) read Python, JavaScript/TypeScript, Java, Kotlin, C# and Go source; What the map reads lists the frameworks. Other languages (Rust, PHP, Ruby and the rest) are scanned by the rules but add no tools, model calls or routes to the map. For Java, Kotlin, C# and Go a route counts only when its own handler, or a function in the same file, reaches a model: a Spring controller that calls an injected service in another file is not linked.

    A route handler, or a function a tools list names, is followed one import away from the file that uses it and no further. A list re-exported through a package’s __init__.py from a third module is missed, and toolsets built at run time (an MCP client’s tool list, a computer-use toolset) name no tool in the source.

    A model’s verdict in Keryn Web is a second opinion. It can be wrong, and it does not change a finding’s proof level. See agentic triage.

    Keryn on your machine never probes a host. The one place Keryn looks at someone else’s address is the optional Keryn Cloud, where you type an address and Keryn’s servers read it; active port checks run only on a domain you verified. What it reports is what the servers observed from the outside, at one moment.

    Policy templates, the controls view and framework mappings cite clauses and controls for reference only. A control marked clear means Keryn’s rules for it found nothing. No Keryn output is a compliance determination.