Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    Baselines and deltas

    View as Markdown

    Review a change by what it adds: the AI components it brings in, and the findings that are new, instead of the whole repository’s history. You will compare against a git branch, save a baseline file, and fail a job when a change adds an AI component.

    • The Keryn CLI (install).
    • A git clone that contains the base commit. In CI that means a full fetch (fetch-depth: 0); see CI.
    • Nothing is fetched: --base reads a ref of the clone you have, or a file you saved.

    keryn diff runs the AI inventory on the working tree and on the base, and prints what differs.

    Terminal window
    keryn diff . --base origin/main
    Keryn — Checks passed (Go); dependency advisories not checked (offline)
    dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
    Analysis: complete · Dependency advisories: not checked (offline) · Policy: checks passed · Approval: not recorded
    base 69a8fd978be3df6646542bac4cd38507eae55067 (origin/main) → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
    Comparison: recorded finding delta; baseline compatibility not independently verified
    Change classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 0 cleared, 0 not comparable
    Cases: 0 current = 0 new + 0 existing + 0 unclassified; 0 cleared from previous assessment
    Newly blocking: 0 (blocking now, not blocking in the base run under this policy)
    Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
    Accounting: 5 recorded rule hits; 5 non-case observations; 0 suppressed rule hits; 0 failed checks
    diff: +2 / -0 components
    + library anthropic (llm-sdk)
    + model claude-sonnet-4-5 (model)
    recorded rule-hit changes: +2 / -0 (includes non-case observations)
    Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entries
    Coverage · source patterns: scanned 3 files; 29/29 requested rules; 0 skipped; 0 excluded
    Coverage · dataflow: not recorded
    Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
    Coverage · known-fix matching: not recorded
    Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
    Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
    Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
    Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
    Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
    Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
    Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
    Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
    Exit code: 0
    counts: 5 rule matches · 5 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
    rules: 29 rules available · 29 supported · 20 applied to the 3 scanned files

    The change in this repository adds a second provider (anthropic) and a model. +2 / -0 components is the inventory delta. Zero current cases means this inventory-only run reports observations rather than vulnerability cases. keryn diff uses the AI inventory rules only, so it is fast.

    2. Fail the job when a change adds a component

    Section titled “2. Fail the job when a change adds a component”
    Terminal window
    keryn diff . --base origin/main --fail-on-added
    Keryn — Policy blocked (Hold); dependency advisories not checked (offline)
    2 component(s) added vs base
    dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
    Analysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recorded
    base 69a8fd978be3df6646542bac4cd38507eae55067 (origin/main) → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
    Comparison: recorded finding delta; baseline compatibility not independently verified
    Change classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 0 cleared, 0 not comparable
    Cases: 0 current = 0 new + 0 existing + 0 unclassified; 0 cleared from previous assessment
    Newly blocking: 0 (blocking now, not blocking in the base run under this policy)
    Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
    Accounting: 5 recorded rule hits; 5 non-case observations; 0 suppressed rule hits; 0 failed checks
    diff: +2 / -0 components
    + library anthropic (llm-sdk)
    + model claude-sonnet-4-5 (model)
    recorded rule-hit changes: +2 / -0 (includes non-case observations)
    Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entries
    Coverage · source patterns: scanned 3 files; 29/29 requested rules; 0 skipped; 0 excluded
    Coverage · dataflow: not recorded
    Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
    Coverage · known-fix matching: not recorded
    Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
    Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
    Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
    Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
    Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
    Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
    Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
    Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
    Exit code: 1
    counts: 5 rule matches · 5 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
    rules: 29 rules available · 29 supported · 20 applied to the 3 scanned files

    --fail-on-added turns an added component into exit 1. It looks at components only. There is no flag that fails on added findings alone: --fail-on still judges every finding, so on a repository that already has findings, gate on components and keep the findings visible as the delta (step 4), or let suppressions carry the accepted ones.

    A branch is not always available: a release job, a scheduled scan, a repository you only receive as a zip. Save what the last good state looked like.

    Terminal window
    keryn scan . --json keryn-out/baseline.json --aibom keryn-out/baseline.cdx.json --quiet

    The scan exits 1 because this fixture has findings; the two files are written either way. --base takes a CycloneDX AI-BOM or a Keryn result JSON. Keep the file where your pipeline keeps artifacts (a release asset, a CI cache). Keryn recognises baseline reports by their contents and records them as skipped in coverage. Keep the baseline outside the checkout if you want it excluded from coverage accounting.

    Now make a change that adds a vector store:

    import chromadb
    store = chromadb.Client().create_collection("tickets")
    def remember(ticket_id: str, text: str) -> None:
    store.add(ids=[ticket_id], documents=[text])
    Terminal window
    keryn diff . --base keryn-out/baseline.cdx.json --fail-on-added
    Keryn — Policy blocked (Hold); dependency advisories not checked (offline)
    1 component(s) added vs base
    dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
    Analysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recorded
    base keryn-out/baseline.cdx.json → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
    Comparison: not comparable: baseline has no findings
    Cases: 0 current = 0 new + 0 existing + 0 unclassified; unknown cleared from previous assessment
    Newly blocking: unknown (no complete base run to compare with)
    Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
    Accounting: 6 recorded rule hits; 6 non-case observations; 0 suppressed rule hits; 0 failed checks
    diff: +1 / -0 components
    + library chromadb (vector-db)
    Exceptions: 0 expired suppression entries; changes since the base unknown (no base run with its exceptions)
    Coverage · source patterns: scanned 4 files; 29/29 requested rules; 0 skipped; 0 excluded
    Coverage · dataflow: not recorded
    Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
    Coverage · known-fix matching: not recorded
    Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
    Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
    Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
    Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
    Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
    Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
    Limitation: Maintainer/install-hook transition not checked without --base evidence; a lockfile alone does not record authenticated maintainer history.
    Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
    Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
    Exit code: 1
    counts: 6 rule matches · 6 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
    rules: 29 rules available · 29 supported · 20 applied to the 4 scanned files

    Only chromadb is new, and the job fails on it. Coverage counts files excluded from this analysis, including earlier Keryn outputs. A CycloneDX baseline compares components only; it cannot establish case changes.

    With a full scan and a result JSON as the base, the console separates case changes from recorded rule-hit changes (which include inventory observations), and the Markdown summary (--markdown) lists the changes for a pull request comment.

    Terminal window
    keryn scan . --base keryn-out/baseline.json --quiet --markdown keryn-out/summary.md
    Terminal window
    grep -A3 "Changes vs base" keryn-out/summary.md
    ### Changes vs base: +1 / -0
    - added `chromadb` (library, vector-db)

    Findings are compared by their fingerprint (keryn-fp/v1: rule, file and the matched code), not by line number, so code that moves usually stays the same finding. Usually: a finding whose own text includes a location can show once as removed and once as added after an unrelated edit above it. Treat the findings delta as a pointer for the reviewer and the component delta as the exact one.

    Scans that compare with a base are never answered from the rescan cache: the comparison is the point.

    Put the base in .keryn.yml so the CLI, the editors and CI agree. The language server then marks every finding that is new against it with [new]:

    .keryn.yml
    apiVersion: keryn.dev/config/v1
    base: origin/main

    A policy can judge changes too: changes.model: block fails a model swap, changes.prompt: review warns on a prompt edit.

    The delta is the set of new edges into your system: a provider, a model, a vector store the code now reaches that the base did not. Reviewing that list is the cheapest control on a pull request, and --fail-on-added makes it a gate. It does not say the existing findings are fine, and it sees only what the code references (static references only, not runtime use).

    • Wire it into a pipeline: CI.
    • Accept a finding on purpose, with a reason and an expiry: suppressions.
    • Decide what a change may add: policies.