# Baselines and deltas

> Judge a change by what it adds, against a git ref or a saved result, and gate on new AI components.

Source: https://keryn.dev/docs/guides/baselines/

## Goal

Review a change by what it **adds**: the AI components it brings in, and the findings that are new, instead of the whole repository's history. You will compare against a git branch, save a baseline file, and fail a job when a change adds an AI component.

## Before you start

- The Keryn CLI ([install](https://keryn.dev/docs/start/install/)).
- A git clone that contains the base commit. In CI that means a full fetch (`fetch-depth: 0`); see [CI](https://keryn.dev/docs/guides/ci/).
- Nothing is fetched: `--base` reads a ref of the clone you have, or a file you saved.

## 1. Compare with a branch

`keryn diff` runs the AI inventory on the working tree and on the base, and prints what differs.

```sh
keryn diff . --base origin/main
```

```text
Keryn — Checks passed (Go); dependency advisories not checked (offline)
  dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
Analysis: complete · Dependency advisories: not checked (offline) · Policy: checks passed · Approval: not recorded
base 69a8fd978be3df6646542bac4cd38507eae55067 (origin/main) → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
Comparison: recorded finding delta; baseline compatibility not independently verified
Change classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 0 cleared, 0 not comparable
Cases: 0 current = 0 new + 0 existing + 0 unclassified; 0 cleared from previous assessment
Newly blocking: 0 (blocking now, not blocking in the base run under this policy)
Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
Accounting: 5 recorded rule hits; 5 non-case observations; 0 suppressed rule hits; 0 failed checks
diff: +2 / -0 components
  + library anthropic (llm-sdk)
  + model claude-sonnet-4-5 (model)
  recorded rule-hit changes: +2 / -0 (includes non-case observations)
Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entries
Coverage · source patterns: scanned 3 files; 29/29 requested rules; 0 skipped; 0 excluded
Coverage · dataflow: not recorded
Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
Coverage · known-fix matching: not recorded
  Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
  Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
  Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
  Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
  Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
  Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
  Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
  Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
Exit code: 0
counts: 5 rule matches · 5 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
rules: 29 rules available · 29 supported · 20 applied to the 3 scanned files
```

The change in this repository adds a second provider (`anthropic`) and a model. `+2 / -0 components` is the inventory delta. Zero current cases means this inventory-only run reports observations rather than vulnerability cases. `keryn diff` uses the AI inventory rules only, so it is fast.

## 2. Fail the job when a change adds a component

```sh
keryn diff . --base origin/main --fail-on-added
```

```text
Keryn — Policy blocked (Hold); dependency advisories not checked (offline)
  2 component(s) added vs base
  dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
Analysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recorded
base 69a8fd978be3df6646542bac4cd38507eae55067 (origin/main) → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
Comparison: recorded finding delta; baseline compatibility not independently verified
Change classes (metric contract 1; comparable): 0 introduced, 0 newly detected, 0 continuing, 0 reopened, 0 cleared, 0 not comparable
Cases: 0 current = 0 new + 0 existing + 0 unclassified; 0 cleared from previous assessment
Newly blocking: 0 (blocking now, not blocking in the base run under this policy)
Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
Accounting: 5 recorded rule hits; 5 non-case observations; 0 suppressed rule hits; 0 failed checks
diff: +2 / -0 components
  + library anthropic (llm-sdk)
  + model claude-sonnet-4-5 (model)
  recorded rule-hit changes: +2 / -0 (includes non-case observations)
Exceptions: 0 now set aside that counted in the base; 0 new case(s) set aside on arrival; 0 no longer set aside; 0 expired entries
Coverage · source patterns: scanned 3 files; 29/29 requested rules; 0 skipped; 0 excluded
Coverage · dataflow: not recorded
Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
Coverage · known-fix matching: not recorded
  Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
  Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
  Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
  Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
  Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
  Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
  Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
  Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
Exit code: 1
counts: 5 rule matches · 5 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
rules: 29 rules available · 29 supported · 20 applied to the 3 scanned files
```

`--fail-on-added` turns an added component into exit 1. It looks at components only. There is no flag that fails on added findings alone: `--fail-on` still judges every finding, so on a repository that already has findings, gate on components and keep the findings visible as the delta (step 4), or let [suppressions](https://keryn.dev/docs/guides/suppressions/) carry the accepted ones.

## 3. Save a baseline

A branch is not always available: a release job, a scheduled scan, a repository you only receive as a zip. Save what the last good state looked like.

```sh
keryn scan . --json keryn-out/baseline.json --aibom keryn-out/baseline.cdx.json --quiet
```

The scan exits 1 because this fixture has findings; the two files are written either way. `--base` takes a CycloneDX AI-BOM or a Keryn result JSON. Keep the file where your pipeline keeps artifacts (a release asset, a CI cache). Keryn recognises baseline reports by their contents and records them as skipped in coverage. Keep the baseline outside the checkout if you want it excluded from coverage accounting.

Now make a change that adds a vector store:

```python
import chromadb

store = chromadb.Client().create_collection("tickets")

def remember(ticket_id: str, text: str) -> None:
    store.add(ids=[ticket_id], documents=[text])
```

```sh
keryn diff . --base keryn-out/baseline.cdx.json --fail-on-added
```

```text
Keryn — Policy blocked (Hold); dependency advisories not checked (offline)
  1 component(s) added vs base
  dependency advisories not checked (offline): 1 lockfile(s); keryn scan --online checks them
Analysis: complete · Dependency advisories: not checked (offline) · Policy: blocked · Approval: not recorded
base keryn-out/baseline.cdx.json → analyzed 69a8fd978be3df6646542bac4cd38507eae55067-dirty
Comparison: not comparable: baseline has no findings
Cases: 0 current = 0 new + 0 existing + 0 unclassified; unknown cleared from previous assessment
Newly blocking: unknown (no complete base run to compare with)
Cases: 0 of 0 shown (limit 20); 0 omitted by display limit
Accounting: 6 recorded rule hits; 6 non-case observations; 0 suppressed rule hits; 0 failed checks
diff: +1 / -0 components
  + library chromadb (vector-db)
Exceptions: 0 expired suppression entries; changes since the base unknown (no base run with its exceptions)
Coverage · source patterns: scanned 4 files; 29/29 requested rules; 0 skipped; 0 excluded
Coverage · dataflow: not recorded
Coverage · dependency advisories: not checked (offline); 1 lockfile(s) (keryn scan --online checks them, sending package names and versions only)
Coverage · known-fix matching: not recorded
  Limitation: Semgrep-compatible SUBSET: no join, no metavariable-pattern/comparison, no cross-file or type inference, no language equivalences/constant propagation; taint is an approximation.
  Limitation: Structural matching uses the bundled language grammars, including Ruby. Unsupported rule/language combinations are listed in coverage; generic text also supports regex rules.
  Limitation: Parity is only claimed where proven by differential tests against native Semgrep CE (results: https://keryn.dev/evals/semgrep-ce/).
  Limitation: Deliberate divergence: each language is matched with its own grammar. Semgrep CE reuses the JavaScript parse of a pattern for every language of a rule that lists javascript, so its Python side never matches keyword arguments, dict literals or eval/require there (reported as a rule warning).
  Limitation: 1 package licence unknown: their lockfiles record none. keryn scan --online looks them up on deps.dev, sending package names and versions only.
  Limitation: 1 lockfile not checked against vulnerability advisories (offline). keryn scan --online checks them, sending package names and versions only.
  Limitation: Maintainer/install-hook transition not checked without --base evidence; a lockfile alone does not record authenticated maintainer history.
  Limitation: 1 npm or PyPI package checked against the shipped malicious-package table (OSV MAL records) and the popularity list; additional ecosystem signals depend on the available data and package contents.
  Limitation: Malicious-package data: PyPI; source date <time>; shipped. Published MAL records are not proof that unlisted packages are safe.
Exit code: 1
counts: 6 rule matches · 6 findings · 0 excluding AI-BOM inventory · 0 blocking before answers
rules: 29 rules available · 29 supported · 20 applied to the 4 scanned files
```

Only `chromadb` is new, and the job fails on it. Coverage counts files excluded from this analysis, including earlier Keryn outputs. A CycloneDX baseline compares components only; it cannot establish case changes.

## 4. Read the findings delta

With a full scan and a result JSON as the base, the console separates case changes from recorded rule-hit changes (which include inventory observations), and the Markdown summary (`--markdown`) lists the changes for a pull request comment.

```sh
keryn scan . --base keryn-out/baseline.json --quiet --markdown keryn-out/summary.md
```

```sh
grep -A3 "Changes vs base" keryn-out/summary.md
```

```text
### Changes vs base: +1 / -0

- added `chromadb` (library, vector-db)
```

Findings are compared by their fingerprint (`keryn-fp/v1`: rule, file and the matched code), not by line number, so code that moves usually stays the same finding. Usually: a finding whose own text includes a location can show once as removed and once as added after an unrelated edit above it. Treat the findings delta as a pointer for the reviewer and the component delta as the exact one.

Scans that compare with a base are never answered from the [rescan cache](https://keryn.dev/docs/guides/rescan-cache/): the comparison is the point.

## 5. Make it the default

Put the base in `.keryn.yml` so the CLI, the editors and CI agree. The language server then marks every finding that is new against it with `[new]`:

```yaml
# .keryn.yml
apiVersion: keryn.dev/config/v1
base: origin/main
```

A [policy](https://keryn.dev/docs/guides/policies/) can judge changes too: `changes.model: block` fails a model swap, `changes.prompt: review` warns on a prompt edit.

## What you just proved

The delta is the set of **new edges into your system**: a provider, a model, a vector store the code now reaches that the base did not. Reviewing that list is the cheapest control on a pull request, and `--fail-on-added` makes it a gate. It does not say the existing findings are fine, and it sees only what the code references (static references only, not runtime use).

## Next

- Wire it into a pipeline: [CI](https://keryn.dev/docs/guides/ci/).
- Accept a finding on purpose, with a reason and an expiry: [suppressions](https://keryn.dev/docs/guides/suppressions/).
- Decide what a change may add: [policies](https://keryn.dev/docs/guides/policies/).
