# The proof ladder

> Every finding states how much evidence stands behind it, from a pattern match to a reproduction.

Source: https://keryn.dev/docs/concepts/proof-ladder/

Every finding carries a proof level. It says how much evidence stands behind the finding, not how severe it is. Severity (`error`, `warning` or `info`) is the rule's rating of the impact if the finding is real. The two are shown side by side and neither replaces the other.

| Level | Meaning |
|---|---|
| `proof 0 inventory` | An AI component, capability or posture fact. Never gates as a vulnerability. |
| `proof 1 pattern` | A code pattern matched. |
| `proof 2 path-in-file` | A path from a source (user input, model output) to a sink inside one file. |
| `proof 3 path-cross-file` | A path across files (Python, TypeScript, JavaScript, Go, Java, C#, C/C++). How far it reaches depends on the language: see [what Keryn does not do](https://keryn.dev/docs/concepts/limits/). |
| `proof 4 known-fix` | The code matches a block that a published advisory's fix removed; cites the CVE or GHSA. |
| `proof 5 reproduced` | Verify reproduced it in an isolated container; cites the evidence hash. |

## The same code at two rungs

This function asks a model for a shell command and runs it. The first rule only looks for `shell=True`. The second follows the model's reply to the call that runs it. Compare the proof levels in the two recorded results.

**Rung 1: a pattern matched** (a live example on the page; the engine runs in the browser, nothing is uploaded)

```python
import subprocess
from anthropic import Anthropic

client = Anthropic()

def fix_it(error_text):
    reply = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=300,
        messages=[{"role": "user", "content": "Give one shell command that fixes: " + error_text}],
    )
    command = reply.content[0].text
    return subprocess.run(command, shell=True, capture_output=True)
```

Rule:

```yaml
rules:
  - id: acme.shell-true
    languages: [python]
    severity: WARNING
    message: A command string is run through a shell.
    pattern: subprocess.run(..., shell=True, ...)
```

Result, computed when the site was built by the same engine as `keryn scan`:

```text
1 finding · 1 of 1 rules ran on 1 file

warning acme.shell-true line 14 [proof 1 pattern]
  A command string is run through a shell.
  | return subprocess.run(command, shell=True, capture_output=True)
```

**Rung 2: a path from source to sink in the file** (a live example on the page; the engine runs in the browser, nothing is uploaded)

```python
import subprocess
from anthropic import Anthropic

client = Anthropic()

def fix_it(error_text):
    reply = client.messages.create(
        model="claude-sonnet-4-5",
        max_tokens=300,
        messages=[{"role": "user", "content": "Give one shell command that fixes: " + error_text}],
    )
    command = reply.content[0].text
    return subprocess.run(command, shell=True, capture_output=True)
```

Rule:

```yaml
rules:
  - id: acme.model-output-reaches-shell
    languages: [python]
    severity: ERROR
    message: Text a model produced is run as a shell command. Treat it as untrusted input.
    mode: taint
    pattern-sources:
      - pattern: $CLIENT.messages.create(...)
    pattern-sinks:
      - pattern: subprocess.run($CMD, ...)
```

Result, computed when the site was built by the same engine as `keryn scan`:

```text
1 finding · 1 of 1 rules ran on 1 file

error acme.model-output-reaches-shell line 14 [proof 2 path-in-file]
  Text a model produced is run as a shell command. Treat it as untrusted input.
  Flow: source (line 8) -> propagation (line 8) -> propagation (line 13)
  | return subprocess.run(command, shell=True, capture_output=True)
```

The second finding lists its flow: the model call, the assignment that carries the reply, and the call that runs it. Removing `shell=True` removes the pattern match. The data-flow rule still reports the path if model output reaches the subprocess call.

A single finding sits on a path. How findings combine into chains, and how a chain is graded, is on [paths, chains and chokepoints](https://keryn.dev/docs/edges/paths-and-chains/).

## Why the ladder exists

A pattern match is often still present after the code was fixed. On Keryn's benchmark of real vulnerability fixes, rules that found the vulnerable version also fired on the fixed version for 85% of the pairs at rung 1, 63% at rung 2 and 9% at rung 4. Treat a rung-1 finding as a lead to check, and a rung-4 or rung-5 finding as evidence.

The default gate counts high-confidence pattern rules and data-flow findings at rung 3 and above. Rung-2 findings gate only for the finding classes where precision was measured on clean code.

> **Note: In the next release**
> A pattern finding that Deep confirmed (rung 2 or above) also counts toward the default gate, whatever its rule's confidence, because two engines agree on the line. See [static analysis](https://keryn.dev/docs/guides/sast/#2-read-what-deep-said-about-a-pattern-finding).

Findings also carry a rule **confidence** (`P1` high, `P2` lower). That rates the rule, not this finding. `--gate-confidence P2` widens the gate to the lower tier.
