GitHub App for pull requests
Get a Check on every pull request that says which findings and which AI components the change adds or removes, computed by the same Keryn CLI you run locally, bound to the exact commits that were compared.
If you only want SARIF and a summary on each pull request, the GitHub Action is the simpler route and does not involve a server.
How it works
Section titled “How it works”The app is three pieces, and only the first is a server:
- The app server, which you host behind your own HTTPS. It receives
pull_requestwebhooks, records them, and talks to the GitHub API. - A workflow in your repository, pinned to a reviewed commit. The app starts it with
workflow_dispatch. It checks out the trusted helper, the exact base commit and the exact head commit, then runskeryn cion the base andkeryn ci --baseon the head. The pull request’s own build, install commands, Actions and scripts are never run: fork source is data. - A reporter step in that workflow, which signs the two result bundles and posts them to the app server’s
/runner-reportendpoint.
The app then checks the run’s identity and provenance, re-reads the pull request to confirm base and head have not moved, rebuilds the gate and the change summary from the native results, and publishes a Check named Keryn source on the head commit.
What leaves your machine
Section titled “What leaves your machine”This is the one place where the pieces of a Keryn scan cross a boundary, so each hop is spelled out.
| From | To | What | Why |
|---|---|---|---|
| GitHub | your app server | the signed pull_request webhook (opened, reopened, synchronize, ready for review), up to 1 MiB |
to learn a pull request changed |
| your app server | api.github.com |
an installation token request, a pull request lookup, a workflow dispatch, and the Check | to start the run and report its result |
| your runner | your app server | the baseline and head result bundles, signed with a separate runner secret, at most 12 MiB, each artifact at most 4 MiB | to let the app build the Check |
What the pieces do not do:
- The scan runs in your own GitHub Actions runner on a checkout of your own repository. Source files are not sent to Keryn or to any service of ours; there is no Keryn-hosted component in this design.
- Installation tokens are used only against
api.github.com, are restricted to the configured repository and permissions, and redirects are refused. - The scanner process in the workflow receives a minimal environment without GitHub or reporter secrets, and
.keryn.ymlin either checkout cannot override the trusted configuration, because the CLI is run from a newly created trusted directory. - The app stores bindings, hashes and short summaries in a mode-0600 state file. It does not store raw runner reports or source artifacts; those stay in your Actions artifacts, with a seven-day retention setting.
Be aware of one more fact: the result bundles pass through the app server’s memory on their way to being checked, and they can contain native finding context. You must trust the host you choose for the app, its ingress and its logging configuration.
What the Check shows
Section titled “What the Check shows”The Check summarises added and removed findings, the rule IDs that now gate, AI components added and removed, and model or prompt changes, and links to your workflow run. It does not publish source snippets, captured prompts, arbitrary finding messages or credential values.
| Evidence | Check conclusion |
|---|---|
| valid base and head evidence, complete coverage, passing full-head gate | success |
| valid evidence with current gating violations | failure |
| incomplete coverage, unsupported or timed-out rules, no files scanned, missing runner evidence, a changed workflow pin or an uncertain dispatch | action_required |
| the pull request’s base or head no longer matches the recorded comparison | cancelled |
| a bad signature, wrong run provenance, tampered artifacts or conflicting reports | rejected; cannot complete a passing Check |
The gate still evaluates the complete current report. A clean difference does not turn a failing repository into a passing one, and a finding that disappears is not proof it was fixed. This first workflow runs source analysis and the AI inventory comparison only: dependency checks, dynamic scans, runtime AI tests and agent capability comparison are not requested and not claimed.
Before you start
Section titled “Before you start”- A GitHub account or organization where you may register an app, and the repositories to install it on.
- A host for the app server with HTTPS ingress that preserves the raw request body. The server binds
127.0.0.1; no tunnel, DNS or certificate is provided. - Node 24 or newer, and a digest-pinned packaged CLI tarball that supports
ci. The setup helper verifies the tarball’s SHA-256 and installs it offline with lifecycle scripts disabled. - The Keryn source checkout. All commands below are run from its
ts/directory.
1. Try the offline replay
Section titled “1. Try the offline replay”node clients/github-app/replay.mjs --out output/github-app-demoThe replay authenticates synthetic pull request and runner messages, dispatches a fork comparison through an injected GitHub API, rejects a green outcome for incomplete evidence, and checks idempotency. It opens no listener and makes no external request. Use a new output directory each time.
2. Generate the installation files
Section titled “2. Generate the installation files”Set CLI_SHA256 to the SHA-256 digest of the CLI tarball you reviewed before generating the files.
node clients/github-app/setup.mjs \ --out output/github-app-install \ --app-url https://checks.example.com \ --repository example/customer \ --cli-url https://releases.example.com/keryn-cli.tgz \ --cli-sha256 "$CLI_SHA256"This writes a manifest, an intentionally incomplete config.json, .github/workflows/keryn-app.yml and a runner helper, and refuses an existing output directory. It registers nothing, deploys nothing and writes nothing into your repository.
3. Register, install and pin
Section titled “3. Register, install and pin”- Review the generated manifest and register the app under the account or organization you intend. It requests Contents: read, Pull requests: read, Actions: write, Checks: write, and subscribes only to
pull_requestevents. Registration is done by you in GitHub; the server does not automate it. - Install it only on the intended repositories, then fill in the app ID, installation ID, numeric repository ID, numeric workflow ID and the exact repository name in
config.json. Repository and installation pairs form an explicit allowlist. - Commit the two generated
.githubfiles to a trusted ref of your repository and protect that ref. Record the reviewed commit asworkflowShaand the ref name asworkflowRef. The app refuses to dispatch if the ref moves, so a deliberate workflow update means reviewing and updating the pin. - Create the repository’s keryn-app environment, restrict it to the reviewed ref, and require a reviewer for fork runs. Put an independent random
KERYN_RUNNER_SECRETof at least 32 characters in that environment and the same value in the app process environment. Forks are disabled by default; enable them only deliberately. - Supply the webhook secret, the app’s private key and the runner secret through environment variables. The configuration and state files never store their values.
4. Run the server
Section titled “4. Run the server”node clients/github-app/app.mjs serve \ --config output/github-app-install/config.json \ --state output/github-app-service/state.json \ --port 8787Route /webhook and /runner-report from your HTTPS ingress to this port. /health reveals only that the process is running. Making Keryn source a required check in branch protection is a separate decision for a repository admin.
Limits to plan for
Section titled “Limits to plan for”- One process owns an exclusive lock on the state file. Do not run several workers against it.
- A workflow dispatch whose response is lost is not retried automatically: the Check becomes uncertain, and a correctly bound runner report can still complete it. Inspect the workflow before you re-dispatch.
- A report must arrive within 24 hours, otherwise the Check becomes
action_required. - State keeps at most 10,000 jobs and 20,000 delivery records and does not evict them on its own. Plan archiving before sustained traffic.
- Merge queues, push-only branches and automated Check re-requests are not implemented.
What you just proved
Section titled “What you just proved”The scan stayed in your runner, the comparison named the exact base and head commits it covered, and the Check carries counts and rule IDs, not source. What is not proved yet is a live installation: registration, environment policy, workflow dispatch, fork Check presentation and HTTPS ingress still need acceptance in your own account, and local replay does not stand in for that.
- Gate a pull request without a server: CI.
- Decide what fails a gate: policies.
- Compare against a base in the terminal: baselines.
- Every surface at a glance: apps and integrations.