Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    GitHub App for pull requests

    View as Markdown

    Get a Check on every pull request that says which findings and which AI components the change adds or removes, computed by the same Keryn CLI you run locally, bound to the exact commits that were compared.

    If you only want SARIF and a summary on each pull request, the GitHub Action is the simpler route and does not involve a server.

    The app is three pieces, and only the first is a server:

    1. The app server, which you host behind your own HTTPS. It receives pull_request webhooks, records them, and talks to the GitHub API.
    2. A workflow in your repository, pinned to a reviewed commit. The app starts it with workflow_dispatch. It checks out the trusted helper, the exact base commit and the exact head commit, then runs keryn ci on the base and keryn ci --base on the head. The pull request’s own build, install commands, Actions and scripts are never run: fork source is data.
    3. A reporter step in that workflow, which signs the two result bundles and posts them to the app server’s /runner-report endpoint.

    The app then checks the run’s identity and provenance, re-reads the pull request to confirm base and head have not moved, rebuilds the gate and the change summary from the native results, and publishes a Check named Keryn source on the head commit.

    This is the one place where the pieces of a Keryn scan cross a boundary, so each hop is spelled out.

    From To What Why
    GitHub your app server the signed pull_request webhook (opened, reopened, synchronize, ready for review), up to 1 MiB to learn a pull request changed
    your app server api.github.com an installation token request, a pull request lookup, a workflow dispatch, and the Check to start the run and report its result
    your runner your app server the baseline and head result bundles, signed with a separate runner secret, at most 12 MiB, each artifact at most 4 MiB to let the app build the Check

    What the pieces do not do:

    • The scan runs in your own GitHub Actions runner on a checkout of your own repository. Source files are not sent to Keryn or to any service of ours; there is no Keryn-hosted component in this design.
    • Installation tokens are used only against api.github.com, are restricted to the configured repository and permissions, and redirects are refused.
    • The scanner process in the workflow receives a minimal environment without GitHub or reporter secrets, and .keryn.yml in either checkout cannot override the trusted configuration, because the CLI is run from a newly created trusted directory.
    • The app stores bindings, hashes and short summaries in a mode-0600 state file. It does not store raw runner reports or source artifacts; those stay in your Actions artifacts, with a seven-day retention setting.

    Be aware of one more fact: the result bundles pass through the app server’s memory on their way to being checked, and they can contain native finding context. You must trust the host you choose for the app, its ingress and its logging configuration.

    The Check summarises added and removed findings, the rule IDs that now gate, AI components added and removed, and model or prompt changes, and links to your workflow run. It does not publish source snippets, captured prompts, arbitrary finding messages or credential values.

    Evidence Check conclusion
    valid base and head evidence, complete coverage, passing full-head gate success
    valid evidence with current gating violations failure
    incomplete coverage, unsupported or timed-out rules, no files scanned, missing runner evidence, a changed workflow pin or an uncertain dispatch action_required
    the pull request’s base or head no longer matches the recorded comparison cancelled
    a bad signature, wrong run provenance, tampered artifacts or conflicting reports rejected; cannot complete a passing Check

    The gate still evaluates the complete current report. A clean difference does not turn a failing repository into a passing one, and a finding that disappears is not proof it was fixed. This first workflow runs source analysis and the AI inventory comparison only: dependency checks, dynamic scans, runtime AI tests and agent capability comparison are not requested and not claimed.

    • A GitHub account or organization where you may register an app, and the repositories to install it on.
    • A host for the app server with HTTPS ingress that preserves the raw request body. The server binds 127.0.0.1; no tunnel, DNS or certificate is provided.
    • Node 24 or newer, and a digest-pinned packaged CLI tarball that supports ci. The setup helper verifies the tarball’s SHA-256 and installs it offline with lifecycle scripts disabled.
    • The Keryn source checkout. All commands below are run from its ts/ directory.
    Terminal window
    node clients/github-app/replay.mjs --out output/github-app-demo

    The replay authenticates synthetic pull request and runner messages, dispatches a fork comparison through an injected GitHub API, rejects a green outcome for incomplete evidence, and checks idempotency. It opens no listener and makes no external request. Use a new output directory each time.

    Set CLI_SHA256 to the SHA-256 digest of the CLI tarball you reviewed before generating the files.

    Terminal window
    node clients/github-app/setup.mjs \
    --out output/github-app-install \
    --app-url https://checks.example.com \
    --repository example/customer \
    --cli-url https://releases.example.com/keryn-cli.tgz \
    --cli-sha256 "$CLI_SHA256"

    This writes a manifest, an intentionally incomplete config.json, .github/workflows/keryn-app.yml and a runner helper, and refuses an existing output directory. It registers nothing, deploys nothing and writes nothing into your repository.

    1. Review the generated manifest and register the app under the account or organization you intend. It requests Contents: read, Pull requests: read, Actions: write, Checks: write, and subscribes only to pull_request events. Registration is done by you in GitHub; the server does not automate it.
    2. Install it only on the intended repositories, then fill in the app ID, installation ID, numeric repository ID, numeric workflow ID and the exact repository name in config.json. Repository and installation pairs form an explicit allowlist.
    3. Commit the two generated .github files to a trusted ref of your repository and protect that ref. Record the reviewed commit as workflowSha and the ref name as workflowRef. The app refuses to dispatch if the ref moves, so a deliberate workflow update means reviewing and updating the pin.
    4. Create the repository’s keryn-app environment, restrict it to the reviewed ref, and require a reviewer for fork runs. Put an independent random KERYN_RUNNER_SECRET of at least 32 characters in that environment and the same value in the app process environment. Forks are disabled by default; enable them only deliberately.
    5. Supply the webhook secret, the app’s private key and the runner secret through environment variables. The configuration and state files never store their values.
    Terminal window
    node clients/github-app/app.mjs serve \
    --config output/github-app-install/config.json \
    --state output/github-app-service/state.json \
    --port 8787

    Route /webhook and /runner-report from your HTTPS ingress to this port. /health reveals only that the process is running. Making Keryn source a required check in branch protection is a separate decision for a repository admin.

    • One process owns an exclusive lock on the state file. Do not run several workers against it.
    • A workflow dispatch whose response is lost is not retried automatically: the Check becomes uncertain, and a correctly bound runner report can still complete it. Inspect the workflow before you re-dispatch.
    • A report must arrive within 24 hours, otherwise the Check becomes action_required.
    • State keeps at most 10,000 jobs and 20,000 delivery records and does not evict them on its own. Plan archiving before sustained traffic.
    • Merge queues, push-only branches and automated Check re-requests are not implemented.

    The scan stayed in your runner, the comparison named the exact base and head commits it covered, and the Check carries counts and rule IDs, not source. What is not proved yet is a live installation: registration, environment policy, workflow dispatch, fork Check presentation and HTTPS ingress still need acceptance in your own account, and local replay does not stand in for that.