Skip to content

    Type a page, a command such as keryn scan, or a rule id.

    Student

    View as Markdown

    You get: a detection rule you understand, a code fix you can check, and a first score in the Keryn Academy. Read the walkthrough in your browser. To run the exercise, check Keryn downloads for a CLI build available to your account.

    The rule playground has a short introduction to custom scans. In the Flask sample below, line 14 puts request data from persona into the system prompt. That lets request data influence the model’s instructions.

    The rule uses mode: taint to follow data within the file. Its pattern-sources match reads from the request, and its pattern-sinks match the content of a system message.

    Save the rule and the app from the CLI exercise. Run the first scan to see the finding at line 14. Then move the persona into the user message and rerun the scan. The custom rule should report zero findings.

    Open the Keryn Academy and choose Start the speed run: a short check across the 14 assessment domains. When you want your lab completions saved and a credential issued, create an account there; the courses, labs and flash cards are open without one.

    The same rule runs in a full scan alongside the bundled library. The rule and app below are what this page’s tests scan:

    rules:
    - id: user-input-in-system-prompt
    languages: [python]
    severity: WARNING
    message: Request data reaches the system prompt; keep user text in the user message.
    mode: taint
    pattern-sources:
    - pattern: request.args.get(...)
    - pattern: request.json[...]
    pattern-sinks:
    - patterns:
    - pattern-inside: '{"role": "system", "content": $C}'
    - pattern: $C
    from flask import Flask, request
    from openai import OpenAI
    app = Flask(__name__)
    client = OpenAI()
    @app.post("/ask")
    def ask():
    persona = request.args.get("persona")
    question = request.json["q"]
    reply = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
    {"role": "system", "content": "You are " + persona},
    {"role": "user", "content": question},
    ],
    )
    return reply.choices[0].message.content
    Terminal window
    keryn rules lint rules/system-prompt.yml
    keryn scan . --rules rules/system-prompt.yml --fail-on none --json result.json -q
    jq -r '.findings[] | select(.ruleId == "user-input-in-system-prompt") | "\(.ruleId) \(.path):\(.line)"' result.json
    rules/system-prompt.yml: 1 rule(s)
    0 error(s), 0 warning(s)
    user-input-in-system-prompt app.py:14

    Update the two messages in your copy of app.py and scan again:

    Terminal window
    python3 - <<'PYTHON'
    from pathlib import Path
    app = Path("app.py")
    source = app.read_text()
    source = source.replace('"You are " + persona', '"You are a helpful travel assistant."')
    source = source.replace('"content": question', '"content": persona + ": " + question')
    app.write_text(source)
    PYTHON
    keryn scan . --rules rules/system-prompt.yml --fail-on none --json result.json -q
    jq '[.findings[] | select(.ruleId == "user-input-in-system-prompt")] | length' result.json
    0