# Semgrep rules: Keryn vs Semgrep CE 1.157.0

> Over a public and authored corpus, do Keryn's pattern rules report the same findings as native Semgrep CE for every rule in the Keryn library?

Source: https://keryn.dev/evals/semgrep-ce/

Over a public and authored corpus, do Keryn's pattern rules report the same findings as native Semgrep CE for every rule in the Keryn library?

| Evaluation detail | Description |
|---|---|
| Keryn engine | Keryn pattern rules at `2d7fca3dc059`: Keryn's Semgrep-compatible rule engine (tree-sitter, Rust/WASM core). |
| Reference | Semgrep CE 1.157.0. Oracle, installed from PyPI. |
| Corpus | Keryn parity corpus (public and authored files), restated from `ts/results/parity-corpus-2026-11-08.json` (run of 2026-11-08) |
| Comparison | Per finding: same rule, file and range. A rule is identical when no finding is missed or extra. |
| Receipt | [`semgrep-ce.json`](https://keryn.dev/evals/semgrep-ce.json), sha256 `4f6bc4230f460fa7` |

## Results

| Measure | Value | Count | Denominator |
|---|---|---|---|
| Rules with identical findings | **100%** | 164 / 164 | rules that reported a finding on either side |
| Library rules the engine executed | **100%** | 1,398 / 1,398 | rules in the Keryn library |
| Semgrep findings Keryn also reported | **100%** | 13,475 / 13,475 | findings native Semgrep reported |
| Keryn findings Semgrep also reported | **100%** | 13,475 / 13,475 | findings Keryn reported |

Cases by status: 164 identical.

## What this does and does not measure

- The 1398 rules of the Keryn library (132 packs) run by both engines on the same 1002 files.
- Rules that report nothing on the corpus on either side are executed but do not appear as cases.
- Restated from the committed weekly parity receipt; numbers are not recomputed here.

## All cases (164)

**Show every case**

| Case | Status | Reference rows | Keryn rows | Matching | Missing | Extra |
|---|---|---|---|---|---|---|
| `javascript.permissions.mcp-tool-param-to-shell` | identical | 2 | 2 | 2 | 0 | 0 |
| `javascript.prompt-injection.user-input-in-system-prompt` | identical | 4 | 4 | 4 | 0 | 0 |
| `javascript.secrets.llm-api-key-logged` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.agents2026.pydantic-ai-tool-argument-to-exec` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.egress.py.secret-to-side-channel` | identical | 17 | 17 | 17 | 0 | 0 |
| `keryn.egress.ts.secret-to-side-channel` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.errdisc.py.exception-detail-to-caller` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.fastapi.cors-wildcard-with-credentials` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.fastapi.llm-route-without-auth` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.fastapi.llm-route-without-rate-limit` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.gaps.dynamic-import-from-variable` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.gaps.instructions-concatenated-with-input` | identical | 2 | 2 | 2 | 0 | 0 |
| `keryn.gaps.joined-path-written-unconfined` | identical | 6 | 6 | 6 | 0 | 0 |
| `keryn.gaps.parameter-url-fetched` | identical | 2 | 2 | 2 | 0 | 0 |
| `keryn.gaps.unbounded-model-loop` | identical | 7 | 7 | 7 | 0 | 0 |
| `keryn.gaps.vector-query-without-tenant-filter` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.inv.lib.ai-sdk-providers` | identical | 9 | 9 | 9 | 0 | 0 |
| `keryn.inv.lib.anthropic` | identical | 33 | 33 | 33 | 0 | 0 |
| `keryn.inv.lib.cpp-ml-runtimes` | identical | 13 | 13 | 13 | 0 | 0 |
| `keryn.inv.lib.crewai-autogen` | identical | 31 | 31 | 31 | 0 | 0 |
| `keryn.inv.lib.dotnet-ai-sdks` | identical | 10 | 10 | 10 | 0 | 0 |
| `keryn.inv.lib.go-ai-sdks` | identical | 6 | 6 | 6 | 0 | 0 |
| `keryn.inv.lib.google-genai` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.inv.lib.jvm-ai-sdks` | identical | 53 | 53 | 53 | 0 | 0 |
| `keryn.inv.lib.langchain` | identical | 72 | 72 | 72 | 0 | 0 |
| `keryn.inv.lib.langgraph` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.inv.lib.llamaindex` | identical | 120 | 120 | 120 | 0 | 0 |
| `keryn.inv.lib.ollama` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.inv.lib.openai` | identical | 56 | 56 | 56 | 0 | 0 |
| `keryn.inv.lib.php-ai-sdks` | identical | 90 | 90 | 90 | 0 | 0 |
| `keryn.inv.lib.rust-ai-crates` | identical | 24 | 24 | 24 | 0 | 0 |
| `keryn.inv.lib.swift-ai-frameworks` | identical | 17 | 17 | 17 | 0 | 0 |
| `keryn.inv.lib.vercel-ai` | identical | 28 | 28 | 28 | 0 | 0 |
| `keryn.inv.lib.vertex-ai` | identical | 2 | 2 | 2 | 0 | 0 |
| `keryn.inv.mcp` | identical | 59 | 59 | 59 | 0 | 0 |
| `keryn.inv.mcp.manifest` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.inv.model.anthropic` | identical | 33 | 33 | 33 | 0 | 0 |
| `keryn.inv.model.google` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.inv.model.open-weights` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.inv.model.openai` | identical | 65 | 65 | 65 | 0 | 0 |
| `keryn.inv.service.endpoint` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.inv.vectordb` | identical | 8 | 8 | 8 | 0 | 0 |
| `keryn.keyliteral.openai` | identical | 21 | 21 | 21 | 0 | 0 |
| `keryn.statetx.ts.action-without-replay-guard` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.vendor.anthropic-content-read-without-stop-reason-check-py` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.vendor.anthropic-messages-without-metadata-user-id-js` | identical | 5 | 5 | 5 | 0 | 0 |
| `keryn.vendor.anthropic-messages-without-metadata-user-id-py` | identical | 15 | 15 | 15 | 0 | 0 |
| `keryn.vendor.openai-call-without-moderation-js` | identical | 35 | 35 | 35 | 0 | 0 |
| `keryn.vendor.openai-call-without-moderation-py` | identical | 11 | 11 | 11 | 0 | 0 |
| `keryn.vendor.openai-chat-without-user-identifier-js` | identical | 33 | 33 | 33 | 0 | 0 |
| `keryn.vendor.openai-chat-without-user-identifier-py` | identical | 11 | 11 | 11 | 0 | 0 |
| `keryn.vendor.openai-content-read-without-refusal-check-js` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.vendor.openai-content-read-without-refusal-check-py` | identical | 4 | 4 | 4 | 0 | 0 |
| `keryn.vfgap.child-process-exec-sync-js` | identical | 5 | 5 | 5 | 0 | 0 |
| `keryn.vfgap.exec-command-shell-go` | identical | 1 | 1 | 1 | 0 | 0 |
| `keryn.vfgap.flask-debug-run-py` | identical | 2 | 2 | 2 | 0 | 0 |
| `keryn.vfgap.process-builder-non-literal-java` | identical | 2 | 2 | 2 | 0 | 0 |
| `keryn.vfgap.process-start-non-literal-cs` | identical | 3 | 3 | 3 | 0 | 0 |
| `keryn.vfgap.subprocess-check-shell-true-py` | identical | 1 | 1 | 1 | 0 | 0 |
| `python.deserialization.pickle-load-model-file` | identical | 1 | 1 | 1 | 0 | 0 |
| `python.mcp.fastmcp-http-transport-without-auth` | identical | 3 | 3 | 3 | 0 | 0 |
| `python.permissions.agent-tool-param-to-shell` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.agentcfg.mcp-env-inline-secret-known-prefix` | identical | 21 | 21 | 21 | 0 | 0 |
| `seceng.agentcfg.mcp-npx-unpinned-package` | identical | 28 | 28 | 28 | 0 | 0 |
| `seceng.agentcfg.mcp-remote-no-authorization` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.agentframework.autogen-shared-workdir-or-history` | identical | 101 | 101 | 101 | 0 | 0 |
| `seceng.agentframework.crewai-memory-without-tenant-scope` | identical | 12 | 12 | 12 | 0 | 0 |
| `seceng.agentframework.crewai-process-without-rate-or-step-limit` | identical | 457 | 457 | 457 | 0 | 0 |
| `seceng.agentrules.pipe-to-shell` | identical | 21 | 21 | 21 | 0 | 0 |
| `seceng.bundle-embedded-system-prompt` | identical | 24 | 24 | 24 | 0 | 0 |
| `seceng.bundle-tool-schema-exposed` | identical | 103 | 103 | 103 | 0 | 0 |
| `seceng.cloudai.azure-openai-public-network-access` | identical | 974 | 974 | 974 | 0 | 0 |
| `seceng.cloudai.bedrock-broad-invoke-model-policy` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.cloudflare-ai-gateway-logs-prompts` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.cloudflare.vectorize-query-without-namespace-filter` | identical | 3 | 3 | 3 | 0 | 0 |
| `seceng.container-agent-no-readonly-rootfs` | identical | 4797 | 4797 | 4797 | 0 | 0 |
| `seceng.container-ai-image-latest-or-unpinned` | identical | 57 | 57 | 57 | 0 | 0 |
| `seceng.container-ai-secret-env-direct` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.deno.edge-secret-logged` | identical | 8 | 8 | 8 | 0 | 0 |
| `seceng.deploy-ai-service-no-rate-limit-hint` | identical | 20 | 20 | 20 | 0 | 0 |
| `seceng.deploy-tool-container-unrestricted-egress` | identical | 4797 | 4797 | 4797 | 0 | 0 |
| `seceng.edge-ai-route-no-rate-limit` | identical | 34 | 34 | 34 | 0 | 0 |
| `seceng.forensics-ai-error-swallowed` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.forensics-model-version-missing` | identical | 109 | 109 | 109 | 0 | 0 |
| `seceng.forensics-no-user-tenant-on-ai-event` | identical | 47 | 47 | 47 | 0 | 0 |
| `seceng.forensics-prompt-hash-missing` | identical | 25 | 25 | 25 | 0 | 0 |
| `seceng.forensics.ai-action-missing-correlation-id` | identical | 215 | 215 | 215 | 0 | 0 |
| `seceng.forensics.tool-call-missing-audit-event` | identical | 5 | 5 | 5 | 0 | 0 |
| `seceng.gateway.log-contains-api-key` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.github-actions-excessive-token-permissions-ai-job` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.github-workflow-dispatch-input-to-shell-or-ai` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.github.actions-unpinned-third-party-action` | identical | 16 | 16 | 16 | 0 | 0 |
| `seceng.gitlab-ci-user-input-to-shell-or-ai` | identical | 5 | 5 | 5 | 0 | 0 |
| `seceng.gitlab-unpinned-ai-container-image` | identical | 3 | 3 | 3 | 0 | 0 |
| `seceng.graphql-schema-exposes-prompt-or-tool-types` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.graphql.ai-mutation-without-auth-context` | identical | 3 | 3 | 3 | 0 | 0 |
| `seceng.iacai.bedrock-agent-iam-wildcard-action-resource` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.java.langchain4j-tool-dangerous-method` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.kotlin-prompt-or-completion-logged` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.langgraph-checkpoint-shared-or-unscoped` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.langgraph-state-persists-sensitive-context` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.laravel-ai-output-direct-db-write` | identical | 8 | 8 | 8 | 0 | 0 |
| `seceng.llm01.missing-safety-sidecar-openai-python` | identical | 24 | 24 | 24 | 0 | 0 |
| `seceng.llm01.taint.request-to-system-prompt-js` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.llm02-system-prompt-in-client` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.llm02.prompt-message-logging` | identical | 16 | 16 | 16 | 0 | 0 |
| `seceng.llm02.prompt-message-logging-rust` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.llm04.python-model-output-to-system-shell` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.llm05.node-child-process-exec` | identical | 5 | 5 | 5 | 0 | 0 |
| `seceng.llm05.python-sql-string-format` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.llm05.python-subprocess-shell-true` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.llm05.react-dangerously-set-inner-html` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.llm05.react-dangerously-set-inner-html-unprotected` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.llm05.taint.model-output-to-child-process-js` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.llm05.taint.model-output-to-dynamic-dispatch-js` | identical | 20 | 20 | 20 | 0 | 0 |
| `seceng.llm05.taint.model-output-to-reflection-dispatch-python` | identical | 4 | 4 | 4 | 0 | 0 |
| `seceng.llm06.agent-broad-tools-without-approval` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.llm06.browser-agent-action` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.llm06.dangerous-tool-without-obvious-approval` | identical | 12 | 12 | 12 | 0 | 0 |
| `seceng.llm06.langchain-tool-missing-args-schema` | identical | 6 | 6 | 6 | 0 | 0 |
| `seceng.llm06.tool-call-without-approval-gate` | identical | 6 | 6 | 6 | 0 | 0 |
| `seceng.llm06.tool-call-without-approval-gate-js` | identical | 21 | 21 | 21 | 0 | 0 |
| `seceng.llm09.missing-hallucination-disclaimer` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.llm09.python-rag-response-missing-citation` | identical | 8 | 8 | 8 | 0 | 0 |
| `seceng.llm09.python-rag-response-missing-confidence` | identical | 8 | 8 | 8 | 0 | 0 |
| `seceng.llm10.openai-call-without-max-tokens` | identical | 34 | 34 | 34 | 0 | 0 |
| `seceng.llm10.unbounded-retry-loop-js` | identical | 4 | 4 | 4 | 0 | 0 |
| `seceng.loop.langchain-agent-no-max-iterations` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.loop.langgraph-recursion-limit-missing` | identical | 41 | 41 | 41 | 0 | 0 |
| `seceng.mcp.command-tool` | identical | 60 | 60 | 60 | 0 | 0 |
| `seceng.mcp.filesystem-read-tool` | identical | 35 | 35 | 35 | 0 | 0 |
| `seceng.mcp.http-sse-server` | identical | 18 | 18 | 18 | 0 | 0 |
| `seceng.mcp.java.tool-process-exec` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.mcp.v3.public-server-bind-no-auth` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.modelsc.mcp-stdio-npx-unpinned-ts` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.postgres-ai-audit-log-raw-prompt` | identical | 44 | 44 | 44 | 0 | 0 |
| `seceng.promptsc-runtime-prompt-edit-route` | identical | 7 | 7 | 7 | 0 | 0 |
| `seceng.promptsc.env-controlled-system-prompt` | identical | 4 | 4 | 4 | 0 | 0 |
| `seceng.promptsc.prompt-template-from-database-without-version` | identical | 19 | 19 | 19 | 0 | 0 |
| `seceng.redaction-tool-arguments-logged` | identical | 3 | 3 | 3 | 0 | 0 |
| `seceng.redaction.prompt-logged-before-redaction-python` | identical | 2 | 2 | 2 | 0 | 0 |
| `seceng.redaction.prompt-logged-before-redaction-ts` | identical | 14 | 14 | 14 | 0 | 0 |
| `seceng.redaction.trace-attributes-raw-ai-context` | identical | 6 | 6 | 6 | 0 | 0 |
| `seceng.redis.ai-cache-without-ttl` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.stream-sensitive-token-leak` | identical | 6 | 6 | 6 | 0 | 0 |
| `seceng.stream.sse-direct-llm-stream-no-filter-ts` | identical | 4 | 4 | 4 | 0 | 0 |
| `seceng.toolauth.audit-missing-for-dangerous-tool` | identical | 47 | 47 | 47 | 0 | 0 |
| `seceng.toolauth.dangerous-tool-no-approval-python` | identical | 4 | 4 | 4 | 0 | 0 |
| `seceng.toolauth.dangerous-tool-no-approval-ts` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.toolauth.model-controlled-user-or-tenant` | identical | 3 | 3 | 3 | 0 | 0 |
| `seceng.vectordb.chroma-query-without-where` | identical | 15 | 15 | 15 | 0 | 0 |
| `seceng.vectordb.pinecone-query-without-namespace-or-filter` | identical | 5 | 5 | 5 | 0 | 0 |
| `seceng.vendor.anthropic-computer-use-without-domain-or-action-policy` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.vendor.openai-assistants-thread-not-user-bound` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.vendor.openai-file-search-vector-store-no-boundary` | identical | 12 | 12 | 12 | 0 | 0 |
| `seceng.vendor.openai-realtime-without-auth-or-rate-limit` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.vercelai-dangerous-tool-without-approval` | identical | 1 | 1 | 1 | 0 | 0 |
| `seceng.vercelai.streamtext-tools-without-maxsteps` | identical | 1 | 1 | 1 | 0 | 0 |
| `vulnfix-ai-ai_cors_wildcard_creds` | identical | 4 | 4 | 4 | 0 | 0 |
| `vulnfix-ai-ai_js_dangerous_html` | identical | 1 | 1 | 1 | 0 | 0 |
| `vulnfix-ai-ai_mcp_bind_all` | identical | 2 | 2 | 2 | 0 | 0 |
| `vulnfix-ai-ai_mcp_js_tool_path` | identical | 2 | 2 | 2 | 0 | 0 |
| `vulnfix-ai-ai_mcp_sse_transport` | identical | 2 | 2 | 2 | 0 | 0 |
| `vulnfix-ai-ai_mcp_tool_command` | identical | 3 | 3 | 3 | 0 | 0 |
