Sent the same canned HTTP responses, does keryn-nuclei report the same matches and extractions as nuclei for each template?
| Evaluation detail |
Description |
| Keryn engine |
keryn-nuclei at 71c1bb2d05b1: Keryn’s sans-IO Nuclei template engine, crates/keryn-nuclei (parse, request plan, matchers, extractors, DSL subset); the eval host (examples/nuclei_host) sends the requests. |
| Reference |
nuclei v3.11.1. Oracle only, run on a development machine against 127.0.0.1 fixture servers (crates/keryn-nuclei/tools/run.sh); never shipped. |
| Corpus |
nuclei-templates (11 Keryn-authored feature fixtures plus 40 upstream templates with synthesised fixture responses), projectdiscovery/nuclei-templates at v10.5.1 (79fb682760ab), restated from crates/keryn-nuclei/results/nuclei-diff.json (run of 2026-11-08), crates/keryn-nuclei/results/nuclei-census.json |
| Comparison |
Per template: the set of distinct (template-id, matcher-name, sorted extracted-results) rows each engine reported; a missing matcher name or extraction counts as empty. A template is identical when both sets are equal, including both being empty. |
| Receipt |
nuclei.json, sha256 95bfcb85c19a50ff |
| Measure |
Value |
Count |
Denominator |
| Templates with identical results |
96.1% |
49 / 51 |
templates run by both engines against a fixture (11 Keryn-authored, 40 upstream) |
| Upstream templates with identical results |
95.0% |
38 / 40 |
upstream nuclei-templates run against a synthesised fixture |
| Nuclei results the engine also produced |
98.0% |
48 / 49 |
distinct result rows nuclei produced |
| Engine results nuclei also produced |
98.0% |
48 / 49 |
distinct result rows keryn-nuclei produced |
| Upstream HTTP templates the engine can execute fully |
69.9% |
8,375 / 11,989 |
templates in nuclei-templates with an HTTP section |
| Upstream HTTP templates the engine can execute fully or in part |
72.4% |
8,678 / 11,989 |
templates in nuclei-templates with an HTTP section |
Cases by status: 2 differs, 49 identical.
- HTTP templates only; DNS, TCP, headless, code and JavaScript protocols are not executed by the engine.
- Every request goes to a 127.0.0.1 fixture server that serves canned responses, so this measures template semantics, not network behaviour against real targets.
- The 40 upstream templates are a sample of the templates whose fixture could be synthesised from the template itself, not the whole pack.
- Template coverage is a separate metric: of the 11989 upstream templates with an HTTP section, 8375 are executable and 303 partial (the template census, which sends nothing).
- Restated from the committed receipts; numbers are not recomputed from a new run.
| Case |
Status |
Reference rows |
Keryn rows |
Matching |
Missing |
Extra |
upstream/CVE-2005-4385 |
differs |
0 |
1 |
0 |
0 |
1 |
upstream/CVE-2025-41393 |
differs |
1 |
0 |
0 |
1 |
0 |
Show every case
| Case |
Status |
Reference rows |
Keryn rows |
Matching |
Missing |
Extra |
keryn-fixture/keryn-fx-case-binary-size |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-env-file |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-favicon |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-header-kval |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-json-version |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-login-chain |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-named-or |
identical |
2 |
2 |
2 |
0 |
0 |
keryn-fixture/keryn-fx-negative |
identical |
0 |
0 |
0 |
0 |
0 |
keryn-fixture/keryn-fx-payloads |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-redirect |
identical |
1 |
1 |
1 |
0 |
0 |
keryn-fixture/keryn-fx-variables |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CNVD-2020-62422 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CNVD-2021-15824 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CNVD-2023-96945 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2000-0760 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2001-0537 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2002-1131 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2004-0519 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2005-4385 |
differs |
0 |
1 |
0 |
0 |
1 |
upstream/CVE-2007-2449 |
identical |
0 |
0 |
0 |
0 |
0 |
upstream/CVE-2008-6465 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2009-1872 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2010-4239 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2011-5106 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2012-4889 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2013-7240 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2014-8799 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2015-4050 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2016-4977 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2017-17736 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2018-18570 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2019-18957 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2020-2096 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2021-31682 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2022-27926 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2023-3844 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2024-33610 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/CVE-2025-41393 |
differs |
1 |
0 |
0 |
1 |
0 |
upstream/CVE-2026-39339 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/aem-crx-package-manager |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/apache-apollo-panel |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/arcgis-api-service |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/avaya-aura-system-manager-panel |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/backpack-admin-panel |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/bmc-remedy-sso-panel |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/checkmarx-cxsast-panel |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/cisco-integrated-login |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/covenant-c2 |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/mantisbt-anonymous-login |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/minio-browser |
identical |
1 |
1 |
1 |
0 |
0 |
upstream/ssl-network-extender |
identical |
1 |
1 |
1 |
0 |
0 |