Given the same HTTP messages, does keryn-passive raise exactly the alerts that ZAP’s own passive scan rule classes raise, field for field?
| Evaluation detail |
Description |
| Keryn engine |
keryn-passive at be1b50c66c74: Keryn’s passive web engine (crates/keryn-passive): ZAP’s passive scan rules reimplemented in Rust class by class, native and WebAssembly (CLI, web app, Chrome extension). |
| Reference |
OWASP ZAP passive scan rules (zap-extensions pscanrules, pscanrulesBeta, pscanrulesAlpha) on ZAP core 2.17.0. Oracle only: the upstream rule classes run unmodified by a small Java harness on a development machine; never shipped. |
| Corpus |
Keryn passive-web fixture corpus (HTTP messages per rule, alerting and quiet), run against zap-extensions pscanrules at the pinned commit, zaproxy/zap-extensions at 1b34a3b77298, restated from crates/keryn-passive/results/conformance.json, crates/keryn-passive/docs/passive-web/upstream-rules.json, crates/keryn-passive/docs/passive-web/rule-status.json |
| Comparison |
Per (fixture, rule, threshold) case at LOW, MEDIUM and HIGH: the alerts each side raises, compared on every field (id, ref, name, risk, confidence, param, evidence, other info, CWE, WASC, URI, description, solution, reference, tags). A rule is identical when all its cases agree, the oracle raised no error, and the WASM build agrees with native. |
| Receipt |
zap-passive.json, sha256 9ddfb4934a59b68a |
| Measure |
Value |
Count |
Denominator |
| Ported rules identical to ZAP on every case |
100% |
63 / 63 |
upstream passive rules ported to keryn-passive |
| Upstream passive rules ported and identical |
98.4% |
63 / 64 |
all passive rules in the pinned upstream inventory |
| Portable upstream rules that are ported |
100% |
63 / 63 |
upstream passive rules that make sense outside ZAP |
| Differential cases that agree on every alert field |
100% |
18,948 / 18,948 |
(fixture, rule, threshold) cases run on both sides |
Cases by status: 63 identical.
- Passive rules only. ZAP’s active scanner, spider and scripts are out of scope.
- The fixtures are authored per rule; they exercise each rule’s alerting and quiet paths but are not traffic captured from real sites.
- 64 upstream passive rules in the pinned inventory; 1 not portable (10116: Reports whether the scanning ZAP install is out of date (ZAP’s own version and update feed); meaningless outside ZAP).
- Restated from the committed differential; numbers are not recomputed from a new run. The rule ids are zero-padded so cases sort numerically.
Show every case
| Case |
Status |
Reference rows |
Keryn rows |
Matching |
Missing |
Extra |
000002 |
identical |
153 |
153 |
153 |
|
|
000003 |
identical |
180 |
180 |
180 |
|
|
010009 |
identical |
132 |
132 |
132 |
|
|
010010 |
identical |
306 |
306 |
306 |
|
|
010011 |
identical |
306 |
306 |
306 |
|
|
010015 |
identical |
171 |
171 |
171 |
|
|
010017 |
identical |
771 |
771 |
771 |
|
|
010019 |
identical |
144 |
144 |
144 |
|
|
010020 |
identical |
771 |
771 |
771 |
|
|
010021 |
identical |
144 |
144 |
144 |
|
|
010023 |
identical |
135 |
135 |
135 |
|
|
010024 |
identical |
189 |
189 |
189 |
|
|
010025 |
identical |
162 |
162 |
162 |
|
|
010026 |
identical |
126 |
126 |
126 |
|
|
010027 |
identical |
1008 |
1008 |
1008 |
|
|
010028 |
identical |
177 |
177 |
177 |
|
|
010029 |
identical |
159 |
159 |
159 |
|
|
010030 |
identical |
177 |
177 |
177 |
|
|
010031 |
identical |
615 |
615 |
615 |
|
|
010032 |
identical |
186 |
186 |
186 |
|
|
010033 |
identical |
120 |
120 |
120 |
|
|
010034 |
identical |
123 |
123 |
123 |
|
|
010035 |
identical |
771 |
771 |
771 |
|
|
010036 |
identical |
177 |
177 |
177 |
|
|
010037 |
identical |
177 |
177 |
177 |
|
|
010038 |
identical |
204 |
204 |
204 |
|
|
010039 |
identical |
177 |
177 |
177 |
|
|
010040 |
identical |
582 |
582 |
582 |
|
|
010041 |
identical |
771 |
771 |
771 |
|
|
010042 |
identical |
771 |
771 |
771 |
|
|
010043 |
identical |
615 |
615 |
615 |
|
|
010044 |
identical |
150 |
150 |
150 |
|
|
010049 |
identical |
249 |
249 |
249 |
|
|
010050 |
identical |
153 |
153 |
153 |
|
|
010052 |
identical |
177 |
177 |
177 |
|
|
010054 |
identical |
306 |
306 |
306 |
|
|
010055 |
identical |
780 |
780 |
780 |
|
|
010056 |
identical |
177 |
177 |
177 |
|
|
010057 |
identical |
123 |
123 |
123 |
|
|
010061 |
identical |
177 |
177 |
177 |
|
|
010062 |
identical |
318 |
318 |
318 |
|
|
010063 |
identical |
204 |
204 |
204 |
|
|
010094 |
identical |
171 |
171 |
171 |
|
|
010096 |
identical |
135 |
135 |
135 |
|
|
010097 |
identical |
177 |
177 |
177 |
|
|
010098 |
identical |
129 |
129 |
129 |
|
|
010099 |
identical |
183 |
183 |
183 |
|
|
010105 |
identical |
174 |
174 |
174 |
|
|
010108 |
identical |
771 |
771 |
771 |
|
|
010109 |
identical |
771 |
771 |
771 |
|
|
010110 |
identical |
156 |
156 |
156 |
|
|
010115 |
identical |
177 |
177 |
177 |
|
|
010202 |
identical |
600 |
600 |
600 |
|
|
020020 |
identical |
147 |
147 |
147 |
|
|
090001 |
identical |
165 |
165 |
165 |
|
|
090002 |
identical |
144 |
144 |
144 |
|
|
090003 |
identical |
594 |
594 |
594 |
|
|
090004 |
identical |
174 |
174 |
174 |
|
|
090005 |
identical |
135 |
135 |
135 |
|
|
090011 |
identical |
171 |
171 |
171 |
|
|
090022 |
identical |
147 |
147 |
147 |
|
|
090033 |
identical |
387 |
387 |
387 |
|
|
110009 |
identical |
126 |
126 |
126 |
|
|