When Keryn says a vulnerable dependency is reachable, or not reachable in the analysed code, is that what the code does?
| Evaluation detail |
Description |
| Keryn engine |
keryn import reachability (ts/packages/sca/src/import-reachability.ts) at b7c1a6278e63: The dependency stage’s import join over first-party source; no network, no Deep pass. |
| Reference |
independently authored truth labels authored-reach-1. Bounded regression expectations, including traps (similar names, type-only and commented-out imports, test-only use, groupId/package mismatch), not a real-world reachability oracle. |
| Corpus |
30 authored cases: 15 npm, 12 PyPI, 3 Maven |
| Comparison |
sca-reachability/1: the engine’s per-finding verdict against an authored truth. used -> reachable, unused -> not-reachable-in-analysed-code, undecidable -> unknown. A wrong not-reachable is counted apart from an abstention. |
| Receipt |
sca-reachability.json, sha256 9d4f1bfff6246917 |
| Measure |
Value |
Count |
Denominator |
| Cases where the verdict equals the authored truth |
93.3% |
28 / 30 |
all authored cases |
| Not-reachable verdicts that are right |
100% |
8 / 8 |
cases the engine called not reachable in analysed code |
| Reachable verdicts that are right |
100% |
14 / 14 |
cases the engine called reachable |
| Used packages found reachable |
93.3% |
14 / 15 |
cases whose vulnerable code is used |
| Unused packages called not reachable |
88.9% |
8 / 9 |
cases whose vulnerable code is not used by shipped code |
| Undecidable cases left unknown |
100% |
6 / 6 |
cases a static import read cannot decide |
Cases by status: 2 differs, 28 identical.
- Static imports and, where an advisory names functions, a name match in importing files.
- No call graph, no data flow, no framework or reflection model; calls made by other packages are never followed.
- Synthetic cases: precision and recall here are not real-world rates.
| Case |
Status |
Tier |
Expected verdict |
Keryn verdict |
mvn-02-group-mismatch |
differs |
|
reachable |
|
py-09-tests-only |
differs |
|
not-reachable-in-analysed-code |
|
Show every case
| Case |
Status |
Tier |
Expected verdict |
Keryn verdict |
mvn-01-group-import |
identical |
|
reachable |
|
mvn-02-group-mismatch |
differs |
|
reachable |
|
mvn-03-not-imported |
identical |
|
unknown |
|
npm-01-default-import |
identical |
|
reachable |
|
npm-02-unused-direct |
identical |
|
not-reachable-in-analysed-code |
|
npm-03-require-subpath |
identical |
|
reachable |
|
npm-04-scoped |
identical |
|
reachable |
|
npm-05-test-only |
identical |
|
not-reachable-in-analysed-code |
|
npm-06-transitive |
identical |
|
unknown |
|
npm-07-dynamic-require |
identical |
|
unknown |
|
npm-08-symbol-used |
identical |
|
reachable |
|
npm-09-symbol-unused |
identical |
|
not-reachable-in-analysed-code |
|
npm-10-reexport |
identical |
|
reachable |
|
npm-11-type-only |
identical |
|
not-reachable-in-analysed-code |
|
npm-12-vue-sfc |
identical |
|
reachable |
|
npm-13-similar-name |
identical |
|
not-reachable-in-analysed-code |
|
npm-14-commented-import |
identical |
|
not-reachable-in-analysed-code |
|
npm-15-dynamic-import-literal |
identical |
|
reachable |
|
py-01-import |
identical |
|
reachable |
|
py-02-pyyaml |
identical |
|
reachable |
|
py-03-pillow-from |
identical |
|
reachable |
|
py-04-unused-direct |
identical |
|
not-reachable-in-analysed-code |
|
py-05-importlib |
identical |
|
unknown |
|
py-06-unknown-mapping |
identical |
|
unknown |
|
py-07-symbol-unused |
identical |
|
not-reachable-in-analysed-code |
|
py-08-symbol-used |
identical |
|
reachable |
|
py-09-tests-only |
differs |
|
not-reachable-in-analysed-code |
|
py-10-top-level-txt |
identical |
|
reachable |
|
py-11-submodule |
identical |
|
reachable |
|
py-12-transitive |
identical |
|
unknown |
|