For advisories of NuGet packages, does Forge report a flow of the advisory’s CWE in the last vulnerable release and no longer report it in the first patched release?
| Evaluation detail |
Description |
| Keryn engine |
keryn-forge at 6b98539c6472: Keryn Forge (crates/keryn-forge) examples/forge_scan, models/cil.json; binary sha256 55b9846b5fdf5957510209747a7a65e170581eb58aee885b237a17412ab8f4fc. |
| Reference |
GitHub Advisory Database (via vulnfix-mine evidence db) + registry release history . Ground truth: the advisory says the vulnerable release has the weakness of the listed CWE and the first patched release does not. |
| Corpus |
dotnet bytecode pairs |
| Comparison |
Per advisory package (one vulnerable release, one patched release, both scanned with forge_scan –accept-missing-deps all). identical (outcome found-and-fixed): the vulnerable release has a flow of the advisory CWE family whose key (rule, source function, sink function) is absent as a flow in the patched release. differs: flow-persists (flow in both), or a miss (outcome = the miss class: no-sink-model, sink-bound-no-source, bound-no-flow, unknown-only, producer-gap). unsupported: no shipped Forge rule judges the advisory CWE. error/timeout: a scan failed. Failures stay in the denominator. |
| Receipt |
forge-cve-dotnet.json, sha256 cc464f50f932afbf |
| Measure |
Value |
Count |
Denominator |
| Advisories where the vulnerable release has the CWE flow and the patched release no longer does |
0% |
0 / 30 |
advisory packages with both releases scanned or failed |
| Of the advisories whose CWE Forge judges, those with a flow in the vulnerable release |
4.8% |
1 / 21 |
cases whose CWE some shipped Forge rule judges and whose scans completed |
| Of the advisories whose CWE Forge judges, those where the flow disappears in the patched release |
0% |
0 / 21 |
cases whose CWE some shipped Forge rule judges and whose scans completed |
| Of the flows found in vulnerable releases, those with none of the same flows in the patched release |
0% |
0 / 1 |
cases with a CWE flow in the vulnerable release |
| Of the judged advisories with a known fix commit, those whose flow touches a class the fix changed |
0% |
0 / 21 |
judged cases with a known fix commit |
| Advisories whose CWE some shipped Forge rule judges |
70.0% |
21 / 30 |
advisory packages in the batch |
| Judged advisories where a sink is bound but no source exists (library entry points unmodelled) |
0% |
0 / 21 |
cases whose CWE some shipped Forge rule judges and whose scans completed |
Cases by status: 21 differs, 9 unsupported.
- Advisories with a known fixed version whose vulnerable and fixed releases are both on the registry; Jenkins/other-registry packages, pom-only packages and artifacts over 25 MB are dropped before scoring (counts in the pairs dropped.json, not cases).
- Libraries have no HTTP request source: unless a framework entry model applies, a library flow can only start at a modelled source, so most misses are
sink-bound-no-source. That is a measured gap, not a data error.
- Artifacts over 1.5 MB are scanned as a slice (packages of the classes the fix changed, same list for both releases); the case records
mode.
- Pair selection is deterministic but biased to advisories with a known fix commit and a CWE label; at most 2 pairs per package.
| Case |
Status |
Analysis outcome |
nuget:AjaxNetProfessional@21.12.21.1..21.12.22.1 |
differs |
Unknown only |
nuget:Autoupdater.NET.Official@1.5.7..1.5.8 |
differs |
Unknown only |
nuget:DotNetNuke.Core@9.13.7-ci0064..9.13.8 |
differs |
Unknown only |
nuget:DotNetNuke.Core@9.13.8..9.13.9 |
differs |
Unknown only |
nuget:DotNetNuke.Web@9.13.8..9.13.9 |
differs |
Unknown only |
nuget:DotNetZip@1.10.1..1.11.0 |
differs |
Bound, no flow |
nuget:DotVVM@4.2.9..4.2.10 |
differs |
Flow persists |
nuget:HtmlSanitizer@5.0.355..5.0.372 |
differs |
No sink model |
nuget:ICG.AspNetCore.Utilities.CloudStorage@6.0.0..8.0.0 |
unsupported |
Weakness not modelled |
nuget:LiteDB@5.0.12..5.0.13 |
differs |
Bound, no flow |
nuget:Microsoft.OpenApi.Kiota.Builder@1.29.0..1.29.1 |
differs |
Bound, no flow |
nuget:NuGet.CommandLine.XPlat@6.2.0..6.2.1 |
unsupported |
Weakness not modelled |
nuget:NuGet.Commands@5.0.0-rtm.5867..5.0.2 |
unsupported |
Weakness not modelled |
nuget:NuGet.Commands@6.2.0..6.2.1 |
unsupported |
Weakness not modelled |
nuget:OWASP.AntiSamy@1.1.1..1.2.0 |
differs |
No sink model |
nuget:OrchardCore@1.2.0..1.2.1 |
differs |
Unknown only |
nuget:OrchardCore@1.3.0..1.4.0 |
differs |
Unknown only |
nuget:Serenity.Net.Core@6.6.6..6.7.0 |
differs |
Bound, no flow |
nuget:Serilog.Enrichers.ClientInfo@2.0.3..2.1.0 |
differs |
No sink model |
nuget:SharpCompress@0.47.4..0.48.0 |
differs |
Bound, no flow |
nuget:SharpZipLib@1.3.2..1.3.3 |
differs |
Bound, no flow |
nuget:Steeltoe.Management.Endpoint@4.1.0..4.2.0 |
unsupported |
Weakness not modelled |
nuget:Steeltoe.Management.Endpoint@4.2.0..4.3.0 |
unsupported |
Weakness not modelled |
nuget:Steeltoe.Management.EndpointCore@3.3.0..3.4.0 |
unsupported |
Weakness not modelled |
nuget:TrueLayer.Client@1.5.0..1.6.0 |
differs |
No sink model |
nuget:Umbraco.Cms.Api.Delivery@16.1.0..16.1.1 |
unsupported |
Weakness not modelled |
nuget:Umbraco.Cms.Api.Management@14.3.2..14.3.3 |
unsupported |
Weakness not modelled |
nuget:Umbraco.Cms.Core@13.1.0..13.1.1 |
differs |
Unknown only |
nuget:Umbraco.Cms.Web.BackOffice@13.1.0..13.1.1 |
differs |
Unknown only |
nuget:YoutubeDLSharp@1.1.1..1.1.2 |
differs |
Bound, no flow |
Show every case
| Case |
Status |
Analysis outcome |
nuget:AjaxNetProfessional@21.12.21.1..21.12.22.1 |
differs |
Unknown only |
nuget:Autoupdater.NET.Official@1.5.7..1.5.8 |
differs |
Unknown only |
nuget:DotNetNuke.Core@9.13.7-ci0064..9.13.8 |
differs |
Unknown only |
nuget:DotNetNuke.Core@9.13.8..9.13.9 |
differs |
Unknown only |
nuget:DotNetNuke.Web@9.13.8..9.13.9 |
differs |
Unknown only |
nuget:DotNetZip@1.10.1..1.11.0 |
differs |
Bound, no flow |
nuget:DotVVM@4.2.9..4.2.10 |
differs |
Flow persists |
nuget:HtmlSanitizer@5.0.355..5.0.372 |
differs |
No sink model |
nuget:ICG.AspNetCore.Utilities.CloudStorage@6.0.0..8.0.0 |
unsupported |
Weakness not modelled |
nuget:LiteDB@5.0.12..5.0.13 |
differs |
Bound, no flow |
nuget:Microsoft.OpenApi.Kiota.Builder@1.29.0..1.29.1 |
differs |
Bound, no flow |
nuget:NuGet.CommandLine.XPlat@6.2.0..6.2.1 |
unsupported |
Weakness not modelled |
nuget:NuGet.Commands@5.0.0-rtm.5867..5.0.2 |
unsupported |
Weakness not modelled |
nuget:NuGet.Commands@6.2.0..6.2.1 |
unsupported |
Weakness not modelled |
nuget:OWASP.AntiSamy@1.1.1..1.2.0 |
differs |
No sink model |
nuget:OrchardCore@1.2.0..1.2.1 |
differs |
Unknown only |
nuget:OrchardCore@1.3.0..1.4.0 |
differs |
Unknown only |
nuget:Serenity.Net.Core@6.6.6..6.7.0 |
differs |
Bound, no flow |
nuget:Serilog.Enrichers.ClientInfo@2.0.3..2.1.0 |
differs |
No sink model |
nuget:SharpCompress@0.47.4..0.48.0 |
differs |
Bound, no flow |
nuget:SharpZipLib@1.3.2..1.3.3 |
differs |
Bound, no flow |
nuget:Steeltoe.Management.Endpoint@4.1.0..4.2.0 |
unsupported |
Weakness not modelled |
nuget:Steeltoe.Management.Endpoint@4.2.0..4.3.0 |
unsupported |
Weakness not modelled |
nuget:Steeltoe.Management.EndpointCore@3.3.0..3.4.0 |
unsupported |
Weakness not modelled |
nuget:TrueLayer.Client@1.5.0..1.6.0 |
differs |
No sink model |
nuget:Umbraco.Cms.Api.Delivery@16.1.0..16.1.1 |
unsupported |
Weakness not modelled |
nuget:Umbraco.Cms.Api.Management@14.3.2..14.3.3 |
unsupported |
Weakness not modelled |
nuget:Umbraco.Cms.Core@13.1.0..13.1.1 |
differs |
Unknown only |
nuget:Umbraco.Cms.Web.BackOffice@13.1.0..13.1.1 |
differs |
Unknown only |
nuget:YoutubeDLSharp@1.1.1..1.1.2 |
differs |
Bound, no flow |