dataset-finetune-integrity-v2
5 rules · Keryn Software Licence · rules 0.8.0 · all packs
How to read a rule
- Severity
- How bad the finding is if it is real:
errorfails a default gate,warningandinfodo not. - Confidence
- P0 exploitable as written; P1 dangerous in context; P2 a heuristic or a posture check: confirm by reading the code.
- False-positive risk
- low: an exact API or flag, so hits are almost always real; medium: the pattern is legitimate in many programs, so check each hit; high: it finds a lead to confirm, so expect benign hits.
- Edge
- The relationship the finding sits on. reaches: content or traffic gets to a thing; calls: one thing invokes another; reads: one thing reads from another; writes: one thing writes to another; sends: data leaves for another party; grants: authority is given or left unchecked; depends_on: something relies on a component it does not verify; trusts: one party believes another without evidence. The boundary it crosses, when there is one: user->prompt (untrusted input reaching a prompt); model->code (model output reaching code, a shell, a query or a page); secret->outside (a secret leaving the trusted side); supply-chain (code, models or prompts pulled in from outside).
- Taxonomy chips
- References for readers who work in OWASP LLM 2025, OWASP Agentic, CWE or NIST AI RMF. They say where a rule belongs, not that a framework is satisfied.
seceng.dataset.hf-dataset-unpinned
Section titled “seceng.dataset.hf-dataset-unpinned”warning python huggingface
Hugging Face dataset is loaded without a pinned revision. Training, eval, and RAG ingestion data should be version-pinned.
Confidence P2: a heuristic or a posture check: confirm by reading the code. False-positive risk medium: the pattern is legitimate in many programs, so check each hit.
Edge inspected: depends_on something relies on a component it does not verify; it crosses the supply-chain boundary (code, models or prompts pulled in from outside).
CWE-829 LLM03 LLM09 ASI05 ASI04 ASI06 MAP MEASURE MANAGE
seceng.dataset.remote-training-data-no-integrity
Section titled “seceng.dataset.remote-training-data-no-integrity”info python, typescript, javascript, bash browser-extension
Training, fine-tuning, eval, or RAG ingestion data is downloaded from a remote URL without visible checksum/signature/manifest validation.
Confidence P1: dangerous in context. False-positive risk high: it finds a lead to confirm, so expect benign hits.
Edge inspected: depends_on something relies on a component it does not verify; it crosses the supply-chain boundary (code, models or prompts pulled in from outside).
CWE-494 LLM03 LLM04 LLM09 ASI05 ASI04 ASI06 MAP MEASURE MANAGE
seceng.dataset.eval-set-from-production-no-redaction
Section titled “seceng.dataset.eval-set-from-production-no-redaction”info python, typescript, javascript dataset-finetune-integrity
Production chat, prompt, transcript, or user data appears to be reused as eval/fine-tuning data without visible redaction or consent.
Confidence P1: dangerous in context. False-positive risk high: it finds a lead to confirm, so expect benign hits.
Edge inspected: calls one thing invokes another; it crosses the model->code boundary (model output reaching code, a shell, a query or a page).
CWE-200 CWE-359 LLM02 LLM09 ASI05 ASI01 ASI06 GOVERN MEASURE
seceng.dataset.hf-unknown-user-repo-for-finetune
Section titled “seceng.dataset.hf-unknown-user-repo-for-finetune”info python huggingface
Fine-tuning or eval pipeline pulls dataset/model from a user-controlled Hugging Face repo pattern. Verify provenance and pin revision.
Confidence P1: dangerous in context. False-positive risk high: it finds a lead to confirm, so expect benign hits.
Edge inspected: depends_on something relies on a component it does not verify; it crosses the supply-chain boundary (code, models or prompts pulled in from outside).
CWE-829 LLM03 LLM09 ASI05 ASI06 ASI04 MAP GOVERN
seceng.dataset.synthetic-data-leaks-source-records
Section titled “seceng.dataset.synthetic-data-leaks-source-records”info python, typescript, javascript dataset-finetune-integrity
Synthetic data generation appears to feed raw source records to an LLM without visible minimization/redaction.
Confidence P1: dangerous in context. False-positive risk high: it finds a lead to confirm, so expect benign hits.
Edge inspected: calls one thing invokes another; it crosses the model->code boundary (model output reaching code, a shell, a query or a page).