Academy Labs/AI Logging & Forensics Lab
Keryn Academy Lab45 minPractitionerDefend

AI Logging & Forensics Lab

Evaluate AI telemetry coverage by analyzing real guardrail denial logs and SOC event streams. Identify forensic gaps, correlate events across log sources, and produce a chain-of-custody and investigatability assessment.

Progress

0/100 points

Status

not-started

Steps

0/4

Mission

Primary objective

Analyze both log sources. Classify each event, correlate across sources, identify what evidence is present and what is missing, and write an investigatability assessment that tells the security team whether they can answer the questions an incident response requires.

Brief

Scenario

Post-incident AI telemetry review

A security operations team has flagged anomalous activity in an AI assistant deployment. You have access to two log sources: a guardrail denial log (freeform) and a SOC event stream (structured JSONL). The events span a 15-minute window. Your job is to determine whether this constitutes an incident, what happened, and whether the logs are sufficient to support a full forensic investigation.

Objectives

  • Classify AI security events from guardrail and SOC log formats.
  • Correlate events across log sources to reconstruct incident timelines.
  • Identify forensic gaps: what information is missing that an investigation would require.
  • Produce a chain-of-custody document and investigatability assessment.

Prerequisites

  • Understand basic log formats: structured JSON events and freeform log lines.
  • Review MITRE ATT&CK and ATLAS event classification concepts.
  • Complete the RAG Security Lab or review AI pipeline architectures.

Expected signals

  • prompt injection attempt
  • PII exposure
  • tool abuse
  • token spike anomaly
  • guardrail denial
  • forensic gap

Prepare

Reading materials

Study tool

Flash cards

Study cards across the four pillars and the 14 assessment domains, with a spaced-repetition style runner.

Checking…

Study tool

Scenario assessment

Scored scenario questions with per-domain results and a remediation plan for your weakest domains.

Checking…

Course print edition

AI Product Management for Secure AI Features

Define, scope, and ship AI products buyers can trust.

Checking…

Course print edition

AI Model Gateways and Secure Platforms

Build the approved path for safe AI adoption.

Checking…

Do not rebuild

Reusable source assets

fixturestatic/read-only

Guardrail denial log

Freeform log file capturing guardrail trigger events with timestamps, request IDs, detected patterns, and applied policies. Four events covering injection detection, PII redaction, and request blocking.

llm-attack-range/ai-security-range/fixtures/logs/guardrail-denials.log

fixturestatic/read-only

SOC event stream

Structured JSONL SOC events: WAF block, guardrail trigger, tool abuse anomaly, and token spike. Each event includes timestamp, event type, user ID, payload snippets, and MITRE IDs.

llm-attack-range/ai-security-range/fixtures/simulation-logs/soc-events.jsonl

Sample inputs

Guardrail denial log · log

From fixtures/logs/guardrail-denials.log. Note: the log captures policy trigger and action but does NOT include the full prompt, user identity, session context, or downstream tool calls.

2026-05-02T10:00:01Z [INFO] Request received: id=req_123
2026-05-02T10:00:02Z [WARN] Guardrail trigger: Prompt Injection detected. Pattern: "Ignore previous instructions". Policy: P01_STRICT
2026-05-02T10:00:02Z [ERROR] Request req_123 blocked. Action: 403 Forbidden.
2026-05-02T10:05:15Z [WARN] Guardrail trigger: PII leak attempt. Field: SSN. Policy: P02_PII_REDACT
SOC event stream · text

From fixtures/simulation-logs/soc-events.jsonl. These events are structured with MITRE ATT&CK/ATLAS IDs where applicable. The TOKEN_SPIKE event has no MITRE mapping — is that a gap?

{"timestamp": "2026-05-02T11:00:00Z", "event_type": "WAF_BLOCK", "source_ip": "192.168.1.50", "payload_snippet": "Ignore previous instructions", "mitre_id": "AML.T0051"}
{"timestamp": "2026-05-02T11:05:00Z", "event_type": "GUARDRAIL_TRIGGER", "user_id": "user_821", "rule": "PII_REDACTION", "action": "REDACTED"}
{"timestamp": "2026-05-02T11:10:00Z", "event_type": "TOOL_ABUSE_ANOMALY", "tool_id": "python_repl", "command": "os.system('rm -rf /')", "status": "BLOCKED"}
{"timestamp": "2026-05-02T11:15:00Z", "event_type": "TOKEN_SPIKE", "user_id": "user_999", "tokens": 128000, "anomaly_score": 0.95}

Track progress

Lab steps

Classify each log event

Read both log sources. For each event, classify it by type (prompt injection attempt, PII exposure, tool abuse, DoS/resource exhaustion, policy violation) and severity (low/medium/high/critical). Note which events have MITRE IDs and which do not.

Evidence prompt: Build a table: timestamp, source, event type, severity, MITRE ID (or 'none'). Flag any event that is missing a classification field you would expect.

Correlate events across sources

The guardrail log and SOC stream cover overlapping time windows. Determine whether any events in the two sources describe the same underlying activity. What is the timeline? Is there a user or session ID that connects events across sources?

Evidence prompt: Describe any correlations you found. If the same event appears in both sources, what additional detail does each source add? What is still unknown about the actor or session?

Identify forensic gaps

An incident investigation needs to answer: Who? What? When? How? Impact? For each unanswered question, identify what log field or log source is missing. Common gaps: no session token in the guardrail log, no full prompt captured, no user identity on the WAF block, no completion log to see what the model returned.

Evidence prompt: List each forensic question and whether the current logs can answer it. For gaps: name the missing field or log source.

Write the investigatability assessment

Produce a structured assessment: can this set of logs support a full incident investigation? Rate each of the five forensic dimensions (attribution, timeline, impact, containment evidence, chain of custody) as sufficient, partial, or insufficient. Fill in the evidence artifact builder below.

Evidence prompt: Fill in all required fields in the evidence artifact builder. The sufficiency rating field determines your overall finding.

Submission draft

Evidence artifact builder

Forensic Log Analysis

Document event classifications, cross-source correlations, forensic gaps, and your investigatability assessment. This artifact supports post-incident review and telemetry improvement planning.

Reference

Framework mappings

NIST AI RMF

MANAGE · Incident response and risk treatment

MITRE ATLAS

AML.T0051 · LLM Prompt Injection

OWASP LLM Top 10

LLM02 · Sensitive Information Disclosure

Self-assessment

Scoring checklist

Score estimate: 0/100

Export

Submit or export your lab evidence

Save a local progress draft, submit the self-scored artifact, or export Markdown for evidence portfolio use.

Next

Continue the Keryn lab path