telemetry-lab

A local, file-based detection workflow lab for reviewer-verifiable telemetry and detection demos.

Latest milestone: v0.6.0 — fourth demo and config-change investigation.

Demos

• telemetry-window-demo
• ai-assisted-detection-demo
• rule-evaluation-and-dedup-demo
• config-change-investigation-demo

Demo	Input	Deterministic core	LLM role	Main artifacts	Guardrails / non-goals
telemetry-window-demo	JSONL / CSV events	Windows Features Alert thresholds	None	features.csv alerts.csv summary.json 3 PNG plots	MVP only No realtime No case management
ai-assisted-detection-demo	JSONL auth / web / process	Normalize Rules Grouping ATT&CK mapping	JSON-only case drafting	rule_hits.json case_bundles.json case_summaries.json case_report.md audit_traces.jsonl	Human verification required No autonomous response No final verdict
rule-evaluation-and-dedup-demo	JSON raw rule hits	Scope resolution Cooldown grouping Suppression reasoning	None	rule_hits_before_dedup.json rule_hits_after_dedup.json dedup_explanations.json dedup_report.md	No realtime No dashboard No AI stage
config-change-investigation-demo	JSONL config changes Policy denials Follow-on events	Normalize Risky-change rules Bounded correlation	None	change_events_normalized.json investigation_hits.json investigation_summary.json investigation_report.md	No realtime No dashboard No AI stage

What This Repo Is

telemetry-lab is a small portfolio repository for constrained detection workflows. It is not a SIEM, dashboard, or monitoring platform; it is organized as four local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.

telemetry-window-demo

telemetry-window-demo turns timestamped event streams into sliding-window feature tables, cooldown-reduced rule-based alerts, PNG timeline plots, and machine-readable run summaries.

ai-assisted-detection-demo

ai-assisted-detection-demo uses deterministic normalization, detection, case grouping, and ATT&CK mapping, then limits the LLM to JSON-only case summarization. Human verification is required, there is no autonomous response, and the demo does not produce a final incident verdict.

rule-evaluation-and-dedup-demo

rule-evaluation-and-dedup-demo starts from raw rule hits and makes cooldown behavior legible. It shows which hits were kept, which were suppressed, how scope was resolved, and why repeated hits collapsed into fewer retained alerts.

config-change-investigation-demo

config-change-investigation-demo follows risky configuration changes into bounded follow-on evidence such as policy denials and service signals. It stays deterministic, file-based, and review-oriented, with no added AI stage.

Quick Run

python -m pip install -e .
python -m telemetry_window_demo.cli run --config configs/default.yaml

Use the same Python interpreter for install, tests, and demo commands. On machines with multiple Python installs, replace python with the intended interpreter path. To run the test suite in a fresh environment, install the dev extra with python -m pip install -e ".[dev]".

Other demo entrypoints:

• python -m telemetry_window_demo.cli run-ai-demo
• python -m telemetry_window_demo.cli run-rule-dedup-demo
• python -m telemetry_window_demo.cli run-config-change-demo

Useful inspection commands:

• python -m telemetry_window_demo.cli summarize --input data/raw/sample_events.jsonl

For CSV inputs, pass a .csv file to --input; use --timestamp-col when the timestamp column is not named timestamp.

The run --config configs/default.yaml command reads data/raw/sample_events.jsonl and regenerates:

• data/processed/features.csv
• data/processed/alerts.csv
• data/processed/summary.json
• data/processed/event_count_timeline.png
• data/processed/error_rate_timeline.png
• data/processed/alerts_timeline.png

With the bundled default sample, the current repo state produces:

• 41 normalized events
• 24 windows
• 12 alerts after a 60 second cooldown

Why it is worth a quick look:

• it shows a full telemetry path from raw events to operator-facing outputs
• the sample inputs and outputs are reproducible in-repo
• a second bundled scenario gives a slightly richer walkthrough without changing the basic CLI flow

Reviewer Path

For a quick coherence pass across the demos:

1. Run python -m telemetry_window_demo.cli run --config configs/default.yaml and confirm data/processed/summary.json reports 41 events, 24 windows, and 12 alerts.
2. Run python -m telemetry_window_demo.cli run-rule-dedup-demo and confirm demos/rule-evaluation-and-dedup-demo/artifacts/dedup_report.md shows 10 raw hits reduced to 6 retained alerts with 4 suppressions.
3. Run python -m telemetry_window_demo.cli run-config-change-demo and confirm demos/config-change-investigation-demo/artifacts/investigation_report.md shows 4 normalized changes, 3 risky changes, and 3 investigations.
4. Run python -m telemetry_window_demo.cli run-ai-demo and confirm demos/ai-assisted-detection-demo/artifacts/case_report.md shows 3 deterministic cases with human verification and no final incident verdict.

Demo Variants

Default sample:

• config: configs/default.yaml
• input: data/raw/sample_events.jsonl
• outputs: data/processed/
• current summary: 41 events, 24 windows, 12 alerts, summary.json included

Richer sample:

• config: configs/richer_sample.yaml
• input: data/raw/richer_sample_events.jsonl
• outputs: data/processed/richer_sample/
• current summary: 28 events, 24 windows, 8 alerts, summary.json included

Input Support

Runtime input support:

• .jsonl
• .csv

Required fields for both formats on every row or record:

• timestamp by default, or the column named by time.timestamp_col in a run config
• event_type
• source
• target
• status

Input and output validation:

• config paths, event inputs, and plot CSV inputs must point to files
• required event fields must be present and non-empty
• custom timestamp columns cannot reuse required event field names
• plot input tables validate required columns, datetime values, numeric ranges, and window bounds

Cooldown behavior:

• repeated alerts are keyed by (rule_name, scope)
• scope prefers the first available entity-like field in this order: entity, source, target, host
• when no entity-like field is present, cooldown falls back to per-rule_name behavior

Repo Guide

• demos/rule-evaluation-and-dedup-demo/README.md explains the third demo and links its committed before/after dedup artifacts
• demos/config-change-investigation-demo/README.md explains the config-change investigation demo and its committed artifacts
• docs/reviewer-path.md maps common review questions to the right demo and artifacts
• docs/sample-output.md summarizes the committed sample artifacts
• docs/roadmap.md sketches the next demo directions
• data/processed/summary.json captures the default run in machine-readable form
• data/processed/richer_sample/summary.json captures the richer scenario pack
• tests/ keeps regression coverage close to the CLI behavior and windowing logic

Next Demo Directions

• add a focused auth/login anomaly triage walkthrough on top of the existing window features
• add a compact config-change drift follow-up scenario using the current deterministic evidence model
• keep sample-output docs and public repo presentation aligned with the checked-in demo state

Scope

This repository is a portfolio prototype, not a production monitoring system.

Limitations

• No real-time ingestion
• No streaming state management
• No alert routing or case management
• No dashboard or service deployment
• Sample-data driven only