Tools · SIEM and detection engineering · 23 / 24
Detection program metrics: alert quality, time to triage, and coverage you can defend
Choose a small set of metrics that show whether detections are useful, compute them from case data, and avoid metrics that reward the wrong behavior.
Metrics worth tracking
Per rule: alert volume, true positive rate (alerts that led to real findings or required action), and false positive rate. For the team: mean time to triage and mean time to escalate. For coverage: techniques with validated detections, and log sources with health monitoring. Trends matter more than single numbers.
Metrics to avoid
Counting rules rewards writing many weak rules. Counting closed alerts rewards closing without investigation. A coverage percentage without validation overstates what you can see. Pair any speed metric with a quality check, such as periodic review of closed cases.
Lab: compute rule quality from cases
Export 90 days of alert dispositions from your case system or create a sample CSV with rule, disposition (true_positive, benign_true_positive, false_positive), created, and triaged timestamps. Compute, per rule, the volume, the false positive rate, and the median triage time. List the three rules with the most false positives and make them the next tuning targets.
import pandas as pd
df = pd.read_csv('alerts.csv', parse_dates=['created', 'triaged'])
df['triage_min'] = (df.triaged - df.created).dt.total_seconds() / 60
summary = df.groupby('rule').agg(
volume=('rule', 'size'),
fp_rate=('disposition', lambda s: (s == 'false_positive').mean()),
median_triage_min=('triage_min', 'median'),
).sort_values('fp_rate', ascending=False)
print(summary.head(10).round(2))Reporting
A short monthly report with the trend of false positive rate, triage time, validated coverage of priority techniques, and log source health tells leadership far more than a raw alert count. Include one or two examples of detections that caught something real.
Diagram
Good and misleading metrics
Measure outcomes, not activity.
- True positive rate per rule
Good: shows usefulness.
- Median time to triage
Good, when paired with quality review.
- Validated technique coverage
Good: backed by tests.
- Number of rules
Misleading: rewards quantity.
- Alerts closed per analyst
Misleading: rewards rushing.
Check yourself
1. Which metric best shows whether a rule is useful?
2. Why is “number of rules” a misleading metric?
3. Which rules should usually be tuned first?