The detection engineering lifecycle: from idea to tested, tuned, and retired rules

Stages

A detection starts as a requirement: a technique you need to see, a gap from an incident, or a new threat report. Next, confirm the data exists: which log source, which fields, and whether they are actually collected everywhere. Then write the logic, test it against both malicious examples and normal history, deploy it with a runbook, and watch its alert volume and true positive rate. Finally, review it periodically and retire it when the data or the threat changes.

Skipping the data check is the most common failure: a rule written for command-line arguments is useless where process command lines are not logged.

What to document

Each rule should record its purpose, ATT&CK technique, data sources and required fields, the logic in plain words, known false positives, test cases, severity, owner, and the response steps. This turns a query into something a teammate can maintain.

• Purpose and threat
• Data source and fields
• Logic in plain language
• Known false positives
• Test evidence
• Owner, severity, runbook link

Lab: write one detection card

Choose a behavior such as a new local administrator being added on a workstation. Write the card first, before any query: purpose, technique T1098 or T1136, data source Windows Security 4732 for the Administrators group, fields needed, expected false positives such as IT onboarding scripts, and a response step. Then write the query in your SIEM lab, generate a test event by adding a test user to the group in a VM, and attach the result as test evidence.

# generate a benign test event in a lab VM (elevated PowerShell)
New-LocalUser -Name labtest -NoPassword
Add-LocalGroupMember -Group Administrators -Member labtest
Remove-LocalGroupMember -Group Administrators -Member labtest
Remove-LocalUser -Name labtest

Measuring health

Track alerts per week, the share closed as true positive, mean time to triage, and whether the rule fired during exercises. A rule that never fires may be broken; a rule that fires constantly and is always closed as benign is training analysts to ignore it.