Detection as code: storing rules in Git with tests and CI checks

Why treat rules as code

When rules live only in a SIEM console, changes are hard to review, easy to break, and impossible to roll back cleanly. In a Git repository, each change is a pull request with a reviewer, a diff, automated tests, and history. Deployment uses the SIEM’s API or Terraform provider, so the repository is the source of truth.

A simple layout

Keep rules in a folder per platform or technique, with test events beside them: one or more samples that should match and some that should not. CI validates syntax, converts Sigma to the target query language, and runs the queries or a local matcher against the samples.

Snippet

detections/
  windows/
    certutil_download/
      rule.yml
      tests/
        match_certutil_urlcache.json
        nomatch_certutil_dump.json
  cloud/
    aws_root_console_login/
      rule.yml
      tests/...
.github/workflows/detections.yml

Lab: a CI job that lints and converts

Create a small repository with two Sigma rules and a GitHub Actions workflow that installs sigma-cli, runs sigma check on every rule, and converts them for your SIEM backend, failing the build on any error. Add a Python test that loads each JSON sample and checks it against a simple field matcher for the rule’s selection, so a change that breaks matching fails CI.

Snippet

name: detections
on: [pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: '3.12' }
      - run: pip install sigma-cli pytest pyyaml
      - run: sigma plugin install splunk
      - run: sigma check detections/
      - run: sigma convert -t splunk -p sysmon detections/ -o build/splunk.spl
      - run: pytest tests/

Deploying

On merge to the main branch, a deploy job pushes converted rules through the SIEM API, such as Splunk saved searches, Sentinel analytics rules via ARM or Terraform, or Elastic detection rules via the Kibana API. Tag each deployed rule with the commit hash so an alert can be traced back to the exact version.

Diagram

Detection as code pipeline

Pull request
Rule change with test samples.
Lint and convert
sigma check and sigma convert in CI.
Test
Matching samples must match; non-matching must not.
Deploy
Merge triggers deployment through the SIEM API with the commit hash.

Check yourself

1. What is the main advantage of keeping rules in Git?

2. Why include samples that should not match?

3. How should rules reach the SIEM in this model?