Tools · SIEM and detection engineering · 24 / 24
Linux detections in a SIEM: auditd and syslog queries for common attacker behavior
Turn auditd and authentication logs into SIEM detections for new sudoers entries, cron changes, reverse-shell-like process patterns, and suspicious use of tools like curl piped into a shell.
Data you need
auditd records system calls and file watches with keys you assign, such as -k sudoers_change. Authentication logs record SSH and sudo activity. Forward both to the SIEM with an agent such as the Elastic Agent auditd integration, the Splunk Add-on for Unix and Linux, or the Azure Monitor Agent. Without file watches and execve logging, most of these detections cannot work.
# /etc/audit/rules.d/hackblue.rules -w /etc/sudoers -p wa -k sudoers_change -w /etc/sudoers.d/ -p wa -k sudoers_change -w /etc/crontab -p wa -k cron_change -w /var/spool/cron/ -p wa -k cron_change -a always,exit -F arch=b64 -S execve -F euid=0 -k root_exec
Detection ideas
Alert on writes to sudoers or cron files outside change windows or by non-configuration-management processes. Alert on web server users (www-data, nginx) executing shells or network tools. Flag command lines that pipe a download into a shell, such as curl ... | sh, on servers. Flag new SSH authorized_keys entries for privileged users.
# Splunk example
index=linux sourcetype=linux_audit key=sudoers_change
| stats count values(exe) as exe values(auid) as auid by host
# KQL example (Syslog / AUOMS or custom table)
Syslog
| where SyslogMessage has "type=EXECVE" and SyslogMessage has_any ("a0=\"curl\"", "a0=\"wget\"")
// auditd writes one EXECVE per process, so curl ... | sh arrives as two events.
// Join on the SYSCALL record's ppid to find the shell; arguments with spaces are hex-encoded.Lab: trigger and catch
On a lab VM, load the audit rules above, forward logs to your SIEM, and perform benign versions of each behavior: edit /etc/sudoers.d/labtest with visudo, add a harmless cron entry, and run sh -c id as www-data with sudo -u www-data. Confirm each produces an event with the expected key and that your queries find them. Remove the test changes afterward.
sudo augenrules --load && sudo auditctl -l sudo visudo -f /etc/sudoers.d/labtest # add a comment line, save (crontab -l 2>/dev/null; echo '# lab test entry') | crontab - sudo -u www-data sh -c id sudo ausearch -k sudoers_change -i | tail sudo rm /etc/sudoers.d/labtest; crontab -l | grep -v 'lab test' | crontab -
Tuning
Configuration management tools such as Ansible, Puppet, or Chef legitimately edit these files. Exclude them by the exact executable and service account, and alert when changes come from interactive sessions instead.
Diagram
Linux behaviors and their data
Each detection needs a specific source.
- sudoers change
auditd file watch on /etc/sudoers and sudoers.d.
- cron persistence
auditd watch on crontab and spool directories.
- web user runs a shell
auditd execve with uid of the web server user.
- curl piped to shell
execve command lines or shell history forwarding.
- new SSH key
File watch on authorized_keys for privileged users.
Check yourself
1. Which auditd rule type records writes to /etc/sudoers?
2. Which event is most suspicious on a web server?
3. How should you exclude Ansible changes from a sudoers alert?