Tools · SIEM and detection engineering · 10 / 24
Reading web server access logs for attacks: scanners, brute force, and odd status codes
Parse nginx or Apache access logs with command-line tools, find scanners by 404 bursts and user agents, spot login brute forcing, and flag suspicious paths and parameters.
The combined log format
Each line contains the client IP, identity, user, timestamp, request line (method, path, protocol), status code, response size, referrer, and user agent. Behind a load balancer or CDN, the real client IP is often in an X-Forwarded-For field you must log explicitly.
What attacks look like
Content discovery scanners produce bursts of 404 responses for paths like /.env, /wp-admin, /.git/config, and /phpmyadmin from one IP. Login brute forcing shows many POST requests to /login with 401 or 200 responses of identical size. Injection probes leave quotes, angle brackets, ../ sequences, or encoded characters in paths and query strings. A sudden 500 spike after such requests means something reacted badly.
Lab: a quick triage script
Use the access log from your own lab web server, or a public sample from a honeypot dataset. Run the commands below to list the top 404 sources, the most-requested missing paths, POST bursts to login endpoints, and requests containing traversal or script-like patterns. Then pick one IP and print its full timeline.
awk '$9==404 {print $1}' access.log | sort | uniq -c | sort -rn | head
awk '$9==404 {print $7}' access.log | sort | uniq -c | sort -rn | head -20
awk '$6=="\"POST" && $7 ~ /login/ {print $1, $9}' access.log | sort | uniq -c | sort -rn | head
grep -E "(\.\./|%2e%2e|<script|%3Cscript|union(%20|\+)select|/etc/passwd)" -i access.log | head
awk -F'"' '{print $6}' access.log | sort | uniq -c | sort -rn | head
grep '^203.0.113.45 ' access.log | awk '{print $4, $6, $7, $9}'From triage to detection
Turn the patterns into SIEM rules: more than N 404s per IP per minute, more than N failed logins per IP or per username, and any request matching the injection pattern list that returned 200 or 500. Rate limiting and a web application firewall reduce noise, but logs remain the record of what was tried and what succeeded.
Diagram
Access log lines and what they mean
Same server, three stories.
Scanner
203.0.113.45 "GET /.git/config HTTP/1.1" 404
Dozens of 404s per minute
Brute force
198.51.100.9 "POST /login HTTP/1.1" 401 (x400)
Identical sizes, many usernames
Probe
192.0.2.33 "GET /download?file=..%2f..%2fetc%2fpasswd" 200
Status 200, size 1.9 KB
Check yourself
1. What does a burst of 404 responses for /.env, /.git/config, and /wp-admin from one IP usually indicate?
2. In the default combined format split by spaces, which awk field holds the status code?
3. Which log line deserves attention first?