HackBlue

Tools · Linux command line for security · 17 / 28

journalctl for security: filtering by unit, time, priority, and user

Query the systemd journal for SSH logins, sudo use, service crashes, and kernel messages in a time window, and export it as JSON for analysis.

10 min read · practice only with authorization

The filters that matter

journalctl reads the systemd journal. -u limits to a unit such as ssh or sshd. --since and --until take times like "2026-11-02 14:00" or "1 hour ago". -p sets a priority such as err or warning. -k shows kernel messages. -b selects a boot, with -b -1 for the previous one. _UID= and _COMM= match fields the journal records for every entry.

Combine them. A precise query is the difference between reading ten lines and scrolling ten thousand.

Snippet
journalctl -u ssh --since "1 hour ago"
journalctl _COMM=sudo --since today
journalctl -p err -b
journalctl -k --since "2026-11-02 14:00" --until "2026-11-02 15:00"

Fields and output formats

Every entry has structured fields. -o verbose shows them all. -o json writes one JSON object per line, ideal for jq or for shipping to a log platform. -o short-iso prints ISO timestamps that sort and compare cleanly across systems.

journalctl -F _SYSTEMD_UNIT lists every value a field has taken, a quick way to see which units logged anything.

Snippet
journalctl -u ssh -o json --since today | jq -r 'select(.MESSAGE|test("Accepted")) | .MESSAGE'
journalctl -F _SYSTEMD_UNIT | sort | head

Lab: reconstruct an hour

In a lab VM, log in over SSH, run a sudo command, restart a service, and fail one SSH password. Then reconstruct that hour using only journalctl: the accepted login with source address, the sudo command, the restart, and the failure. Write a four-line timeline with ISO timestamps.

Check whether your journal survives reboots. If /var/log/journal does not exist, the journal may be in memory only. Set Storage=persistent in /etc/systemd/journald.conf and restart systemd-journald.

Snippet
journalctl --since "1 hour ago" -o short-iso -u ssh -u nginx _COMM=sudo
ls -ld /var/log/journal
sudo sed -i "s/^#\?Storage=.*/Storage=persistent/" /etc/systemd/journald.conf
sudo systemctl restart systemd-journald
journalctl --disk-usage

Protecting the journal

Local logs can be erased by anyone who gains root. Forward them to a central system as they are written. journald can also seal logs with Forward Secure Sealing (journalctl --setup-keys) so tampering is detectable with journalctl --verify.

A sudden gap in the journal, or a shrinking disk usage figure, is itself an indicator worth investigating.

Diagram

Building a timeline from the journal

  1. Window

    Pick --since and --until around the event.

  2. Narrow

    Add -u units, _COMM, and -p priority.

    Command
    journalctl -u ssh --since "1 hour ago" -o short-iso
  3. Export

    Use -o json and jq for structured filtering.

  4. Persist

    Make sure the journal survives reboot and is forwarded.

Check yourself

1. Which option shows only kernel messages?

2. Your journal disappears after every reboot. What setting fixes it?

3. Which output format is best for piping to jq?