HackBlue

Tools · Python and Bash for security automation · 17 / 20

Scheduling security scripts with systemd timers and cron, with logging and alerts

Run your checks on a schedule as a dedicated user, capture output in the journal, alert on failure with OnFailure, and avoid the common cron pitfalls.

9 min read · practice only with authorization

systemd timers

A timer unit triggers a service unit on a schedule. Compared with cron, timers log to the journal automatically, can run missed jobs after downtime with Persistent=true, add randomized delays, and support OnFailure to trigger another unit, such as a notification.

Snippet
# /etc/systemd/system/certwatch.service
[Unit]
Description=TLS certificate expiry check
OnFailure=notify-failure@%n.service
[Service]
Type=oneshot
User=secchecks
ExecStart=/usr/bin/xargs -a /opt/secchecks/hosts.txt /opt/secchecks/.venv/bin/python /opt/secchecks/certwatch.py
NoNewPrivileges=true
ProtectSystem=strict
PrivateTmp=true

# /etc/systemd/system/certwatch.timer
[Unit]
Description=Daily certificate check
[Timer]
OnCalendar=*-*-* 07:00:00
RandomizedDelaySec=15m
Persistent=true
[Install]
WantedBy=timers.target

If you use cron

cron runs with a minimal environment, so use absolute paths, set PATH at the top of the crontab, and redirect output to a log or to logger so failures are visible. Use flock to prevent overlapping runs of long jobs.

Snippet
PATH=/usr/local/bin:/usr/bin:/bin
MAILTO=""
0 7 * * * flock -n /tmp/certwatch.lock /opt/secchecks/run.sh 2>&1 | logger -t certwatch

Lab: schedule and fail on purpose

Create a dedicated user, install one of your scripts from earlier lessons, and add the service and timer units. Start the service once manually and read the journal. Add a host that will fail the check and confirm the notification unit runs. List timers to see the next run time.

Snippet
sudo useradd --system --home /opt/secchecks --shell /usr/sbin/nologin secchecks
sudo systemctl daemon-reload
sudo systemctl start certwatch.service
journalctl -u certwatch.service -n 20 --no-pager
sudo systemctl enable --now certwatch.timer
systemctl list-timers certwatch.timer

Security of the scheduler

Scheduled jobs are also an attacker persistence technique, so keep yours documented, owned by root, and run as unprivileged users. Script files should not be writable by the account that runs them. Review timers and crontabs regularly, as covered in the Linux security hub.

Diagram

cron versus systemd timer

Both work; timers add observability.

cron

Logging. Manual redirection

Missed runs. Skipped

Sandboxing. None by default

systemd timer

Logging. Journal automatically

Missed runs. Persistent=true catches up

Sandboxing. ProtectSystem, NoNewPrivileges, PrivateTmp

Check yourself

1. Which timer option runs a missed job after the machine was off?

2. Which unit setting triggers a notification when the job fails?

3. Why do cron jobs often fail when they work in an interactive shell?