Linux persistence checklist: authorized_keys, shell profiles, ld.so.preload, and more

The usual places

Beyond cron and systemd, look at SSH authorized_keys for every user, shell startup files such as .bashrc, .profile, and /etc/profile.d/, /etc/ld.so.preload which forces a library into every program, PAM modules in the auth stack, sudoers drop-ins, new accounts with UID 0, rc.local on older systems, and udev rules that run commands when devices appear.

Each of these is legitimate in normal use, which is why attackers choose them. The check is always the same: compare with what you expect, and look at recent changes.

One command per location

Run these as root and save the output. On a clean system, /etc/ld.so.preload usually does not exist at all, so its presence is a strong signal. PAM files should match package versions, which debsums or rpm -V can confirm.

Look for recently modified files across all of them with find -mtime -7.

Snippet

sudo find / -name authorized_keys -not -path '/proc/*' -exec ls -l {} \; 2>/dev/null
sudo ls -la /etc/profile.d/ ; sudo grep -l -E 'curl|wget|nc |base64' /home/*/.bashrc /root/.bashrc /etc/profile.d/* 2>/dev/null
cat /etc/ld.so.preload 2>/dev/null || echo 'no ld.so.preload'
sudo grep -rn 'pam_exec\|pam_script' /etc/pam.d/
ls -l /etc/rc.local /etc/udev/rules.d/ 2>/dev/null
sudo find /etc /root /home -xdev -type f -mtime -7 2>/dev/null | head -50

Lab: plant three, find three

On a lab VM, add a line to a test user’s .bashrc that writes to a log file on each login, add a second public key to their authorized_keys, and create a harmless udev rule that logs when a USB device is added. Then give the commands above to yourself as if you were a responder, and see which you find first.

Record how long each took. The ones that took longest are where your monitoring should improve.

After you find one

Finding one persistence mechanism usually means there are others. Check every location, preserve copies with timestamps, then remove. Rotate credentials and keys used on the host, since the attacker may have copied them.

For important servers, rebuilding from a known-good image is often safer than cleaning. Persistence you did not find survives cleaning but not a rebuild.

Diagram

Persistence locations and their tells

Normal features, abused.

authorized_keys
Unknown key comments or keys added recently.
Shell startup files
Network commands or encoded strings in .bashrc.
/etc/ld.so.preload
Usually absent; presence is suspicious.
PAM stack
pam_exec or unknown modules in auth lines.
udev rules
RUN+= commands for common device events.

Check yourself

1. Why is the existence of /etc/ld.so.preload notable?

2. You removed one backdoor key. What should you assume?

3. Which file runs commands in every interactive bash session for a user?