HackBlue

Tools · Linux command line for security · 14 / 28

How to investigate a suspicious process on Linux with ps, pstree, and top

Find a process using too much CPU or running from an odd path, see its parent and children, its user, start time, and command line.

11 min read · practice only with authorization

See every process with context

ps auxf lists every process with user, PID, CPU and memory share, start time, and the full command, drawn as a tree so you can see which process started which. pstree -p shows the same hierarchy more compactly. top or htop sorts live by CPU, which is how a cryptominer usually reveals itself.

The questions are always the same: who is running it, what started it, where is the binary, and what is it talking to.

Snippet
ps auxf | less
pstree -p -u | less
ps -eo pid,ppid,user,lstart,etime,cmd --sort=-%cpu | head

Signs worth a closer look

A process running from /tmp, /dev/shm, or a hidden directory such as /var/tmp/.x is unusual. So is a web server user like www-data owning a shell or a network tool. A name that imitates a kernel thread, such as [kworker] in square brackets but with a real executable path, is a common disguise; genuine kernel threads have no executable.

Unexpected parent-child pairs matter too. sshd spawning bash is normal. nginx or a database spawning bash is not.

Lab: trace a fake miner

In a lab VM, copy /bin/sleep to /dev/shm/.cache/kworker and run it as a background job to imitate a disguised process. Find it with ps by sorting or grepping, confirm its executable path through /proc/PID/exe, and see its parent with ps -o ppid. Note how the name looks innocent but the path does not.

Before killing a suspicious process in a real incident, capture what you can: the command line, open files, network connections, and a copy of the binary. Killing first destroys evidence.

Snippet
mkdir -p /dev/shm/.cache && cp /bin/sleep /dev/shm/.cache/kworker
/dev/shm/.cache/kworker 3600 &
ps -eo pid,ppid,user,cmd | grep -v grep | grep kworker
ls -l /proc/$(pgrep -f .cache/kworker)/exe
sudo ls -l /proc/*/exe 2>/dev/null | grep -E "/tmp/|/dev/shm/"

What to record

Write down PID, PPID, user, start time, full command line, executable path, and its hash. Those let someone else confirm your finding and search other hosts for the same binary.

Then move to network connections and persistence, covered in the next tutorials, to learn how it got there and whether it will come back.

Diagram

Triage of one suspicious process

  1. Spot

    Sort by CPU or look for odd paths and users.

    Command
    ps -eo pid,ppid,user,cmd --sort=-%cpu | head
  2. Place

    Find its parent and children with pstree.

  3. Locate

    Read /proc/PID/exe for the real binary path.

  4. Preserve

    Record details and copy the binary before stopping it.

Check yourself

1. A process is named [kworker/0:1] but /proc/PID/exe points to /dev/shm/.cache/kworker. What does that suggest?

2. Which parent-child pair is most suspicious?

3. Why capture details before killing a suspicious process?