Forensic
| Mission | Members | Contacts
| Incident handling | Alerts | Monitoring | Security challenges | Procedures | Dissemination
FORENSIC HOWTO
release of 19 may 2011, edited by Heiko Reese <Heiko.Reese(at)kit.edu>
Linux Forensics HowTo
This document is a constant work-in-progress. Comments and additions are always welcome.
This document describes a best-effort approach for preserving and analyzing compromized Linux installations. Because there are many different Linux userlands (aka distributions), some commands may require a different syntax or different commands (most notably when package management is involved) to achive the same goal. To follow the instructions in this document, at least a basic understanding of the procedures presented here is necessary.
Forensic analysis consists of (at least) these phases:
* Identify the system. * Gather data. * Analyzie the data.
Identify compromized systems
TODO: (logs, monitoring, netflows, suspicious or erratic behaviour, external notification, etc...)
Gather data
The data aquisition process is twofold: first, gather information from the running (live) system. After that, analyze the »cold« system.
If the system runs as a virtual machine, freeze/pause it and create dumps/images from the filesysems/blockdevices and the memory.
Try not to write to the local filesystem. Put all gathered data onto external drives, network shares or into a ramdisk.
Collect data about the system's state (consult the manpages if you are unsure about what you are doing): {{{
- -------------
mkdir incident_data cd incident_data ps -auxwwwe > ps_auxwwwe.txt netstat --program --notrim --verbose -n > netstat_pTvn.txt netstat --program --notrim --verbose > netstat_pTv.txt w > w.txt last > last.txt lastlog > lastlog.txt cat /proc/mounts > proc_mounts.txt arp -n > arp_n.txt ip neigh show > ip_neigh_show.txt ip route list > ip_route_list.txt ip link show > ip_link_show.txt lsof -b -l -P -X -n -o -R -U > lsof_blPXnoRU.txt for i in t p c t l; do ipcs -a -${i} > ipcs_a_${i}.txt;done
- -------------
}}}
If there are suspicious processes that need further analysis, preserver the original binary and dump the program's memory: {{{
- -------------
export PID=12345 # <- INSERT PROCESS-ID (PID) HERE kill -STOP ${PID} # stop process cp /proc/${PID}/exe ${PID}.exe
- some distributions have a script called "gcore" which does this in batch-mode
gdb -p ${PID}
# type "gcore", then "detach" and "quit" # The program's memory is now saved as core.PID.
ls -l /dev/shm
- look for shared-memory-segments owned by the process
- by doing
grep '/dev/shm' /proc/${PID}/maps
- copy them if deemed neccessary
tar cvf proc_${PID}.tar /proc/${PID}/{auxv,cgroup,cmdline,comm,environ,limits,maps,sched,schedstat,sessionid,smaps,stack,stat,statm,status,syscall,wchan} kill -9 ${PID} # kill process
- -------------
}}}
Create a list of all files in the system: {{{
- -------------
mkdir /mnt/root_ro mount --bind / /mnt/root_ro mount -o remount,ro /mnt/root_ro
- do not combine the two previous steps, this won't work on some older kernels
find /mnt/root_ro -xdev > find_root_ro_xdev.txt umount /mnt/root_ro
- -------------
}}}
Install/copy chkrootkit (http://www.chkrootkit.org), rkhunter (http://rkhunter.sourceforge.net) and ossec-rootcheck (http://www.ossec.net/main/rootcheck) to the machine.
Remount all „real“ filesystems as read-only ({{{mount -o remount,ro MOUNTPOINT}}}). This is best done manually by the administratior. You may use this very simple heuristic as an alternative if needed:
grep -E ' (ext[234]
Run chkrootkit, rkhunter and ossec-rootcheck.
Some package management-systems have checksums of their installed packages. Debian-based systems offer {{{debsums}}} and Redhat has {{{rpm -Va}}}. Save the output.
Copy the collected data someplace save or remove/unount the external storage/network drive.
Do not shutdown as usual! Disconnect the power the the system.
Remove the harddisks and create images (use http://www.gnu.org/software/ddrescue/ddrescue.html).
Data analysis
In order to proceed from this point on, answer these questions:
* What do you already know? How can you use this knowledge to proceed? * What is your next goal? Finding the breakin point? Understand malicious code/backdoors? Find culprits? Identify other systems involved? * Do you have the resources/manpower to analyze now? Or should these resources be used to mitigate the threat (assuming that is still exists)?
TODO: (explain procedures for standard problems)
Here are a few pointers to helpful software and services:
* TODO: malware-foo * TODO: network intelligence * TODO: binary analysis