Each record / log entry contain a msg field, composed of a timestamp and an unique ID. Multiple records generated as part of the same Auditd event can share the same msg field. For example, cat /etc/passwd can generate SYSCALL + EXECVE records for the execution of cat and a PATH record for the access to the /etc/passwd file.
The type field contains the type of the record:
- User authentication and access: USER\_LOGIN\_SUCCESS, USER\_LOGIN\_FAILED, USER\_AUTH\_SUCCESS, USER\_AUTH\_FAILED, USER\_START\_SUCCESS, USER\_START\_FAILED, SESSION\_TERMINATED.
- Process execution: EXECVE and SYSCALL.
- Filesystem access: PATH (for relative or absolute file access), CWD (current working directory, useful to reconstruct full path if a relative path has been recorded in PATH records) and OPENAT.
- Commands entered in a TTY console: TTY or by users: USER\_CMD.
- Full command-line of process: PROCTITLE. The associated proctitle field MAY be encoded in hexadecimal.
- Network socket connections: SOCKADDR. The associated saddr field contains IP and port information, and can be interpreted directly at event generation (if log\_format = ENRICHED is set), or with ausearch -i or simple scripting.
- Account activity: ADD\_USER or ADD\_GROUP.
- More record types are listed in the RedHat documentation.
If present, the auid field defines the ID of the user upon login and remains the same even if the user's identity changes (for instance with su).
If present, uid / gid and euid / egid fields define the user / group IDs and the effective user / group IDs of the audited process.
If present, the tty and ses fields define respectively the terminal and session from which the audited process was invoked.
For SYSCALL records, the aX field(s) define the arguments / parameters of the syscall, represented by unsigned long long integers and as such cannot be used to determine the values taken by the arguments.
Configuration file notably defining the path of the log files:/etc/auditd.conf
Configuration defining the rules to apply:/etc/audit/audit.rules
Rules best practice:
Current log files (default location):/var/log/audit/audit.log/var/log/audit/audit.log.1
Rotated log archives (default location):/var/log/audit/audit.log.\*.gz
The aureport and ausearch utilities can (and if possible should) be used to search the auditd log files.
Example:aureport -i \[--login | --executable | ...] \[--summary] -if \
System-wide configuration file:/etc/environment
Initialization scripts can also be used to define system-wide or user scoped environment variables.
Configuration:/etc/fstab
Mount logs (such as Mounting operation / keyword):/var/log/dmesg
/etc/timezone/etc/adjtime/etc/localtime
Example of a configuration file writing logs to common files:
auth,authpriv.\* /var/log/auth.log
*.*;auth,authpriv.none -/var/log/syslog
kern.\* -/var/log/kern.log
mail.\* -/var/log/mail.log
/etc/syslog.conf/etc/rsyslog.conf/etc/rsyslog.d/*.conf/etc/syslogÂng.conf/etc/syslogÂng/*
ext2ext3
Current log file:/var/log/apt/history.log
Rotated log archives:/var/log/apt/history.log.\*.gz
Current log files:/var/log/dpkg.log/var/log/dpkg.log.1
Rotated log archives:/var/log/dpkg.log.\*.gz
Authentication information and sudo commands. More precisely, usage of authorization systems: successful or unsuccessful logins, sudo commands, etc.
Usually generated by the AUTH and AUTHPRIV facilities of the syslog daemon. AUTH regroups the authentication events / messages while AUTHPRIV regroups the elevation of privileges events / messages (such as commands executed through sudo).
Notably includes:
- Successful or unsuccessful logins to the sshd deamon. The authentication types (password, pubkey, etc.) or reason of failure (unknown user, invalid password) is specified.
- Commands executed with elevated privileges using sudo.
Location of the auth logs depend of the syslog daemon configuration (refer to the "Syslog daemon configuration" artefact below for more information).
\[Debian / Ubuntu based systems]
Default location:/var/log/auth.log/var/log/auth.log.1
Rotated log archives:/var/log/auth.log.\*.gz
\[RedHat / CentOS based systems]
Default location (for AUTHPRIV logs):/var/log/secure
utmp / utmpx: currently logged users.wtmp / wtmpx: all current and past logins, with additional details on system reboots, etc.btmp / btmpx: all bad login attempts.
The \*tmpx files are extended database files that supersede the \*tmp files on some distributions.
Linux:/var/run/utmp/var/log/wtmp/var/log/btmp
Solaris:/var/adm/utmp (deprecated)/var/adm/utmpx/var/adm/wtmp (deprecated)/var/adm/wtmpx
FreeBSD 9.0:/var/run/utx.active (utmp equivalent)/var/log/utx.log (wtmp equivalent)
Possible SSH outgoing connections.
System-wide or user scoped known SSH keys for remote hosts. Usually collected, and user-validated, from the remote hosts when connecting for the first time.
The remote hosts hostname and IP address can be either stored in clear-text or hashed if HashKnownHosts is set to "yes" in the SSH client ssh\_config configuration file.
Even if the hosts information are hashed, the following command can be used to check whether the specified hostname is present in the given known hosts file:ssh-keygen -l -f \.
Additionally, John can be used to bruteforce known hosts files:john --format=known\_hosts \nmap -sL -Pn -n 192.168.0.0/16 172.16.0.0/12 10.0.0.0/8 | grep '^Nmap scan report for' | cut -d ' ' -f 5 > IP\_list.txtjohn --wordlist=IP\_list.txt --format=known\_hosts \
One of the few endpoint disk artefacts to identify outgoing SSH connections.
System-wide known hosts:/etc/ssh/known\_hosts\
Scheduled jobs that are configured using the at command-line utility to be run exactly one time. The jobs are executed as shell (bash, zsh, etc.) scripts.
By default, any user can create at jobs.
Each at jobs is represented by a file, which contains metadata information as comments, the environment variables for the execution, and the configured shell script / commands.
The filename follows a specific format (\[a=]\) and gives additional information about the job. The following information can be deduced from the file name and the file itself:
- File created or last modified timestamp => when the at job was created.
- Username, uid, and gid of the user that created the job as shell comments in the file.
- Filename first char: a => job is pending or = => job is running.
- Filename 5 next chars => job id.
- Filename 8 next (and last) chars => hex-encoded minutes since epoch timestamp. Can be converted to retrieve the epoch timestamp of execution by converting to decimal and multiplying by 60.
Configured at jobs locations, each files representing a single at job:/var/spool/at//var/spool/cron/atjobs/
Configuration files that define the users that can or cannot create at jobs:/etc/at.allow/etc/at.deny
Output of currently running at jobs, saved as email text files:/var/spool/at/spool//var/spool/cron/atspool/
Number of at jobs that have been created (already executed, executing, or scheduled):/var/spool/at/.SEQ/var/spool/cron/atjobs/.SEQ
Trace of previous at jobs execution can be found in:
- Session opening by the atd daemon events in syslog or journal logs.
- at jobs email sent events in local email logs.
Scheduled jobs that, unlike at jobs, are executed repeatedly at a given frequency. cron jobs are executed at a pre-determined time (such as five minutes after midnight every day or at 2:15pm on the first of every month). The jobs are executed as shell (bash, zsh, etc.) scripts.
If the system is not running when a cron job is planned, the job will not be executed following the system boot.
By default, only root can define system-wide cron jobs (in /etc/crontab or under /etc/cron.d). But users can be allowed to use the crontab command, and will have their respective jobs created in the /var/spool/cron/crontabs/\ file.
Usage rights of cron are defined by the /etc/cron.allow and /etc/cron.deny files:
- If cron.allow exists, only root and users listed in this file can use cron (and cron.deny is ignored).
- If only cron.deny exists, all users except the ones listed in this file can use cron.
- If neither of the files exist, only root can use cron (default state).
Each cron job is represented by a line in a crontab file.
The line format is a follow:\.
The specifiers can be replaced by a wildcard *, that represents "all". For example, a line with only wildcards (* \* \* \* \* \) means that the command will be executed every minute of every hour of every day of every month.
System-wide cron jobs:/etc/crontab file.crontab files under /etc/cron.d/.
User specific cron jobs, with one crontab file per user (that has configured jobs):/var/spool/cron/crontabs/\.
Usage rights of cron:/etc/cron.allow/etc/cron.deny
Scheduled jobs that are executed with a frequency specified in days or in a time interval (daily, weekly, monthly, or annually). Unlike cron jobs, anacron jobs are executed after a system boot if the system was not running during the last scheduled execution.
Depending on the Linux distribution, anacron itself may be run as a systemd timer (/lib/systemd/system/anacron.timer), a system cron job (/etc/cron.d/anacron), and / or at boot (/etc/init.d/anacron).
By default, cron "dot" directories (/etc/cron.{daily,weekly,monthly}) are often run by anacron.
Each anacron job is represented by a line in an anacrontab file.
The line format is a follow: \
System-wide anacron jobs: /etc/anacrontabcron "dot" directories, that contain shell scripts directly:/etc/cron.daily//etc/cron.hourly//etc/cron.monthly//etc/cron.weekly/
Deprecated mechanism, in favor of Systemd / init.d, to define and start services as shell scripts at the system startup.
Scripts are configured to be executed at different run levels, from 0 (stop) to 6 (reboot) through 1 (maintenance mode) and 2-5 (multi-users mode, such as desktop startup, etc.).
RC scripts locations:/etc/rc.local/etc/rc.common/etc/rc<0-6>.d/*/etc/rcS.d/*
User scoped initialization script:\\\\\
System-wide initialization scripts:/etc/profile/etc/profile.d/\*/etc/skel/.profile (Not used if \ or \ exist).
Executed if an interactive shell is opened:\\
Executed at the end of the session:\/etc/zlogout\
Configuration of the SSH authorization keys:/etc/ssh/sshd\_config AuthorizedKeysFiledirective.
Default SSH authorization keys location:\\
Each XDG autostart entry is represented by a file, which contains the following notable keys:
- Type key that specifies the entry type (application, link, or directory).
- Name key that indicates an arbitrary name assigned by the autostart entry creator.
- Exec key that defines the application and command line arguments to be executed.
User scoped initialization autostart entries:\
System-wide autostart entries: /etc/xdg/autostart/*.desktop
Simply put, webshells are script files that are executed by a webserver. Webshells are notably leveraged to:
- execute code / commands on the underlying operating system following the exploitation of a web vulnerability (unrestricted file upload, remote code execution, etc.)
- maintain persistence following the compromise of an host exposing a webserver (usually Internet facing).
Usual locations:/var/www/html/usr/local/www//etc/nginx/etc/apache2/srv//srv/www...
Webshells can be uncovered by:
- Yara rules aimed at webshells such as Neo23x0's gen\_webshells.yar or thor-webshells.yar:yara -r \
- Reviewing added files or modifications in legitimate files using code repository or a fresh install of the application if possible.
- Manually by looking for known webshell patterns (Runtime.getRuntime().exec, eval, system, etc.), obfuscated script files, or files modified during the targeted timeframe.
- Reviewing the webserver access logs if available, looking for exploitation IoCs, unusual requests, large response size, etc.
wget utility's HTTP Strict Transport Security (HSTS) history.HSTS is a mechanism to only allow access to a particular website in HTTPS if that website was accessed in HTTPS once and defines an HSTS policy. The HSTS policy to follow is define by the web server through the Strict-Transport-Security HTTP response header. The web browser or utility has to store the websites accessed in HTTPS (with HSTS implemented) for the duration specified in the header to support HSTS.
wget's HSTS history is implemented as a plaintext file, with an entry per line.
For each entry, the following notable information are available:
- Hostname of the accessed website
- Created timestamp in UTC (in epoch format) that defines when the entry was created. As the entry is overwritten upon new access to a website defining an HSTS policy, the created timestamp matches the last access to the website.
Debian / Ubuntu:/var/log/apache2/access.log/var/log/apache2/error.log
RHEL / Red Hat / CentOS / Fedora :/var/log/httpd/access\_log/var/log/httpd/error\_log
FreeBSD:/var/log/httpd-access.log/var/log/httpd-error.log
Custom definition for access (CustomLog section) or error (ErrorLog section) logs:/etc/httpd/conf/httpd.conf/etc/apache2/apache2.conf/usr/local/etc/apache22/httpd.conf