A Comprehensive Guide to Auditd in Linux

Introduction to Auditd

Auditd is the userspace component of the Linux Auditing System included in the Linux kernel. It logs information about system activity to provide accountability, detect intrusions, and support forensic analysis. Understanding auditd is key for any Linux administrator concerned about security and compliance.

As a background daemon, auditd collects data specified by audit rules and writes them to log files for examination. The audit daemon itself is controlled by the auditd.conf file. Additional utilities like auditctl, ausearch, and aureport give admins control over auditing configuration, search, and reporting.

Key Components of Linux Audit System

Audit Daemon (auditd) – The core auditd daemon manages auditing activity in the kernel and writes logs. It‘s highly configurable via the /etc/audit/auditd.conf file.

Audit Dispatcher (audispd) – Plugins for forwarding event notifications. Used for log centralization.

Audit Rules (auditctl) – The /etc/audit/rules.d/ files and auditctl tool control what events get logged through kernel filters.

Audit Logs – By default written to /var/log/audit/audit.log. Contains event data specified in audit rules.

Analysis Tools – ausearch, aureport, aulast tools search logs and create reports.

Together these components allow comprehensive auditing tailored to security policy requirements.

Installing Auditd

Auditd is included in most Linux distributions. To install on Debian/Ubuntu:

$ sudo apt install auditd audispd-plugins

RHEL/CentOS:

$ sudo yum install audit audit-libs

Enable and start the auditd service:

$ sudo systemctl enable auditd
$ sudo systemctl start auditd

Verify it is running with systemctl status auditd.

Configuring Audit Rules

Audit rules control what events and data get captured to logs. Rules are created using the auditctl command and read on auditd start from /etc/audit/rules.d/.

Here is the syntax for auditctl:

auditctl -a action,filter -S system_call -F field=value -k key_name

Common options:

• -w file – Watch file or directory
• -p permissions – Filter by permission type
• -k key – Tag events with a keyword
• -a , -D – Add or delete rule

For example, to audit permission changes on /etc/shadow:

auditctl -w /etc/shadow -p wa -k shadow_perms

This adds a rule filtering write and attribute change attempts, tagged with key "shadow_perms".

Understanding Audit Rule Syntax

Audit rules match events in the kernel using a series of selectors. Important ones include:

• -S syscall – Match system call
• -F arch=b64 – Filter on architecture
• -a always,exit – Filter exit status

Multiple selectors can be combined to precisely target specific events.

Here we log execute attempts on setuid binaries:

-a always,exit -F arch=b64 -S execve -F auid>=1000
     -F uid=0 -C uid!=euid

This breaks down to:

• always,exit: Log on exit
• arch=b64: 64-bit systems
• execve: execve syscalls
• -F uid=0: Root uid
• -C uid!=euid: Setuid (effective uid diff from real)

Understanding these selectors allows crafting of very customized rules. See man auditctl for details.

Monitoring Common Attack Vectors

Some standard audit use cases:

• Monitor write access to /etc/shadow (Contains user passwords)
• Log exec of setuid binaries
• Track reads of SSL private keys
• Watch login-related files like PAM modules
• Log firewall rule changes

Attacks often involve tampering with authentication systems, gaining root privilege, or lateral movement. Crafting rules to explicitly log these components can expose malicious activity.

Searching Audit Logs with ausearch

The ausearch tool queries logs for events matching specified criteria. For example, to search auth logs by date:

# ausearch -m USER_LOGIN -ts today

Common ausearch filters:

• -ts TIME – Timestamp
• -m MESSAGE – Match string message
• -p PID – Process ID
• -ui USER_ID – User ID
• -k KEY – Keyword rule filter
• -sv SUCCESSVALUE – Match success/fail

iese filters allow slicing logs to extract only relevant events.

Here we filter sshd messages for failed logins in the past hour:

# ausearch -m sshd -sv no -ts recent

Understanding how to leverage ausearch filters takes time but allows precise hunting in verbose audit logs.

Generating Reports with aureport

While ausearch outputs raw event data, aureport creates summary reports for analysis.

Generating a login report:

# aureport -l
Login Report
===============================================
# date time auid host term exe arg success event
......
Feb 16 12:00:01 dev1 /usr/sbin/sshd 100000 sys_ptrace yes 7934
Feb 16 14:03:33 dev1 /usr/libexec/cockpit-ws 100000 unsuccesful yes 83294 

Other common reports:

• -a: Summary of auth events
• -k KEY: Report on custom rule filter
• -au: Activity by user

Reports provide high level overviews and statistics from audit logs. Examining aureport output often signals suspicious outliers warranting further ausearch investigation.

Auditing Containers and Virtual Machines

The primary auditd daemon runs in the host Linux OS. Audit rules apply to the whole system.

To audit containers and VMs, best practice is enabling auditing in the virtualization layer:

Docker – Use the Linux Audit Integration to collect events

KVM – QEMU guest agent for syscall auditing in VMs

This keeps the logs centralized while adding context like virtual instance ID. The hypervisor then feeds the logs into auditd.

Use Cases for Linux Auditing

Linux auditing provides value in several domains:

Security Monitoring – Auditd gives visibility into critical files and system calls indicative of compromise like lateral movement. Alerting on unexpected access attempts detects intruders.

Forensics – Detailed audit records speeds up incident investigations by providing an evidence trail of what happened in a breach.

Compliance – Standards like PCI DSS, HIPAA, and ISO 27001 often require auditing controls for periodic review.

Policy Enforcement – Audit rules combined with SELinux policies enforce permissions by logging and denying unauthorized access attempts.

Anomaly Detection – Profiling normal behavior allows detecting abnormal events through analytics on historical Linux audit data.

Troubleshooting – Auditing writes down a sequential record of system activity invaluable for diagnosing outages and performance issues.

Essentially any Linux server can benefit from expanded monitoring and logging using auditd.

Auditd Performance Considerations

The amount of auditing enabled involves tradeoffs around log volume and performance. Storing audit records is IO intensive – producing excess data impacts the filesystem and disks.

Volume – Too many verbose audit rules can overwhelm infrastructure. Start small.

Log Rotation – Configure log rotation in auditd.conf to compress/delete old logs.

Separate Partition – Use dedicated partition for audit logs to avoid filling up OS filesystem.

Also consider configuring plugins like audispd to offload logs to a centralized server live.

Evaluating average event rate and log size over time allows planning retention policies and storage required.

Advanced Auditd Configuration

The /etc/audit/auditd.conf file tunes audit daemon behavior using a series of options.

To ignore specific audit messages:

ignore_errors = daemons.deny

Adjust the maximum log rate:

max_log_rate = 10

Enable audispd plugin for centralized logging:

q_depth = 160
overflow_action = SYSLOG

There are over 20 other advanced directives. See man 5 auditd.conf for the full reference.

Auditing on Other Operating Systems

While Linux provides a built-in, advanced auditing framework in the kernel, other OSes take different approaches:

• Windows auditing relies on centralized Event Log data and policy rules
• macOS uses the OpenBSM subsystem inspired by Sun’s BSM
• Commercial UNIX vendors like Solaris and HP-UX include bundled audit daemons
• AWS CloudTrail provides instance logging through an API

These alternatives present their own pros and cons. For example, Windows has a familiar interface but is less flexible than Linux auditd rules. Those migrating from other platforms should understand the differences.

Following Auditd Best Practices

When developing an auditing program, adhere to guidelines like the CIS Benchmarks:

• Enable auditing of at least writes for /etc/passwd
• Log privilege escalation via su or sudo
• Alert on audit configuration changes
• Monitor unsuccessful resource access attempts
• Collect logs centrally with audispd

The Center for Internet Security releases Linux benchmark guidelines based on long-standing security expertise.

Gartner, SANS, and other leading institutions publish security best practice recommendations as well relevant to auditd.

The Importance of Auditing for Breach Detection

Per IBM and Ponemon Institute‘s 2020 Cost of a Data Breach Report, the average breach victim organization has 280 days of system compromise before detecting a breach. But organizations that had an auditing program discovered breaches faster:

Breaches detected through auditing or other logging averaged 207 days to identify – 73 fewer days. Early detection translates directly to lower data breach costs overall.