sar / sysstat (Performance Recording and Analysis)
'sar' is a performance recording and analysis tool included in the sysstat package. It collects system statistics — such as CPU, memory, I/O, and network usage — in time-series format and can read past data from saved files. This makes it useful for post-incident analysis, such as investigating high load that occurred overnight or a server failure noticed the next morning. Periodic collection is automated via a systemd timer or cron, and collected data is stored under /var/log/sa/ organized by date.
Syntax
# -----------------------------------------------
# Basic syntax of sar
# -----------------------------------------------
# sar [options] [interval [count]]
# → Omitting interval and count displays statistics from the default collection file
# Example: sar -u 1 5
# → Displays CPU usage in real time, once per second, 5 times
# sar [options] -f {file_path}
# → Reads past data from a saved collection file (/var/log/sa/saDD)
# Example: sar -u -f /var/log/sa/sa20
# sar [options] -s {start_time} -e {end_time}
# → Displays statistics within a specified time range (use with -f)
# Example: sar -u -f /var/log/sa/sa20 -s 14:00:00 -e 16:00:00
# -----------------------------------------------
# CPU statistics
# -----------------------------------------------
# sar -u
# → Displays overall CPU usage (%user / %system / %idle, etc.)
# Example: sar -u 1 10
# sar -u ALL
# → Displays detailed CPU statistics including guest CPU usage
# sar -P {cpu_number|ALL}
# → Displays statistics per CPU core or for all cores
# Example: sar -P ALL 1 3
# -----------------------------------------------
# Memory statistics
# -----------------------------------------------
# sar -r
# → Displays memory usage (kbmemfree / kbmemused / %memused / kbbuffers / kbcached)
# Example: sar -r 1 5
# sar -r ALL
# → Displays detailed memory statistics including slab cache
# sar -W
# → Displays swap usage (kbswpfree / kbswpused / %swpused)
# -----------------------------------------------
# I/O statistics
# -----------------------------------------------
# sar -b
# → Displays block I/O transfer rates (tps / rtps / wtps / bread/s / bwrtn/s)
# sar -d
# → Displays per-device I/O statistics (tps / %util / await, etc.)
# Example: sar -d -p 1 5 (-p displays device names in a human-readable format)
# -----------------------------------------------
# Network statistics
# -----------------------------------------------
# sar -n DEV
# → Displays packet counts and byte counts per network interface
# Example: sar -n DEV 1 5
# sar -n EDEV
# → Displays network error statistics (rxerr/s / txerr/s / rxdrop/s, etc.)
# sar -n SOCK
# → Displays socket usage counts (TCP / UDP / RAW / fragments)
# sar -n TCP
# → Displays TCP connection statistics (active/s / passive/s / iseg/s / oseg/s)
# -----------------------------------------------
# Load average and task statistics
# -----------------------------------------------
# sar -q
# → Displays load averages (runq-sz / plist-sz / ldavg-1 / ldavg-5 / ldavg-15)
# sar -A
# → Displays all available statistics (output is large; use with filtering options)
Option Reference
| Option | Description |
|---|---|
-u | Displays overall CPU usage (%user, %system, %iowait, %idle, etc.). |
-u ALL | Displays detailed CPU statistics including guest CPU usage. |
-P {number|ALL} | Displays CPU usage per specified core. Use ALL to list all cores. |
-r | Displays memory usage (free, used, buffers, cache, etc.). |
-r ALL | Displays detailed memory statistics including slab cache. |
-W | Displays swap space usage and utilization percentage. |
-b | Displays block I/O transfer rates (tps, read/write rates, etc.). |
-d | Displays per-device I/O statistics (tps, %util, await, etc.). Use -p for human-readable device names. |
-n DEV | Displays packet counts and byte counts per network interface. |
-n EDEV | Displays network errors (receive errors, transmit errors, drops, etc.). |
-n SOCK | Displays socket usage counts (TCP, UDP, RAW, fragments). |
-n TCP | Displays TCP connection statistics (active/passive connections, segment counts, etc.). |
-q | Displays load averages (1-min, 5-min, 15-min) and the run queue length. |
-A | Displays all available statistics. |
-f {file} | Reads past data from a saved collection file (/var/log/sa/saDD). |
-s {HH:MM:SS} | Specifies the start time for the displayed statistics. Use with -f. |
-e {HH:MM:SS} | Specifies the end time for the displayed statistics. Use with -f. |
-o {file} | Saves real-time collection data to a file in binary format. |
-h | Displays values in human-readable units (K, M, G). |
Examples
Reviewing past CPU usage logs (post-incident analysis)
# ----------------------------------------------- # Investigating the cause of high load from yesterday on the server # ----------------------------------------------- # Logs are stored under /var/log/sa/ organized by date # The numeric part of the filename is the day of the month (DD) ls /var/log/sa/ # Review CPU usage for the 25th (sa25) in time-series order sar -u -f /var/log/sa/sa25 # Narrow down to 14:00-16:00 where high load is suspected sar -u -f /var/log/sa/sa25 -s 14:00:00 -e 16:00:00
Run the following command:
$ sar -u -f /var/log/sa/sa25 -s 14:00:00 -e 16:00:00 Linux 5.14.0-284.el9.x86_64 (nerv-core01) 03/25/2026 _x86_64_ (4 CPU) 14:00:01 AM CPU %user %system %iowait %steal %idle 14:10:01 AM all 8.23 3.41 0.52 0.00 87.84 14:20:01 AM all 12.67 4.89 0.71 0.00 81.73 14:30:01 AM all 43.15 18.22 1.03 0.00 37.60 14:40:01 AM all 91.84 31.57 2.14 0.00 -25.55 14:50:01 AM all 89.42 29.38 1.98 0.00 -0.78 15:00:01 AM all 15.33 5.61 0.63 0.00 78.43 15:10:01 AM all 7.88 3.12 0.47 0.00 88.53 Average: all 38.36 13.74 1.07 0.00 46.83
The following example demonstrates this:
# Check per-core CPU usage to see if load is concentrated on a specific core sar -P ALL -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00
Run the following command:
$ sar -P ALL -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00 Linux 5.14.0-284.el9.x86_64 (nerv-core01) 03/25/2026 _x86_64_ (4 CPU) 14:30:01 AM CPU %user %system %iowait %steal %idle 14:30:01 AM 0 99.00 98.00 0.00 0.00 -97.00 14:30:01 AM 1 2.13 1.04 0.00 0.00 96.83 14:30:01 AM 2 1.94 0.88 0.00 0.00 97.18 14:30:01 AM 3 2.07 0.97 0.00 0.00 96.96 Average: 0 94.21 89.12 0.00 0.00 -83.33 Average: 1 2.04 0.99 0.00 0.00 96.97
Reviewing memory usage over time
# ----------------------------------------------- # Investigating a memory growth trend on the server # ----------------------------------------------- # Review memory statistics for the 25th # kbmemfree: free memory (KB) / kbmemused: used memory (KB) # %memused: usage percentage / kbbuffers: buffers / kbcached: page cache sar -r -f /var/log/sa/sa25 # Also check swap usage sar -W -f /var/log/sa/sa25
Running these commands produces the following output:
$ sar -r -f /var/log/sa/sa25 Linux 5.14.0-284.el9.x86_64 (rei-unit00) 03/25/2026 _x86_64_ (4 CPU) 00:00:01 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit 02:00:01 AM 3201448 4987256 60.92 124388 2341204 6782340 41.23 04:00:01 AM 2987312 5201392 63.53 135742 2489312 6998124 42.56 06:00:01 AM 2341208 5847496 71.38 148912 2654108 7412680 45.09 08:00:01 AM 1452304 6736400 82.25 158224 2701440 7998240 48.64 10:00:01 AM 512440 7676264 93.74 162104 2723888 9112320 55.42 10:10:01 AM 87312 8101392 98.93 163440 2731200 9512040 57.86 10:20:01 AM 12480 8176224 99.85 163920 2734112 9601240 58.40 Average: 1824929 7365788 89.89 151819 2620783 8174200 49.64 $ sar -W -f /var/log/sa/sa25 10:00:01 AM pswpin/s pswpout/s 10:10:01 AM 45.32 102.87 10:20:01 AM 312.44 589.21
Reviewing network statistics
# ----------------------------------------------- # Checking network load on the server # ----------------------------------------------- # Review send/receive statistics per network interface # rxpck/s: received packets/sec / txpck/s: transmitted packets/sec # rxkB/s: received bytes/sec / txkB/s: transmitted bytes/sec sar -n DEV -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00 # Check whether any network errors have occurred sar -n EDEV -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00
Run the following command:
$ sar -n DEV -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00 Linux 5.14.0-284.el9.x86_64 (asuka-unit02) 03/25/2026 _x86_64_ (4 CPU) 14:30:01 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 14:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 14:30:01 AM eth0 4521.34 3892.12 5842.31 4124.67 0.00 0.00 0.00 14:40:01 AM eth0 4389.22 3741.88 5672.44 3988.12 0.00 0.00 0.00 14:50:01 AM eth0 4612.78 3934.45 5924.89 4201.34 0.00 0.00 0.00 Average: eth0 4507.78 3856.15 5813.21 4104.71 0.00 0.00 0.00 $ sar -n EDEV -f /var/log/sa/sa25 -s 14:30:00 -e 15:00:00 14:30:01 AM IFACE rxerr/s txerr/s coll/s rxdrop/s txdrop/s 14:30:01 AM eth0 0.00 0.00 0.00 0.00 0.00 Average: eth0 0.00 0.00 0.00 0.00 0.00
Real-time monitoring and load average
# ----------------------------------------------- # Real-time monitoring on the server # ----------------------------------------------- # Display CPU usage in real time: 1-second interval, 10 times sar -u 1 10 # Display load average: 2-second interval, 5 times # runq-sz: number of processes waiting to run / plist-sz: total processes and threads # ldavg-1: 1-min average / ldavg-5: 5-min average / ldavg-15: 15-min average sar -q 2 5 # Display and save collection data to a file at the same time sar -u -o /tmp/kaji_monitor_$(date +%Y%m%d).dat 5 12
Run the following command:
$ sar -q 2 5 Linux 5.14.0-284.el9.x86_64 (misato-nerv) 03/25/2026 _x86_64_ (4 CPU) 03:45:12 PM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 blocked 03:45:14 PM 1 412 0.24 0.31 0.28 0 03:45:16 PM 2 413 0.24 0.31 0.28 0 03:45:18 PM 1 413 0.24 0.31 0.28 0 03:45:20 PM 0 413 0.22 0.30 0.28 0 03:45:22 PM 1 413 0.22 0.30 0.28 0 Average: 1 413 0.23 0.31 0.28 0
Enabling sysstat and configuring data collection
# ----------------------------------------------- # Install the sysstat package and enable automatic collection # ----------------------------------------------- # RHEL / AlmaLinux / Rocky Linux sudo dnf install sysstat # Debian / Ubuntu sudo apt install sysstat # Start the sysstat service and enable it to start on boot # The systemd timer (sysstat.service + sysstat-collect.timer) # collects data every 10 minutes by default sudo systemctl enable --now sysstat # Check collection status sudo systemctl status sysstat # List collected files (DD is the day of the month) ls -lh /var/log/sa/
Run the following command:
$ ls -lh /var/log/sa/ total 8.2M -rw-r--r-- 1 root root 412K Mar 23 23:50 sa23 -rw-r--r-- 1 root root 412K Mar 24 23:50 sa24 -rw-r--r-- 1 root root 209K Mar 25 15:50 sa25 -rw-r--r-- 1 root root 412K Mar 23 23:50 sar23 -rw-r--r-- 1 root root 412K Mar 24 23:50 sar24
Overview
'sar' (System Activity Reporter) is a performance recording tool provided by the sysstat package. Its standout feature is the ability to periodically collect a wide range of system statistics — including CPU, memory, I/O, and network usage — and read past data from files in time-series order. Enabling the sysstat service with systemctl causes data to accumulate in /var/log/sa/saDD every 10 minutes by default. For post-incident analysis, you can efficiently narrow down statistics to the time of the problem by specifying a file with -f and filtering the time range with -s/-e. For real-time monitoring, specify an interval and count when running the command. When you want to interactively monitor instantaneous values, combining sar with journalctl to cross-reference logs is even more effective. For long-term trend analysis, it is recommended to save collected data with -o so you can read it back later with -f.
If you find any errors or copyright issues, please contact us.
シェアする