--- title: "System activity reporting with sar" source: "https://docs.nexcess.com/hosting/performance/server-optimization/system-activity-reporting-with-sar/" description: "sar monitors Linux system activity (CPU, memory, I/O) over time. It's crucial for historical performance analysis and troubleshooting server bottlenecks." vertical: "Hosting" date: "2025-05-28" last_modified: "2026-07-30" --- # System activity reporting with sar sar (System Activity Report) It is a powerful command-line utility for collecting, reporting, and saving system activity information. Installed by default on many Linux distributions like CentOS, it’s particularly useful for tracking down the root cause of performance issues, especially those related to disk I/O, CPU, and memory. sar maintains its historical data in a series of files located in the `/var/log/sa/` directory. ##### Viewing historical data `sar` automatically collects data at regular intervals and stores it in daily files. To see a list of the previous month’s reports, you can use the `ls` command: ``` ll /var/log/sa/ ``` **Example Output:** ``` -rw-r--r-- 1 root root 484640 Aug 1 15:51 sa01 -rw-r--r-- 1 root root 484176 Aug 2 23:50 sa02 -rw-r--r-- 1 root root 228576 Aug 3 11:00 sa03 ... -rw-r--r-- 1 root root 491873 Aug 1 23:53 sar01 -rw-r--r-- 1 root root 491907 Aug 2 23:53 sar02 ``` You’ll notice files named `saXX` (binary data files) and `sarXX` (text reports, if generated). `XX` represents the day of the month. ##### Basic usage: CPU activity Running `sar` without any flags will show you a breakdown of CPU usage from the current day. This output is similar to what you might see from `iostat`‘s CPU section. **Command:** ``` sar ``` **Example Output:** ``` 12:30:01 PM CPU %user %nice %system %iowait %steal %idle 12:40:01 PM all 0.13 0.01 0.08 0.02 0.00 99.76 12:50:01 PM all 0.13 0.02 0.07 0.02 0.00 99.76 01:00:01 PM all 0.12 0.01 0.08 0.03 0.00 99.77 01:10:01 PM all 0.22 0.01 0.09 0.05 0.00 99.63 Average: all 0.15 0.01 0.08 0.03 0.00 99.73 ``` This basic report is useful for quickly assessing overall CPU load and identifying if `iowait` (time spent waiting for I/O) is a significant factor. ##### Specific metrics with flags sar offers numerous flags to retrieve specific types of system activity: ##### CPU load averages (-q) The `-q` flag displays the CPU load averages over time, similar to the output from `uptime` or `top`. **Command:** ``` sar -q ``` **Example Output:** ``` 12:00:05 AM runq-sz plist-sz ldavg-1 ldavg-5 ldavg-15 12:10:04 AM 0 385 5.12 7.26 7.90 12:20:15 AM 1 402 13.63 9.29 8.08 Average: 1 362 8.03 9.36 8.60 ``` - `runq-sz`: Run queue size (number of tasks waiting for CPU). - `plist-sz`: Number of tasks in the task list. - `ldavg-1`, `ldavg-5`, `ldavg-15`: Load average over the last 1, 5, and 15 minutes. ##### Memory and swap usage (-r) Use the `-r` flag to see historical memory and swap usage, both in percentages and kilobytes. This helps in diagnosing RAM-related bottlenecks. **Command:** ``` sar -r ``` **Example Output:** ``` 12:30:01 PM kbmemfree kbmemused %memused kbbuffers kbcached kbswpfree kbswpused %swpused kbswpcad 12:40:01 PM 70076 1687976 96.01 191540 984628 1959852 68 0.00 0 Average: 69309 1688743 96.06 191548 984871 1959852 68 0.00 0 ``` - `kbmemfree`: Amount of free memory available. - `kbmemused`: Amount of used memory. - `%memused`: Percentage of used memory. - `kbbuffers`: Memory used by kernel buffers. - `kbcached`: Memory used by the page cache. - `kbswpfree`, `kbswpused`, `%swpused`: Free, used, and percentage of used swap space. - `kbswpcad`: Amount of cached swap memory. ##### Swap space utilization (-s) The `-S` flag provides detailed information about swap space utilization. **Command:** ``` sar -S ``` **Example Output:** ``` 12:00:02 AM kbswpfree kbswpused %swpused kbswcad %swpcad 12:10:01 AM 1884668 212480 10.13 3200 1.51 Average: 188468 212480 10.13 3200 1.51 ``` ##### Swap activity (-w) To see actual page-in and page-out activity to and from swap space, use the `-W` flag. This helps identify if your system is actively swapping, which can indicate memory pressure. **Command:** ``` sar -W ``` **Example Output:** ``` 12:30:01 PM pswpin/s pswpout/s 12:40:01 PM 0.00 0.00 Average: 0.00 0.00 ``` - `pswpin/s`: Number of pages swapped in per second. - `pswpout/s`: Number of pages swapped out per second. ##### Time range and processor specifics `sar` also allows you to specify time ranges and monitor individual processors: ##### Specifying a time range (-s and -e ) The `-s` (start time) and `-e` (end time) flags can be used to specify a particular time range for your `sar` output. Both flags follow the format `HH:MM:SS`. **Command:** ``` sar -s 09:00:00 -e 10:30:00 ``` ##### Per-processor usage (-P) The `-P` flag allows you to show CPU usage for a specific processor, or all processors using `-P ALL`. You can also specify multiple processor IDs separated by commas (e.g., `-P 0,1,2`). **Command (All Processors):** ``` sar -P ALL ``` **Example Output (`-P ALL`):** ``` 12:00:05 AM CPU %user %nice %system %iowait %steal %idle 12:10:04 AM all 18.88 0.70 5.27 75.15 0.00 0.00 12:10:04 AM 0 18.88 0.70 5.27 75.15 0.00 0.00 ``` **Command (Specific Processors):** ``` sar -P 0,1,2 ``` **Example Output (`-P 0,1,2`):** ``` 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:10:01 AM 0 3.63 0.00 0.29 0.05 0.00 96.03 12:10:01 AM 1 4.76 0.00 0.33 0.04 0.00 94.88 12:10:01 AM 2 0.97 0.00 0.07 0.00 0.00 98.96 ``` ##### Checking logs for a different day (-f) While `sar` by default shows the current day’s data, you’ll often need to examine older logs. You can do this using the `-f` flag, specifying the path to the daily data file (`saXX` from `/var/log/sa/`). **Command:** ``` sar -f /var/log/sa/sa01 ``` You can combine the `-f` flag with other `sar` flags (like `-W` for swap activity or `-r` for memory) to get specific information from historical data: **Command:** ``` sar -f /var/log/sa/sa01 -W ``` ##### Helpful tip: Daily averages one-liner Here’s a useful one-liner function to quickly see the `sar` information on a per-day average for the recorded month. Copy and paste this into your terminal: ``` testfunction () { sar -$1 |head -3;ls /var/log/sa/sa[0-9]*|xargs -I '{}' sar -$1 -f {}| grep -E -o '(../../....|Average:.*)'; } ``` Then, run the function with the desired `sar` flag (e.g., `W` for swap info or `q` for load average): ``` testfunction W ``` Or for load average: ``` testfunction q ``` ### Conclusion `sar` is an incredibly versatile and powerful tool for in-depth system performance monitoring and historical analysis. By understanding its various flags and how to interpret its output, you can effectively track CPU, memory, and disk I/O trends, allowing you to proactively identify and resolve performance bottlenecks on your server. For any advanced diagnostics or assistance, Liquid Web’s Heroic Support℠ team is always available.