Article copied from : http://www.softpanorama.org/Utilities/vmstat.shtml (because there is a bullshit of Ad from Google that avoid to read some chapter)
The first tool to use for locating performance bottlenecks is the vmstat command. It provides compact table-based snapshot of various system resources utilization. vmstat does not count itself as a running process. The following metrics might help to determine the overloaded system are
- r: The number of processes in the run queue waiting for CPU. Value usually should not exceed number of processors (or cores). Double digit reading often signals that there is a bottleneck somewhere in the system. In a response sensitive servers (Web servers, terminal servers) this parameter should be close to zero. In a nutshell, non-zero values represent the size of CPU queue — tasks waiting on busy CPU(s) or tasks waiting because of data-serialization issues. In you encounter number higher then the number of CPUs on the system you should probably have a look on the system and see what process is taking most of the cpu (use the top command).
- b: The number of processes/threads sleeping (blocked processes usually waiting for IO). A sleeping process is created when a program makes a request that can’t be immediately fulfilled. High number might suggest I/O bottleneck.
- so: Amount of memory swapped to disk (/s). Most production systems should not swap memory to disk during normal operation. Non-zero value usually means that the amount of physical memory is insufficient for optimal operations of the system with a given load.
- wa: Time spent waiting for IO. Double digits value suggests I/O bottleneck.
For analysis of long intervals output from sar might be a good alternative. In both cases one can pipe output into Perl script to produce alerts. For example if a single cpu dual core Linux 2.6 server demonstrated strange behavior with uptime averages above 12 (case that happened to the author). In this case to lessen the amount of collected information you can use threshold 4 for « r »; exceeding this threshold triggers writing vmstat or sar information to log and/or and piping it into Perl script for detailed analysis.
If the vmstat command is used without any options, it reports the current set of statistics and exits. If a single numeric argument T is supplied it is assumed to be an interval (in seconds) between iterations of the report and vmstat runs in a loop outputting the statistics each T seconds. Optionally the second numerical parameter can be specified which provides the number of iterations of the time loop. For example vmstat 2 10 means output statistics each 2 sec and make 10 iterations before existing.
vmstat reports virtual memory statistics regarding kernel thread, virtual memory, disk, trap, and CPU activity. Traditionally those parts of the table have headers
- procs
- memory (self-explainable columns swpd, free, buff and cache):
- swpd Virtual memory used, in kilobytes.
- free Idle memory, in kilobytes.
- buff Memory used as buffers, in kilobytes.
- cache Cache memory, in kilobytes.
- inactive Inactive memory, in kilobytes, displayed with -a.
- active Active memory, in kilobytes; displayed with -a.
- swap (columns si and so — Memory swapped in/out from disk each second, in kilobytes.)
- io
- system
- cpu
On MP (multi-processor) systems, vmstat averages the number of CPUs into the output. For per-processor statistics, see mpstat(1M). vmstat only supports statistics for certain devices. For more general system statistics, you can also use e sar and iostat. command.
Without options, vmstat displays a one-line summary of the virtual memory activity since the system was booted.
