The Solaris vmstat command is very handy in case you need statistics on kernel thread, virtual memory, disk, trap and CPU activity.

If your Solaris box has several processors (multi-processor unit), the vmstat averages the number of processors into the output. If you need per-processor statistics, you should use mpstat command instead.

The vmstat command can also display statistics on swapping, cache flushing and interupts.


$ vmstat 3
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd -- -- -- in sy cs us sy id
0 0 0 4188292 329716 194 1653 0 0 0 0 4 3 0 0 0 410 2186 530 2 6 92
0 0 0 4043704 182476 9 22 0 0 0 0 0 24 0 0 0 437 425 259 1 4 96
0 0 0 4043704 182348 0 0 0 0 0 0 0 0 0 0 0 385 412 216 0 4 96
0 0 0 4043704 182344 0 0 0 0 0 0 0 43 0 0 0 481 406 290 0 4 96
0 0 0 4043704 182344 0 0 0 0 0 0 0 0 0 0 0 389 425 217 1 3 96

The previous example showed basic set of statistics that were gathered in three second interval. Let’s take a closer look on each of the columns in the output.

kthr (kernel thread state change)

  • r – The number of kernel threads in the dispatch queue
  • b – The number of blocked kernel threads that are waiting for resources
  • w- The number of swapped out processes that are waiting for processing resources to finish

If r divided by the number of CPUs is greater than one we have possible CPU bottleneck and should consider to add some more CPUs to the server or do load balancing. The load balancing can be done either by rescheduling tasks to execute during off-peak hours or adjust priorities (nice values) of existing tasks.

In other words, whenever the value in r column exceeds the number of CPUs on the server, tasks are forced to wait for execution.

Hight number in b column indicates waiting for I/O (disk, network, user). Most likely it would be slow disk.

memory

  • swap – Amount of swap space currently available (Kbytes)
  • free – size of the free list (Kbytes)

page

  • re – Pages reclaimed from the free list.
  • mf – Minor faults (page in memory but not mapped)
  • pi – Kbytes paged in
  • po – Kbytes paged out
  • fr – Kbytes freed
  • de – Anticipated memory that is needed by recently swapped-in processes
  • sr – Scan rate. It is a total number of pages scanned.

Any nonzero number in pi column indicates shortage of RAM memory. It shows that server exceeded the available RAM storage and RAM memory content is read in from the swap disk.

disk faults

  • in – Interrupts per second
  • sy – System calls per second
  • cs – CPU context switch rate

cpu

  • us – User time (%). It is percentage of cpu used for running non-kernel code.
  • sy – System time (%) It is percentage of cpu used for running kernel code.
  • id – Idle time (%)

If us and sy together is often more than 80%, then CPU is approaching its limits.

The sy value should not be higher than us, because it means the system is actually spending less time on real work.