Linux with Operating System Concepts

Header
Meaning
r
Number of processes waiting for run time
b
Number of processes in uninterruptible sleep
swpd
Amount of virtual memory used (in KB)
free
Amount of free RAM (in KB)
buff
Amount of RAM used as buffers (in KB)
cache
Amount of RAM used to cache hard-disk data (in KB)
si
Average KB/second of data swapped into memory from disk
so
Average KB/second of data swapped out of memory to disk
bi
Average blocks/second of data swapped into memory from disk
bo
Average blocks/second of data swapped out from memory to disk
in
Number of interrupts/second
cs
Number of context switches/second
us
User time (nonkernel)
sy
System (kernel) time
id
Idle time
wa
Wait time
st
Cycle-stealing time

---

592
◾
Linux with Operating System Concepts
there is data movement between a disk drive and memory. If a disk transfer is taking place, 
the CPU may be forced to wait as the transfer will be slower than CPU access. Thus, cycles 
are stolen away from the CPU. Cycle stealing used to be more predominant but modern 
computers utilize multiple buses and multiple memory modules these days to reduce any 
such contention. As can be seen in this example, the value for cycle stealing (st) is 0.
The snapshot of the vmstat command shown previously indicates a system with a light 
load. There are few processes running, and a more than sufficient amount of main memory 
so that disk swapping is not needed at the moment. In this system, a majority of the CPU 
time is spent idling rather than either running the user processes or system processes.
The program free displays the free and used memory. In some ways, this is like the 
report provided by df (see Chapter 10) but the information provided is about memory 
rather than the file system. And like df, the information output by free is the current uti-
lization, unlike vmstat that reports on an average over a period of time.
In free’s output (see below), it specifies Mem (memory) and Swap as well as 
−
/
+
buffers/
cached. 
Mem
and 
Swap
refer to main memory and swap space, respectively. The line that 
contains 
−
/
+
buffers/cached
describes the amount of main memory that is allocated 
for a buffer in support of some application(s) or as disk cache.
The report specifies the total amount of memory, how much is free, and how much is 
used. You are also given, for memory, the amount that is being shared, the amount that is 
being buffered, and the amount that is cached. In the example output shown below, notice 
that swap space is only one-half a GB. This is unusual in that swap space (virtual memory) 
is usually at least the size of main memory. This particular Linux system was placed into a 
virtual machine of limited size so that swap space was similarly limited in size.
total 
used 
free 
shared buffers cached
Mem: 1020648 
907160 113488 0 
65436 
471488
−
/
+
buffers/cached: 370236 
650412
Swap: 524280 
8 
524272
14.4.5 I/O System-Monitoring Tools
There are a number of “stat”-type programs used to report on I/O performance. These are 
iostat to report on the file system, lpstat to report on the printer, and network programs 
such as ip, ss, netstat, nstat, rtacct, and nmap. We have explored the stat, df, and du (file 
system) commands in Chapter 10.
Both nstat and rtacct provide network interface statistics. These are useful if you want 
to examine the types of messages that have been received, for instance, 
IcmpInErrors
, 
IpOutRequests
, 
TcpActiveOpens
, 
UdpOutDatagrams
, and 
Ip6InDelivers
. 
The lnstat program provides network-routing cache statistics and additional network sta-
tistics from the Linux kernel, as stored in the directory /
proc/net/stat
. You are able to 
obtain information from the ARP cache, the RT cache, and the ndisk cache.
The nmap program is a network exploration tool. Its main function is to search for net-
work hosts and scan the ports that are available. It does this by sending out IP packets and 
examining the responses. By determining port access, it can report on the services that the 

---

Maintaining and Troubleshooting Linux
◾
593
given computer offers (e.g., ssh, http). Additionally, it can report on firewall activity, operat-
ing system type, and numerous other features of a given computer. With nmap, you can 
investigate a network’s security (or the individual computer’s network access security) as well 
as monitor network components. You can also use nmap to accumulate network statistics.
The only argument required for nmap is the address to be investigated. This can be an 
IP address or an IP alias of a single device, or a range of network addresses. For instance, 
nmap 10.11.12.5-205
would investigate all devices in the range from 10.11.12.5 through 
10.11.12.205. We can also specify a full subnet using 
nmap 10.11.12
. In response, for 
each device contacted, you will receive a report. The report will provide details of acces-
sible ports indicating their status (open, closed), the service implemented on that port, and 
the version (if available). For instance, the command 
nmap 10.11.12.13
might receive 
a report like the following:
Nmap scan report for 10.11.12.13
Host is up (0.000120s latency).
Not shown: 993 closed ports
PORT 
STATE SERVICE
22/tcp open ssh
25/tcp closed smtp
80/tcp open http
111/tcp open rpcbind
113/tcp closed auth
631/tcp open ipp
The nmap program has a number of options available. By using –A, you can obtain more 
detail on version types for the various services and the operating system. For instance, 
from the above report, we might instead see the following entries for ssh and http:
22/tcp 
open
ssh 
OpenSSH 5.3 (protocol 2.0)
| ssh-hostkey: 1024... (DSA)
| 2048... (RSA)
80/tcp 
open
http Apache httpd 2.2.4 ((CentOS))
The … in the above entries are specifications of keys, omitted for space.
Other options allow you to specify the scan type in place of the default scan using Ping. 
There are a wide variety of IP-based scans that can be used including TCP SYN, ACK, 
Maimon scans, UDP scans, and FTP bounce scans. Another group of options is available 
for obtaining timing and performance information. And yet other options are available for 
analyzing firewall protection and can help you analyze the security of the network. There 
is not enough space to cover nmap in detail here; so, you are encouraged to explore it on 
your own should you find yourself serving as a network administrator.
The iostat program gives you the flexibility to obtain three types of reports: CPU uti-
lization, device utilization, and network filesystem utilization. The CPU utilization pro-
vides the percentage of CPU time used for ordinary and niced user applications, system 

---

594
◾
Linux with Operating System Concepts
operations, cycle-stealing time, I/O wait time, and idle time. The device report lists each 
type of connected block device. This report displays the average number of transfers per 
second, blocks read per second, blocks written per second, and total number of blocks 
read and written.
If there are any mounted file systems (including USB- or CD-ROM-mounted devices), 
iostat also provides a report for each of these indicating the filesystem name, number of 
blocks read and written per second, and total number of blocks read and written. You 
can obtain more detailed information on any one device by using 
-x