Lesson 3	Kernel messages
Objective	Display Recent and Old Kernel Messages

Display Recent and Old Kernel Messages

During the Linux boot process, the standard system logging facilities are not available, so kernel messages can only be written to the console^[1] and stored in memory (thekernel ring buffer)^[2] for subsequent writing to disk.
Once the standard system logging facilities^[3] start, they take over management of kernel messages and write the messages the kernel created during boot-up to /var/log/dmesg. From this point on, you can read the kernel messages from the standard system logfile /var/log/messages by looking for the keyword kernel.

The dmesg command

Even with the standard system logging facilities running, the kernel still uses its ring buffer to log system messages. If you want to view recent kernel messages without going to the standard logfiles, use the dmesg command.

Displaying kernel Messages

View the table below to see the various methods to show recent and old kernel messages, the logfiles the command reads from (if any), and a description.
View the table below to see the last 10 messages created by a typical Linux kernel.

Kernel Messages

Command	Logfile	Description
`less /var/log/dmesg`	`/var/log/dmesg`	Displays the kernel's boot-up messages.
`grep kernel /var/log/messages \| less`	`/var/log/messages`	Displays all the kernel's run-time messages.
`dmesg \| less`		Displays the contents of the kernel's ring buffer. Because the kernel holds the ring buffer in its memory, `dmesg` doesn't need to query a logfile.

For additional information, check the man pages for klogd and dmesg.

Kernel Debugging Techniques

Debugging the kernel can be achieved using very simple and straight forward techniques and some time, patience and perseverance. This page describes techniques to help debug the kernel.

Using printk

The simplest, and probably most effective way to debug the kernel is via printk().
This enables one to print messages to the console, and it very similar to printf(). Note that printk() can slow down the execution of code which can alter the way code runs, for example, changing the way race conditions occur.

Changing the ring buffer size

The internal kernel console message buffer can sometimes be too small to capture all of the printk messages, especially when debug code generates a lot of printk messages. If the buffer fills up, it wraps around and one can lose valueable debug messages. To increase the internal buffer, use the kernel boot parameter:

log_buf_len=N

where N is the size of the buffer in bytes, and must be a power of 2.

Changing debug levels

One can specify the type of printk() log level by pre-pending the 1st printk() argument with one of the following:

KERN_EMERG    /* system is unusable                   */
KERN_ALERT    /* action must be taken immediately     */
KERN_CRIT     /* critical conditions                  */
KERN_ERR      /* error conditions                     */
KERN_WARNING  /* warning conditions                   */
KERN_NOTICE   /* normal but significant condition     */
KERN_INFO     /* informational                        */
KERN_DEBUG    /* debug-level messages                 */

for example, printk(KERN_DEBUG "example debug message\n");

Serial Console

Serial console enables one to dump out console messages over a serial cable. Most modern PCs do not have legacy serial ports, so instead, one can use a USB serial dongle instead. A "null serial cable" or "universal file transfer cable" is needed to connect the target computer with the host. Most commonly this will be a DB9 female to DB9 female null serial cable. In addition, one needs to enable USB serial support as a kernel build configuration:

CONFIG_USB_SERIAL_CONSOLE=y
CONFIG_USB_SERIAL=y

and enable the appropriate driver, e.g.:

CONFIG_USB_SERIAL_PL2303=y

and boot this kernel with

console=ttyUSB0,9600n8

one may need to adjust the baud rate appropriately. Note: Generally, there is NO hardware or software flow control on serial console drivers, which means one may get dropped characters when running very high speed tty baud rates, such as 115200 baud.

The next lesson explains how to examine a system log.

[1] Console: The console is the terminal display.

[2] Kernel ring buffer: The kernel ring buffer is an area of kernel memory with fixed size that holds the kernel's log message. As the kernel logs messages, older logs are overwritten.

[3] Facility: Facilities are simply programs that can be configured to send notices to the system log. Common facilities include user, kern, mail, daemon, auth, lpr, news, uucp, and cron.