[lug] Ubuntu 14.04 LTS syslog/rsyslog Stopped Logging kern.log

stimits at comcast.net stimits at comcast.net
Mon Oct 27 17:27:11 MDT 2014

Sadly my email program is limited in reply formats, or I'd "interlace" my comments.
Here is the /var/log/apt/history file relevant section (no other updates are ever mentioned on any kind of logging facility, so this seems to be the one and only logging related update):
Start-Date: 2014-10-11  22:15:15Commandline: apt-get upgradeUpgrade: apt:armhf (1.0.1ubuntu2.4.1, 1.0.1ubuntu2.5), rsyslog:armhf (7.4.4-1ubuntu2.1, 7.4.4-1ubuntu2.3), libapt-pkg4.12:armhf (1.0.1ubuntu2.4.1, 1.0.1ubuntu2.5), bash:armhf (4.3-7ubuntu1.4, 4.3-7ubuntu1.5), apt-utils:armhf (1.0.1ubuntu2.4.1, 1.0.1ubuntu2.5), libapt-inst1.5:armhf (1.0.1ubuntu2.4.1, 1.0.1ubuntu2.5), linux-libc-dev:armhf (3.13.0-36.63, 3.13.0-37.64)End-Date: 2014-10-11  22:15:42
As you can see it uses rsyslog and the system is ARMv7 (quad core ARM15) running Ubuntu LTS 14.04 (this is a full distribution, not cut down).
The process does log to other files for other events, but I'm not sure which ones are controlled through rsyslogd. I have rebooted many times with no change. Running ps shows rsyslogd is indeed running. However, there is a pattern, the following in /var/log/ are 0 size:
Of those size 0 files, kern.log and syslog are listed in /etc/rsyslogd.d/default.conf as this:
*.*;auth,authpriv.none          -/var/log/syslog
kern.*                          -/var/log/kern.log
In /dev the "log" file does exist:
srw-rw-rw- 1 root root 0 Feb  1  2000 log=
I have a very old version of /etc/rsyslogd.conf from last spring, the only diff between this and the active conf is that currently this is added in:
# Enable non-kernel facility klog messages$KLogPermitNonKernelFacility on
The /etc/rsyslogd.d/50-default.conf is unchanged since last spring. 
The in-memory kernel buffer must be how dmesg command is getting kernel messages, as dmesg file and all other log files are missing kernel module info messages in any form.
I'm starting to wonder if someone maintaining Ubuntu LTS decided this ARMv7 flavor needed to save disk space, but I'm having troubles finding out where such a configuration change would be. Perhaps rsyslogd itself has been changed or broken. Always getting messages piped and grepped from dmesg is a pain...can anyone think of any other "smoking guns" to determine if rsyslogd is broken versus package update intentionally causing the change in behavior? Is it time to report a bug?
> On 10/26/2014 05:37 PM, stimits at comcast.net wrote:>> Hi,>> >> In the past few days I did an apt-get upgrade on an Ubuntu 14.04 LTS system which I'm trying to work on kernel modules for. Logging of all kernel messages suddenly ceased, >and kern.log is now empty. If I use a printk with KERN_INFO, I can still get the output via dmesg, but am scratching my head as to where the actual log file is...I've grepped and >gunzip'd and done all I can in /var/log/, and never figured out where the KERN_INFO actually goes. tail -f can no longer be used, as not even the /var/log/dmesg file contains >these messages...somehow the dmesg binary command magically pulls KERN_INFO out of the air even when its specific log file does not have this information.
>I suppose you have checked to see if the *syslog* dameon is running. Did something like>systemd get pulled in that wasn't there before? A reboot might also give you more>information. Does the /dev/log socket exist and is at least world readable?
>It is strange that a find /etc -iname '*syslog*' wouldn't find anything.>There are other loggers, e.g. metalog. Maybe the logging system was changed>or deselected during the upgrade -- you don't actually have to have a logging>service/daemon running for a system to run.
>IIRC, the kernel has an in-memory message buffer (the size is set in the kernel config>somewhere). I suspect dmesg could read from that when the /var/log/dmesg data file>doesn't exist.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lug.boulder.co.us/pipermail/lug/attachments/20141027/5908bef8/attachment.html>

More information about the LUG mailing list