Running syslog in a container

Modern applications utilize logging with a whole set of different interfaces and backends like logstash / elasticsearch or similar applications running on cloud environments like azure monitor. Many linux based applications are built with OS logging facilities in mind - especially syslog. And syslog is doing really well here.

Propably most sysadmins that are renewing components nowadays will use some approach of containers to achieve better maintainability and deterministic deployments. On this path you might come across the requirement to provide syslog to an application that is running in a container.

Depending on your environment you are either building a custom image or using a pre-built like from docker hub or some internal registry. This guide will cover integrating rsyslog in your own image.

Installing rsyslog

When building the container you just need to add rsyslog using the packet manager of your current distribution.

On debian based images (debian/ubuntu) this can be done:

RUN apt-get update && apt-get install -y --no-install-recommends rsyslog && rm -rf /var/lib/apt/lists/*

This will install rsyslog and clean apt installation caches afterwards.

Container startup

If you're container is currently running a single application you'll need to modify this. Create a file called run.sh and modify your Dockerfile:

# copy startup script
COPY run.sh /run.sh

# startup command
CMD [ "/run.sh" ]

Apply chmod +x to run.sh to the file to enable execution (either in Dockerfile or source).

The script looks as follows:

#!/bin/bash

# start rsyslog
echo 'Starting rsyslog'
# remove lingering pid file
rm -f /run/rsyslogd.pid
# start service
service rsyslog start

# start rspamd
echo 'Starting xxx'
/usr/bin/xxx

Replace /usr/bin/xxx with the application & arguments your container should run actually.

This will trigger startup of rsyslog daemon on container start. If the container has been stopped but not removed, there might be a remaining pid file that prevents rsyslogd from starting. Therefore we're removing the file on each startup manually.

The rsyslog configuration may be mapped using a container volume to provide desired runtime configuration for the daemon.

This is everything that is required to get rsyslog p and running.

One more thing: Activation of imklog fails

If you're inspecting your container logs you might see the following message:

rsyslogd: imklog: cannot open kernel log (/proc/kmsg): Operation not permitted. rsyslogd: activation of module imklog failed

This means that the imklog (kernel log) module cannot be activated. Running workloads in a container focuses on application logs and doesn't bother with kernel logging (which is job of the container host).

To disable this warning, just disable the imklog module of rsyslog.

# disably imklog in container
#module(load="imklog")

If you really need to access kernel logs you might run container in privileged mode and then load the module. If you're running regular application workloads there's no valid use case when this is required, so my recommendation is to disable the kernel log module.

Happy logging!