1. Introduction
Event logging is a central source of information for IT operations and security. The syslog-ng application collects logs from
many different sources, processes and filters them in real-time, and finally it stores the logs or forwards them for further
analysis. The same feature set comes in handy also in a containerized environment.
Containerization, and Docker in particular, changed the way we distribute and run applications. Containers provide isolated
environments, which make it possible to run applications with conflicting dependencies on the same host. There are even
dedicated container hosts, like Atomic Host, which do not allow you to install any applications on the host directly, only in
containers. You can install your central syslog-ng server in a container and enjoy all benefits of containerization.
You can use syslog-ng also for collecting Docker logs. Docker already provides many drivers for logging, even for central log
collection. On the other hand, remote logging drivers arrive with a minimalist feature set and you are not able to use the
docker logs command anymore. To have the best of both worlds, you can use the journald logging driver in Docker, and use
syslog-ng to read Docker logs from journald and forward log messages to your central log server or other destinations.
There are many types of software that log to files or pipes instead of their stdout, the place where Docker expects them.
Fortunately, by using Docker volumes, you can share data among containers and syslog-ng can collect these logs as well. The
use of the wildcard-file source gives you additional flexibility.
The following sections show you how to use syslog-ng in these scenarios.
•
•
•
Run your central log server in Docker
Collect Docker infrastructure logs using syslog-ng
Collect logs from containers using Docker volumes
2. Procedure – Run your central log server in Docker
Purpose:
You can run your central log server in a Docker container. To deploy a syslog-ng log server running in a Docker container, you
can download a ready-to-use image from the Docker Hub, already passing one million pulls. The image is based on the latest
LTS Ubuntu and the latest stable version of syslog-ng. It has all modules, including Java modules and experimental modules
from the incubator.
You can download the image and run it in a container of your host machine using a single command.
Note
As a central log server, the syslog-ng image exposes three different ports where it can receive log messages:
• syslog UDP: 514
• syslog TCP: 601
• syslog TLS: 6514
To be able to use them, you need to enable these ports both in the syslog-ng configuration (syslog-ng.conf) and in
the command line starting the Docker container.
syslog-ng
client
Dockler host
syslog-ng container
file
destination
syslog-ng
client
syslog-ng
client
network source
directory mapped
from the Docker host
Figure 1. Run your central syslog-ng server in Docker
syslog-ng.com
Steps:
Step 1.
Configure syslog-ng.
If you do not have a syslog-ng configuration file at hand for testing, create one. Here is a simple syslog-ng.conf,
which listens to the legacy syslog protocol on UDP port 514, the new syslog protocol on TCP port 601, and stores
any incoming log messages in the file /var/log/syslog.
@version: 3.12
source s_net {
network(
ip(“0.0.0.0”)
transport(“udp”)
);
syslog(
ip(“0.0.0.0”)
);
};
destination d_file {
file(“/var/log/syslog”);
};
log {source(s_net); destination(d_file); };
Step 2.
You can map files or directories from your host into the container. If you use this simple configuration file where
all the settings are in a single file without any encryption keys, mapping the configuration file is the easiest. In any
other scenario, you should map a complete directory for the configuration. The example docker command will
map the configuration file.
Step 3.
If you store all your log messages in a database, there is not much need for persistent storage for your container.
If your central log server also stores data, there is a good chance that you will want to have access to those logs
even if you switch to another syslog-ng image. In this case, you should map a directory from the host machine,
so your log storage is independent from your Docker containers.
If you have your syslog-ng.conf under /data/syslog-ng/conf and plan to store your logs in the
/data/syslog-ng/logs directory, you can use the following command line to get started:
docker run -it -v
/data/syslog-ng/conf/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf -v
/data/syslog-ng/logs:/var/log -p 514:514 -p 601:601 --name syslog-ng
oneidentity/syslog-ng:latest -edv
When you first execute this command, it can take a few minutes until syslog-ng is up and running, as the image is
downloaded over the Internet. On subsequent executions, Docker will use the local copy and start immediately.
The name of the container will be “syslog-ng” and Docker will use the latest available syslog-ng image from the
Docker Hub.
It does the following:
1. Starts the container in interactive mode.
2. Maps two network ports from the host to the container.
3. Maps the configuration file and log directory.
4. Adds some debug options to syslog-ng.
Step 4.
Test your configuration.
Using the docker ps command from another terminal, check if your container is up and running. You can see
more information about the image, including the opened network ports.
linux-pzl9:~ # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES b947a3411c1a oneidentity/syslog-ng:latest “/usr/sbin/syslog-ng “ 18 seconds
ago Up 17 seconds 0.0.0.0:514->514/tcp, 514/udp, 0.0.0.0:601->601/tcp,
6514/tcp syslog-ng
The loggen command can generate you a few sample logs. If you used the above configuration and directories,
you should see a flood of messages on the screen where you started the container and also new messages in the
file /data/syslog-ng/logs/syslog linux-pzl9:~ # loggen -i -S-P localhost 601 average rate = 1006.53
msg/sec, count=10066, time=10.000, (average) msg size=260, bandwidth=255.42 kB/sec.
syslog-ng.com
Please complete the form to gain access to this content
