Linux online course

rsyslog: Linux System Logging Configuration and Remote Log Forwarding

Learn how rsyslog collects Linux messages, matches facility and severity rules, writes local logs, forwards events remotely, and validates configuration changes.

rsyslog is a Linux logging implementation that receives, filters, formats, stores, and forwards log events. Its daemon process is called rsyslogd. Debian-family and Red Hat-family distributions commonly use rsyslog, although some installations use other logging components alongside it.

This lesson explains the traditional syslog-style configuration used by rsyslog. You will learn how messages are classified by facility and severity, how selectors choose messages, how actions deliver them to files or users, and how to forward selected events to another host.

Useful background includes basic Linux services, permissions, hostname resolution, and log files under /var/log. See Linux file structure and managing file ownership for related fundamentals.

What rsyslog does

The traditional syslog model defines a common way for programs and system components to classify and deliver messages. A traditional daemon named syslogd implemented this model. rsyslog builds on that model while adding modular inputs and outputs, flexible rule processing, and filters based on message content.

rsyslogd can collect messages from:

• Applications that submit messages through the local logging interface.
• Kernel components and kernel message sources.
• System services and other daemons.
• Network clients, when a network input module and receiver configuration are enabled.

After receiving a message, rsyslog can write it to a local file, send it to a device, display it on logged-in user terminals, or forward it to another logging server. It can also use modules for additional input, output, parsing, and filtering behavior.

rsyslog configuration layout

The primary file commonly includes fragments with a directive such as:

$IncludeConfig /etc/rsyslog.d/*.conf

The pattern tells rsyslog to load matching configuration files from the supplementary directory. Exact default files vary by distribution, so inspect the active configuration rather than assuming every system has identical files.

Ordering matters. Configuration fragments are read in an order determined by the include pattern and filenames. If several rules match the same message, each matching rule can process it. A broad wildcard rule in an earlier or later file may therefore add another destination instead of replacing a more specific rule. Use filenames with clear numeric prefixes when you need predictable organization.

Modules and feature loading

A module is a loadable rsyslog component that provides an input, output, parser, or other capability. Examples include modules for the local logging socket, kernel messages, network input, and output destinations.

Older configurations use the legacy $ModLoad directive:

$ModLoad module_name

Replace module_name with an installed module appropriate to the feature. A leading # comments out a directive and disables it:

# $ModLoad module_name

For example, a system may load a local socket input module to receive messages from local applications, or a network input module to receive messages from remote clients. A sender-side forwarding rule is not enough to build centralized logging: the receiver must also load and configure an appropriate network input and permit the required traffic.

Global directives

Global directives affect how rsyslog operates generally rather than selecting one facility and severity combination. Legacy configurations may specify the owner and group of newly created log files:

$FileOwner syslog
$FileGroup adm

$FileOwner sets the user owner, and $FileGroup sets the group. Actual account names differ between distributions. These settings affect who can read or manage newly created files, so choose them consistently with your operational and security requirements.

Global configuration can also include fragments:

$IncludeConfig /etc/rsyslog.d/*.conf

File permissions, directory permissions, ownership, and the privileges available to the service all affect whether a destination file can be created and written.

Rules, selectors, and actions

A traditional rsyslog rule has two main parts:

1. A selector, which identifies messages that match.
2. An action, which specifies what to do with matching messages.

The selector generally has the form facility.priority. In this syntax, priority is the traditional name for the urgency field commonly called severity.

A severity name normally acts as a threshold. A rule using alert matches alert and more urgent levels, especially emerg. It does not match less urgent levels such as crit, err, or warning. The usual severity order from most to least urgent is emerg, alert, crit, err, warning, notice, info, and debug.

The facility identifies the source category. For example, kern represents kernel-originated messages and mail represents mail-related messages. The combination determines which events are recorded or delivered.

Facilities and severities in practice

• kern: messages associated with the kernel.
• mail: messages associated with mail services.
• alert: a high-severity event requiring immediate attention.
• emerg: the most severe level, indicating that the system is unusable.

Thus, kern.alert means kernel messages at alert severity and above, while mail.* means every severity for the mail facility. The selector *.emerg means emergency messages from every facility.

Local logging destinations

Kernel alerts in a dedicated file

To write kernel alert-level and more urgent messages to a separate file:

kern.alert    /var/log/kern.log

This gives administrators a focused location for critical kernel events. Because alert is a threshold, emergency kernel messages also match this rule.

All mail facility messages

To collect every severity from the mail facility:

mail.*        /var/log/mail

The wildcard is in the severity position, so this is facility-specific collection rather than severity-specific collection.

Emergency messages to logged-in users

The action * can deliver a message to all logged-in text-mode users:

*.emerg       *

Here the first wildcard selects every facility, and emerg selects emergency messages. The final asterisk is the all-users terminal destination, not another selector.

Rules can overlap. A mail message at emergency severity matches both:

mail.*        /var/log/mail
*.emerg       *

It can therefore be written to /var/log/mail and announced to logged-in text-console users. Matching one rule does not inherently stop later rules from processing the same event.

Remote log forwarding

To forward selected messages to another logging host, replace a local file action with a remote host action. Traditional syntax uses one @ before the destination hostname:

kern.alert    @suse1

This rule forwards kernel alert-level and more urgent events to the host named suse1. The local file form and remote form are different actions:

The sender must resolve suse1 to an address and reach it over the network. Firewalls and routing must allow the logging traffic. On the receiver, rsyslog must be running with a network input configured to accept remote messages. Configure remote reception deliberately because accepting network logs exposes a service to other hosts.

Applying and validating changes

Place a new rule in the main file or in an included fragment such as a suitably named file under /etc/rsyslog.d/. Then validate the configuration before relying on it. The exact validation command can vary with the installed rsyslog version and packaging, so consult the local daemon's supported options and review service output for parse errors.

Restart the service after editing:

sudo service rsyslog restart

On systemd-based systems, the equivalent is commonly:

sudo systemctl restart rsyslog
sudo systemctl status rsyslog

Generate a controlled test message with logger:

logger -p mail.alert "rsyslog rule test"

Then inspect the intended local destination:

sudo tail -n 50 /var/log/mail

For remote forwarding, inspect both the sender and receiver. Confirm that the receiver's log contains the test event rather than assuming that a successful sender restart proves delivery.

Troubleshooting rsyslog rules

A new rule has no effect

• Confirm that rsyslog was restarted or reloaded after the edit.
• Verify that the fragment is included by /etc/rsyslog.conf.
• Check that the generated event uses the facility and severity in the selector.
• Run configuration validation and inspect service journal output for syntax or startup errors.
• Use logger to create a controlled test instead of waiting for a rare production event.

The expected file is missing or cannot be written

• No matching event may have occurred yet; send a matching test message.
• Check that the destination directory exists and is accessible.
• Review $FileOwner, $FileGroup, and directory permissions.
• Confirm the privileges and service account used by rsyslog on that distribution.

Remote forwarding does not work

• Verify that the destination hostname resolves correctly.
• Check routing, firewall rules, and network reachability.
• Confirm that the receiver is running and listening for the expected logging input.
• Check sender and receiver service logs.
• Generate a known matching event, such as a message using the intended facility and severity.

A message appears in too many destinations

Look for overlapping rules. A broad rule such as *.emerg can match an event already selected by a facility-specific rule such as mail.*. Decide whether multiple destinations are intended before narrowing or removing a rule.

Emergency notifications are not visible

• Confirm that the message is classified as emerg.
• Confirm that *.emerg * is loaded.
• Remember that the destination targets logged-in text-mode users; it is not a general desktop notification mechanism.
• Test emergency notification carefully in a non-production environment.

End-to-end configuration example

The following traditional rules demonstrate local collection, user notification, and remote forwarding. Use only the destinations appropriate for your system:

# Critical kernel messages locally
kern.alert    /var/log/kern.log

# Every mail facility message locally
mail.*        /var/log/mail

# Emergency messages from every facility to logged-in users
*.emerg       *

# Critical kernel messages to a central host
kern.alert    @suse1

After saving the configuration, validate it, restart rsyslog, check the service status, generate a suitable test event, and inspect the local and remote destinations. Remember that the last rule requires a receiver-side network input, name resolution, and working connectivity.

Exam-relevant points

• rsyslogd is the daemon; rsyslog is the logging implementation.
• A selector chooses messages; an action specifies their destination or handling.
• A facility identifies the originating subsystem, while severity expresses urgency.
• kern.alert matches kernel alert and emergency messages.
• mail.* matches every severity for the mail facility.
• *.emerg matches emergency messages from every facility.
• Multiple matching rules can send one message to multiple destinations.
• @hostname is traditional single-@ remote-forwarding notation.
• Remote forwarding requires both a sender rule and a configured receiver.
• Configuration inclusion and filename order affect which rules are loaded and how overlapping rules are organized.