Linux online course

Rotate, Compress, and Manage Linux Log Files with logrotate

Learn how to configure Linux logrotate to rotate logs by time or size, retain limited copies, compress old files, run scripts, and manage application logs.

Log files record events from the operating system, services, and applications. Because many logs are written continuously, an unattended file can grow until the filesystem runs out of free space. This lesson explains how logrotate automates rotation, retention, compression, deletion, and service actions.

This lesson assumes familiarity with Linux paths, administrative access, basic shell and editor use, logs, services, and basic cron concepts. For background, see Linux file structure and managing Linux file ownership.

Why log rotation is necessary

Rotation is the process of retiring the current active log and starting or creating a replacement while preserving an older copy. For example, an active access_log might become access_log.1, while a new empty access_log receives new entries.

Without rotation, continuously written files can consume all available capacity on a filesystem. Rotation applies a policy that can:

• replace or rename the active log;
• retain a defined number of older copies or retain them for a defined period;
• compress older copies to reduce disk usage;
• delete copies that exceed the retention policy; and
• notify or reload a service after its log changes.

Retention is the number or duration of rotated copies kept before they are removed. Rotation is therefore both a naming operation and a disk-space management policy.

What logrotate does

logrotate is a Linux utility for applying automated log rotation and retention policies. It can rotate logs, create replacement files, compress old logs, remove expired copies, mail a log before it is removed, and execute shell commands before or after rotation.

logrotate is normally invoked regularly by a scheduler such as cron. The scheduler starts logrotate; logrotate then reads its configuration and determines whether each configured log should be rotated.

logrotate

Rotation is the maintenance step after an application or system component is generating log entries. A rotation policy cannot create useful entries by itself; the relevant service must first be configured to write to the target path.

Configuration locations and organization

Location: /etc/logrotate.conf
Role: Primary configuration file.
Typical contents: Global defaults and an include directive for additional policies.

Location: /etc/logrotate.d/
Role: Directory for separate package-specific or application-specific policies.
Typical contents: One configuration file per service or package.

Location: A service-specific file such as an Apache policy under /etc/logrotate.d/
Role: Defines behavior for one service's logs.
Typical contents: Service paths, schedules, compression, retention, and reload actions.

/etc/logrotate.conf commonly establishes defaults for later log definitions. The include directive loads additional configuration files or a directory of files:

weekly
rotate 4
create
dateext
#compress
include /etc/logrotate.d

Here, weekly is a default interval, rotate 4 retains four older copies, create creates a replacement file, and dateext requests date-based suffixes. The commented compress line does not enable compression. The include directive loads policies from /etc/logrotate.d/.

A log-specific stanza can override applicable global defaults. This lets a high-volume application use a size threshold or a different schedule while ordinary logs use the global policy.

Logrotate stanza syntax

A stanza is a block containing one or more log paths followed by directives enclosed in braces:

/var/log/example.log {
    weekly
    rotate 4
    compress
    create
}

One definition controls the policy for the specified log file or files. Every directive inside the braces applies to that target and supersedes an applicable global default.

Multiple paths can be listed before the opening brace:

/var/log/service-a.log /var/log/service-b.log {
    daily
    rotate 7
}

Rotation schedules and size triggers

The interval directives express time-based rotation:

Option: daily
Example: daily
Effect: Rotate according to a daily interval.

Option: weekly
Example: weekly
Effect: Rotate according to a weekly interval.

Option: monthly
Example: monthly
Effect: Rotate according to a monthly interval.

Option: size
Example: size 1k
Effect: Rotate when the log reaches the configured maximum-size threshold.

Option: Default numeric extensions
Example: access_log.1
Effect: Older copies receive numbered suffixes.

Option: dateext
Example: access_log-YYYYMMDD
Effect: Older copies receive date-based suffixes.

size sets a maximum-log-size trigger. Values can be expressed in bytes or with suffixes for kilobytes, megabytes, and gigabytes, such as 1k, 20M, or 2G. A small threshold is useful for demonstrating growth-based rotation:

/var/log/example_log {
    size 1k
}

This policy is based on the file reaching 1 KB rather than waiting only for a daily, weekly, or monthly interval. In a real policy, choose a threshold appropriate to the log's write rate and available disk space.

Retention and deletion

The rotate directive specifies how many older rotated copies to retain:

/var/log/application.log {
    weekly
    rotate 4
}

With rotate 4, the policy keeps four older copies, subject to the naming and processing options in the stanza. When a copy reaches the end of its retention lifecycle, it is removed. If mail is configured, logrotate can mail that expiring log before removal.

Retention must balance troubleshooting needs against disk capacity. Keeping more copies provides a longer history but consumes more space, even when compression is enabled.

Naming rotated files

By default, rotated files use numbered suffixes. A sequence may look like:

access_log
access_log.1
access_log.2
access_log.3

On rotation, older numbers are shifted as needed and the active file is retired. The dateext directive uses a date suffix instead of a numeric suffix:

access_log
access_log-20140930

For a web server, a date-stamped access log makes the period covered by a file immediately visible. The exact date format depends on logrotate's date-extension behavior and the rotation date.

Compression and replacement files

compress compresses older rotated logs. gzip is the default compression method, so a retained file commonly ends with .gz:

/var/log/web/access_log {
    weekly
    rotate 4
    compress
}

Compression reduces the disk space used by retained logs while preserving historical data. Compression can be enabled in global defaults or in one stanza, and it can be disabled for a policy when uncompressed files are required.

create creates a new empty active log after rotation. This is useful when the application expects the original path to exist. File mode, owner, and group can also be supplied with a more specific create form when required by the service:

/var/log/application.log {
    weekly
    create
}

Some applications keep an open file descriptor and need an explicit reload or reopen action after rotation. In that case, combine create with a postrotate hook.

Handling missing logs

missingok tells logrotate not to report an error when a configured log file is unavailable:

/var/log/optional-application.log {
    daily
    missingok
}

A file may be absent because the application has not started, has not generated an entry, uses a different path, or intentionally creates the file only under certain conditions. Verify the path first; use missingok when absence should not make an unattended rotation job fail.

Pre- and post-rotation scripts

prerotate and postrotate are hooks for shell commands run immediately before or after rotation. Each script block ends with endscript.

/var/log/application.log {
    weekly
    prerotate
        # commands to run before rotation
    endscript
    postrotate
        # commands to run after rotation
    endscript
}

Use prerotate for preparation or notification before the file changes. Use postrotate to notify or reload a service so it closes the old file and begins writing to the new active file. Commands must be appropriate for the service and should be tested with administrative care.

Mailing expiring logs

The mail directive takes an email-address argument. It can send a rotated log when that log reaches the end of its retention lifecycle, before logrotate removes it:

/var/log/security-events.log {
    weekly
    rotate 4
    mail admin@example.invalid
}

Mail delivery must be configured on the system for this feature to be useful. The directive does not by itself configure an email transport.

Core logrotate directives

Directive: compress
Purpose: Compress older rotated logs.
Typical value: None.
Effect: Uses gzip by default and reduces retained-log size.

Directive: size
Purpose: Set a maximum-size trigger.
Typical value: 1k, 20M, or 2G.
Effect: Rotation can be triggered by file growth.

Directive: dateext
Purpose: Use date-based names.
Typical value: None.
Effect: Replaces numeric-style suffixes with date suffixes.

Directive: daily, weekly, monthly
Purpose: Set a time interval.
Typical value: The directive itself.
Effect: Establishes the time-based rotation schedule.

Directive: missingok
Purpose: Ignore an absent target.
Typical value: None.
Effect: Suppresses an error for a missing log.

Directive: prerotate and postrotate
Purpose: Run shell commands around rotation.
Typical value: A script block ending in endscript.
Effect: Supports notifications, preparation, reloads, and reopen actions.

Directive: endscript
Purpose: End a script block.
Typical value: One marker after the commands.
Effect: Marks the end of a prerotate or postrotate block.

Directive: mail
Purpose: Mail an expiring log.
Typical value: An email address.
Effect: Sends a log when it reaches the end of retention.

Directive: rotate
Purpose: Set the retained-copy count.
Typical value: 4.
Effect: Older copies beyond the count are removed or mailed first.

Directive: create
Purpose: Create a replacement active file.
Typical value: None, or file attributes in a detailed form.
Effect: Leaves a new empty log at the active path.

Directive: include
Purpose: Load additional configuration.
Typical value: /etc/logrotate.d.
Effect: Brings service-specific policies into the configuration.

Global defaults and per-log overrides

A global file might establish a default-style policy like this:

weekly
rotate 4
create
dateext
#compress
include /etc/logrotate.d

An individual stanza can override the weekly schedule. For example, this policy makes /var/log/wtmp monthly:

/var/log/wtmp {
    monthly
}

The stanza's monthly directive supersedes the applicable global weekly default for that target. Other inherited defaults may still apply unless the stanza changes them.

Service-specific policies

Installed packages can provide files in /etc/logrotate.d/. Apache is a typical example: its access and error logs may need a service-specific schedule, retention count, compression policy, and post-rotation reload action rather than the same settings used for every system log.

# Conceptual Apache policy location
/etc/logrotate.d/apache2

Do not assume every distribution uses the same service name or exact file contents. Inspect the installed policy and adapt only the relevant settings. The important organization principle is that the package owns a focused policy while /etc/logrotate.conf supplies shared defaults and includes the directory.

Creating a custom application policy

Suppose an application writes entries to /var/log/example_log. Create a separate policy file under /etc/logrotate.d/, such as /etc/logrotate.d/example-app, with administrative privileges:

/var/log/example_log {
    size 1k
    compress
    dateext
}

Each directive has a specific purpose:

• /var/log/example_log identifies the log managed by this stanza.
• size 1k requests rotation when the file reaches 1 KB, so growth rather than only a calendar interval triggers the policy.
• compress compresses older rotated copies with gzip by default.
• dateext gives rotated copies date-based suffixes instead of the usual numeric suffixes.

A more complete production policy might add retention, a replacement file, and a service action:

/var/log/example_log {
    size 1k
    rotate 4
    compress
    dateext
    missingok
    create
    postrotate
        # reload or signal the application here
    endscript
}

Before relying on the policy, confirm that the application actually writes to /var/log/example_log, that the path is correct, and that the service can write to the replacement file. A rotation configuration is maintenance for generated logs, not a substitute for configuring log generation.

Rotation lifecycle

1. An application or system component writes entries to an active log.
2. A scheduled invocation or a size condition causes logrotate to evaluate the policy.
3. Any configured prerotate commands run.
4. The active log is renamed or retired as a rotated copy.
5. The rotated copy may receive a numeric or date-based name and may be compressed.
6. A new active log may be created at the original path.
7. Any configured postrotate commands run so a service can reopen or reload its log.
8. Copies beyond the retention count are removed, or mailed before removal if configured.

Troubleshooting logrotate policies

The configured file does not exist

Check whether the application has created the file and verify the configured path. If the file is intentionally absent at times, add missingok to suppress the unavailable-file error.

Disk consumption remains high

Rotation may be too infrequent, the size threshold may be too large, too many copies may be retained, or compression may be disabled. Review the interval and size settings, reduce rotate where appropriate, and enable compress for retained logs.

The application uses the global schedule unexpectedly

The stanza may not contain its own interval or size rule, or its policy file may not be loaded. Add the intended daily, weekly, monthly, or size directive and confirm that include /etc/logrotate.d loads the directory.

Filenames are numbered instead of date-stamped

dateext may be absent, disabled, or ineffective for the applicable stanza. Enable it globally or inside the individual log definition.

The service stops writing after rotation

The service may still hold the old file open or may need to reopen its log. Add a suitable postrotate script containing the service-specific reload, reopen, or signal action.

Old logs are not compressed

Check whether compress is enabled in the global defaults or applicable stanza. An explicit disable setting or a policy override can prevent compression.

Exam-relevant notes

• /etc/logrotate.conf contains global defaults and commonly includes /etc/logrotate.d/.
• /etc/logrotate.d/ is commonly used for package-specific and application-specific policies.
• A stanza is a log path or group of paths followed by directives inside braces.
• rotate 4 means retain four older rotated copies.
• daily, weekly, and monthly are time-based intervals; size 1k is a size-based trigger.
• Numeric names such as access_log.1 are the default style; dateext requests names such as access_log-20140930.
• compress uses gzip by default for older logs, and create creates a replacement active file.
• missingok suppresses an error when the target log is absent.
• prerotate and postrotate contain commands and must end with endscript.
• mail can send an expiring log before it is removed, provided mail delivery is configured.