back

Text File Input Module

Module Name:    imfile

Author: Rainer Gerhards <rgerhards@adiscon.com>

Description:

Provides the ability to convert any standard text file into a syslog message. A standard text file is a file consisting of printable characters with lines being delimited by LF.

The file is read line-by-line and any line read is passed to rsyslog's rule engine. The rule engine applies filter conditions and selects which actions needs to be carried out. Empty lines are not processed, as they would result in empty syslog records. They are simply ignored.

As new lines are written they are taken from the file and processed. Please note that this happens based on a polling interval and not immediately. The file monitor support file rotation. To fully work, rsyslogd must run while the file is rotated. Then, any remaining lines from the old file are read and processed and when done with that, the new file is being processed from the beginning. If rsyslogd is stopped during rotation, the new file is read, but any not-yet-reported lines from the previous file can no longer be obtained.

When rsyslogd is stopped while monitoring a text file, it records the last processed location and continues to work from there upon restart. So no data is lost during a restart (except, as noted above, if the file is rotated just in this very moment).

State Files

Rsyslog must keep track of which parts of the monitored file are already processed. This is done in so-called “state files”. These files are always created in the rsyslog working directory (configurable via $WorkDirectory).

To avoid problems with duplicate state files, rsyslog automatically generates state file names according to the following scheme:

As a concrete example, consider file /var/log/applog is being monitored. The corresponding state file will be named imfile-state:-var-log-applog.

Note that it is possible to set a fixed state file name via the deprecated “stateFile” parameter. It is suggested to avoid this, as the user must take care of name clashes. Most importantly, if “stateFile” is set for file monitors with wildcards, the same state file is used for all occurrences of these files. In short, this will usually not work and cause confusion. Upon startup, rsyslog tries to detect these cases and emit warning messages. However, the detection simply checks for the presence of “*” and as such it will not cover more complex cases.

Note that when $WorkDirectory is not set or set to a non-writable location, the state file will not be generated. In those cases, the file content will always be completely re-sent by imfile, because the module does not know that it already processed parts of that file.

Metadata

The imfile module supports message metadata. It supports the following data items

Metadata is only present if enabled. By default it is enabled for input() statements that contain wildcards. For all others, it is disabled by default. It can explicitly be turned on or off via the addMetadata input() parameter, which always overrides the default.

Configuration Directives:

Module Directives

Action Directives

Caveats/Known Bugs:

Currently none.

Sample:

The following sample monitors two files. If you need just one, remove the second one. If you need more, add them according to the sample ;). This code must be placed in /etc/rsyslog.conf (or wherever your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.

Legacy Configuration Directives:

Note: in order to preserve compatibility with previous versions, the LF escaping in multi-line messages is turned off for legacy-configured file monitors (the "escapeLF" input parameter). This can cause serious problems. So it is highly suggested that new deployments use the new input() statement and keep LF escaping turned on.

Sample:

The following sample monitors two files. If you need just one, remove the second one. If you need more, add them according to the sample ;). This code must be placed in /etc/rsyslog.conf (or wherever your distro puts rsyslog's config files). Note that only commands actually needed need to be specified. The second file uses less commands and uses defaults instead.

[rsyslog.conf overview] [manual index] [rsyslog site]

This documentation is part of the rsyslog project.
Copyright © 2008-2014 by Rainer Gerhards and Adiscon. Released under the GNU GPL version 3 or higher.