.. SPDX-FileCopyrightText: Copyright 2012- David Sommerseth SPDX-License-Identifier: GPL-2.0-only .. For notes on how to document Python in RST form, see e.g.: .. http://sphinx.pocoo.org/domains.html#the-python-domain Configuring logactio ==================== Logactio makes use of a single ini-styled configuration file. The default location for this file is */etc/logactio.cfg*. This file uses a separate section, identified by *[section-name]*. There are four different groups of sections which you need to configure, *IgnoreList*, *Reporter*, *Logfile* and *Rule*. Each of these group names have a *label*, where the group name and the label are separated by a colon (:) .. index:: pair: Configuration; IgnoreList Configuring Ignore Lists ************************ An ignore list holds a set of IPv4 and/or IPv6 addresses or CIDR ranges. Once defined, a reporter or rule section may reference it by name. When a regex match contains an IP address that belongs to the ignore list, the reporter or rule is silently skipped for that event. An ignore list section is declared as ``[IgnoreList:NAME]`` and contains a single key named ``addresses``. The value may list addresses separated by commas, newlines (with continuation indentation), or both. .. code-block :: ini [IgnoreList:localnet] addresses: 127.0.0.1, 192.168.0.0/24, 10.0.0.0/8, ::1, fd00::/8 Each entry can be either: * a single IPv4 or IPv6 address (e.g. ``192.168.1.1`` or ``::1``) * a CIDR network range (e.g. ``10.0.0.0/8`` or ``2001:db8::/32``) To attach one or more ignore lists to a reporter or rule, add the ``ignore-list`` key with a comma-separated list of names: .. code-block :: ini [Reporter:nftblock] module: NftablesSet nft-table: filter nft-set: blocklist ignore-list: localnet, trusted-hosts [Rule:ssh-brute] logfile: messages regex: .*sshd.*Failed password.* from (\S+) threshold: 5 ignore-list: localnet, trusted-hosts When ``ignore-list`` is set on a *Reporter*, events are filtered for that reporter only — other reporters attached to the same rule still fire normally. When set on a *Rule*, the entire rule (all its reporters) is skipped for matching addresses. .. index:: pair: Configuration; Reporters Configuring Reporters ********************* There are several different reporters available in logactio, and each of them take different configuration parameters. If no reporter is configured, a built-in reporter called *Default* will be used. * The Default reporter This reporter will only write data via the configured logging methods. This reporter also does not have any configuration settings and does not require any specific declaration. * :ref:`lnk-httpreporter` This reporter will send the extracted log data to a web server. Both HTTP and HTTPS may be used. * :ref:`lnk-ntfyreporter` This reporter will send the extracted log data as a notification to an ntfy.sh-compatible server, using HTTPS POST with a configurable message template. * :ref:`lnk-smtpreporter` This reporter will send the extracted log data via SMTP to one or more e-mail recipients. Support for SSL and STARTTLS in addition to SMTP-AUTH are availble as well. * :ref:`lnk-IPTipset` This "reporter" can add extracted IP addresses from a log file and add it to an ipset table. This table can be used by iptables to for example block access. * :ref:`lnk-NftablesSet` This "reporter" can add extracted IP addresses from a log file and add them to a pre-existing nftables set. .. index:: pair: Configuration; HTTPreporter .. _lnk-httpreporter: HTTPreporter ------------ This reporter takes the following configuration variables * method This defaults to **GET** if this is not set. But can be set to **POST** if you want the reports to be sent via HTTP POST instead of HTTP GET. * url This is a required setting. This is the URL where to submit the reports. * format This defaults to **plain** if not set. Controls how the report data is encoded in the request: * **plain**: data is URL-encoded (``application/x-www-form-urlencoded``). Works with both GET and POST. * **json**: data is sent as a JSON object (``application/json``). Requires **method** to be set to **POST**. .. index:: pair: Examples; HTTPreporter HTTPreporter example ~~~~~~~~~~~~~~~~~~~~~~ .. code-block :: ini [Reporter:HTTP-DEMO-1] module: HTTPreporter method: POST url: http://logactio.example.com/alert/ [Reporter:HTTP-DEMO-2] module: HTTPreporter url: http://logactio.example.com/info/ [Reporter:HTTP-DEMO-3] module: HTTPreporter method: POST url: http://logactio.example.com/alert/json/ format: json Here three reporters are configured. *HTTP-DEMO-1* will use HTTP POST with URL-encoded data, *HTTP-DEMO-2* will use HTTP GET with URL-encoded data, and *HTTP-DEMO-3* will use HTTP POST sending a JSON body. .. index:: pair: Configuration; ntfyReporter .. _lnk-ntfyreporter: ntfyReporter ------------ This reporter sends notifications to an `ntfy.sh `_ compatible server using HTTPS POST. The message body is rendered from a configurable template, allowing you to format the notification using the rule name and regex capture groups from the matching log line. The required configuration variables are: * server The hostname of the ntfy.sh server to send notifications to. The connection is always made over HTTPS. * topic The ntfy.sh topic to post notifications to. This is the channel that subscribers will need to listen on to receive the notifications. Optional settings are: * template A format string controlling the notification message body. The following placeholders are available: * ``{rulename}`` — the name of the rule that triggered * ``{0}`` — the full regex match (all capture groups pipe-joined) * ``{1}``, ``{2}``, … — individual regex capture groups (1-indexed) If not set, the default template is ``[{rulename}] {0}``. * priority The ntfy.sh notification priority. Valid values are ``min``, ``default``, ``high``, ``urgent``, and ``max``. Defaults to ``default``. * title Sets the ``Title`` header on the ntfy.sh request, which is displayed as the notification title on supported clients. Supports the same template placeholders as *template* (``{rulename}``, ``{0}``, ``{1}``, etc). * tags Sets the ``Tags`` header on the ntfy.sh request. A comma-separated list of emoji tags (e.g. ``warning,desktop``). .. index:: pair: Examples; ntfyReporter ntfyReporter example ~~~~~~~~~~~~~~~~~~~~ .. code-block:: ini [Reporter:ntfy] module: ntfyReporter server: https://ntfy.sh topic: my-topic priority: high tags: warning,bell title: Alert: {rulename} template: Warning: Logged event {rulename} from {1} in module {2} [Logfile:applog] file: /var/log/app.log reporters: ntfy [Rule:ntfy-warn] logfile: applog regex: Warning: Unexpected input from (.*) in (.*) event: .* This configures a reporter named *ntfy*. When the *ntfy-warn* rule triggers, a notification is posted to ``https://ntfy.sh/my-topic`` with the message body rendered from the template. For example, a log line containing:: .. code-block:: text Aug 14 09:30:19 appserver parser-app[10395]: WARN: Unprocessed event -- Warning: Unexpected input from john.doe in Auth:Login:ChPwd module. event: Invalid characters in new password will produce the following notification: .. code-block:: text Warning: Logged event ntfy-warn from john.doe in module Auth:Login:ChPwd .. index:: pair: Configuration; SMTPreporter .. _lnk-smtpreporter: SMTPreporter ------------ This reporter requires the following configuration variables: * sender The e-mail address which will be used in the "From:" field when sending mails * recipients This contains a comma separated list with all e-mail addresses who will get the logactio reports. * smtp_host This declares which SMTP server to use when sending the reports. In addition the SMTPreporter supports these optional variables: * subject_prefix The default subject prefix is set to 'LogActio Alert: '. By setting this variable, the subject prefix will be changed accordingly. * smtp_port The default value is set to port 25. * smtp_username If the SMTP server requires authentication to relay messages, this variable sets the SMTP user name. To use this feature, you must also set the smtp_password. * smtp_password This sets the SMTP password to use for the authentication * sslmode This is not set by default, so everything goes in clear text. If your SMTP server supports either SSL or STARTTLS, you can set it to SSL or STARTTLS. In SSL mode the SMTP library expects to the server to do the SSL handshake before the SMTP commands can be sent. In STARTTLS mode, the SMTP library will connect to the SMTP server in clear text and if the server supports STARTTLS, it will send the STARTTLS command and start the SSL handshake. .. index:: pair: Examples; SMTPreporter SMTPreporter example ~~~~~~~~~~~~~~~~~~~~~~ .. code-block :: ini [Reporter:SMTP-DEMO-1] module: SMTPreporter sender: logactio@example.com recipients: john.doe@example.com, jane.doe@example.com smtp_host: localhost [Reporter:SMTP-DEMO-2] module: SMTPreporter sender: john.doe@example.com recipients: bob.external@acme.com smtp_host: smtp.example.com smtp_port: 587 sslmode: STARTTLS smtp_username: logactioSMTP smtp_password: S3cretP4ssw0rd subject_prefix: Issues at Example Corp: Here two more reporters are configured. SMTP-DEMO-1 will use the SMTP MTA running locally on the system, and this will not require any authentication or SSL functionality. When this reporter is triggered, it will send mails to john.doe and jane.doe with the From field set to logactio@example.com. The SMTP-DEMO-2 reporter will send mails using an external SMTP server using port 587 and which requires STARTTLS and authentication. The subject line is also prefixed differently. .. index:: pair: Configuration; IPTipset .. _lnk-IPTipset: IPTipset -------- To use this module, you need ipset and iptables installed and available. This will also require logactio to run with root privileges, otherwise it lacks privileges to update the ipset and iptables rules. .. IMPORTANT:: This module expects only a single regex group to be extracted. This value **has** to be an IP address, otherwise the IPTipset module will fail when executing the ipset command. The required configuration variables are: * ipset-name This is the name of the ipset table to this specific section will use. Optional settings are: * ipset-create Boolean variable which tells wether the IPTipset module should create the ipset table when starting or not. The values of "true" (case or "1" will be understood as True, all other values as False. * ipset-hashsize, ipset-timeout, ipset-counters These variables will be sent further to the 'ipset create' command and will set hashsize, timeout and the counters flag. ipset-counters is a boolean flag, where the value of "true" (case insensitive) or "1" will be understood as True, all other values as False. The default value for ipset-hashsize is 1024, ipset-timeout is not defined and ipset-counters is False by default. * ipset-save This will enable the IPTipset module to save the state of the ipset table when exiting. If this file exists when LogActio is started again later on, this state will be loaded automatically. * iptables-chains If set, LogActio will insert the ipset checks into the listed iptables chains. Multiple chains can be listed, separated by comma. * iptables-insert-points This is optional but only available if iptables-chains are used. This defines where in the different iptables chains the ipset rules will be inserted. By default it will be inserted at the top of the chain. The syntax is CHAIN:NUM, where CHAIN is the name provided in *iptables-chains* and NUM is the value used when calling iptables. The result is something like 'iptables -I CHAIN NUM ...'. Multiple chains can be defined, separated by comma. * iptables-jump This is required when using *iptables-chains*. This defines what to do when iptables get a positive match against the ipset table. This sets the '-j' options when iptables commands are executed. .. index:: pair: Examples; IPTipset IPTipset example ~~~~~~~~~~~~~~~~ .. code-block :: ini [Reporter:ipsetblock] module: IPTipset ipset-name: BlockList ipset-create: True ipset-hashsize: 2048 ipset-timeout: 3600 ipset-counters: False ipset-save: /var/lib/ipset/logactio-ipset.save iptables-chains: INPUT,OUTPUT,FORWARD iptables-insert-points: INPUT:2,FORWARD:4 iptables-jump: DROP This will configure a reported named ipsetblock. Whenever this reporter is called, it will add an IP address to an ipset table named 'Blocklist'. This list will be created when LogActio starts, with a hashsize of 2048 and a timeout of 1 hour (3600 seconds). The ipset table state will be saved to /var/lib/ipset/logactio-ipset.save. When starting LogActio the following commands will be run: .. code-block :: shell # ipset --exist create BlockList hash:ipset hashsize 2048 timeout 3600 # iptables -I INPUT 2 -m set --match-set BlockList src -j DROP # iptables -I OUTPUT -m set --match-set BlockList src -j DROP # iptables -I FORWARD 4 -m set --match-set BlockList src -j DROP .. index:: pair: Configuration; NftablesSet .. _lnk-NftablesSet: NftablesSet ----------- To use this module, you need nftables installed and available. This will also require logactio to run with root privileges, otherwise it lacks privileges to update the nftables set. The nftables set must be pre-created before starting logactio. This module will not create or destroy the set. .. IMPORTANT:: This module expects only a single regex group to be extracted. This value **has** to be an IP address, otherwise the NftablesSet module will fail when executing the nft command. The required configuration variables are: * nft-table The name of the nftables table containing the set. * nft-set The name of the nftables set to add IP addresses to. Optional settings are: * nft-family The nftables family of the table. Defaults to **inet**. * nft-save If set, the state of the nftables set is saved to this file in native nft format on every update. Restoring the set from this file at startup is left to an external tool (e.g. a systemd service or init script). .. index:: pair: Examples; NftablesSet NftablesSet example ~~~~~~~~~~~~~~~~~~~ .. code-block :: ini [Reporter:nftblock] module: NftablesSet nft-family: inet nft-table: filter nft-set: blocklist nft-save: /var/lib/nftables/logactio-blocklist.nft This configures a reporter named nftblock. Whenever triggered, it adds an IP address to the pre-existing ``inet filter blocklist`` nftables set. The current set state is saved to the given file after each update. To use native expiry, declare the set with ``flags timeout`` and set a default ``timeout`` in the nftables ruleset — nft will then expire entries automatically without any additional logactio configuration. .. index:: single: Log files Configuring log files ********************* The configuration sections for the log files are similar to the reporters. For each configured logfile section logactio will start a separate worker thread watching the file for new content using inotify. There are two configuration variables logactio supports for logfiles: * file This is mandatory, and declares the log file it should watch. * reporters This is optional, but declares the default reporter module(s) to use if an event happens to this file. You may list more reporter modules, separated by comma. Logfile example --------------- A typical configuration for log file sections would look something like this: .. code-block :: ini [Logfile:messages] file: /var/log/messages reporters: SMTP-DEMO-1 [Logfile:maillog] file: /var/log/maillog reporters: HTTP-DEMO-1, HTTP-DEMO-3 In this example we have configured two Logfile groups, *messages* and *maillog*. Any event happening in /var/log/messages will by default be reported using the configured SMTP-DEMO-1 reporter setup. On changes in the *maillog*, both the HTTP-DEMO-1 and HTTP-DEMO-3 reporters will be triggered. .. index:: single: Rules Configuring watch rules *********************** If you only configure Logfile and Reporter sections, logactio will not trigger at all. You need to configure some rules what logactio should react to. The rules are based on regular expressions. And if there is a match on the log lines received, each of these lines will be acted upon separately. A Rules section consists of two required configuration variables: * logfile This is the log file this rule is to be used against * regex This is the regular expression which needs to match to cause the reporter to be triggered. You can also use regex groups, like (.*), to extract information from the log line which will be sent to the reporter. If you use multiple groups, all of them will be sent to reporter. * threshold: This sets how many times this event should match before triggering the reporter. * threshold-type: This defines how the threshold counter works. By default, it is set to *rule*. This will increase the "hit counter" each time this watch rule is triggered. By setting the *threshold-type* parameter to *exact* it will also consider the regex groups defined in the *regex* parameter. When using the *exact* type, it will have individual threshold counters per group contents for each time this rule is triggered. The optional settings are: * reporters This overrides the default reporters configured in the log file's Logfile section. This can be used to add exceptions or report an event differently in special cases. For example you might want developers to get an automatic mail if their program causes an exception which is logged. While a system- administrator might only want reports if someone tries to log into a system unsuccessfully more than 3 times. Setting up different Rule sections with different reporters and thresholds brings you this power, even if everything is logged to the same file. * time-frame This extends the threshold trigger to also consider a time frame before trigging an action. If the threshold is set to 3 and time-frame is set to 10, logactio will not trigger an action unless there are 3 events within the last 10 seconds. * rate-limit This will restrict logactio from any flood actions. If this value is set to 10 and you have log changes which matches this rule once every second, logactio will only perform the configured action once per 10 second. * reset-rule-rate-limits This takes a comma separated list of rule names, but only for the same log file this rule uses. This can be used to "unlock" another rule's rate-limit restriction. This is useful where you might report connection issues only once an hour, even though the failed reconnection attempts are logged every minute. But in the moment the connection really is restored you can trigger a logactio action informing the connection is back again. But if this connection drops after a few minutes again - it might be the "connection-failed" rule won't trigger before an hour later. By adding reset-rule-rate-limits on the "connection-is-back" rule, it can reset the "connection-failed"'s rate-limit check, so it that rule will trigger instantly. Rule examples ------------- .. code-block :: ini [Rule:iptables] logfile: messages regex: .* (.*) kernel: .* IN=(\S*) OUT=.* MAC=.* SRC=(\S* )DST=(\S* ).* PROTO=(\S* )SPT=(\d* )DPT=(\d* ).* threshold: 1 [Rule:lost-connection] logfile: maillog regex: lost connection after (.*) from (.*) reporters: HTTP-DEMO-2 threshold: 5 We declare two rules here, one which looks for a certain pattern which matches iptables and uses the default reporters. Each time log line in /var/log/messages matches this rule, the reporters are triggered. It will report the hostname of the server this happened, the input device, source and destination IP addresses as well as protocol, source and destination ports. The second rule looks for connection issues in /var/maillog and will report these every 5th issue using only the configured HTTP-DEMO-2 reporter.