Rootless Promtail with Ansible Enhancing Security and Efficiency

The Ansible Promtail Role allows you to effortlessly deploy and manage Promtail, agent which ships contents of local logs to private Loki. Role is tailored for systems from the Red Hat family. Before diving into the details, I highly recommend reading the role's README.md file for a comprehensive understanding.

👀 README.md - promtail Role

This blog post is primarily focused on explaining the rationale behind constructing this role in a specific manner and highlighting the importance of not running Promtail as root.

What is Promtail ?

Promtail is a component of the Loki logging stack, designed to collect log data from applications running on a system. Acting as a log forwarder, tailing log files or accessing system journals, and sends the log entries to a central Loki server for storage and analysis. Promtail is highly efficient, leveraging the power of labels and service discovery from Prometheus, making it easy to configure and integrate with existing monitoring setups.

Avoid Running Promtail as root

Why avoid running Promtail as root? While it may seem easier to configure with elevated privileges, doing so introduces significant security risks. Granting unnecessary root access increases the attack surface, making your system vulnerable to potential exploitation.

On the other hand, opting for a rootless approach ensures better security and adheres to modern best practices. By minimizing privileges and restricting access to essential resources, you achieve a more robust and secure Promtail setup.

How can Promtail access system logs without root ?

Utilizing setcap

Setcap command-line utility is powerful that allow to set specific capabilities for executable files, granting them certain privileges without the need for full root access. For instance, running the command setcap 'cap_dac_read_search=+ep' /usr/bin/promtail adds the capability to read and search directories and files to the /usr/bin/promtail executable. As a result, Promtail gains the ability to perform these actions without relying on full root privileges.

However, there's a significant problem with this approach. Granting such capabilities to Promtail can potentially give it access to sensitive files like /etc/shadow, which are typically restricted to the root user. This introduces unnecessary risks and provides more permissions than needed for Promtail to fulfill its intended purpose.

Due to these security concerns, I have not implemented this setcap approach.

Utilizing ACL

Linux Access Control Lists (ACLs) provide an effective way to manage Promtail's access to specific logs. By leveraging the promtail_acl_log_file_permission and promtail_acl_log_dir_permission variables, Ansible can be instructed to assign the correct permissions to read and access only the specified logs.

1. Granting Access to Log Directories: To allow Promtail access to a directory and all its subdirectories, you can configure the promtail_acl_log_dir_permission variable. For instance, if you have an application that manages logs separately and is not handled by logrotate, this configuration ensures that any new log files created in the specified directory inherit the default permissions, providing Promtail with read access.

promtail_acl_log_dir_permission:
  - "/var/log"

2. Granting Access to Specific Log Files: If you want to grant Promtail read access to specific logs, you can use the promtail_acl_log_file_permission variable. Specify the exact paths to the logs you wish to grant access to. This approach guarantees that ACLs are applied to the specified log files, granting Promtail the necessary access.

promtail_acl_log_file_permission:
  - "/var/log/secure"
  - "/var/log/messages"
  - "/var/log/cron"
  - "/var/log/dnf.log"
  - "/var/log/boot.log"

Since I cannot predict which logs you want to configure, I've decided to avoid modifying the system logrotate file located at /etc/logrotate.d/rsyslog. To overcome some limitations of logrotate, this role creates a dummy directory called /var/log/dummy_promtail_acl, along with an empty log file within it.

The following configuration is generated based on the provided variables promtail_acl_log_dir_permission and promtail_acl_log_file_permission:

# CONTENT OF /etc/logrotate.d/promtail_acl
/var/log/dummy_promtail_acl/dummy_promtail_acl.log
{
    copytruncate
    postrotate
        /usr/bin/setfacl -R -m d:u:promtail:rx /var/log
        /usr/bin/setfacl -m u:promtail:r /var/log/secure
        /usr/bin/setfacl -m u:promtail:r /var/log/messages
        /usr/bin/setfacl -m u:promtail:r /var/log/cron
        /usr/bin/setfacl -m u:promtail:r /var/log/dnf.log
        /usr/bin/setfacl -m u:promtail:r /var/log/boot.log
    endscript
}

This configuration ensures that the permissions for Promtail are maintained after log rotation.

---

Utilizing ACLs and carefully managing access to log files enhances Promtail's security, ensuring it has the necessary permissions to fulfill its logging responsibilities effectively. By implementing this approach, the role becomes easily shareable across RedHat Family systems, while also restricting Promtail's access to only what is required.

👀 Are you curious about how easily you can deploy this role? 👉 Quick Start

---

Thanks for reading. I'm entering the void. 🛸 ➡️ 🕳️