securityContext Option Needed to Access /var/log when SELinux is Enabled

[ImranR98]

To read Traefik logs, CrowdSec needs access to the /var/log/containers directory, but this is inaccessible if the host has SELinux (like on Fedora).

I found (source) that the solution (without disabling SELinux) is to add this option to the pod spec:

securityContext:
  seLinuxOptions:
    type: container_logreader_t

I assume this can't be added to the Helm chart by default (would probably error out on non-SELinux systems but I haven't checked), but it should still be possible to set this in values.yaml but doesn't appear to be. Could this option be added?

Note: I'm grabbing Traefik logs by adding this to values.yaml (maybe there's an alternative?):

agent:
  acquisition:
    - namespace: production
      podName: traefik-*
      program: traefik

[github-actions]

@ImranR98: Thanks for opening an issue, it is currently awaiting triage.

If you haven't already, please provide the following information:

• kind : bug, enhancementor documentation
• area : agent, appsec, configuration, cscli, local-api

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the forked project rr404/oss-governance-bot repository.

[github-actions]

@ImranR98: There are no 'kind' label on this issue. You need a 'kind' label to start the triage process.

• /kind bug
• /kind documentation
• /kind enhancement

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the forked project rr404/oss-governance-bot repository.

[ImranR98]

/kind enhancement
/area configuration

[ImranR98]

Note: Aside from this, there is something else preventing the agent from actually acting on the log files (it never bans my IP even when being hammered by nikto). Works fine without SELinux. Unclear what the cause is, there may be additional SELinux settings to add.

Update: This is because the file can be read but not watched for changes. Not sure how to proceed from here.

Update 2: Even after enabling watch through a custom SELinux policy, CrowdSec just can't seem to pick up Traefik logs. Looks like the Helm chart is currently just totally incompatible with SELinux.

Update 3: Finally got a custom SELinux policy that seems to work at least in basic testing (obviously not ideal since the policy needs to be applied on each node). This script applies the policy and patches CrowdSec:

#!/bin/bash -e

# https://github.com/crowdsecurity/helm-charts/issues/190

# Define the policy file and module names
POLICY_FILE="/tmp/allow_logreader_to_watch_logs.te"
MODULE_FILE="/tmp/allow_logreader_to_watch_logs.mod"
PACKAGE_FILE="/tmp/allow_logreader_to_watch_logs.pp"

cat <<EOF >$POLICY_FILE # Create the policy file
module allow_logreader_to_watch_logs 1.0;

require {
    class file { watch watch_reads };
    class dir { watch };
    type container_logreader_t;
    type container_log_t;
};

allow container_logreader_t container_log_t:file { watch watch_reads };
allow container_logreader_t container_log_t:dir { watch };
EOF
checkmodule -M -m -o "$MODULE_FILE" "$POLICY_FILE" # Compile the policy file into a module
semodule_package -m $MODULE_FILE -o $PACKAGE_FILE  # Package the module into a policy package
sudo semodule -i $PACKAGE_FILE                     # Install the policy package into SELinux

# Clean up
rm -f $POLICY_FILE $MODULE_FILE $PACKAGE_FILE

# Apply the required label to the deployment
kubectl patch -n production daemonset crowdsec-agent --type='strategic' -p '{
  "spec": {
    "template": {
      "spec": {
        "securityContext": {
          "seLinuxOptions": {
            "type": "container_logreader_t"
          }
        }
      }
    }
  }
}'

[ImranR98]

If it turns out to be impossible to get past the SELinux issues, maybe we could have a way to grab Traefik logs through an existing PVC instead of the chart automatically creating one for /var/log? That way, Traefik could be configured to send the logs to that location, bypassing all these SELinux restrictions on /var/log. I'm assuming such an option does not currently exist since all online docs point to the agent.acquisition.podName method of connecting to Traefik. But maybe the agent.extraVolumeMounts could be used somehow?

[srkoster]

@ImranR98 your issues seems a bit related to me issue with Talos and Crowdsec, see #196 . Do you see the following log entries in your agent container log files just after the startup has been completed? If so, check my issue for a possible solution.

time="2024-11-08T07:41:11Z" level=info msg="Force add watch on /var/log/containers" type=file
time="2024-11-08T07:41:11Z" level=error msg="Could not create watch on directory /var/log/containers : no such file or directory" type=file