Create more directories in /var in base image

[cgwalters]

Following ostreedev/ostree@f81b9fa we now have improved semantics for including content in /var in the container image by default.

There is still an inherent tension between this and systemd-tmpfiles.

However, we already have a few things in the image like /var/tmp because without it too many things fail.

Let's go ahead and extend that set to at least:

• /var/roothome
• /var/home
• /var/log/journal

[cgwalters]

OK so the failure in tests appears to be due to:

    [    5.541009] audit: type=1400 audit(1709059381.739:4): avc:  denied  { create } for  pid=512 comm="systemd-random-" name="random-seed" scontext=system_u:system_r:init_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=0

I'm not reproducing this booting our stock image (using podman-bootc run).

Notably this test is using bootc install to-filesystem --replace-mode=alongside though...that could be relevant...I wonder if security context leakage from the initial install is bubbling up here.

[shi2wei3]

Using bootc install to-disk lead to the same error, but this won't happen on anaconda

sudo podman run --rm --privileged --pid=host --security-opt label=type:unconfined_t --tls-verify=false \
                $image \
                bootc install to-disk --skip-fetch-check --karg=console=ttyS0,115200n8 "$dev"

From the test result for centos-bootc on RHTAP, I can see this problem is fixed after #357 got merged.

[cgwalters]

@shi2wei3 Are we still seeing this then?

[shi2wei3]

Upstream and downstream rhtap pipeline is broken recently, i'll report back as soon as it fixed. What I can tell is the centos-bootc github CI test result is PASS.

[shi2wei3]

@cgwalters I can confirm this issue still exists on bootc base image. CI job doesn't impact because it doesn't use bootc base image, it add RUN dnf install cloud-init on top of the base image, although I didn't find how it affect the creation of /var/log/journal, but the bootc installed disk image do have /var/log/journal on first boot. So this issue is still reproducible with bootc base image.

[shi2wei3]

This issue is created in related to the issue of /var/log/journal is not created on firstboot, so I leave the followup comment here, but if we found it's a separated issue, feel free to create a new issue.

Recently I found the journal log issue only happen on base images (rpm-ostree compose image) without any layering modifications.
That explain why it only happen on centos-bootc and rhel-bootc, not happen on centos-bootc-dev, because centos-bootc-dev is a derived image on top of centos-bootc.
What I found is any derived images with RUN command with dnf or systemctl would workaround this problem.

[cgwalters]

For the journal
https://issues.redhat.com/browse/RHEL-29331

What I found is any derived images with RUN command with dnf or systemctl would workaround this problem.

Right...it's because dnf writes to /var/log by default and creates that directory.

[cgwalters]

But yes we should definitely just pre-create these directories. The problem is that rpm-ostree is really too opinionated here...tempting to just add a "really mutate anything" equivalent of RUN.

[cgwalters]

I guess we need to add /var/run -> /run by default, as #140 (comment) came up again with NetworkManager apparently. Ah wait, it's not NM but it's one of the deps. A quick hack on my workstation:

$ rpm -qa | (while read r; do if rpm -ql $r |grep -q /var/run; then echo $r; fi; done)
filesystem-3.18-6.fc39.x86_64
avahi-0.8-24.fc39.x86_64
pam-1.5.3-3.fc39.x86_64
pcsc-lite-2.0.1-1.fc39.x86_64
pptp-1.10.0-17.fc39.x86_64
system-config-printer-udev-1.5.18-5.fc39.x86_64
audit-3.1.2-8.fc39.x86_64
spice-vdagent-0.22.1-4.fc39.x86_64

And I bet it was pptp for you or so.

[cgwalters]

So fixing this is unfortunately annoying because rpm-ostree is extremely opinionated, so...we could do coreos/rpm-ostree#4882 or just try to wedge in a container layer here too.

[ggiguash]

I guess we need to add /var/run -> /run by default, as #140 (comment) came up again with NetworkManager apparently. Ah wait, it's not NM but it's one of the deps. A quick hack on my workstation:

$ rpm -qa | (while read r; do if rpm -ql $r |grep -q /var/run; then echo $r; fi; done)
filesystem-3.18-6.fc39.x86_64
avahi-0.8-24.fc39.x86_64
pam-1.5.3-3.fc39.x86_64
pcsc-lite-2.0.1-1.fc39.x86_64
pptp-1.10.0-17.fc39.x86_64
system-config-printer-udev-1.5.18-5.fc39.x86_64
audit-3.1.2-8.fc39.x86_64
spice-vdagent-0.22.1-4.fc39.x86_64

And I bet it was pptp for you or so.

This is a sublist of packages using /var/run directly. There is another set of issues because of the packages using /run and /var/run interchangeably. NetworkManager is one of those packages in conjunction with DBus.
See containers/bootc#410.
Bottom line is that it would be much easier if we had all these /var symlinks in the base image by default.