RPM package zeros out log file on upgrade

[PeterS242]

I believe the /var/log/salt/master and /var/log/salt/minion files are likely included in the RPM package(s) as a zero-length file and thus wipe out an already-existing log file on install/upgrade.

I upgraded from 2016.11.6 to 2017.7.0 just moments ago and saw this during the upgrade:

[root@linux ~]# la /var/log/salt
total 26968
drwxr-xr-x.  2 root root    4096 Jul 17 03:35 .
drwxr-xr-x. 19 root root    8192 Jul 17 03:35 ..
-rw-r--r--.  1 root root       0 Jun 22 16:22 master
-rw-r--r--.  1 root root 2162596 Jul 19 14:04 minion
-rw-r--r--.  1 root root 2693188 Jun  4 03:20 minion-20170604.gz
-rw-r--r--.  1 root root 2502729 Jun 12 03:20 minion-20170612.gz
-rw-r--r--.  1 root root 1853171 Jun 18 03:40 minion-20170618.gz
-rw-r--r--.  1 root root 2787595 Jun 26 03:00 minion-20170626.gz
-rw-r--r--.  1 root root  651254 Jul  2 03:00 minion-20170702.gz
-rw-r--r--.  1 root root 2316647 Jul  9 03:40 minion-20170709.gz
-rw-r--r--.  1 root root 2959578 Jul 17 03:20 minion-20170717.gz
-rw-r--r--.  1 root root 9188383 Mar  9 17:44 minion.rpmsave
[root@linux ~]# la /var/log/salt
total 24404
drwxr-xr-x.  2 root root    4096 Jul 19 14:04 .
drwxr-xr-x. 19 root root    8192 Jul 17 03:35 ..
-rw-r--r--.  1 root root       0 Jul 12 18:07 master
-rw-r--r--.  1 root root       0 Jul 12 18:07 minion
-rw-r--r--.  1 root root 2693188 Jun  4 03:20 minion-20170604.gz
-rw-r--r--.  1 root root 2502729 Jun 12 03:20 minion-20170612.gz
-rw-r--r--.  1 root root 1853171 Jun 18 03:40 minion-20170618.gz
-rw-r--r--.  1 root root 2787595 Jun 26 03:00 minion-20170626.gz
-rw-r--r--.  1 root root  651254 Jul  2 03:00 minion-20170702.gz
-rw-r--r--.  1 root root 2316647 Jul  9 03:40 minion-20170709.gz
-rw-r--r--.  1 root root 2959578 Jul 17 03:20 minion-20170717.gz
-rw-r--r--.  1 root root 9188383 Mar  9 17:44 minion.rpmsave
[root@linux ~]# 

I would expect it to act like other packages and not wipe out the log file(s). I think this should be done with specific flags in the RPM spec-file rather than packaging a zero-length file. As it is, we lose any logged entries prior to the install/update.

[PeterS242]

I think that maybe the assumption being made is that RPM needs to show that the log file is owned by the package when I don't believe this is a requirement. On my system I actually show a few that have ownership but I really think that is an exception. See attachment, output.txt.

output.txt

[PeterS242]

Upgrade from 2017.7.0 to 2017.7.1 still zero-ing out log files. The following is the /var/log/salt/ directory before the upgrade.

[root@linux salt]# la
total 1012092
total 1810376
drwxr-xr-x.  2 root root      4096 Aug 13 03:42 .
drwxr-xr-x. 13 root root     12288 Aug 13 03:42 ..
-rw-------.  1 root root      1119 Aug 16 13:54 key
-rw-------.  1 root root      1216 Jun 27 03:22 key-20170627.gz
-rw-------.  1 root root      2119 Jul 11 03:10 key-20170711.gz
-rw-------.  1 root root      2090 Jul 16 03:08 key-20170716.gz
-rw-------.  1 root root      2137 Jul 23 03:29 key-20170723.gz
-rw-------.  1 root root      2517 Jul 30 03:15 key-20170730.gz
-rw-------.  1 root root       211 Aug  6 03:22 key-20170806.gz
-rw-------.  1 root root      1839 Aug 13 03:42 key-20170813.gz
-rw-------.  1 root root 816819428 Aug 16 13:56 master
-rw-------.  1 root root  20416750 Jul  2 03:39 master-20170702.gz
-rw-------.  1 root root  82743064 Jul  9 04:39 master-20170709.gz
-rw-------.  1 root root 214730291 Jul 16 03:08 master-20170716.gz
-rw-------.  1 root root 238289524 Jul 23 03:29 master-20170723.gz
-rw-------.  1 root root 128091104 Jul 30 03:15 master-20170730.gz
-rw-------.  1 root root 114160755 Aug  6 03:22 master-20170806.gz
-rw-------.  1 root root 112644662 Aug 13 03:42 master-20170813.gz
-rw-------.  1 root root    656412 Aug 16 13:54 minion
-rw-------.  1 root root    667463 Jul  2 03:39 minion-20170702.gz
-rw-------.  1 root root   2306202 Jul  9 04:39 minion-20170709.gz
-rw-------.  1 root root   2589642 Jul 16 03:08 minion-20170716.gz
-rw-------.  1 root root   1787214 Jul 23 03:29 minion-20170723.gz
-rw-------.  1 root root   2346552 Jul 30 03:15 minion-20170730.gz
-rw-------.  1 root root    540291 Aug  6 03:22 minion-20170806.gz
-rw-------.  1 root root    552040 Aug 13 03:42 minion-20170813.gz
-rw-------.  1 root root 114378847 Aug  7 17:43 ssh
[root@linux salt]# 

The following is the /var/log/salt/ directory after the upgrade.

[root@linux salt]# la
total 1012092
drwxr-xr-x.  2 root root      4096 Aug 16 13:56 .
drwxr-xr-x. 13 root root     12288 Aug 13 03:42 ..
-rw-------.  1 root root      1119 Aug 16 13:54 key
-rw-------.  1 root root      1216 Jun 27 03:22 key-20170627.gz
-rw-------.  1 root root      2119 Jul 11 03:10 key-20170711.gz
-rw-------.  1 root root      2090 Jul 16 03:08 key-20170716.gz
-rw-------.  1 root root      2137 Jul 23 03:29 key-20170723.gz
-rw-------.  1 root root      2517 Jul 30 03:15 key-20170730.gz
-rw-------.  1 root root       211 Aug  6 03:22 key-20170806.gz
-rw-------.  1 root root      1839 Aug 13 03:42 key-20170813.gz
-rw-r--r--.  1 root root     43073 Aug 16 13:56 master
-rw-------.  1 root root  20416750 Jul  2 03:39 master-20170702.gz
-rw-------.  1 root root  82743064 Jul  9 04:39 master-20170709.gz
-rw-------.  1 root root 214730291 Jul 16 03:08 master-20170716.gz
-rw-------.  1 root root 238289524 Jul 23 03:29 master-20170723.gz
-rw-------.  1 root root 128091104 Jul 30 03:15 master-20170730.gz
-rw-------.  1 root root 114160755 Aug  6 03:22 master-20170806.gz
-rw-------.  1 root root 112644662 Aug 13 03:42 master-20170813.gz
-rw-r--r--.  1 root root         0 Aug  7 14:51 minion
-rw-------.  1 root root    667463 Jul  2 03:39 minion-20170702.gz
-rw-------.  1 root root   2306202 Jul  9 04:39 minion-20170709.gz
-rw-------.  1 root root   2589642 Jul 16 03:08 minion-20170716.gz
-rw-------.  1 root root   1787214 Jul 23 03:29 minion-20170723.gz
-rw-------.  1 root root   2346552 Jul 30 03:15 minion-20170730.gz
-rw-------.  1 root root    540291 Aug  6 03:22 minion-20170806.gz
-rw-------.  1 root root    552040 Aug 13 03:42 minion-20170813.gz
-rw-------.  1 root root 114378847 Aug  7 17:43 ssh
[root@linux salt]# 

Note that the logs master and minion are now either zero length or have been reset from zero-on.

[PeterS242]

This is still an issue. I just now downgraded my Master from 3000.3.1 to 3000.2.1 (trying to work around another issue, saltstack/salt#57750) and my /var/log/salt/master file was zero'ed out as part of the process.

It is normal for RPMs to take ownership of subdirectories that are made in the system, like /var/log/salt/ for instance, however it is not normal (or necessary) for an RPM to take ownership of individual log files.

As an example (Red Hat Apache RPM:)

[root@linux ~]# rpm -qls httpd | egrep log
normal        /etc/httpd/logs
normal        /etc/logrotate.d/httpd
normal        /usr/lib64/httpd/modules/mod_log_config.so
normal        /usr/lib64/httpd/modules/mod_log_forensic.so
normal        /usr/lib64/httpd/modules/mod_logio.so
normal        /usr/sbin/rotatelogs
normal        /usr/share/man/man8/rotatelogs.8.gz
normal        /var/log/httpd
[root@linux ~]#

I would highly recommend removing the following lines from the SPEC files.

touch %{buildroot}%{_var}/log/salt/minion
touch %{buildroot}%{_var}/log/salt/master

This is causing RPM to consider these files to be created by the RPM packaging process rather than the Salt Master or Minion apps and in this way produces zero-byte files which wipe out the previous files on the filesystem when upgrading.

[PeterS242]

Actually after digging through the documentation for SPEC files, if you still want to have explicit entries in the RPM for these log files, I believe it would probably be more appropriate to move each individual log file entry, /var/log/salt/minion and /var/log/salt/master, into the %files section of each of the respective RPMs, but use the %ghost tag.

This is described here https://docs.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html-single/RPM_Guide/index.html

Another special modifier, %ghost, tells the rpm command that the file should not be included in the package. You can use this to name the needed attributes for a file that the program, when installed, will create. For example, you may want to ensure that a program’s log file has certain attributes.

The MAN page for RPM specifies this similarly as shown here https://linux.die.net/man/8/rpm (note the description of GHOST)

The format of the output is a string of 8 characters, a possible attribute marker:

c %config configuration file.
d %doc documentation file.
g %ghost file (i.e. the file contents are not included in the package payload).
l %license license file.
r %readme readme file.

I believe %ghost would mark the log file as being owned by the package as SaltStack appears to want, but the contents of the log file is not used (the zero byte file.)

I guess, time permitting, I will try to patch this SPEC and generate a pull request...

[bryceml]

when you make the pr, make it on the salt-pack-py3 repo in the master folder too. This repo is for python 2 and other than CVE releases, all new builds will be python3 only. 3001 was python3 only for example.