[BUG] Publisher ACL throws Authentication error after upgrade to 3007.1

[bastian-src]

Description
I've been debugging this for a while, coming from #66228. However, the promoted fix does not work for me. It resolves the prior error such that my non-root user can execute the salt command, but I'm getting the following error afterwards:

<DATE> 10:50:38,269 [salt.transport.tcp:306 ][DEBUG   ][767223] PubClient conencted to <salt.transport.tcp.PublishClient object at 0x7f6fe1bc1360> '/var/run/salt/master/master_event_pub.ipc'
<DATE> 10:50:38,270 [salt.transport.tcp:1102][DEBUG   ][767219] Subscriber at  connected
<DATE> 10:50:39,292 [salt.auth        :365 ][WARNING ][767231] Authentication failure of type "user" occurred.
<DATE> 10:50:39,292 [salt.master      :2393][WARNING ][767231] Authentication failure of type "user" occurred.

---

I've debugged the issue and the error message originates in the auth module:
https://github.com/saltstack/salt/blob/master/salt/auth/__init__.py#L364

The key dictionary contains the expected users and corresponding keys. However the auth_key variable (coming from load["key"]) is an empty string (not None).

@dwoz can you maybe help me why this might be the case?

Setup

Contents of /etc/salt/master.d/auth.conf:

publisher_acl:
  me:
    - .*

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD
• classic packaging
• onedir packaging
• used bootstrap to install

Steps to Reproduce the behavior

Follow instructions at https://docs.saltproject.io/salt/user-guide/en/latest/topics/security.html#publisher-acls to set up publisher-acl and directory permissions.

Expected behavior

Non-root user can start jobs such as test.ping and permissions to do so are not reset when (re)starting the salt-master service. Was working on 3006.3 before upgrade to 3007.1.

Versions Report

salt --versions-report

Salt Version:
          Salt: 3007.1
 
Python Version:
        Python: 3.10.14 (main, Apr  3 2024, 21:30:09) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.4
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.16.0
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: almalinux 8.10 Cerulean Leopard
        locale: utf-8
       machine: x86_64
       release: 4.18.0-553.22.1.el8_10.x86_64
        system: Linux
       version: AlmaLinux 8.10 Cerulean Leopard
 

[bastian-src]

Resolved after running:

rm /var/cache/salt/master/.*key

as documented here: https://docs.saltproject.io/en/latest/ref/publisheracl.html#permission-issues