You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Description urllib.parse.urlunparse() may or may not expand file schema with an empty netloc to file:/// or file: due to some contention around schema-based handling for security reasons. Apparently the implemented behaviour changed between python 3.12.5 and 3.12.7 (I've not tried to reproduce with 3.12.6)
This inconsistency breaks salt.utils.url.create() which assumes it would always unparse to file:///path.
(More in Additional context)
Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
Please be as specific as possible and give set-up details.
on-prem machine
VM (Virtualbox, KVM, etc. please specify)
VM running on a cloud service, please be explicit and add details
container (Kubernetes, Docker, containerd, etc. please specify)
or a combination, please be explicit
jails if it is FreeBSD
classic packaging
onedir packaging
used bootstrap to install
Steps to Reproduce the behavior
Run salt-call state.show_sls '*' with any populated state tree with salt installed under python 3.12.7.
States will fail due to errors like:
- Specified SLS top in saltenv base is not available on the salt master or through
a configured fileserver
2024-10-08 10:40:20,154 (salt.fileclient) DEBUG: Could not find file 'salt://s' in saltenv 'base'
Expected behavior state.show_sls '*' should work.
Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
Salt Version:
Salt: 3007.1Python Version:
Python: 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]Dependency Versions:
cffi: 1.16.0cherrypy: 18.8.0dateutil: 2.8.2docker-py: Not Installedgitdb: 4.0.10gitpython: 3.1.43Jinja2: 3.1.4libgit2: 1.7.2looseversion: 1.3.0M2Crypto: Not InstalledMako: Not Installedmsgpack: 1.0.7msgpack-pure: Not Installedmysql-python: Not Installedpackaging: 23.1pycparser: 2.21pycrypto: Not Installedpycryptodome: 3.19.1pygit2: 1.14.1python-gnupg: 0.5.2PyYAML: 6.0.1PyZMQ: 25.1.2relenv: Not Installedsmmap: 5.0.1timelib: Not InstalledTornado: 6.3.3ZMQ: 4.3.4Salt Package Information:
Package Type: Not InstalledSystem Versions:
dist:
locale: utf-8machine: AMD64release: 10system: Windowsversion: 10 10.0.19045 SP0 Multiprocessor Free
Additional context
This is how salt.utils.url.create is currently implemented:
defcreate(path, saltenv=None):
""" join `path` and `saltenv` into a 'salt://' URL. """path=path.replace("\\", "/")
ifsalt.utils.platform.is_windows():
path=salt.utils.path.sanitize_win_path(path)
path=salt.utils.data.decode(path)
query=f"saltenv={saltenv}"ifsaltenvelse""url=salt.utils.data.decode(urlunparse(("file", "", path, "", query, "")))
return"salt://{}".format(url[len("file:///") :])
urlunparse() does not consistently produce the expected file:/// prefix under some versions of python:
PS D:\salt\> py -3.12
Python 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlunparse
>>> urlunparse(("file", "", "path/to/thing", "", "q=v", ""))
'file:path/to/thing?q=v'
PS D:\salt\> py -3.11
Python 3.11.9 (tags/v3.11.9:de54cf5, Apr 2 2024, 10:12:12) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlunparse
>>> urlunparse(("file", "", "path/to/thing", "", "q=v", ""))
'file:///path/to/thing?q=v'
PS D:\salt\> py -3.10
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from urllib.parse import urlunparse
>>> urlunparse(("file", "", "path/to/thing", "", "q=v", ""))
File "<stdin>", line 1
urlunparse(("file", "", "path/to/thing", "", "q=v", ""))
IndentationError: unexpected indent
>>> urlunparse(("file", "", "path/to/thing", "", "q=v", ""))
'file:///path/to/thing?q=v'
I have monkeypatched my local (embedded) salt to workaround this. I've not been able to submit a PR as pre-commits hooks are not working in my local environments, and I'm not willing to spend much more hours digging into this specific problem.
Description
urllib.parse.urlunparse()
may or may not expandfile
schema with an emptynetloc
tofile:///
orfile:
due to some contention around schema-based handling for security reasons. Apparently the implemented behaviour changed between python 3.12.5 and 3.12.7 (I've not tried to reproduce with 3.12.6)This inconsistency breaks
salt.utils.url.create()
which assumes it would always unparse tofile:///path
.(More in Additional context)
Setup
(Please provide relevant configs and/or SLS files (be sure to remove sensitive info. There is no general set-up of Salt.)
Please be as specific as possible and give set-up details.
Steps to Reproduce the behavior
Run
salt-call state.show_sls '*'
with any populated state tree with salt installed under python 3.12.7.States will fail due to errors like:
Increasing
salt.fileclient
verbosity gives errors like:Expected behavior
state.show_sls '*'
should work.Versions Report
salt --versions-report
(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)Additional context
This is how
salt.utils.url.create
is currently implemented:urlunparse()
does not consistently produce the expectedfile:///
prefix under some versions of python:Which ends up producing broken URIs like this:
I have monkeypatched my local (embedded)
salt
to workaround this. I've not been able to submit a PR as pre-commits hooks are not working in my local environments, and I'm not willing to spend much more hours digging into this specific problem.The text was updated successfully, but these errors were encountered: