Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove direct pex imports from the codebase #21515

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

krishnan-chandra
Copy link
Contributor

@krishnan-chandra krishnan-chandra commented Oct 10, 2024

Closes #21401. This removes any direct imports of pex in the codebase but retains it as a dependency, because certain tests depend on it.

@krishnan-chandra krishnan-chandra added category:bugfix Bug fixes for released features backend: Python Python backend-related issues labels Oct 10, 2024
Comment on lines +228 to +261
# This method is copied from the pex package, located at pex.variables.Variables._get_kv().
# It is copied here to avoid a hard dependency on pex.
def _get_kv(variable: str) -> Optional[list[str]]:
kv = variable.strip().split("=")
if len(list(filter(None, kv))) == 2:
return kv
else:
return None


# This method is copied from the pex package, located at pex.variables.Variables.from_rc().
# It is copied here to avoid a hard dependency on pex.
def _read_pex_rc(rc: Optional[str] = None) -> dict[str, str]:
"""Read pex runtime configuration variables from a pexrc file.

:param rc: an absolute path to a pexrc file.
:return: A dict of key value pairs found in processed pexrc files.
"""
ret_vars = {}
rc_locations = [
os.path.join(os.sep, "etc", "pexrc"),
os.path.join("~", ".pexrc"),
os.path.join(os.path.dirname(sys.argv[0]), ".pexrc"),
]
if rc:
rc_locations.append(rc)
for filename in rc_locations:
try:
with open(os.path.expanduser(filename)) as fh:
rc_items = map(_get_kv, fh)
ret_vars.update(dict(filter(None, rc_items)))
except OSError:
continue
return ret_vars
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This involves reading the pexrc file, which kind of explains why we were directly importing from pex before. I'm not sure if it's worth keeping a hard dependency just for this, but maybe it is in case that logic changes in the future.

@@ -143,7 +141,7 @@ def provide_chroot(existing):

# Default to resolving with whatever we're currently running with.
interpreter_constraints = (
InterpreterConstraints([f"=={interpreter.identity.version_str}"]) if interpreter else None
InterpreterConstraints([f"=={python_version}.*"]) if python_version else None
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note - this does subtly change because we are no longer using Pex to hard-pin the patch version of Python in this constraint; we only pin the minor version. That could potentially be a problem if this behavior is particularly material to the test.

@krishnan-chandra krishnan-chandra added category:internal CI, fixes for not-yet-released features, etc. and removed category:bugfix Bug fixes for released features labels Oct 14, 2024
@krishnan-chandra
Copy link
Contributor Author

Fixed the failing test - we can't remove pex as a dependency completely, as there are tests that rely on pex being present as an executable within the environment. But this should have cut out any direct imports, at least.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend: Python Python backend-related issues category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

excise direct Pex imports
1 participant