Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate polars warning regarding python fork() #9245

Open
larsevj opened this issue Nov 18, 2024 · 1 comment
Open

Investigate polars warning regarding python fork() #9245

larsevj opened this issue Nov 18, 2024 · 1 comment

Comments

@larsevj
Copy link
Contributor

larsevj commented Nov 18, 2024

When running ert with polars 14, you get the following warning:

/Users/LEVJE/.local/share/uv/python/cpython-3.12.7-macos-aarch64-none/lib/python3.12/asyncio/subprocess.py:224: RuntimeWarning: Using fork() can cause Polars to deadlock in the child process.
In addition, using fork() with Python in general is a recipe for mysterious
deadlocks and crashes.

The most likely reason you are seeing this error is because you are using the
multiprocessing module on Linux, which uses fork() by default. This will be
fixed in Python 3.14. Until then, you want to use the "spawn" context instead.

See https://docs.pola.rs/user-guide/misc/multiprocessing/ for details.

  transport, protocol = await loop.subprocess_exec(

Should investigate if this is at all relevant for ert, and if not silence the warning.

@larsevj larsevj moved this to Todo in SCOUT Nov 18, 2024
@berland
Copy link
Contributor

berland commented Nov 18, 2024

This is probably related to usages of fork() that is not immediately followed by an exec(). If all our forks stem from launching shell subprocesses, this warning is not relevant to Ert. Some background info: https://lisper.in/fork-exec-python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

2 participants