Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-pthread not set in cross-compiled python gcc flags #36

Open
virtuald opened this issue Oct 3, 2020 · 2 comments
Open

-pthread not set in cross-compiled python gcc flags #36

virtuald opened this issue Oct 3, 2020 · 2 comments

Comments

@virtuald
Copy link
Contributor

virtuald commented Oct 3, 2020

Problem

Without -pthread set when compiling, bad things happen, such as this:

ImportError: /usr/local/lib/python3.8/site-packages/scipy/fft/_pocketfft/pypocketfft.cpython-38-arm-linux-gnueabi.so: undefined symbol: pthread_atfork

On my native python (Fedora 32, Python 3.8):

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'gcc -pthread',
 'CXX': 'g++ -pthread',
 'LDSHARED': 'gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now  '
             '-g  -Wl,-z,relro -Wl,--as-needed  -Wl,-z,now  -g'}

However in the cross-python:

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc',
 'CXX': 'arm-frc2020-linux-gnueabi-c++',
 'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -shared'}

Potential fix

Unfortunately, python's configure script runs a program to determine if -pthread is needed. However, there's a partial fix for this. If one adds ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes to the cross ./configure argument, you get:

>>> import pprint, distutils.sysconfig
>>> pprint.pprint({i: distutils.sysconfig.get_config_vars(i)[0] for i in ('CC', 'CXX', 'LDSHARED')})
{'CC': 'arm-frc2020-linux-gnueabi-gcc -pthread',
 'CXX': 'arm-frc2020-linux-gnueabi-c++',
 'LDSHARED': 'arm-frc2020-linux-gnueabi-gcc -pthread -shared'}

It seems like there isn't a way to override the CXX version without changing configure.ac (unless you have a good idea), filed an issue against CPython @ https://bugs.python.org/issue41916

Action for crossenv

Probably all/some of this discussion should be integrated into the cross-compilation documentation? Not sure how you'd like that to be done.

@benfogle
Copy link
Owner

This sounds like a toolchain/cpython issue more than a crossenv issue?

When creating the cross environment, would adding --cxx='arm-frc2020-linux-gnueabi-c++ -pthread' be a workaround? If that works then that should be better documented.

@virtuald
Copy link
Contributor Author

Sorry for the delayed response! I haven't tried that workaround.

Instead, I ended up patching configure.ac/configure and that seems to have fixed the issue (PR @ python/cpython#22525). With that patch applied, setting the configure cache values ac_cv_pthread_is_default=no ac_cv_pthread=yes ac_cv_cxx_thread=yes fixes the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants