Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip install git+https://github.com/alkaline-ml/pmdarima.git throws ModuleNotFoundError: No module named 'distutils.msvccompiler' #582

Open
harupy opened this issue Sep 18, 2024 · 5 comments
Open

pip install git+https://github.com/alkaline-ml/pmdarima.git throws ModuleNotFoundError: No module named 'distutils.msvccompiler' #582

harupy opened this issue Sep 18, 2024 · 5 comments

Comments

@harupy
Copy link

harupy commented Sep 18, 2024

This is not a bug. In https://github.com/mlflow/mlflow, we use pip install git+https://github.com/alkaline-ml/pmdarima.git to install the development version of pmdarima and run tests against it. This command started failing with the following error recently:

% docker run --rm python:3.8 pip install git+https://github.com/harupy/pmdarima.git@pin-setuptools
Collecting git+https://github.com/harupy/pmdarima.git@pin-setuptools
  Cloning https://github.com/harupy/pmdarima.git (to revision pin-setuptools) to /tmp/pip-req-build-m7yplexv
  Running command git clone --filter=blob:none --quiet https://github.com/harupy/pmdarima.git /tmp/pip-req-build-m7yplexv
  Running command git checkout -b pin-setuptools --track origin/pin-setuptools
  Switched to a new branch 'pin-setuptools'
  branch 'pin-setuptools' set up to track 'origin/pin-setuptools'.
  Resolved https://github.com/harupy/pmdarima.git to commit 0e207106ce8562a1db57a0ff19b794255d8ae062
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.5 MB/s eta 0:00:00
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.4 MB/s eta 0:00:00
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.5 MB/s eta 0:00:00
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.3 MB/s eta 0:00:00
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 10.3 MB/s eta 0:00:00
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 9.8 MB/s eta 0:00:00
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'done'
  Created wheel for pmdarima: filename=pmdarima-0.0.0-cp38-cp38-linux_aarch64.whl size=2104950 sha256=4d028a38259c984bc32b523d18e8910c5411d8d4a38d63216d083fec5003b6b5
  Stored in directory: /tmp/pip-ephem-wheel-cache-54mjx6wz/wheels/b9/40/4a/88c839cd3c48e189fdc633b304d2bb581af11d55d1f2f7ee37
Successfully built pmdarima
Installing collected packages: pytz, urllib3, tzdata, threadpoolctl, six, packaging, numpy, joblib, Cython, scipy, python-dateutil, patsy, scikit-learn, pandas, statsmodels, pmdarima
Successfully installed Cython-3.0.11 joblib-1.4.2 numpy-1.24.4 packaging-24.1 pandas-2.0.3 patsy-0.5.6 pmdarima-0.0.0 python-dateutil-2.9.0.post0 pytz-2024.2 scikit-learn-1.3.2 scipy-1.10.1 six-1.16.0 statsmodels-0.14.1 threadpoolctl-3.5.0 tzdata-2024.1 urllib3-2.2.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
                                                                                                    
(mlflow) @ mlflow [pmdarima-python-3.9] -- INSERT --
% docker run --rm python:3.8 pip install git+https://github.com/alkaline-ml/pmdarima.git
Collecting git+https://github.com/alkaline-ml/pmdarima.git
  Cloning https://github.com/alkaline-ml/pmdarima.git to /tmp/pip-req-build-0gm1driz
  Running command git clone --filter=blob:none --quiet https://github.com/alkaline-ml/pmdarima.git /tmp/pip-req-build-0gm1driz
  Resolved https://github.com/alkaline-ml/pmdarima.git to commit c33a01768dfaf65cadfa666420e75548b8587011
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 6.7 MB/s eta 0:00:00
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.4 MB/s eta 0:00:00
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.3 MB/s eta 0:00:00
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 10.7 MB/s eta 0:00:00
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error
  
  × Building wheel for pmdarima (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Partial import of pmdarima during the build process.
      Requirements: ['joblib>=0.11', 'Cython>=0.29,!=0.29.18,!=0.29.31', 'numpy>=1.21.2', 'pandas>=0.19', 'scikit-learn>=0.22', 'scipy>=1.3.2', 'statsmodels>=0.13.2', 'urllib3', 'setuptools>=38.6.0,!=50.0.0', 'packaging>=17.1  # Bundled with setuptools, but want to be explicit']
      Adding extra setuptools args
      Traceback (most recent call last):
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 421, in build_wheel
          return self._build_with_temp_dir(
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 503, in run_setup
          super().run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 318, in run_setup
          exec(code, locals())
        File "<string>", line 371, in <module>
        File "<string>", line 360, in do_setup
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 24, in <module>
          from numpy.distutils.command import config, config_compiler, \
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/command/config.py", line 19, in <module>
          from numpy.distutils.mingw32ccompiler import generate_manifest
        File "/tmp/pip-build-env-rfjwdxcl/overlay/lib/python3.8/site-packages/numpy/distutils/mingw32ccompiler.py", line 29, in <module>
          from distutils.msvccompiler import get_build_version as get_build_msvc_version
      ModuleNotFoundError: No module named 'distutils.msvccompiler'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pmdarima
Failed to build pmdarima
ERROR: Could not build wheels for pmdarima, which is required to install pyproject.toml-based projects

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip
@harupy harupy mentioned this issue Sep 18, 2024
Closed
39 tasks
@harupy
Copy link
Author

harupy commented Sep 18, 2024

I'm investigating a fix on https://github.com/harupy/pmdarima/tree/pin-setuptools.

harupy@0e20710 seems to work.

% docker run --rm python:3.8 pip install git+https://github.com/harupy/pmdarima.git@0e207106ce8562a1db57a0ff19b794255d8ae062
Collecting git+https://github.com/harupy/pmdarima.git@0e207106ce8562a1db57a0ff19b794255d8ae062
  Cloning https://github.com/harupy/pmdarima.git (to revision 0e207106ce8562a1db57a0ff19b794255d8ae062) to /tmp/pip-req-build-sr6q8f4e
  Running command git clone --filter=blob:none --quiet https://github.com/harupy/pmdarima.git /tmp/pip-req-build-sr6q8f4e
  Running command git rev-parse -q --verify 'sha^0e207106ce8562a1db57a0ff19b794255d8ae062'
  Running command git fetch -q https://github.com/harupy/pmdarima.git 0e207106ce8562a1db57a0ff19b794255d8ae062
  Running command git checkout -q 0e207106ce8562a1db57a0ff19b794255d8ae062
  Resolved https://github.com/harupy/pmdarima.git to commit 0e207106ce8562a1db57a0ff19b794255d8ae062
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting Cython!=0.29.18,!=0.29.31,>=0.29
  Using cached Cython-3.0.11-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB)
Collecting packaging>=17.1
  Using cached packaging-24.1-py3-none-any.whl (53 kB)
Collecting urllib3
  Downloading urllib3-2.2.3-py3-none-any.whl (126 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 126.3/126.3 kB 4.7 MB/s eta 0:00:00
Collecting numpy>=1.21.2
  Downloading numpy-1.24.4-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (14.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 10.3 MB/s eta 0:00:00
Collecting scikit-learn>=0.22
  Downloading scikit_learn-1.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting joblib>=0.11
  Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 301.8/301.8 kB 10.1 MB/s eta 0:00:00
Requirement already satisfied: setuptools!=50.0.0,>=38.6.0 in /usr/local/lib/python3.8/site-packages (from pmdarima==0.0.0) (57.5.0)
Collecting statsmodels>=0.13.2
  Downloading statsmodels-0.14.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (10.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.5/10.5 MB 10.4 MB/s eta 0:00:00
Collecting pandas>=0.19
  Using cached pandas-2.0.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (11.7 MB)
Collecting scipy>=1.3.2
  Downloading scipy-1.10.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (31.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 31.0/31.0 MB 10.2 MB/s eta 0:00:00
Collecting tzdata>=2022.1
  Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Collecting pytz>=2020.1
  Using cached pytz-2024.2-py2.py3-none-any.whl (508 kB)
Collecting python-dateutil>=2.8.2
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.5.0-py3-none-any.whl (18 kB)
Collecting patsy>=0.5.4
  Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: pmdarima
  Building wheel for pmdarima (pyproject.toml): started
  Building wheel for pmdarima (pyproject.toml): finished with status 'done'
  Created wheel for pmdarima: filename=pmdarima-0.0.0-cp38-cp38-linux_aarch64.whl size=2104962 sha256=130266f6b132ec5a8bc2f66a84f33af41a6a5868e25cf5e7d5bcd7ae2fbc0ef3
  Stored in directory: /root/.cache/pip/wheels/d4/b5/f7/3057ff446df49ded9a2d2898468572a9d244d3be0d27538486
Successfully built pmdarima
Installing collected packages: pytz, urllib3, tzdata, threadpoolctl, six, packaging, numpy, joblib, Cython, scipy, python-dateutil, patsy, scikit-learn, pandas, statsmodels, pmdarima
Successfully installed Cython-3.0.11 joblib-1.4.2 numpy-1.24.4 packaging-24.1 pandas-2.0.3 patsy-0.5.6 pmdarima-0.0.0 python-dateutil-2.9.0.post0 pytz-2024.2 scikit-learn-1.3.2 scipy-1.10.1 six-1.16.0 statsmodels-0.14.1 threadpoolctl-3.5.0 tzdata-2024.1 urllib3-2.2.3
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: pip install --upgrade pip

@harupy
Copy link
Author

harupy commented Sep 18, 2024

Looks like this is an issue in numpy.

@harupy
Copy link
Author

harupy commented Sep 18, 2024

pmdarima/pmdarima/_build_utils/__init__.py

Line 7 in c33a017

from numpy.distutils.system_info import get_info
looks like the culprit.

@harupy
Copy link
Author

harupy commented Sep 18, 2024

Can we pin setuptools?

@harupy
Copy link
Author

harupy commented Oct 2, 2024

Any updates on this issue?

@harupy harupy mentioned this issue Oct 17, 2024
Merged
39 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@harupy