Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: update to 3.12 #18078

Merged
merged 18 commits into from
Oct 6, 2024
Merged

python: update to 3.12 #18078

merged 18 commits into from
Oct 6, 2024

Conversation

landfillbaby
Copy link
Member

@landfillbaby landfillbaby commented Oct 2, 2023

@landfillbaby
Copy link
Member Author

idk someone else can make it work

@licy183
Copy link
Member

licy183 commented Oct 3, 2023

We should build a host python for python3.12. Ubuntu 22.04 hasn't packaged it yet. Some other repositories such as ppa/deadsnakes may be an alternative in the future, but it hasn't packaged python3.12 yet.

IMO we had better hold it to 3.11 until some other python libraries support it. Bumping to python3.12 will at least break the build process of numpy because python3.12 remove support for distutils but meson-python (which numpy uses for python3.12) hasn't supported cross-compile.

@romanovj
Copy link

romanovj commented Oct 7, 2023

I think that termux should stick to 3.11and 3.12 could be provided by tur-repo

@TomJo2000
Copy link
Member

I think that termux should stick to 3.11and 3.12 could be provided by tur-repo

The opposite will be the case.
Termux is a rolling release and the main repo hosts the latest versions of packages.
Older versions are only provided in special cases such as LTS releases, or on the TUR.

@showierdata9978
Copy link

atleast allow people to install it as python3.12

@Randrianasulu
Copy link

installing CI artefacts resulted in pip setup error:

~ $ dpkg -i debs/python*.deb                                                                       
(Reading database ... 125977 files and directories currently installed.)                           
Preparing to unpack .../python-ensurepip-wheels_3.12.0_all.deb ...                               
  Unpacking python-ensurepip-wheels (3.12.0) over (3.12.0) ...                                       Preparing to unpack .../python-numpy-static_1.26.1-1_aarch64.deb ...                              
 Unpacking python-numpy-static (1.26.1-1) over (1.26.1-1) ...                                       Preparing to unpack .../python-numpy_1.26.1-1_aarch64.deb ...                                      Unpacking python-numpy (1.26.1-1) over (1.26.1-1) ...                                              Preparing to unpack .../python-pillow_10.1.0-1_aarch64.deb ...                                     Unpacking python-pillow (10.1.0-1) over (10.1.0-1) ...                                             Preparing to unpack debs/python-pip_23.3-1_all.deb ...                                             Unpacking python-pip (23.3-1) over (23.3-1) ...                                                    Preparing to unpack .../python-static_3.12.0_aarch64.deb ...                                       Unpacking python-static (3.12.0) over (3.12.0) ...                                                 Preparing to unpack .../python-tkinter_3.12.0_aarch64.deb ...                                      Unpacking python-tkinter (3.12.0) over (3.12.0) ...                                                Preparing to unpack debs/python_3.12.0_aarch64.deb ...                                             Unpacking python (3.12.0) over (3.12.0) ...                                                        Setting up python (3.12.0) ...                                                                     Setting up python-ensurepip-wheels (3.12.0) ...                                                    Setting up python-numpy (1.26.1-1) ...                                                             
Setting up python-pillow (10.1.0-1) ...                                                            
Setting up python-pip (23.3-1) ...                                                                 
pip setup...                                                                                       
Traceback (most recent call last):                                                                   File "/data/data/com.termux/files/usr/bin/pip", line 5, in <module>                                  
from pip._internal.cli.main import main                                                          File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/pip/_internal/cli/main.py", line 10, in <module>                                                                                     from pip._internal.cli.autocompletion import autocomplete                                        File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>                                                                          
 from pip._internal.cli.main_parser import create_main_parser                                     File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/pip/_internal/cli/main_parser.py", line 5, in <module>                                                                               import subprocess                                                                                File "/data/data/com.termux/files/usr/lib/python3.12/subprocess.py", line 104, in <module>        
   from _posixsubprocess import fork_exec as _fork_exec                                           
ImportError: dlopen failed: cannot locate symbol "_Py_NoneStruct" referenced by "/data/data/com.termux/files/usr/lib/python3.12/lib-dynload/_posixsubprocess.cpython-312.so"...                     
  Setting up python-static (3.12.0) ...                                                              
Setting up python-tkinter (3.12.0) ...                                                             
Setting up python-numpy-static (1.26.1-1) ...                                                      
Processing triggers for man (1.14.6-1) ...

@sylirre
Copy link
Member

sylirre commented Dec 19, 2023

@Randrianasulu Using wildcards with dpkg won't result in proper installation order of dependencies.

@Randrianasulu
Copy link

@Randrianasulu Using wildcards with dpkg won't result in proper installation order of dependencies.

well, I tried to reinstall with
pkg reinstall ~/debs/python-ensurepip-wheels_3.12.0_all.deb

and then.python and python-static, and then again ensurepip and python-pip still giving me same error ....

@sylirre
Copy link
Member

sylirre commented Dec 19, 2023

pkg install ./*.deb is more correct variant as it would pass debs to apt and it would handle correct installation order.

However yes, it looks like python was built with issues.

Importing subprocess module results in error:

~/debs $ python3
Python 3.12.0 (main, Oct 17 2023, 08:53:28) [Clang 14.0.7 (https://android.googlesource.com/toolchain/llvm-project 4c603efb0 on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/data/com.termux/files/usr/lib/python3.12/subprocess.py", line 104, in <module>
    from _posixsubprocess import fork_exec as _fork_exec
ImportError: dlopen failed: cannot locate symbol "_Py_NoneStruct" referenced by "/data/data/com.termux/files/usr/lib/python3.12/lib-dynload/_posixsubprocess.cpython-312.so"...
>>> 

I guess similar errors may appear with other modules that use native extensions.

Also it looks like Python was built without readline support, i.e. cursor moving by arrows and history navigation doesn't work (raw escape sequences appear when using arrows):

>>> print("test^[[D^[[A^[[C^[[B

Basically this PR is unusable.

@landfillbaby
Copy link
Member Author

landfillbaby commented Dec 19, 2023

yeah i kind of abandoned this since python 3.12 drops so much support. someone else can take over if they want

@yubrajbhoi
Copy link

yubrajbhoi commented Jan 6, 2024

Looks like some old patches were not applied. Here are all the patches: https://github.com/yubrajbhoi/termux-python-12/tree/main/patches

I have tested them with Python 3.12.0 and 3.12.1. Every module seems to be working, including subprocess. Also, AFAIK distutils is still available from setuptools package. Can't we use that?

I used that to build Numpy for Python-3.12.1 using the setup.py script and it worked.

@licy183 licy183 force-pushed the python3.12 branch 2 times, most recently from 5763a69 to cda6787 Compare January 11, 2024 08:57
@licy183
Copy link
Member

licy183 commented Jan 15, 2024

It tests fine on my device.

@landfillbaby @sylirre Could you please test it again?

The setuptools should be hardcoded to 67.6.1 if crossenv wants to be used, even 67.7 can't be used due to some weird errors...

@sylirre
Copy link
Member

sylirre commented Jan 15, 2024

@licy183 Looks good, tested few modules and can't find issues.

@landfillbaby
Copy link
Member Author

landfillbaby commented Jan 25, 2024

did some testing as asked, LGTM ! :) sorry i've not been active recently

@licy183
Copy link
Member

licy183 commented Feb 11, 2024

Next, we should find all the revdeps of python, check whether it supports python3.12 and revbump if needed.

@codenamedpkt

This comment was marked as off-topic.

fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078)
fornwall added a commit that referenced this pull request Oct 6, 2024
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
…dio,vision})

Revbump some python packages after python 3.12 (#18078).
fornwall added a commit that referenced this pull request Oct 6, 2024
Revbump some python packages after python 3.12 (#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Update to numpy 2, and update reverse dependencies:

- matplotlib from 3.9.1 to 3.9.2
- python-contourpy from 1.2.1 to 1.3.0
- python-numpy from 1.26.5 to 2.1.1
- python-onnxruntime from 1.19.0 to 1.19.2
- python-scipy from 1.14.0 to 1.14.1
- python-torch from 2.1.2 to 2.4.1
- python-torchaudio from 2.0.1 to 2.4.1
- python-torchvision from 0.15.1 to `0.19.1
- Revbump: python-pyarrow, inkscape and opencv

Some of these packages does not currently build before this PR. Getting these updated and buildable is a nice step to prepare for [python 3.12](termux#18078).

Fixes termux#21059, termux#21110, termux#21194 and termux#21284.

Co-authored-by: Chongyun Lee <[email protected]>
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078)
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
…dio,vision})

Revbump some python packages after python 3.12 (termux#18078).
nguynkhn pushed a commit to nguynkhn/termux-packages that referenced this pull request Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

python 3.12.0 request