Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] dymamic crossenv + python3*-wheels + python310-313 updates #6282

Open
wants to merge 114 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
2e34e11
python311: Remove optional wheels (to be migrated to its own spk)
th0ma7 Oct 14, 2024
c0a147d
python313: Initial commit, fails to build on readline
th0ma7 Oct 14, 2024
f6d064e
python-wheels: First attempt, to be further developed
th0ma7 Oct 14, 2024
1c414e6
py311: Re-enable lto
th0ma7 Oct 14, 2024
9026ffd
python313: First working build
th0ma7 Oct 14, 2024
b4088e7
python313: Remove pgo optimizations as non-functional in cross-builds
th0ma7 Oct 15, 2024
56df365
python-wheels: First integration of additional wheels for testing
th0ma7 Oct 15, 2024
228c277
mariadb-connector-c: Update from version 3.3.5 to 3.3.11 and fix buil…
th0ma7 Oct 15, 2024
26ed69e
python313: No longer support armv5 arch due to c++11 support needed
th0ma7 Oct 16, 2024
2a7c397
spksrc.crossenv.mk: First iteration of on-demand crossenv creation
th0ma7 Oct 30, 2024
2d71fc9
spksrc.crossenv.mk: Print crossenv resulting package list
th0ma7 Oct 30, 2024
69d9376
py310-311-313: Remove crossenv creation
th0ma7 Nov 2, 2024
f0b3025
mk/crossenv: Remove aiohttp example as obsolete
th0ma7 Nov 2, 2024
866b550
mk/crossenv: disable meson and scikit-build-core temporarily
th0ma7 Nov 2, 2024
7fbc3de
spksrc.crossenv.mk: Misc fixes
th0ma7 Nov 2, 2024
8c6a914
python-module.mk: Make compatible with spksrc.crossenv.mk
th0ma7 Nov 2, 2024
68f335f
wheel.mk: Make compatible with spksrc.crossenv.mk
th0ma7 Nov 2, 2024
e7fb214
crossenv.mk: Allow building from PYTHON_PACKAGE dependend spk
th0ma7 Nov 4, 2024
a4e3a21
py310-311: Fix lto vs pgo optimizations
th0ma7 Nov 5, 2024
7933961
borgbackup: Package bump to force build
th0ma7 Nov 5, 2024
ff60f1f
wheel.mk: Use $name-$version using - instead of == for crossenv
th0ma7 Nov 5, 2024
1960dee
python.mk: do not symlink crossenv and python-cc.mk
th0ma7 Nov 5, 2024
65f5c37
python.mk: Move mandatory PYO3_* variables for rust cross-compiling
th0ma7 Nov 11, 2024
e6f9edf
crossenv.mk: Fix crossenv creation and pip tool calls
th0ma7 Nov 11, 2024
2e3d093
crossenv/requirements-default.txt: Update to latest wheel versions
th0ma7 Nov 13, 2024
dad5dac
spk.mk: Add a crossenvclean clean-up option
th0ma7 Nov 13, 2024
4793833
python.mk + crossenv.mk: Functional versions
th0ma7 Nov 14, 2024
b2a1e37
python.mk: Further clean-up and removal of duplicate variables
th0ma7 Nov 14, 2024
cd7da96
py311-312-312: Update pure requirements to latest versions
th0ma7 Nov 14, 2024
918b69f
py310-311-313: Further clean-up Makefile from unused variables
th0ma7 Nov 14, 2024
f9550f1
py310-311: No longer need to remove test folder
th0ma7 Nov 14, 2024
843f168
py313: Fix symlink creation
th0ma7 Nov 14, 2024
ca55696
crossenv.mk: Include missing OPENSSL_*_DIR
th0ma7 Nov 14, 2024
d680421
spk.mk: Update crossenvclean and add crossenvcleanall
th0ma7 Nov 14, 2024
91ab834
spk.mk: crossenvclean remove cookie file
th0ma7 Nov 15, 2024
6fbb394
crossenv: Fix python version and use status cookie
th0ma7 Nov 15, 2024
4d4cf2c
crossenv-default: Enforce using msgpack==1.0.5 for py313 builds
th0ma7 Nov 15, 2024
8802e36
fix crossenv installation by packages using prebuilt python
hgy59 Nov 16, 2024
41030d0
fix python311 for qoriq
hgy59 Nov 17, 2024
116d4eb
spksrc.python-wheel.mk: fix include of python-cc.mk
hgy59 Nov 18, 2024
9ed71e1
rdiff-backup: add crossenv for frozenlist
hgy59 Nov 18, 2024
2bcf2d1
rdiff-backup: add crossenv for yarl
hgy59 Nov 18, 2024
5e83ec7
fix for version less crossenv requirements
hgy59 Nov 18, 2024
ec63c6c
python.mk: revert support for ADDITIONAL_CROSSENV_WHEELS
hgy59 Nov 20, 2024
5c1b243
Revert "borgbackup: Package bump to force build"
hgy59 Nov 20, 2024
7bea4a5
use python -m build to create wheels with python-wheel.mk
hgy59 Nov 20, 2024
a473d10
ffsync: fix build of cross/cryptography
hgy59 Nov 23, 2024
8a052a7
spksrc.python-wheel.mk: Fix wheel building
th0ma7 Nov 24, 2024
9b650db
DTLSSocket: Update from 0.1.16 to 0.2.2
th0ma7 Nov 24, 2024
35d0f4b
homeassistant: Fix building bcrupt and cryptography
th0ma7 Nov 24, 2024
ec24c01
spksrc.crossenv.mk: Ensure Python=* has hostpython to avoid errors
th0ma7 Nov 24, 2024
b595a7d
crossenv/requirements-numpy-1.24.4.txt: Definition for numpy
th0ma7 Nov 24, 2024
0276404
python.mk: Remove zlib at source to avoir re-building
th0ma7 Nov 24, 2024
dd306d8
dtlssocket: Add -std=c99 for gcc < 5
th0ma7 Nov 24, 2024
1825e05
python.mk: Erase only $(ARCH)-$(TCVERSION) related files
th0ma7 Nov 25, 2024
f3f4dc3
deluge: Update wheels and fix build
th0ma7 Nov 25, 2024
89dcd4c
boost_1.82: Allow builds from python.mk
th0ma7 Nov 25, 2024
d3e67a7
cross/cryptography: crossenv already has cffi and pycparser
th0ma7 Nov 25, 2024
77f44c0
crossenv.mk: Add debug traces to find issue on gh
th0ma7 Nov 25, 2024
6f45c95
crossenv.mk: Testing alternative method to get python version dir
th0ma7 Nov 25, 2024
94c6249
crossenv.mk: Send msg to status file
th0ma7 Nov 25, 2024
d7edc67
wheel.mk: Remove trailing # comment using $${wheel%% *}
th0ma7 Nov 26, 2024
708218a
crossenv.mk: Use $(WORK_DIR)/pip caching dir for crossenv creation
th0ma7 Nov 26, 2024
3b29da6
python-wheels: include wheels in package
hgy59 Nov 26, 2024
2b552da
update github build action
hgy59 Nov 26, 2024
c261b65
adjust variable GH_DEPENDENCY_FOLDERS
hgy59 Nov 26, 2024
2f537de
real fix for #6284
hgy59 Nov 26, 2024
194e2c1
fix github prepare script
hgy59 Nov 26, 2024
1431ce3
crossenv.mk: Miscelanious cleanups
th0ma7 Nov 26, 2024
c56d379
mariadb-connector-c: Fix compilation for some archs
th0ma7 Nov 27, 2024
403cd2e
fix package name substitution in prepare.sh
hgy59 Nov 27, 2024
349858c
Update mk/spksrc.crossenv.mk - typo
th0ma7 Nov 27, 2024
9e204f1
python311-wheels: Rename to reference exact matching python version
th0ma7 Nov 27, 2024
eb4ae3c
native/python313: Sync wheels with default crossenv versions
th0ma7 Nov 27, 2024
475c83a
python312: Initial package
th0ma7 Nov 27, 2024
10dec2b
mariadb-connector-c: Fix build using spksrc.python.mk
th0ma7 Nov 27, 2024
646df32
prepare.sh: keep python311-wheels to build
hgy59 Nov 27, 2024
15eb382
fix python312/PLIST
hgy59 Nov 27, 2024
eb79654
cleanup python
hgy59 Nov 28, 2024
02c60b6
fix INFO file generation
hgy59 Nov 29, 2024
38f9eed
python packages: further cleanup
hgy59 Nov 29, 2024
e46e087
wheel-env.mk: Bump default abi3 to match py311 with cp37
th0ma7 Dec 1, 2024
61f9213
crossenv.mk: Reduce total amount of makefile variable definitions
th0ma7 Dec 1, 2024
0fe7b1c
crossenv.mk: Empty MAKEFLAGS= variable when calling crossenv recipe
th0ma7 Dec 1, 2024
e57b178
crossenv.mk: Fix missing variable definition
th0ma7 Dec 1, 2024
d13479a
tc.mk: Enhance meson toolchain file declarations
th0ma7 Dec 2, 2024
5a0e6df
wheel.mk: Add meson wheel compiling ability
th0ma7 Dec 2, 2024
00e030e
python311-wheels: Add new compatible numpy versions
th0ma7 Dec 2, 2024
fd06e19
python312-wheels: Wheel testing package against py312
th0ma7 Dec 2, 2024
6c27459
Update spk/python312-wheels/Makefile
th0ma7 Dec 2, 2024
d5c8c39
numpy-1.25.1: Adding crossenv requirement file
th0ma7 Dec 2, 2024
f626555
crossenv.mk: Fix missing definitions for HOST and BUILD_ARCH
th0ma7 Dec 3, 2024
4ab716f
wheel.mk: sort wheelhouse requirement files to avoid duplicates
th0ma7 Dec 3, 2024
a5eb355
python31*-wheels: various adjustments
th0ma7 Dec 3, 2024
3eeab10
restore python311-wheels/src/requirements-crossenv-greenlet-v3.txt
hgy59 Dec 3, 2024
5bc2003
build.sh: add python312 and python313 to packages to keep
hgy59 Dec 3, 2024
d6e796c
python.mk: Add xz (lzma) to the library exception list
th0ma7 Dec 3, 2024
227669c
python311-wheels: Sync greenlet requirement files with py312-wheels
th0ma7 Dec 3, 2024
6041539
python.mk: Use *_EXCLUDE variables to simplify later additions
th0ma7 Dec 3, 2024
a2edf55
python310: Update from version 3.10.15 to 3.10.16
th0ma7 Dec 3, 2024
cfed98d
python311: Update from version 3.11.10 to 3.11.11
th0ma7 Dec 3, 2024
8364c6f
python312-python313: Sync requirement files with py310-311
th0ma7 Dec 3, 2024
fce0233
python312: Update from version 3.12.7 to 3.12.8
th0ma7 Dec 4, 2024
487239e
python313: Update from version 3.13.0 to 3.13.1
th0ma7 Dec 4, 2024
3866121
py311-312-wheels: Fix greenlet 3.x support with gcc <= 8
th0ma7 Dec 4, 2024
c3f9abc
crossenv.mk: Move PATH and LD_LIBRARY_PATH to be crossenv specific
th0ma7 Dec 5, 2024
540fae4
define build-python when not in path
hgy59 Dec 6, 2024
6ec2f8d
minimize specific crossenv for frozenlist and yarl
hgy59 Dec 6, 2024
cfee21f
remove unsupported configure args
hgy59 Dec 6, 2024
8232883
revert fullpath for build-python for python310 and python311
hgy59 Dec 6, 2024
2ffb5e7
python312-wheels: fix build
hgy59 Dec 6, 2024
92aff79
revert unintended change
hgy59 Dec 6, 2024
fc5a2ce
spksrc.wheel.mk: add PATH for build of pure-python-wheels
hgy59 Dec 6, 2024
54e8a0a
migrate requirements for crossenv from mk/crossenv to spk/python31*/c…
hgy59 Dec 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 13 additions & 16 deletions .github/actions/prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@
# Evaluate packages to build and referenced source files to download.
#
# Functions:
# - Evaluate all packages to build depending on files defined in ${GH_FILES}.
# - Build all packages defined by ${USER_SPK_TO_BUILD} and ${GH_SPK_PACKAGES}
# - Evaluate additional packages to build depending on changed folders defined in ${GH_DEPENDENT_PACKAGES}.
# - synocli-videodriver is moved to head of packages to build first if triggered by its ffmpeg5-7
# - python310-311 and ffmpeg5-7 are moved to head of remaining packages to build when triggered by its own or a dependent.
# - python310-313 and ffmpeg5-7 are moved to head of remaining packages to build when triggered by its own or a dependent.
# - Referenced native and cross packages of the packages to build are added to the download list.

set -o pipefail
Expand All @@ -18,12 +19,8 @@ echo "::group:: ---- find dependent packages"
make setup-synocommunity
DEFAULT_TC=$(grep DEFAULT_TC local.mk | cut -f2 -d= | xargs)

# filter for changes made in the spk directories and take unique package name (without spk folder)
SPK_TO_BUILD+=" "
SPK_TO_BUILD+=$(echo "${GH_FILES}" | tr ' ' '\n' | grep -oP "^spk/\K[^\/]*" | sort -u | tr '\n' ' ')

# filter for changes made in the cross and native directories and take unique package name (including cross or native folder)
DEPENDENT_PACKAGES=$(echo "${GH_FILES}" | tr ' ' '\n' | grep -oP "(cross|native)/[^\/]*" | sort -u | tr '\n' ' ')
# all packages to build from changes or manual definition
SPK_TO_BUILD="${USER_SPK_TO_BUILD} ${GH_SPK_PACKAGES} "

# get dependency list
# dependencies in this list include the cross or native folder (i.e. native/python cross/glib)
Expand All @@ -37,20 +34,20 @@ do
done

# search for dependent spk packages
for package in ${DEPENDENT_PACKAGES}
for package in ${GH_DEPENDENCY_FOLDERS}
do
echo "===> Searching for dependent package: ${package}"
packages=$(echo "${DEPENDENCY_LIST}" | grep " ${package} " | grep -o ".*:" | tr ':' ' ' | sort -u | tr '\n' ' ')
echo "===> Found: ${packages}"
SPK_TO_BUILD+=${packages}
SPK_TO_BUILD+=" ${packages}"
done

# fix for packages with different names
if [ "$(echo ${SPK_TO_BUILD} | grep -ow nzbdrone)" != "" ]; then
SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -vw "nzbdrone" | tr '\n' ' ')" sonarr3"
if [ "$(echo ${SPK_TO_BUILD} | grep -o ' nzbdrone ')" != "" ]; then
SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -v "^nzbdrone$" | tr '\n' ' ')" sonarr3"
fi
if [ "$(echo ${SPK_TO_BUILD} | grep -ow python)" != "" ]; then
SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -vw "python" | tr '\n' ' ')" python2"
if [ "$(echo ${SPK_TO_BUILD} | grep -o ' python ')" != "" ]; then
SPK_TO_BUILD=$(echo "${SPK_TO_BUILD}" | tr ' ' '\n' | grep -v "^python$" | tr '\n' ' ')" python2"
fi

# remove duplicate packages
Expand Down Expand Up @@ -86,8 +83,8 @@ do
fi
done

# for python (310, 311) find all packages that depend on them
for py in python310 python311; do
# for python (310, 311, 312, 313) find all packages that depend on them
for py in python310 python311 python312 python313; do
python_dependent_packages=$(find spk/ -maxdepth 2 -mindepth 2 -name "Makefile" -exec grep -Ho "PYTHON_PACKAGE = ${py}" {} \; | grep -Po ".*spk/\K[^/]*" | sort | tr '\n' ' ')

# If packages contain a package that depends on python (or is python), then ensure
Expand Down
25 changes: 15 additions & 10 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,26 +97,31 @@ jobs:
fetch-depth: 0
persist-credentials: false

- name: Get changed files for pull request
- name: Get changed spk_packages and dependency_folders for pull request
if: github.event_name == 'pull_request'
id: getfile_pr
id: getchanges_pr
run: |
git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | xargs
echo "files=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | xargs)" >> $GITHUB_OUTPUT
git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs
echo "spk_packages=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs
echo "dependency_folders=$(git diff --no-commit-id --name-only -r origin/master...${{github.event.pull_request.head.sha}} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT

- name: Get changed files for push
- name: Get changed spk_packages and dependency_folders of last commit for push
if: github.event_name == 'push'
id: getfile
id: getchanges_push
run: |
git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | xargs
echo "files=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | xargs)" >> $GITHUB_OUTPUT
git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs
echo "spk_packages=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "^spk/\K[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT
git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs
echo "dependency_folders=$(git diff-tree --no-commit-id --name-only -r ${{ github.sha }} | grep -oP "(cross|native)/[^\/]*" | sort -u | xargs)" >> $GITHUB_OUTPUT

- name: Evaluate dependencies
id: dependencies
run: ./.github/actions/prepare.sh
env:
GH_FILES: ${{ steps.getfile.outputs.files }} ${{ steps.getfile_pr.outputs.files }}
SPK_TO_BUILD: ${{ github.event.inputs.package }}
GH_SPK_PACKAGES: ${{ steps.getchanges_push.outputs.spk_packages }} ${{ steps.getchanges_pr.outputs.spk_packages }}
GH_DEPENDENCY_FOLDERS: ${{ steps.getchanges_push.outputs.dependency_folders }} ${{ steps.getchanges_pr.outputs.dependency_folders }}
USER_SPK_TO_BUILD: ${{ github.event.inputs.package }}

# Set default values for all builds (manual or automated)
- name: Set default values for generate matrix
Expand Down
6 changes: 3 additions & 3 deletions cross/boost_1.82/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ WITH_PYTHON_LIBRARY =
ifneq ($(findstring python,$(BOOST_LIBRARIES)),)
WITH_PYTHON_LIBRARY = 1
# The packages depending on boost with python, must build python before.
include $(WORK_DIR)/python-cc.mk
include $(WORK_DIR)/crossenv/build/python-cc.mk
PYTHON_NAME = $(basename $(notdir $(PYTHON_INTERPRETER)))
else ifneq ($(findstring $(BOOST_LIBRARIES), all),)
# exclude python library if "all" selected
Expand All @@ -43,7 +43,7 @@ ADDITIONAL_CXXFLAGS = -Wno-deprecated-declarations

ifneq ($(strip $(WITH_PYTHON_LIBRARY)),)
CONFIGURE_ARGS += --with-python=$(WORK_DIR)/../../../native/$(PYTHON_NAME)/work-native/install/usr/local/bin/$(PYTHON_NAME)
ADDITIONAL_CXXFLAGS += -I$(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) -fpermissive
ADDITIONAL_CXXFLAGS += -I$(or $(PYTHON_STAGING_INSTALL_PREFIX),$(STAGING_INSTALL_PREFIX))/$(PYTHON_INC_DIR) -fpermissive
endif

include ../../mk/spksrc.common.mk
Expand Down Expand Up @@ -75,7 +75,7 @@ boost_pre_compile:
@rm -rf $(WORK_DIR)/$(PKG_DIR)/user-config.jam
@echo "using gcc : ${TC_GCC} : ${CXX} : <address-model>\"$(ADDRESS_MODEL)\" <cflags>\"$(CFLAGS)\" <cxxflags>\"$(CXXFLAGS) $(ADDITIONAL_CXXFLAGS)\" <linkflags>\"$(LDFLAGS)\" <link>\"shared\" ; " > $(WORK_DIR)/$(PKG_DIR)/user-config.jam
ifneq ($(strip $(WITH_PYTHON_LIBRARY)),)
@echo "using python : $(PYTHON_VERSION) : : $(STAGING_INSTALL_PREFIX)/$(PYTHON_INC_DIR) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam
@echo "using python : $(PYTHON_VERSION) : : $(or $(PYTHON_STAGING_INSTALL_PREFIX),$(STAGING_INSTALL_PREFIX))/$(PYTHON_INC_DIR) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam
endif
@echo "project : requirements <cxxflags>$(OFLAGS) ;" >> $(WORK_DIR)/$(PKG_DIR)/user-config.jam

Expand Down
11 changes: 2 additions & 9 deletions cross/cryptography/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,13 @@ PKG_NAME = cryptography
PKG_VERS = 41.0.3
PKG_EXT = tar.gz
PKG_DIST_NAME = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
PKG_DIST_SITE = https://files.pythonhosted.org/packages/source/c/$(PKG_NAME)
PKG_DIST_SITE = https://files.pythonhosted.org/packages/source/c/cryptography
PKG_DIR = $(PKG_NAME)-$(PKG_VERS)

DEPENDS = cross/openssl3 cross/cffi
DEPENDS = cross/openssl3 cross/libffi

HOMEPAGE = https://github.com/pyca/cryptography
COMMENT = Provide cryptographic recipes and primitives to Python developers
LICENSE = BSD/ASL2

###
### This cross/cryptography is needed to generate python-limited abi3 wheels
### Ref: https://github.com/pyca/cryptography/issues/7671
###

WHEELS_BUILD_ARGS = --py-limited-api=$(PYTHON_LIMITED_API)

include ../../mk/spksrc.python-wheel.mk
14 changes: 9 additions & 5 deletions cross/dtlssocket/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,22 @@
PKG_NAME = DTLSSocket
PKG_VERS = 0.1.16
PKG_NAME = dtlssocket
PKG_VERS = 0.2.2
PKG_EXT = tar.gz
PKG_DIST_NAME = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
PKG_DIST_SITE = https://files.pythonhosted.org/packages/58/42/0a0442118096eb9fbc9dc70b45aee2957f7546b80545e2a05bd839380519
PKG_DIST_SITE = https://files.pythonhosted.org/packages/97/b6/702c4197c37c7036d0b7d67a2963c83a4b7098525959566dc6e2284c9f89
PKG_DIR = $(PKG_NAME)-$(PKG_VERS)

BUILD_DEPENDS = cross/cython

HOMEPAGE = https://git.fslab.de/jkonra2m/tinydtls-cython
COMMENT = DTLSSocket is a Cython wrapper for tinydtls with a Socket like interface.
LICENSE = Eclipse Public License 1.0

PRE_CONFIGURE_TARGET = dtlssocket_pre_configure

include ../../mk/spksrc.common.mk

ifeq ($(call version_le, $(TC_GCC), 5),1)
ADDITIONAL_CFLAGS = -std=c99
endif

include ../../mk/spksrc.python-wheel.mk

.PHONY: dtlssocket_pre_configure
Expand Down
6 changes: 3 additions & 3 deletions cross/dtlssocket/digests
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
DTLSSocket-0.1.16.tar.gz SHA1 9b4eaa3e9a949f202844a27ba54b2f05d73aa410
DTLSSocket-0.1.16.tar.gz SHA256 30b108ae45fce1c033e3ef6c2ddd7e7418062ae374228e3a7fa969b254366a39
DTLSSocket-0.1.16.tar.gz MD5 15dfeca9678e381c7bb6e11e4c864fd5
dtlssocket-0.2.2.tar.gz SHA1 0be2a954ca56fe7cca3a9fbc5a9e65194f0ab5a5
dtlssocket-0.2.2.tar.gz SHA256 4e76d715726e0c46dc09e35da9b6717b063c2389b06c1723decc3ba38b7387f4
dtlssocket-0.2.2.tar.gz MD5 d1f686febd12e67ec7a2efe81bf1eaee
34 changes: 30 additions & 4 deletions cross/mariadb-connector-c/Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
PKG_NAME = mariadb-connector-c
PKG_VERS = 3.3.5
# Version 3.3.11 is the last known version to work with aarch64-6.2.4
# Assembly error: Error: CFI instruction used without previous .cfi_startproc
PKG_VERS = 3.3.11
PKG_EXT = tar.gz
PKG_DIST_NAME = v$(PKG_VERS).$(PKG_EXT)
PKG_DIST_FILE = $(PKG_NAME)-$(PKG_VERS).$(PKG_EXT)
Expand All @@ -10,23 +12,47 @@ HOMEPAGE = https://mariadb.com/kb/en/mariadb-connector-c/
COMMENT = MariaDB C Connector
LICENSE = LGPL

#USE_NATIVE_CMAKE_LEGACY = 1
#CMAKE_USE_TOOLCHAIN_FILE = OFF
CMAKE_DISABLE_EXE_LINKER_FLAGS = 1

include ../../mk/spksrc.cross-cmake.mk

ifeq ($(call version_lt, $(TC_GCC), 5.0),1)
ifeq ($(call version_gt, $(TC_GCC), 4.9),1)
ADDITIONAL_CFLAGS += -Wno-error=maybe-uninitialized
# fails on evansport-6.2.4
ADDITIONAL_CFLAGS += -Wno-error=strict-aliasing
endif

ifeq ($(call version_lt, $(TC_GCC), 4.9),1)
ADDITIONAL_CFLAGS += -std=gnu99
# fails on 88f6281-6.2.4
ADDITIONAL_CFLAGS += -Wno-error=missing-field-initializers
ADDITIONAL_CFLAGS += -Wno-error=uninitialized
# fails on hi3535-6.2.4
ADDITIONAL_CFLAGS += -Wno-error=array-bounds
endif

# MySQL specific:
CMAKE_ARGS += -DWITH_UNIT_TESTS=OFF
CMAKE_ARGS += -DDEFAULT_CHARSET=utf8
CMAKE_ARGS += -DINSTALL_LIBDIR=lib

# Creates the following two synlinks:
# - libmysqlclient.so -> libmariadb.so.3
# - libmysqlclient_r.so -> libmariadb.so.3
# Which in turns conflict with mysql-connector-c
#CMAKE_ARGS += -DWITH_MYSQLCOMPAT=ON
hgy59 marked this conversation as resolved.
Show resolved Hide resolved

# Does not handle pkgconfig well and fails to
# find openssl3 include and libraries using
# spksrc.python.mk thus using OPENSSL_STAGING_PREFIX
DEPENDS += cross/openssl3
CMAKE_ARGS += -DWITH_SSL=ON
ifneq ($(wildcard $(OPENSSL_STAGING_PREFIX)),)
CMAKE_ARGS += -DOPENSSL_CRYPTO_LIBRARY:FILEPATH=$(OPENSSL_STAGING_PREFIX)/lib/libcrypto.so
CMAKE_ARGS += -DOPENSSL_SSL_LIBRARY:FILEPATH=$(OPENSSL_STAGING_PREFIX)/lib/libssl.so
CMAKE_ARGS += -DOPENSSL_INCLUDE_DIR:PATH=$(OPENSSL_STAGING_PREFIX)/include
endif

DEPENDS += cross/zlib
CMAKE_ARGS += -DWITH_EXTERNAL_ZLIB=ON
DEPENDS += cross/libiconv
Expand Down
6 changes: 3 additions & 3 deletions cross/mariadb-connector-c/digests
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
mariadb-connector-c-3.3.5.tar.gz SHA1 29ad65e81a87e41a93e8984f2c3d1662dcec7c1c
mariadb-connector-c-3.3.5.tar.gz SHA256 c0fda1fa6e52dc85de27156cd847088a72d40d9de6514f7efa57c8d93134a54c
mariadb-connector-c-3.3.5.tar.gz MD5 b8967cf63d4fc5660f230762c0e39ee2
mariadb-connector-c-3.3.11.tar.gz SHA1 2fa1fe042bcc65297d6d22196685701faf3e96d1
mariadb-connector-c-3.3.11.tar.gz SHA256 6ecada1f3fc32ad95efcd2c982b319191f5e6479f3f521bc4b714e6158f5e415
mariadb-connector-c-3.3.11.tar.gz MD5 700afe8ebff69bdf66b3c37747d1d527
103 changes: 8 additions & 95 deletions cross/python310/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -28,23 +28,20 @@ CONFIGURE_ARGS += --without-ensurepip
CONFIGURE_ARGS += --enable-loadable-sqlite-extensions
CONFIGURE_ARGS += --with-computed-gotos=yes
CONFIGURE_ARGS += --with-build-python
CONFIGURE_ARGS += --disable-test-modules

include ../../mk/spksrc.archs.mk

# optionally generate optimized code
ifeq ($(strip $(PYTHON_OPTIMIZE)),1)
CONFIGURE_ARGS += --enable-optimizations
# old compilers fail with unrecognized command line options: -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none
ifneq ($(findstring $(ARCH),$(OLD_PPC_ARCHS)),$(ARCH))
# Some tests (like test_base64) must find libpython shared library at runtime.
# python: error while loading shared libraries: libpython3.10.so.1.0: cannot open shared object file: No such file or directory
ENV += LD_LIBRARY_PATH=$(WORK_DIR)/$(PKG_DIR)
#
# Enable Link-Time Optimization
CONFIGURE_ARGS += --with-lto
endif
else
# test modules are required for optimization only
CONFIGURE_ARGS += --disable-test-modules
#
# pgo optimization can only be run native platform and now
# exit on errors which had no impact (and effects) previously.
# https://github.com/python/cpython/pull/111950
#CONFIGURE_ARGS += --enable-optimizations
endif

# older gcc does not know -Wno-unused-result
Expand All @@ -70,44 +67,17 @@ CONFIGURE_ARGS += --with-system-expat
DEPENDS += cross/libffi
CONFIGURE_ARGS += --with-system-ffi

# Mandatory PYO3_* variables for rust cross-compiling
ENV += PYO3_CROSS_LIB_DIR=$(STAGING_INSTALL_PREFIX)/lib/
ENV += PYO3_CROSS_INCLUDE_DIR=$(STAGING_INSTALL_PREFIX)/include/
# Mandatory of using OPENSSL_*_DIR starting with
# cryptography version >= 40
# https://docs.rs/openssl/latest/openssl/#automatic
ENV += OPENSSL_LIB_DIR=$(STAGING_INSTALL_PREFIX)/lib/
ENV += OPENSSL_INCLUDE_DIR=$(STAGING_INSTALL_PREFIX)/include/


ADDITIONAL_CFLAGS = -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -L $(STAGING_INSTALL_PREFIX)/lib -I $(STAGING_INSTALL_PREFIX)/include

POST_PATCH_TARGET = python310_post_patch
PRE_CONFIGURE_TARGET = python310_pre_configure
COMPILE_TARGET = python310_compile
INSTALL_TARGET = python310_install
POST_INSTALL_TARGET = python310_post_install

include ../../mk/spksrc.cross-cc.mk

HOST_ARCH = $(shell uname -m)
BUILD_ARCH = $(shell expr "$(TC_TARGET)" : '\([^-]*\)' )
PYTHON_NATIVE = $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/bin/python3
PIP_NATIVE = $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/bin/pip
PYTHON_NATIVE = $(abspath $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/bin/python3)
HOSTPYTHON = $(WORK_DIR)/$(PKG_DIR)/hostpython
HOSTPYTHON_LIB_NATIVE = $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/$(PKG_DIR)/build/lib.linux-$(HOST_ARCH)-$(PKG_VERS_MAJOR_MINOR)
PYTHON_LIB_NATIVE = $(WORK_DIR)/$(PKG_DIR)/build/lib.linux-$(HOST_ARCH)-$(PKG_VERS_MAJOR_MINOR)
PYTHON_SITE_PACKAGES_NATIVE = $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/lib/python$(PKG_VERS_MAJOR_MINOR)/site-packages
PYTHON_LIB_CROSS = $(WORK_DIR)/$(PKG_DIR)/build/lib.linux-$(BUILD_ARCH)-$(PKG_VERS_MAJOR_MINOR)
PYTHON_LIB_DIR = lib/python$(PKG_VERS_MAJOR_MINOR)
PYTHON_INC_DIR = include/python$(PKG_VERS_MAJOR_MINOR)

# Required so that the python3 binaries and libraries generated by native compilation
# take precedence over current environment python3 when generating
# target architecture resources
PATH := $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/bin:$(PATH)
LD_LIBRARY_PATH := $(WORK_DIR)/../../../native/$(PKG_NAME)/work-native/install/usr/local/lib:$(LD_LIBRARY_PATH)
export PATH LD_LIBRARY_PATH

.PHONY: python310_post_patch
python310_post_patch:
Expand All @@ -134,60 +104,3 @@ python310_install:
@install -m 755 -d $(STAGING_INSTALL_PREFIX)/etc
@install -m 644 src/mime.types $(STAGING_INSTALL_PREFIX)/etc/
$(RUN) _PYTHON_HOST_PLATFORM=$(TC_TARGET) $(MAKE) install prefix=$(STAGING_INSTALL_PREFIX)

# default wheels to install in crossenv
CROSSENV_WHEELS = cffi==1.17.0
#
# Cython version >= 3.x breaks PyYAML wheel
# https://github.com/yaml/pyyaml/issues/601
#
# It also breaks numpy at crossenv creation time using github-action
# https://numpy.org/devdocs/reference/distutils_status_migration.html
# Python 3.12 considerations: scikit-build-core, meson-python
#CROSSENV_WHEELS += Cython==3.0.2
CROSSENV_WHEELS += Cython==0.29.37
CROSSENV_WHEELS += flit==3.9.0
CROSSENV_WHEELS += scikit-build==0.18.1
CROSSENV_WHEELS += setuptools-rust==1.10.2
CROSSENV_WHEELS += setuptools-scm==8.1.0
# For future use when building numpy >= 1.26
#CROSSENV_WHEELS += meson-python==1.5.2
#CROSSENV_WHEELS += scikit-build-core==0.10.7
ifneq ($(findstring $(ARCH),$(OLD_PPC_ARCHS)),$(ARCH))
CROSSENV_WHEELS += cryptography==43.0.1
CROSSENV_WHEELS += maturin==1.7.4
CROSSENV_WHEELS += poetry==1.8.3
endif

# Create the crossenv in preparation for
# cross-compiling all the necessary wheels
.PHONY: python310_post_install
python310_post_install: $(WORK_DIR)/python-cc.mk
mkdir -p $(PYTHON_LIB_CROSS)
cp -R $(HOSTPYTHON_LIB_NATIVE) $(PYTHON_LIB_CROSS)/../
@$(RUN) $(PYTHON_NATIVE) -m crossenv $(STAGING_INSTALL_PREFIX)/bin/python$(PKG_VERS_MAJOR_MINOR) --cc $(TC_PATH)$(TC_PREFIX)gcc --cxx $(TC_PATH)$(TC_PREFIX)c++ --ar $(TC_PATH)$(TC_PREFIX)ar --sysroot $(TC_SYSROOT) --env LIBRARY_PATH= --manylinux manylinux2014 $(WORK_DIR)/crossenv/
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) wget --no-verbose https://bootstrap.pypa.io/get-pip.py
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) build-python get-pip.py "pip==24.2" --no-setuptools --no-wheel --disable-pip-version-check
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) python get-pip.py "pip==24.2" --no-setuptools --no-wheel --disable-pip-version-check
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) build-pip --disable-pip-version-check install "setuptools==68.1.2" "wheel==0.44.0" "pip-tools==7.4.1"
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) pip --disable-pip-version-check install "setuptools==68.1.2" "wheel==0.44.0" "pip-tools==7.4.1"
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) build-pip --disable-pip-version-check install $(CROSSENV_WHEELS)
. $(WORK_DIR)/crossenv/bin/activate && $(RUN) pip --disable-pip-version-check install $(CROSSENV_WHEELS)
ifneq ($(PYTHON_LIB_NATIVE),$(PYTHON_LIB_CROSS))
cp $(PYTHON_LIB_CROSS)/_sysconfigdata_*.py $(PYTHON_LIB_NATIVE)/_sysconfigdata.py
endif

$(WORK_DIR)/python-cc.mk:
@echo HOSTPYTHON=$(HOSTPYTHON) > $@
@echo CROSSENV=$(WORK_DIR)/crossenv/bin/activate >> $@
@echo HOSTPYTHON_LIB_NATIVE=$(HOSTPYTHON_LIB_NATIVE) >> $@
@echo PYTHON_LIB_NATIVE=$(PYTHON_LIB_NATIVE) >> $@
@echo PYTHON_SITE_PACKAGES_NATIVE=$(PYTHON_SITE_PACKAGES_NATIVE) >> $@
@echo PYTHON_INTERPRETER=$(INSTALL_PREFIX)/bin/python$(PKG_VERS_MAJOR_MINOR) >> $@
@echo PYTHON_VERSION=$(PKG_VERS_MAJOR_MINOR) >> $@
@echo PYTHON_LIB_CROSS=$(PYTHON_LIB_CROSS) >> $@
@echo PYTHON_LIB_DIR=$(PYTHON_LIB_DIR) >> $@
@echo PYTHON_INC_DIR=$(PYTHON_INC_DIR) >> $@
@echo PIP=$(PIP_NATIVE) >> $@
@echo CROSS_COMPILE_WHEELS=1 >> $@
@echo ADDITIONAL_WHEEL_BUILD_ARGS=--no-build-isolation >> $@
Loading
Loading