Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for python install on self-hosted runners #852

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 0 additions & 56 deletions .github/workflows/pr_push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,64 +16,8 @@ permissions:
contents: read

jobs:
CodeChecks:
uses: ./.github/workflows/reusable_checks.yml
DocsBuild:
uses: ./.github/workflows/reusable_docs_build.yml
FastBuild:
name: Fast builds
needs: [CodeChecks, DocsBuild]
uses: ./.github/workflows/reusable_fast.yml
Build:
name: Basic builds
needs: [FastBuild]
uses: ./.github/workflows/reusable_basic.yml
DevDax:
needs: [FastBuild]
uses: ./.github/workflows/reusable_dax.yml
Sanitizers:
needs: [FastBuild]
uses: ./.github/workflows/reusable_sanitizers.yml
Qemu:
needs: [FastBuild]
uses: ./.github/workflows/reusable_qemu.yml
Benchmarks:
needs: [Build]
uses: ./.github/workflows/reusable_benchmarks.yml
ProxyLib:
needs: [Build]
uses: ./.github/workflows/reusable_proxy_lib.yml
GPU:
needs: [Build]
uses: ./.github/workflows/reusable_gpu.yml
Valgrind:
needs: [Build]
uses: ./.github/workflows/reusable_valgrind.yml
MultiNuma:
needs: [Build]
uses: ./.github/workflows/reusable_multi_numa.yml
Coverage:
# total coverage (on upstream only)
if: github.repository == 'oneapi-src/unified-memory-framework'
needs: [Build, DevDax, GPU, MultiNuma, Qemu, ProxyLib]
uses: ./.github/workflows/reusable_coverage.yml
secrets: inherit
with:
trigger: "${{github.event_name}}"
Coverage_partial:
# partial coverage (on forks)
if: github.repository != 'oneapi-src/unified-memory-framework'
needs: [Build, Qemu, ProxyLib]
uses: ./.github/workflows/reusable_coverage.yml
CodeQL:
needs: [Build]
permissions:
contents: read
security-events: write
uses: ./.github/workflows/reusable_codeql.yml
Trivy:
needs: [Build]
permissions:
contents: read
security-events: write
uses: ./.github/workflows/reusable_trivy.yml
85 changes: 29 additions & 56 deletions .github/workflows/reusable_codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,78 +17,51 @@ jobs:
security-events: write
env:
VCPKG_PATH: "${{github.workspace}}/build/vcpkg/packages/hwloc_x64-windows;${{github.workspace}}/build/vcpkg/packages/tbb_x64-windows;${{github.workspace}}/build/vcpkg/packages/jemalloc_x64-windows"
PYTHON_PATH: "aa"

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
os: [windows-latest]
include:
- os: ubuntu-latest
# Windows doesn't recognize 'CMAKE_BUILD_TYPE', it uses '--config' param in build command
extra_build_option: '-DCMAKE_BUILD_TYPE=Release'
- os: windows-latest
runs-on: ${{matrix.os}}
runs-on: ["DSS-CUDA", "DSS-WINDOWS"]

steps:
- name: Checkout repository
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
fetch-depth: 0

- name: Setup newer Python
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5.0.0
with:
python-version: "3.10"

- name: Initialize CodeQL
uses: github/codeql-action/init@b7bf0a3ed3ecfa44160715d7c442788f65f0f923 # v3.23.2
with:
languages: cpp

- name: "[Win] Initialize vcpkg"
if: matrix.os == 'windows-latest'
uses: lukka/run-vcpkg@5e0cab206a5ea620130caf672fce3e4a6b5666a1 # v11.5
with:
vcpkgGitCommitId: 3dd44b931481d7a8e9ba412621fa810232b66289
vcpkgDirectory: ${{env.BUILD_DIR}}/vcpkg
vcpkgJsonGlob: '**/vcpkg.json'

- name: "[Win] Install dependencies"
if: matrix.os == 'windows-latest'
- name: aaa
run: |
vcpkg install
python3 -m pip install -r third_party/requirements.txt
ls ${{github.workspace}}../../../..
echo ${{env.PYTHON_PATH}}
echo "PYTHON_PATH=${{env.PYTHON_PATH}}/python/python.exe" >> $GITHUB_ENV

- name: "[Lin] Install apt packages"
if: matrix.os == 'ubuntu-latest'
- name: bbb
run: |
sudo apt-get update
sudo apt-get install -y cmake clang libhwloc-dev libnuma-dev libjemalloc-dev libtbb-dev

# Latest distros do not allow global pip installation
- name: "[Lin] Install Python requirements in venv"
if: matrix.os == 'ubuntu-latest'
echo ${{env.PYTHON_PATH}}

- name: Check for Python >= 3.10
id: check_python
run: |
python3 -m venv .venv
. .venv/bin/activate
echo "$PATH" >> $GITHUB_PATH
python3 -m pip install -r third_party/requirements.txt

- name: Configure CMake
run: >
cmake
-B ${{env.BUILD_DIR}}
${{matrix.extra_build_option}}
-DCMAKE_INSTALL_PREFIX="${{env.INSTL_DIR}}"
-DCMAKE_PREFIX_PATH="${{env.VCPKG_PATH}}"
-DUMF_FORMAT_CODE_STYLE=OFF
-DUMF_DEVELOPER_MODE=ON
-DUMF_BUILD_LIBUMF_POOL_JEMALLOC=ON
-DUMF_BUILD_LEVEL_ZERO_PROVIDER=ON
-DUMF_BUILD_CUDA_PROVIDER=ON
-DUMF_TESTS_FAIL_ON_SKIP=ON

- name: Build
run: cmake --build ${{env.BUILD_DIR}} --config Release -j

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@b7bf0a3ed3ecfa44160715d7c442788f65f0f923 # v3.23.2
echo "python_exists=false" >> $GITHUB_OUTPUT
$pythonCommand = Get-Command python -ErrorAction SilentlyContinue
if ($pythonCommand) {
$pythonVersion = python --version 2>&1
$versionPattern = 'Python (\d+)\.(\d+)\.(\d+)'
echo $pythonVersion
if ($pythonVersion -match $versionPattern) {
$major = [int]$matches[1]
$minor = [int]$matches[2]
if ($major -gt 3 -or ($major -eq 3 -and $minor -ge 10)) {
echo "python_exists=true" >> $GITHUB_OUTPUT
}
}
}
shell: pwsh

Loading