Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[testing] docker workflows #2141

Closed
wants to merge 56 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
57cd953
add build workflow
dsikka Feb 27, 2024
4d93b33
update to run on all containers
dsikka Feb 27, 2024
80fc6a5
try different context
dsikka Feb 27, 2024
aafb68c
try new context
dsikka Feb 27, 2024
2a64768
update to use local file
dsikka Feb 27, 2024
d894187
add missing step
dsikka Feb 27, 2024
53424eb
try again
dsikka Feb 27, 2024
79e6213
update pip
dsikka Feb 27, 2024
a4e21f2
try again
dsikka Feb 27, 2024
bf69446
fix typo
dsikka Feb 27, 2024
df6002c
update workflow name
dsikka Feb 27, 2024
5303ef7
add tag and push step
dsikka Feb 27, 2024
9c9921e
bump up version
dsikka Feb 27, 2024
c2ea3ca
typo
dsikka Feb 27, 2024
b98d66b
update date tag
dsikka Feb 28, 2024
74ebf9a
try again'
dsikka Feb 28, 2024
86e2a93
try again
dsikka Feb 28, 2024
1ef694b
try again
dsikka Feb 28, 2024
69fc928
add caching
dsikka Feb 28, 2024
cc138f4
add docker dev build steps
dsikka Feb 28, 2024
f95fb34
update ref
dsikka Feb 28, 2024
90d8c2e
debug
dsikka Feb 28, 2024
8ee95c7
try again
dsikka Feb 28, 2024
d4f8a83
try again
dsikka Feb 28, 2024
cbf692c
update
dsikka Feb 28, 2024
b983123
try again
dsikka Feb 28, 2024
fa41fd6
update docker
dsikka Feb 28, 2024
f8518b0
try again
dsikka Feb 28, 2024
7931d06
try once more
dsikka Feb 28, 2024
ab4a8f6
try again
dsikka Feb 28, 2024
f65c4a9
try again
dsikka Feb 28, 2024
29bfffc
update
dsikka Feb 28, 2024
67eef3a
typo
dsikka Feb 28, 2024
1c80e70
try again
dsikka Feb 28, 2024
a17f9b4
try again
dsikka Feb 28, 2024
4b38e44
typo
dsikka Feb 28, 2024
40314f1
try again
dsikka Feb 28, 2024
9e5925d
typo
dsikka Feb 28, 2024
c9e37e8
typo
dsikka Feb 28, 2024
d461a6d
typo
dsikka Feb 28, 2024
0ae27f9
try again
dsikka Feb 28, 2024
ea48f1a
try again
dsikka Feb 28, 2024
d32d287
try again
dsikka Feb 28, 2024
6133a22
hardcode
dsikka Feb 28, 2024
5450796
try again
dsikka Feb 28, 2024
930df4b
update
dsikka Feb 28, 2024
76cd0f8
update
dsikka Feb 28, 2024
a625dfc
try again
dsikka Feb 28, 2024
683744c
add release container flows
dsikka Mar 4, 2024
ddcf458
update container names
dsikka Mar 4, 2024
293919c
update build
dsikka Mar 4, 2024
fa89675
add clean step
dsikka Mar 4, 2024
9f881ae
try making clone smaller
dsikka Mar 4, 2024
de9f43c
combine run commands
dsikka Mar 4, 2024
7e6e0ff
restructure
dsikka Mar 4, 2024
9f9e198
just install dev
dsikka Mar 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions .github/workflows/build-container.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Build Docker Container
on:
pull_request:
types: [opened, synchronize, reopened]
branches:
- main
- 'release/**'
push:
branches:
- 'main'
release:
types: [created, published]
schedule:
- cron: '0 2 * * *'

# potentially update to change criteria; only run when wheel build is complete
# use outptus from wheel build to determie if release/dev/nightly
env:
RELEASE: ${{ github.event_name =='release' || (github.base_ref == 'release/**' && github.event_name == 'pull_request')}}
DEV: ${{ github.base_ref == 'main' && github.event_name == 'pull_request'}}
NAME: ${{ github.event.number }}

permissions:
contents: read
packages: write

jobs:
build-container:
name: Build sparseml container
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v2
with:
buildkitd-flags: --debug
- name: Get current date
id: date
run: echo "::set-output name=date::$(date +'%Y%m%d')"
- name: Get the current version
if: ${{ env.RELEASE == 'true' }}
id: version
run: echo "##[set-output name=version;]$(echo ${GITHUB_REF_NAME#*/})"
- name: Login to Github Packages
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build Dev Docker Container
if: ${{ env.DEV == 'true' }}
uses: docker/build-push-action@v4
with:
context: ./docker/containers/docker_dev
build-args: |
BRANCH=${{github.head_ref}}
push: true
tags: ghcr.io/neuralmagic/test-sparseml-dev:${{ env.NAME }}
- name: Build RELEASE Docker Container
if: ${{ env.RELEASE == 'true' }}
uses: docker/build-push-action@v4
with:
context: ./docker/containers/docker_release
build-args: |
VERSION=${{ steps.version.outputs.version }}
push: true
tags: ghcr.io/neuralmagic/test-sparseml:latest, ghcr.io/neuralmagic/test-sparseml:${{ steps.version.outputs.version }}
- name: Build Nightly Docker Container
if: ${{ env.DEV == 'false' && env.RELEASE == 'false'}}
uses: docker/build-push-action@v4
with:
context: ./docker/containers/docker_nightly
push: true
tags: ghcr.io/neuralmagic/test-sparseml-nightly:latest, ghcr.io/neuralmagic/test-sparseml-nightly:${{ steps.date.outputs.date }}
2 changes: 1 addition & 1 deletion docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ RUN \
fi;

FROM cuda_builder AS container_branch_all
ARG VENV
ARG VENV
ENV PATH="${VENV}/bin:$PATH"
ENV PIP_DEFAULT_TIMEOUT=200
ARG VERSION
Expand Down
15 changes: 15 additions & 0 deletions docker/containers/docker_dev/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
ARG SOURCE=ghcr.io/neuralmagic/cuda-python3.8
ARG BRANCH

FROM $SOURCE

ARG BRANCH

RUN python3.8 -m pip install --upgrade pip \
&& python3.8 -m pip install --upgrade setuptools \
&& git clone https://github.com/neuralmagic/sparseml.git --depth 1 --single-branch -b ${BRANCH} \
&& python3.8 -m pip install -e "./sparseml[dev]"

HEALTHCHECK CMD python3.8 -c 'import sparseml'
RUN python3.8 -m pip list | grep sparseml
CMD bash
11 changes: 11 additions & 0 deletions docker/containers/docker_nightly/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
ARG SOURCE=ghcr.io/neuralmagic/cuda-python3.8

FROM $SOURCE

RUN python3.8 -m pip install --upgrade pip \
&& python3.8 -m pip install --upgrade setuptools \
&& python3.8 -m pip install --no-cache-dir "sparseml-nightly[onnxruntime,torchvision,transformers,yolov5,ultralytics]"

HEALTHCHECK CMD python3.8 -c 'import sparseml'
RUN python3.8 -m pip list | grep sparseml
CMD bash
14 changes: 14 additions & 0 deletions docker/containers/docker_release/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
ARG SOURCE=ghcr.io/neuralmagic/cuda-python3.8
ARG VERSION

FROM $SOURCE

ARG VERSION

RUN python3.8 -m pip install --upgrade pip \
&& python3.8 -m pip install --upgrade setuptools \
&& python3.8 -m pip install --no-cache-dir "sparseml[onnxruntime,torchvision,transformers,yolov5,ultralytics]==$VERSION"

HEALTHCHECK CMD python3.8 -c 'import sparseml'
RUN python3.8 -m pip list | grep sparseml
CMD bash
Loading