Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically build docker images for each Brian release #1503

Merged
merged 35 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
05c75d1
Initial commit
bdevans Feb 12, 2024
e038100
Tidy up
bdevans Feb 14, 2024
55ace74
Add README with instructions
bdevans Feb 15, 2024
40eeebe
Fix sudo
bdevans Feb 21, 2024
7da7e7a
Remove unnecessary use of `buildx`
bdevans Feb 21, 2024
cde781f
Update instructions
bdevans Feb 21, 2024
bb3e4e4
Rename
bdevans Feb 26, 2024
124304e
Update to use slim image and tidy comments
bdevans Feb 26, 2024
1d2409c
Add test instructions
bdevans Feb 26, 2024
43ce0a5
Merge remote-tracking branch 'refs/remotes/origin/docker_images' into…
bdevans Feb 26, 2024
5b56339
Add docker pulls badge
bdevans Mar 7, 2024
d0e94c5
Move docker image action into release
mstimberg Mar 7, 2024
2a15a55
Rename publish action
mstimberg Mar 7, 2024
9e93193
Add publish status badge
bdevans Mar 7, 2024
50ac8ce
Merge remote-tracking branch 'refs/remotes/origin/docker_images' into…
bdevans Mar 7, 2024
50db33d
Use local wheels in docker build
mstimberg Mar 7, 2024
85d1bb6
Only push tags to docker hub
mstimberg Mar 7, 2024
b224f62
Fix use of download-artifact action
mstimberg Mar 7, 2024
fb78e25
Use recommended invocation of pip
bdevans Mar 7, 2024
7051fea
Use compatible download/upload actions
mstimberg Mar 7, 2024
b3f9562
Remove deployment environment (for now)
mstimberg Mar 7, 2024
15f8870
Separate package names for upload-artifact@v4
mstimberg Mar 7, 2024
93bbd60
Even more distinct package names for artifacts
mstimberg Mar 7, 2024
873d355
download with correct package names
mstimberg Mar 7, 2024
9b7f966
Only use manylinux wheels
mstimberg Mar 7, 2024
9c7f64b
Fix expression syntax
mstimberg Mar 7, 2024
43eb956
Misunderstood how the new pattern/merge feature was supposed to work …
mstimberg Mar 8, 2024
4b6962b
Upload docker image as artifact
mstimberg Mar 8, 2024
244d0cc
Cannot use load for multi-arch image, trying again
mstimberg Mar 8, 2024
86a863b
Push docker image to brian-dev repository
mstimberg Mar 8, 2024
67d0242
Document docker build process
mstimberg Mar 8, 2024
f437aa6
Fix formatting for multi-line command
mstimberg Mar 8, 2024
84b2ad8
Use uname -m for architecture
mstimberg Mar 11, 2024
77c7263
Install libgsl-dev for GSL support
mstimberg Mar 11, 2024
6f8a3d8
Rearrange
bdevans Mar 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Build and publish to TestPyPI or PyPI
name: Build and publish
on: [push, pull_request]

jobs:
Expand Down Expand Up @@ -130,3 +130,61 @@ jobs:
path: dist/
- name: Publish distribution release 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

build-docker:
name: Build docker image for release
runs-on: ubuntu-latest
environment: release
needs: build-linux
steps:
- name: Checkout repository
uses: actions/checkout@v4
# https://github.com/actions/checkout/
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
# https://github.com/docker/metadata-action
with:
images: |
briansimulator/brian
flavor: latest=true
tags: |
# type=semver,pattern={{raw}}
type=ref,event=tag
labels: |
org.opencontainers.image.title="Brian Docker Image"
org.opencontainers.image.description="Docker image for Brian - a free, open source simulator for spiking neural networks"
org.opencontainers.image.url=https://hub.docker.com/r/briansimulator/brian
org.opencontainers.image.source=https://github.com/brian-team/brian2
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# https://github.com/docker/setup-qemu-action
with:
platforms: 'amd64,arm64'
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# https://github.com/docker/setup-buildx-action
- name: Login to DockerHub
uses: docker/login-action@v3
# https://github.com/docker/login-action
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: load distribution 📦
uses: actions/download-artifact@v4
with:
name: packages
pattern: "*cp312*.whl"
path: dist/
- name: Build (and potentially push) the Docker image
uses: docker/build-push-action@v5
# https://github.com/docker/build-push-action
with:
context: .
file: docker/Dockerfile
build-args: |
'BASE_IMAGE_TAG=3.12-bookworm'
platforms: 'amd64,arm64'
push: github.repository == 'brian-team/brian2' && startsWith(github.ref, 'refs/tags')
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
6 changes: 6 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ Stimberg, M, Brette, R, Goodman, DFM. “Brian 2, an Intuitive and Efficient Neu
.. image:: https://img.shields.io/conda/vn/conda-forge/brian2.svg
:target: https://anaconda.org/conda-forge/brian2

.. image:: https://github.com/brian-team/brian2/actions/workflows/publish.yml/badge.svg
:target: https://github.com/brian-team/brian2/actions/workflows/publish.yml
:alt: Publish status on GitHub Actions

.. image:: https://img.shields.io/debian/v/python3-brian/testing
:alt: Debian package
Expand All @@ -40,6 +43,9 @@ Stimberg, M, Brette, R, Goodman, DFM. “Brian 2, an Intuitive and Efficient Neu
:alt: AUR version
:target: https://aur.archlinux.org/packages/python-brian2

.. image:: https://img.shields.io/docker/pulls/briansimulator/brian
:alt: Docker Pulls


.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.8099373.svg
:target: https://doi.org/10.5281/zenodo.8099373
Expand Down
62 changes: 62 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# docker buildx build --push --platform linux/amd64,linux/arm64/v8 -o type=image -t briansimulator/brian -f docker/Dockerfile .
# docker run -it --init --rm -p 8888:8888 briansimulator/brian

ARG BASE_IMAGE_TAG=3.12-slim-bookworm
FROM python:${BASE_IMAGE_TAG}

LABEL maintainer="Ben Evans <[email protected]>" \
org.opencontainers.image.vendor="The Brian Development Team" \
org.opencontainers.image.licenses="CeCILL 2.1" \
org.opencontainers.image.title="Brian Docker Image" \
org.opencontainers.image.description="Docker image for Brian - a free, open source simulator for spiking neural networks" \
org.opencontainers.image.homepage="https://briansimulator.org" \
org.opencontainers.image.url="https://hub.docker.com/r/briansimulator/brian" \
org.opencontainers.image.source="https://github.com/brian-team/brian2" \
org.opencontainers.image.documentation="https://brian2.readthedocs.io/"

# Install dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
build-essential \
g++ \
git \
sudo \
&& rm -rf /var/lib/apt/lists/*

# Copy wheels into the image
COPY dist /tmp/dist

# Install Brian2 and recommended packages
ARG TARGETARCH
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be preferable to get the architecture from the image directly with something like: dpkg --print-architecture?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make clear that there is no misunderstanding: this is not an argument that you need to set, it is coming from docker (https://docs.docker.com/build/building/variables/#multi-platform-build-arguments). I understood that I have to declare them like this (but I didn't actually test it):

These arguments are useful for doing cross-compilation in multi-platform builds. They're available in the global scope of the Dockerfile, but they aren't automatically inherited by build stages. To use them inside stage, you must declare them:

But there might be another way that is easier – ideally something that uses the same terminology as the pip wheel file names…

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough - six of one, half a dozen of the other I guess 🤷‍♂️

RUN pip install --no-cache-dir --only-binary=:all: \
scipy \
matplotlib \
jupyterlab \
pytest \
pytest-xdist \
&& if [ "${TARGETARCH}" = "arm64" ]; then WHEEL_ARCH=aarch64; else WHEEL_ARCH=x86_64; fi \
&& pip install /tmp/dist/Brian2*_${WHEEL_ARCH}.whl brian2tools \
&& rm -rf /tmp/dist

# Create a non-root user
ARG USER="monty" \
GROUP="monty" \
PASSWORD="monty"
RUN groupadd ${GROUP} && \
useradd -ms /bin/bash -g ${GROUP} -G sudo ${USER} && \
echo "${USER}:${PASSWORD}" | chpasswd
ENV HOME="/home/${USER}"
RUN chown -R ${USER}:${USER} ${HOME}
USER ${USER}

# Copy tutorial notebooks and example scripts to home directory
WORKDIR ${HOME}
RUN git clone https://github.com/brian-team/brian2.git \
&& mv brian2/examples examples \
&& mv brian2/tutorials tutorials \
&& chmod -R +x tutorials \
&& chmod -R +x examples \
&& find . -name '*.ipynb' -exec jupyter trust {} \; \
&& rm -rf brian2

EXPOSE 8888
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--no-browser"]
26 changes: 26 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
Instructions
============

To build the image manually for testing on a local host for the native architecture:

`docker build -t briansimulator/brian -f docker/Dockerfile .`

Alternatively, to test multi-architecture builds and push to docker hub, first login:

`docker login`

Then execute:

`docker buildx build --push --platform linux/amd64,linux/arm64/v8 -o type=image -t briansimulator/brian -f docker/Dockerfile .`

Finally, to run the image:

`docker run -it --init -p 8888:8888 briansimulator/brian`

Or if you prefer a simple `bash` terminal rather than JupyterLab:

`docker run -it --init briansimulator/brian /bin/bash`

Or to run the tests:

`docker run -it --init --rm briansimulator/brian python -c 'import brian2; brian2.test(test_standalone="cpp_standalone")'`
Loading