Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch to SDK container for builds #525

Merged
merged 16 commits into from
Nov 20, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 5 additions & 17 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,19 +1,7 @@
# syntax=docker/dockerfile:1.1.3-experimental

FROM fedora:30 AS origin
RUN dnf makecache && dnf -y update

FROM origin AS base
RUN dnf -y groupinstall "C Development Tools and Libraries" \
&& dnf -y install \
rpmdevtools dnf-plugins-core createrepo_c \
cmake git meson perl-ExtUtils-MakeMaker python which \
bc hostname intltool grub2-tools gperf kmod rsync wget \
elfutils-devel libcap-devel openssl-devel \
&& useradd builder

FROM origin AS util
RUN dnf -y install createrepo_c e2fsprogs gdisk grub2-tools kpartx lz4 veritysetup dosfstools mtools
ARG SDK
FROM ${SDK} as sdk

# The experimental cache mount type doesn't expand arguments, so our choices are limited.
# We can either reuse the same cache for all builds, which triggers overlayfs errors if
Expand All @@ -24,11 +12,11 @@ FROM scratch AS cache
ARG PACKAGE
ARG ARCH
# We can't create directories via RUN in a scratch container, so take an existing one.
COPY --chown=1000:1000 --from=base /tmp /cache
COPY --chown=1000:1000 --from=sdk /tmp /cache
# Ensure the ARG variables are used in the layer to prevent reuse by other builds.
COPY --chown=1000:1000 .dockerignore /cache/.${PACKAGE}.${ARCH}

FROM base AS rpmbuild
FROM sdk AS rpmbuild
ARG PACKAGE
ARG ARCH
ARG NOCACHE
Expand Down Expand Up @@ -70,7 +58,7 @@ RUN --mount=source=.cargo,target=/home/builder/.cargo \
FROM scratch AS rpm
COPY --from=rpmbuild /home/builder/rpmbuild/RPMS/*/*.rpm /output/

FROM util AS imgbuild
FROM sdk AS imgbuild
ARG PACKAGES
ARG ARCH
ARG NOCACHE
Expand Down
31 changes: 30 additions & 1 deletion Makefile.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ DOCKER_BUILDKIT = "1"

[env.development]
IMAGE = "aws-k8s"
# Defined here to allow us to override ${BUILDSYS_ARCH} on the command line.
BUILDSYS_SDK_IMAGE = "thar/sdk-${BUILDSYS_ARCH}:0.0.1"
# Permit pulling directly Upstream URLs when lookaside cache results in MISSes.
BUILDSYS_ALLOW_UPSTREAM_SOURCE_URL = "true"
# Extra flags used when spawning containers.
Expand All @@ -28,11 +30,38 @@ script = [
'''
mkdir -p ${BUILDSYS_OUTPUT_DIR}
mkdir -p ${GO_MOD_CACHE}

for cmd in bash curl docker gunzip ; do
if ! command -v ${cmd} >/dev/null 2>&1 ; then
echo "required program '${cmd}' not found" >&2
exit 1
fi
done
'''
]

[tasks.fetch]
dependencies = ["fetch-workspaces", "fetch-vendored"]
dependencies = [
"fetch-sdk",
"fetch-workspaces",
"fetch-vendored",
]

[tasks.fetch-sdk]
dependencies = ["setup"]
script_runner = "bash"
script = [
'''
set -o pipefail
if ! docker image inspect ${BUILDSYS_SDK_IMAGE} >/dev/null 2>&1 ; then
if ! curl https://thar-upstream-lookaside-cache.s3.us-west-2.amazonaws.com/${BUILDSYS_SDK_IMAGE}.tar.gz \
| gunzip | docker load ; then
echo "failed to load '${BUILDSYS_SDK_IMAGE}'" >&2
exit 1
fi
fi
'''
]

[tasks.fetch-workspaces]
dependencies = ["setup"]
Expand Down
Loading