You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(The repo this issue is based on is private. I will describe the issue first, then try to recreate in a public minimal repo for further clarity and reproducibility.)
Each of the services has individual Cargo.toml files and its own Dockerfile.
I've tried adding cargo-chef to one of these services (currently, we are either not being cache friendly or using a cargo new approach), say, serviceZ. cargo-chef does excellently at caching external dependencies for the entire workspace, but because it skelefies libA, libB, and libC, it has to rebuild these local library dependencies for each change in serviceZ. (The actual repo is too large to individually specify what dependencies to copy over and doing so would lead to keeping dependencies between the project and the Cargo.toml file in sync.) These end up consuming most of the compilation time: using cargo chef can bring subsequent runs from 404s down to 270s, but using a cargo new approach brings subsequent runs down from 404s to 70s. I can also visually see that we are recompiling local dependencies when running cargo build in the cargo chef approach while only serviceZ is compiled in the cargo new approach.
This is related to #74 and #64 in that I want very fine control over what I skelefy.
A quick aside: the cargo new approach suffers from changes in serviceZ invalidating the entire COPY src src line, leading to discarding the dependency cache on source changes. The way I've gotten around it is to separate this process into several stages, similar to the approach in cargo chef, but instead of sharing the recipe.json file between stages, I'm sharing an entire directory that should be the same between runs:
FROM rust as assembler
WORKDIR /usr/app
COPY src src
RUN rm -rf src/serviceZ
RUN USER=root cargo new --bin ./src/serviceZ
COPY src/serviceZ/Cargo.toml src/serviceZ/
# because the above stage should be the same between changes to `serviceZ`, # we can cache copying it to other stagesFROM rust as cacher
WORKDIR /usr/app
COPY Cargo.lock Cargo.lock
COPY rust-patches rust-patches
COPY vendored vendored
COPY .cargo .cargo
COPY Cargo.toml Cargo.toml
COPY --from=assembler /usr/app/src ./src
RUN cargo build --offline -p serviceZ && rm target/debug/deps/serviceZ*
FROM rust as builder
# actually copies over contents of serviceZ and builds it using cached items in cacher stage
...
The text was updated successfully, but these errors were encountered:
(The repo this issue is based on is private. I will describe the issue first, then try to recreate in a public minimal repo for further clarity and reproducibility.)
I have a project directory with structure
Each of the services has individual
Cargo.toml
files and its ownDockerfile
.I've tried adding
cargo-chef
to one of these services (currently, we are either not being cache friendly or using acargo new
approach), say,serviceZ
.cargo-chef
does excellently at caching external dependencies for the entire workspace, but because it skelefieslibA
,libB
, andlibC
, it has to rebuild these local library dependencies for each change inserviceZ
. (The actual repo is too large to individually specify what dependencies to copy over and doing so would lead to keeping dependencies between the project and theCargo.toml
file in sync.) These end up consuming most of the compilation time: using cargo chef can bring subsequent runs from404s
down to270s
, but using acargo new
approach brings subsequent runs down from404s
to70s
. I can also visually see that we are recompiling local dependencies when runningcargo build
in thecargo chef
approach while onlyserviceZ
is compiled in thecargo new
approach.This is related to #74 and #64 in that I want very fine control over what I skelefy.
A quick aside: the
cargo new
approach suffers from changes inserviceZ
invalidating the entireCOPY src src
line, leading to discarding the dependency cache on source changes. The way I've gotten around it is to separate this process into several stages, similar to the approach incargo chef
, but instead of sharing therecipe.json
file between stages, I'm sharing an entire directory that should be the same between runs:The text was updated successfully, but these errors were encountered: