Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cargo-chef doesn't cache local dependencies #75

Open
prestontw opened this issue Aug 5, 2021 · 0 comments
Open

cargo-chef doesn't cache local dependencies #75

prestontw opened this issue Aug 5, 2021 · 0 comments

Comments

@prestontw
Copy link
Contributor

(The repo this issue is based on is private. I will describe the issue first, then try to recreate in a public minimal repo for further clarity and reproducibility.)

I have a project directory with structure

Cargo.toml  # [virtual manifest](https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-manifest) cargo.toml
src
  - libA
  - libB
  - libC
  - serviceZ
  - serviceY
  - serviceX
  - ...
...

Each of the services has individual Cargo.toml files and its own Dockerfile.

I've tried adding cargo-chef to one of these services (currently, we are either not being cache friendly or using a cargo new approach), say, serviceZ. cargo-chef does excellently at caching external dependencies for the entire workspace, but because it skelefies libA, libB, and libC, it has to rebuild these local library dependencies for each change in serviceZ. (The actual repo is too large to individually specify what dependencies to copy over and doing so would lead to keeping dependencies between the project and the Cargo.toml file in sync.) These end up consuming most of the compilation time: using cargo chef can bring subsequent runs from 404s down to 270s, but using a cargo new approach brings subsequent runs down from 404s to 70s. I can also visually see that we are recompiling local dependencies when running cargo build in the cargo chef approach while only serviceZ is compiled in the cargo new approach.

This is related to #74 and #64 in that I want very fine control over what I skelefy.

A quick aside: the cargo new approach suffers from changes in serviceZ invalidating the entire COPY src src line, leading to discarding the dependency cache on source changes. The way I've gotten around it is to separate this process into several stages, similar to the approach in cargo chef, but instead of sharing the recipe.json file between stages, I'm sharing an entire directory that should be the same between runs:

FROM rust as assembler
WORKDIR /usr/app
COPY src src
RUN rm -rf src/serviceZ
RUN USER=root cargo new --bin ./src/serviceZ
COPY src/serviceZ/Cargo.toml src/serviceZ/
# because the above stage should be the same between changes to `serviceZ`, 
# we can cache copying it to other stages

FROM rust as cacher
WORKDIR /usr/app
COPY Cargo.lock Cargo.lock
COPY rust-patches rust-patches
COPY vendored vendored
COPY .cargo .cargo
COPY Cargo.toml Cargo.toml
COPY --from=assembler /usr/app/src ./src
RUN cargo build --offline -p serviceZ && rm target/debug/deps/serviceZ*

FROM rust as builder
# actually copies over contents of serviceZ and builds it using cached items in cacher stage
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant