CODEWITHSUNDEEP
rustrust-cargo
Tomas Aschan
Tomas Aschan

Reputation: 60564

Can Cargo download and build dependencies without also building the application?

Is there a way to tell Cargo to install and build all my dependencies, but not attempt to build my application?

I thought cargo install would do that, but it actually goes all the way to building my app too. I want to get to a state where cargo build would find all dependencies ready to use, but without touching the /src directory.

What I'm really trying to accomplish:

I'm trying to build a Docker image for a Rust application, where I'd like to do the following steps:

Build time (docker build .):

  1. import a docker image with rust tooling installed
  2. add my Cargo.toml and Cargo.lock files
  3. download and build all dependencies
  4. add my source directory to the image
  5. build my source code

Run time (docker run ...):

  1. run the application

I've tried the following Dockerfile, but the indicated step builds my application as well (which of course fails since the source directory isn't there yet):

FROM jimmycuadra/rust

ADD Cargo.toml /source
ADD Cargo.lock /source

RUN cargo install # <-- failure here

ADD src /source/src
RUN cargo build

ENTRYPOINT cargo run

The reason I want to separate the install dependencies step from actually building my application, is that if I don't change the dependencies, I want Docker to be able use a cached image with all dependencies already installed and built. Thus, I can't ADD /src /source/src until after installing the dependecies, as that would invalidate the cached image when I change my own code.

Upvotes: 64

Views: 30698

Answers (6)

RikH
RikH

Reputation: 3404

It can be done via cargo init, cargo build, and cargo install. For example, for a project called foo, define the following Dockerfile:

FROM rust:slim-bullseye

# Build dependencies only.
RUN cargo init foo
COPY Cargo.toml foo/
RUN cargo build --release; \
    rm -rf foo

# Install `foo`.
COPY . .
RUN echo "// force Cargo cache invalidation" >> foo/src/main.rs; \
    cargo install --path foo

CMD ["foo"]

Here, cargo init creates the placeholder files that Cargo expects, cargo build builds the dependencies which were specified in the Cargo.toml, and cargo install creates the foo binary. For some reason, the Docker kept building the default project created by cargo init foo. This problem is resolved above by forcing an update for main.rs by appending // force Cargo cache invalidation.

To avoid slow builds due to large build contexts and large layers, make sure that unimportant folders such as target are ignored via .dockerignore. For example, define the following .dockerignore:

**/*.lock
LICENSE
README.md
target

Upvotes: 1

Maya
Maya

Reputation: 1527

The cargo-chef tool is designed to solve this problem. Here's an example from the README on how you can use it in the Dockerfile:

FROM lukemathwalker/cargo-chef as planner
WORKDIR app
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM lukemathwalker/cargo-chef as cacher
WORKDIR app
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM rust as builder
WORKDIR app
COPY . .
# Copy over the cached dependencies
COPY --from=cacher /app/target target
COPY --from=cacher $CARGO_HOME $CARGO_HOME
RUN cargo build --release --bin app

FROM rust as runtime
WORKDIR app
COPY --from=builder /app/target/release/app /usr/local/bin
ENTRYPOINT ["/usr/local/bin/app"]

Upvotes: 5

user96425
user96425

Reputation: 83

I just wanted to post this here so others will see it going forward. There's an experimental tool for Docker I've just started using called cargo-wharf (https://github.com/denzp/cargo-wharf/tree/master/cargo-wharf-frontend). It's a Docker BuildKit frontend that caches built cargo dependencies for you. If you only change one of your source files, that's the only thing that gets rebuilt when you call docker build. You use it by annotating your Cargo.toml file, then directing Docker to your Cargo.toml instead of a Dockerfile. Go check it out, it's exactly what I wanted. (I am in no way affiliated with the project.)

Upvotes: 0

Dmitry Vasilev
Dmitry Vasilev

Reputation: 6508

Based on a GitHub comment

FROM rust:1.37

WORKDIR /usr/src

# Create blank project
RUN USER=root cargo new PROJ

# We want dependencies cached, so copy those first.
COPY Cargo.toml /usr/src/PROJ/
COPY Cargo.lock /usr/src/PROJ/

WORKDIR /usr/src/PROJ

# This is a dummy build to get the dependencies cached.
RUN cargo build --release

# Now copy in the rest of the sources
COPY MyPROJECT/src /usr/src/PROJ/src/

# This is the actual build.
RUN cargo build --release \
    && mv target/release/appname /bin \
    && rm -rf /usr/src/PROJ

WORKDIR /

EXPOSE 8888

CMD ["/bin/appname"]

Upvotes: 5

Shepmaster
Shepmaster

Reputation: 430634

There is no native support for building just the dependencies in Cargo, as far as I know. There is an open issue for it. I wouldn't be surprised if you could submit something to Cargo to accomplish it though, or perhaps create a third-party Cargo addon. I've wanted this functionality for cargo doc as well, when my own code is too broken to compile ;-)

However, the Rust playground that I maintain does accomplish your end goal. There's a base Docker container that installs Rustup and copies in a Cargo.toml with all of the crates available for the playground. The build steps create a blank project (with a dummy src/lib.rs), then calls cargo build and cargo build --release to compile the crates:

RUN cd / && \
    cargo new playground
WORKDIR /playground

ADD Cargo.toml /playground/Cargo.toml
RUN cargo build
RUN cargo build --release
RUN rm src/*.rs

All of the downloaded crates are stored in the Docker image's $HOME/.cargo directory and all of the built crates are stored in the applications target/{debug,release} directories.

Later on, the real source files are copied into the container and cargo build / cargo run can be executed again, using the now-compiled crates.

If you were building an executable project, you'd want to copy in the Cargo.lock as well.

Upvotes: 27

Zac Delventhal
Zac Delventhal

Reputation: 4010

If you add a dummy main or lib file, you can use cargo build to just pull down the dependencies. I'm currently using this solution for my Docker based project:

COPY Cargo.toml .
RUN mkdir src \
    && echo "// dummy file" > src/lib.rs \
    && cargo build

I'm using --volumes, so I'm done at this point. The host volumes come in and blow away the dummy file, and cargo uses the cached dependencies when I go to build the source later. This solution will work just as well if you want to add a COPY (or ADD) later and use the cached dependencies though.

Upvotes: 11

Related Questions