Add a warning about plugins and the agent in docker #379
Conversation
pages/agent/v3/docker.md.erb
Outdated
| @@ -15,9 +15,9 @@ You can run the Buildkite Agent inside a Docker container using the [`buildkite/ | |||
|
|
|||
| ## 🚨 Incompatibilities with plugins | |||
|
|
|||
| Whilst the agnt runs well in docker, there are presently some difficulties when combining your agent in a docker container and using many of our [plugins](https://buildkite.com/plugins). These can largely be mitigated by ensuring that the `builds-path` from the host is bind mounted into your agent container with the same path as on the host and having the buildkite-agent installed on the host so that it can be mounted into containers. | |||
| While the agent runs well in Docker, there are presently some difficulties when running an agent in a Docker container and using many of our [plugins](https://buildkite.com/plugins). These issues can largely be mitigated by ensuring that the `builds-path` from the host is bind mounted into your agent container with the same path as on the host, and having the `buildkite-agent` installed on the host so that it can be mounted into containers. | |||
There was a problem hiding this comment.
I feel like the last part of the paragraph could do with an explicit example of how to run the agent in a plugin-compatible way. For example (super rough and unchecked), a common configuration is:
docker run \
# Make sure the build checkout dir exists on the host machine, not just the agent container
-v /var/lib/buildkite/builds:/var/lib/buildkite/builds \
# Make sure we use the same path in the container
-e “BUILDKITE_BUILDS_PATH=/var/lib/buildkite/builds” \
# Some Buildkite plugins, like the Docker plugin, mount the buildkite-agent binary into the job’s container to do artifact and meta-data
-v “$(which buildkite-agent):/usr/local/bin/buildkite-agent” \
# Let the agent run Docker commands, so people can use the Docker and Docker Compose Buildkite plugins
-v “/var/run/docker.sock:/var/run/docker.sock” \
buildkite/agent:3 startPeople can then figure out how to do something like that using ECS/k8s etc if they need.
Should we also add a caveat that they should understand what container security protections they’re bypassing by setting these particular options?
There was a problem hiding this comment.
Wow I really made a mess of english in that line huh.
I think your suggestion makes a heap of sense @toolmantim, that's where I was hoping to end up but wanted to chat about it a bit first.
|
@toolmantim I took some of your suggestions and included an example. I also shuffled things around a bit so that the caveats are around accessing docker within a build vs specifically plugins. |
|
Those changes look awesome @lox! I think the issues justify the amount of words/attention we're giving it, and that people running big buildkite/agent Docker setups (especially on platforms like k8s and things) need to be aware of the path-of-least-pain when setting it up… otherwise it'll be developers discovering the pain points months later when they try to use one of our Docker plugins. It's also a big motivator to have a better answer! 😁 |
|
Yup, I agree @toolmantim. Anything else before we ship it? |
|
Not from me, only @harrietgrace’s edits I think! |
|
I updated a couple of little words, nothing major. Happy for it to go out! Awesome additions @lox 👏🏻 🚢🚀 |
This adds a really prominent warning about using the agent in docker and our plugins.
The issue at buildkite-plugins/docker-buildkite-plugin#61 provides some more context.