Feature Request: Support for Buildstream (buildstream.build)

[jmt-lab]

Hi, this product is really nice especially since it has an easy configuration to work with. However it runs into some compatibility issues with buildstream (buildstream.build). I figured since there is no issue here yet I'd go ahead and create one. The two issues i've run into and some solutions i've found:

• Direct use of nativelink as the artifact cache server fails when buildstream tries to initialize remote caches with:

[--:--:--][        ][    main:core activity                 ] WARNING Failed to initialize remote http://localhost:50051: Remote initialisation failed with status UNKNOWN: FetchBlob: 2: Stream removed

I was able to work around this funny enough by putting a buildbox-casd local cache via

buildbox-casd --cas-remote grpc://localhost:50051 --cas-instance main --bind 127.0.0.1:50052 /tmp/casd

Then it looks like everything was happy

• And similar to with buildbarn it runs into this fun shenanigans:

Staging the input root as the filesystem root[](https://docs.buildstream.build/2.3/using_configuring_remote_execution.html#staging-the-input-root-as-the-filesystem-root)

BuildStream requires that the input root given to the remote execution service be treated as the absolute filesystem root.

This is because BuildStream provides guarantees that all build dependencies, including the base runtime and compilers, are defined by elements and run within a sandboxed and isolated build environment, but the [REAPI](https://github.com/bazelbuild/remote-apis) was originally developped without this determinism and control in mind. Instead, typically it is up to the user to configure a cluster to use a docker image to build payloads with, rather than allowing the REAPI client to control the entire sandbox.

Unfortunately the ability to dictate that the input root be treated as the filesystem root in a container on remote workers in the cluster is not yet standardized in the REAPI protocol.

Note

The input root is referred to as the input_root_digest member of the Action message as defined in the [protocol](https://github.com/bazelbuild/remote-apis/blob/main/build/bazel/remote/execution/v2/remote_execution.proto)

This manifests in having permission denied on the work environment since it expects to use absolute paths, One way to address this could be to allow some sort of method to tell nativelink workers to run everything through buildbox-run-bubblewrap or something of the sort.

[MarcusSorealheis]

Thank you for raising the issue. We will prioritize this it internally and revert back.

[jmt-lab]

Thanks! Let me know if there is anything you want me to try, I'm also going to poke around and see what I can find out myself.

[allada]

Any chance you have a cluster configuration yaml (k8 or docker or something) we can use to debug this?

[jmt-lab]

I used the docker-compose example you have in your repo for my testing and setup buildstream with the following:

~/.config/buildstream.conf

artifacts:
  servers:
  - url: http://localhost:50051 (or cas port)
     type: push
remote-execution:
  execution-service:
    url: http://localhost:50051
  action-cache-service:
    url: http://localhost:50051
  storage-service:
    url: http://localhost:50051

I also used the base usage tutorial for a simplet est project: https://docs.buildstream.build/2.3/using_tutorial.html specifically the running commands hello.c example