concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit

Overview

asciicinema example

BuildKit

GoDoc Build Status Go Report Card codecov

BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner.

Key features:

  • Automatic garbage collection
  • Extendable frontend formats
  • Concurrent dependency resolution
  • Efficient instruction caching
  • Build cache import/export
  • Nested build job invocations
  • Distributable workers
  • Multiple output formats
  • Pluggable architecture
  • Execution without root privileges

Read the proposal from https://github.com/moby/moby/issues/32925

Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056cc5317

Join #buildkit channel on Docker Community Slack

ℹ️ If you are visiting this repo for the usage of BuildKit-only Dockerfile features like RUN --mount=type=(bind|cache|tmpfs|secret|ssh), please refer to frontend/dockerfile/docs/syntax.md.

ℹ️ BuildKit has been integrated to docker build since Docker 18.06 . You don't need to read this document unless you want to use the full-featured standalone version of BuildKit.

Used by

BuildKit is used by the following projects:

Quick start

ℹ️ For Kubernetes deployments, see examples/kubernetes.

BuildKit is composed of the buildkitd daemon and the buildctl client. While the buildctl client is available for Linux, macOS, and Windows, the buildkitd daemon is only available for Linux currently.

The buildkitd daemon requires the following components to be installed:

The latest binaries of BuildKit are available here for Linux, macOS, and Windows.

Homebrew package (unofficial) is available for macOS.

$ brew install buildkit

To build BuildKit from source, see .github/CONTRIBUTING.md.

Starting the buildkitd daemon:

You need to run buildkitd as the root user on the host.

$ sudo buildkitd

To run buildkitd as a non-root user, see docs/rootless.md.

The buildkitd daemon supports two worker backends: OCI (runc) and containerd.

By default, the OCI (runc) worker is used. You can set --oci-worker=false --containerd-worker=true to use the containerd worker.

We are open to adding more backends.

To start the buildkitd daemon using systemd socket activiation, you can install the buildkit systemd unit files. See Systemd socket activation

The buildkitd daemon listens gRPC API on /run/buildkit/buildkitd.sock by default, but you can also use TCP sockets. See Expose BuildKit as a TCP service.

Exploring LLB

BuildKit builds are based on a binary intermediate format called LLB that is used for defining the dependency graph for processes running part of your build. tl;dr: LLB is to Dockerfile what LLVM IR is to C.

  • Marshaled as Protobuf messages
  • Concurrently executable
  • Efficiently cacheable
  • Vendor-neutral (i.e. non-Dockerfile languages can be easily implemented)

See solver/pb/ops.proto for the format definition, and see ./examples/README.md for example LLB applications.

Currently, the following high-level languages has been implemented for LLB:

Exploring Dockerfiles

Frontends are components that run inside BuildKit and convert any build definition to LLB. There is a special frontend called gateway (gateway.v0) that allows using any image as a frontend.

During development, Dockerfile frontend (dockerfile.v0) is also part of the BuildKit repo. In the future, this will be moved out, and Dockerfiles can be built using an external image.

Building a Dockerfile with buildctl

buildctl build \
    --frontend=dockerfile.v0 \
    --local context=. \
    --local dockerfile=.
# or
buildctl build \
    --frontend=dockerfile.v0 \
    --local context=. \
    --local dockerfile=. \
    --opt target=foo \
    --opt build-arg:foo=bar

--local exposes local source files from client to the builder. context and dockerfile are the names Dockerfile frontend looks for build context and Dockerfile location.

Building a Dockerfile using external frontend:

External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in ./frontend/dockerfile/cmd/dockerfile-frontend but will move out of this repository in the future (#163). For automatic build from master branch of this repository docker/dockerfile-upstream:master or docker/dockerfile-upstream:master-labs image can be used.

buildctl build \
    --frontend gateway.v0 \
    --opt source=docker/dockerfile \
    --local context=. \
    --local dockerfile=.
buildctl build \
    --frontend gateway.v0 \
    --opt source=docker/dockerfile \
    --opt context=git://github.com/moby/moby \
    --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org

Building a Dockerfile with experimental features like RUN --mount=type=(bind|cache|tmpfs|secret|ssh)

See frontend/dockerfile/docs/experimental.md.

Output

By default, the build result and intermediate cache will only remain internally in BuildKit. An output needs to be specified to retrieve the result.

Image/Registry

buildctl build ... --output type=image,name=docker.io/username/image,push=true

To export the cache embed with the image and pushing them to registry together, type registry is required to import the cache, you should specify --export-cache type=inline and --import-cache type=registry,ref=.... To export the cache to a local directy, you should specify --export-cache type=local. Details in Export cache.

buildctl build ...\
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=inline \
  --import-cache type=registry,ref=docker.io/username/image

Keys supported by image output:

  • name=[value]: image name
  • push=true: push after creating the image
  • push-by-digest=true: push unnamed image
  • registry.insecure=true: push to insecure HTTP registry
  • oci-mediatypes=true: use OCI mediatypes in configuration JSON instead of Docker's
  • unpack=true: unpack image after creation (for use with containerd)
  • dangling-name-prefix=[value]: name image with [email protected]<digest> , used for anonymous images
  • name-canonical=true: add additional canonical name [email protected]<digest>
  • compression=[uncompressed,gzip,estargz,zstd]: choose compression type for layers newly created and cached, gzip is default value. estargz should be used with oci-mediatypes=true.
  • force-compression=true: forcefully apply compression option to all layers (including already existing layers).
  • buildinfo=[all,imageconfig,metadata,none]: choose build dependency version to export (default all).

If credentials are required, buildctl will attempt to read Docker configuration file $DOCKER_CONFIG/config.json. $DOCKER_CONFIG defaults to ~/.docker.

Local directory

The local client will copy the files directly to the client. This is useful if BuildKit is being used for building something else than container images.

buildctl build ... --output type=local,dest=path/to/output-dir

To export specific files use multi-stage builds with a scratch stage and copy the needed files into that stage with COPY --from.

...
FROM scratch as testresult

COPY --from=builder /usr/src/app/testresult.xml .
...
buildctl build ... --opt target=testresult --output type=local,dest=path/to/output-dir

Tar exporter is similar to local exporter but transfers the files through a tarball.

buildctl build ... --output type=tar,dest=out.tar
buildctl build ... --output type=tar > out.tar

Docker tarball

# exported tarball is also compatible with OCI spec
buildctl build ... --output type=docker,name=myimage | docker load

OCI tarball

buildctl build ... --output type=oci,dest=path/to/output.tar
buildctl build ... --output type=oci > output.tar

containerd image store

The containerd worker needs to be used

buildctl build ... --output type=image,name=docker.io/username/image
ctr --namespace=buildkit images ls

To change the containerd namespace, you need to change worker.containerd.namespace in /etc/buildkit/buildkitd.toml.

Cache

To show local build cache (/var/lib/buildkit):

buildctl du -v

To prune local build cache:

buildctl prune

Garbage collection

See ./docs/buildkitd.toml.md.

Export cache

BuildKit supports the following cache exporters:

  • inline: embed the cache into the image, and push them to the registry together
  • registry: push the image and the cache separately
  • local: export to a local directory
  • gha: export to GitHub Actions cache

In most case you want to use the inline cache exporter. However, note that the inline cache exporter only supports min cache mode. To enable max cache mode, push the image and the cache separately by using registry cache exporter.

Inline (push image and cache together)

buildctl build ... \
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=inline \
  --import-cache type=registry,ref=docker.io/username/image

Note that the inline cache is not imported unless --import-cache type=registry,ref=... is provided.

ℹ️ Docker-integrated BuildKit (DOCKER_BUILDKIT=1 docker build) and docker buildxrequires --build-arg BUILDKIT_INLINE_CACHE=1 to be specified to enable the inline cache exporter. However, the standalone buildctl does NOT require --opt build-arg:BUILDKIT_INLINE_CACHE=1 and the build-arg is simply ignored.

Registry (push image and cache separately)

buildctl build ... \
  --output type=image,name=localhost:5000/myrepo:image,push=true \
  --export-cache type=registry,ref=localhost:5000/myrepo:buildcache \
  --import-cache type=registry,ref=localhost:5000/myrepo:buildcache

--export-cache options:

  • type=registry
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • ref=docker.io/user/image:tag: reference
  • oci-mediatypes=true|false: whether to use OCI mediatypes in exported manifests. Since BuildKit v0.8 defaults to true.

--import-cache options:

  • type=registry
  • ref=docker.io/user/image:tag: reference

Local directory

buildctl build ... --export-cache type=local,dest=path/to/output-dir
buildctl build ... --import-cache type=local,src=path/to/input-dir

The directory layout conforms to OCI Image Spec v1.0.

--export-cache options:

  • type=local
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • dest=path/to/output-dir: destination directory for cache exporter
  • oci-mediatypes=true|false: whether to use OCI mediatypes in exported manifests. Since BuildKit v0.8 defaults to true.

--import-cache options:

  • type=local
  • src=path/to/input-dir: source directory for cache importer
  • digest=sha256:deadbeef: digest of the manifest list to import.
  • tag=customtag: custom tag of image. Defaults "latest" tag digest in index.json is for digest, not for tag

GitHub Actions cache (experimental)

buildctl build ... \
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=gha \
  --import-cache type=gha

Following attributes are required to authenticate against the Github Actions Cache service API:

  • url: Cache server URL (default $ACTIONS_CACHE_URL)
  • token: Access token (default $ACTIONS_RUNTIME_TOKEN)

ℹ️ This type of cache can be used with Docker Build Push Action where url and token will be automatically set. To use this backend in a inline run step, you have to include crazy-max/ghaction-github-runtime in your workflow to expose the runtime.

--export-cache options:

  • type=gha
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • scope=buildkit: which scope cache object belongs to (default buildkit)

--import-cache options:

  • type=gha
  • scope=buildkit: which scope cache object belongs to (default buildkit)

Consistent hashing

If you have multiple BuildKit daemon instances but you don't want to use registry for sharing cache across the cluster, consider client-side load balancing using consistent hashing.

See ./examples/kubernetes/consistenthash.

Metadata

To output build metadata such as the image digest, pass the --metadata-file flag. The metadata will be written as a JSON object to the specified file. The directory of the specified file must already exist and be writable.

buildctl build ... --metadata-file metadata.json
{"containerimage.digest": "sha256:ea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae14",...}

Systemd socket activation

On Systemd based systems, you can communicate with the daemon via Systemd socket activation, use buildkitd --addr fd://. You can find examples of using Systemd socket activation with BuildKit and Systemd in ./examples/systemd.

Expose BuildKit as a TCP service

The buildkitd daemon can listen the gRPC API on a TCP socket.

It is highly recommended to create TLS certificates for both the daemon and the client (mTLS). Enabling TCP without mTLS is dangerous because the executor containers (aka Dockerfile RUN containers) can call BuildKit API as well.

buildkitd \
  --addr tcp://0.0.0.0:1234 \
  --tlscacert /path/to/ca.pem \
  --tlscert /path/to/cert.pem \
  --tlskey /path/to/key.pem
buildctl \
  --addr tcp://example.com:1234 \
  --tlscacert /path/to/ca.pem \
  --tlscert /path/to/clientcert.pem \
  --tlskey /path/to/clientkey.pem \
  build ...

Load balancing

buildctl build can be called against randomly load balanced the buildkitd daemon.

See also Consistent hashing for client-side load balancing.

Containerizing BuildKit

BuildKit can also be used by running the buildkitd daemon inside a Docker container and accessing it remotely.

We provide the container images as moby/buildkit:

  • moby/buildkit:latest: built from the latest regular release
  • moby/buildkit:rootless: same as latest but runs as an unprivileged user, see docs/rootless.md
  • moby/buildkit:master: built from the master branch
  • moby/buildkit:master-rootless: same as master but runs as an unprivileged user, see docs/rootless.md

To run daemon in a container:

docker run -d --name buildkitd --privileged moby/buildkit:latest
export BUILDKIT_HOST=docker-container://buildkitd
buildctl build --help

Podman

To connect to a BuildKit daemon running in a Podman container, use podman-container:// instead of docker-container:// .

podman run -d --name buildkitd --privileged moby/buildkit:latest
buildctl --addr=podman-container://buildkitd build --frontend dockerfile.v0 --local context=. --local dockerfile=. --output type=oci | podman load foo

sudo is not required.

Kubernetes

For Kubernetes deployments, see examples/kubernetes.

Daemonless

To run the client and an ephemeral daemon in a single container ("daemonless mode"):

docker run \
    -it \
    --rm \
    --privileged \
    -v /path/to/dir:/tmp/work \
    --entrypoint buildctl-daemonless.sh \
    moby/buildkit:master \
        build \
        --frontend dockerfile.v0 \
        --local context=/tmp/work \
        --local dockerfile=/tmp/work

or

docker run \
    -it \
    --rm \
    --security-opt seccomp=unconfined \
    --security-opt apparmor=unconfined \
    -e BUILDKITD_FLAGS=--oci-worker-no-process-sandbox \
    -v /path/to/dir:/tmp/work \
    --entrypoint buildctl-daemonless.sh \
    moby/buildkit:master-rootless \
        build \
        --frontend \
        dockerfile.v0 \
        --local context=/tmp/work \
        --local dockerfile=/tmp/work

Opentracing support

BuildKit supports opentracing for buildkitd gRPC API and buildctl commands. To capture the trace to Jaeger, set JAEGER_TRACE environment variable to the collection address.

docker run -d -p6831:6831/udp -p16686:16686 jaegertracing/all-in-one:latest
export JAEGER_TRACE=0.0.0.0:6831
# restart buildkitd and buildctl so they know JAEGER_TRACE
# any buildctl command should be traced to http://127.0.0.1:16686/

Running BuildKit without root privileges

Please refer to docs/rootless.md.

Building multi-platform images

Please refer to docs/multi-platform.md.

Contributing

Want to contribute to BuildKit? Awesome! You can find information about contributing to this project in the CONTRIBUTING.md

Issues
  • Dockerfile heredocs

    Dockerfile heredocs

    relates to https://github.com/moby/moby/issues/34423

    As mentioned in #2121, I've been making progress towards implementing heredocs in Dockerfiles, and thought it might be time to open a PR for it :tada:

    I've essentially got all the functionality I think we'd need before wanting to merge, though I'm sure there's some fixes/tests to write before that.

    Things that definitely need resolving before a merge is really possible:

    • [x] Gate the feature behind a build tag, as suggested by @tonistiigi
    • [x] Warn/error/do something if a heredoc is used in a place it's not expected (e.g. an ENV command)
    • [x] Handle RUN heredocs in Windows more elegantly (doesn't look particularly doable with cmd so the current hacky approach, might be the best?)
    • [x] Tests! Currently only the parsing stages are tested, so we some some more complex integration tests.

    I'd really appreciate any feedback anyone has on the current design and implementation!

    opened by jedevc 57
  • Failed to compute cache key in newer version

    Failed to compute cache key in newer version

    This is a docker issue but it seems to be related to BuildKit only. this is something that was still working in docker ~19.03.10 but stopped functioning in 20.10.0+. I managed to bring down my DockerFile to a minimal repro:

    This works (A.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer
    

    This also works (B.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
    

    This no longer works (C.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
    COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer
    

    Output from running A and C after eachother:

    C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f A.Dockerfile .
    [+] Building 3.6s (7/7) FINISHED
     => [internal] load build definition from A.Dockerfile                                                                                                                                                           0.0s
     => => transferring dockerfile: 132B                                                                                                                                                                             0.0s
     => [internal] load .dockerignore                                                                                                                                                                                0.0s
     => => transferring context: 2B                                                                                                                                                                                  0.0s
     => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                2.9s
     => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
     => => resolve docker.io/library/composer:2.0.8                                                                                                                                                                  0.5s
     => CACHED [stage-0 1/2] FROM docker.io/library/php:[email protected]:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                           0.0s
     => [stage-0 2/2] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                           0.1s
     => exporting to image                                                                                                                                                                                           0.1s
     => => exporting layers                                                                                                                                                                                          0.0s
     => => writing image sha256:ea6d75bc9ad24e800c8083e9ea6b7774f2bd9610cb0e61b3640058c9c7fe34c6                                                                                                                     0.0s
    
    C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f C.Dockerfile .
    [+] Building 1.0s (8/8) FINISHED
     => [internal] load build definition from C.Dockerfile                                                                                                                                                           0.0s
     => => transferring dockerfile: 221B                                                                                                                                                                             0.0s
     => [internal] load .dockerignore                                                                                                                                                                                0.0s
     => => transferring context: 2B                                                                                                                                                                                  0.0s
     => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                0.2s
     => FROM docker.io/mlocati/php-extension-installer:latest                                                                                                                                                        0.0s
     => => resolve docker.io/mlocati/php-extension-installer:latest                                                                                                                                                  0.0s
     => => sha256:ccf3a05d8241580ad9d2a6c884a735bb248e90942ab23e0f8197f851a999ddac 526B / 526B                                                                                                                       0.0s
     => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
     => [stage-0 1/3] FROM docker.io/library/php:[email protected]:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                                  0.0s
     => CACHED [stage-0 2/3] COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/                                                                                                   0.0s
     => ERROR [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                     0.0s
    ------
     > [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer:
    ------
    failed to compute cache key: "/usr/bin/composer" not found: not found
    

    This doesn't happen consistently in my build, sometimes everything builds fine and there are no issues. I'm using windows 10 (20H2) and the latest version of Docker Desktop that includes Docker version 20.10.2, build 2291f61, but I have also seen this happen on Linux with the same version

    needs-investigation 
    opened by DRoet 55
  • buildctl: new CLI (

    buildctl: new CLI ("Option C+")

    See https://github.com/moby/buildkit/pull/807#issuecomment-468146089

    Fix #774

    Signed-off-by: Akihiro Suda [email protected]

    opened by AkihiroSuda 54
  • buildkit + gcr.io private repos (credHelpers) do not stack

    buildkit + gcr.io private repos (credHelpers) do not stack

    Docker 18.09-ce here.

    I have FROM directive in my dockerfile pointing to a private registry:

    FROM gcr.io/...
    

    Running DOCKER_BUILDKIT=1 docker build . with this Dockerfile never finishes (after 5 minutes I hit CTRL-C). Without buildkit it builds fine in seconds.

    My ~/.docker/config.json is as follows:

    {
      "credHelpers": {
        "us.gcr.io": "gcloud",
        "staging-k8s.gcr.io": "gcloud",
        "asia.gcr.io": "gcloud",
        "gcr.io": "gcloud",
        "marketplace.gcr.io": "gcloud",
        "eu.gcr.io": "gcloud"
      }
    }
    

    After waiting long time and pressing CTRL-C, the following error is printed (exact image names scrambled with ...):

    ------
     > [stage-1 1/4] FROM gcr.io/...:
    ------
    failed to copy: httpReaderSeeker: failed open: unexpected status code https://gcr.io/v2/...: 403 Forbidden
    

    Bug?

    opened by haizaar 38
  • Support schema1 push for quay?

    Support schema1 push for quay?

    Astonishingly Quay.io still does not support schema2: https://github.com/bazelbuild/rules_docker/issues/102

    DEBU[0011] do request                                    digest=sha256:eb300a827decea6de23bda3e4ec5a60dcb3fb59bd01792fe3b54c08c10f68214 mediatype="application/vnd.docker.distribution.manifest.v2+json" request.headers=map[Content-Type:[application/vnd.docker.distribution.m
    anifest.v2+json]] request.method=PUT size=1245 url="https://quay.io/v2/****/****/manifests/latest"
    DEBU[0012] fetch response received                       digest=sha256:eb300a827decea6de23bda3e4ec5a60dcb3fb59bd01792fe3b54c08c10f68214 mediatype="application/vnd.docker.distribution.manifest.v2+json" response.headers=map[Server:[nginx/1.13.12] Date:[Thu, 24 May 2018 03:1
    1:16 GMT] Content-Type:[application/json] Content-Length:[131]] size=1245 status="415 Unsupported Media Type" url="https://quay.io/v2/****/****/manifests/latest"
    ERRO[0012] /moby.buildkit.v1.Control/Solve returned error: unexpected status: 415 Unsupported Media Type
    

    Do we want to support pushing as schema1?

    I hesitate to add support for such deprecated format, but probably we should do if there are also other registry implementations that lack support for schema2.

    cc @alexellis cc @dmcgowan @stevvooe

    enhancement area/containerd 
    opened by AkihiroSuda 37
  • always display image hashes

    always display image hashes

    It's tough to debug docker building when I can't just get into the previously successful intermediate build image and run the next command manually...

    docker run -it --rm hash_id bash
    # execute the next RUN line here manually.
    

    I would therefore argue that image hashes should always display, just like they do in the current docker.

    question 
    opened by TrentonAdams 36
  • Bridge network

    Bridge network

    adds the support for bridge networking for containerd & runc workers. fixes #28 Needs a review/suggestion Temporary Interface naming.

    NOTE: Still "docker0" is hard-coded, need to provide user input.

    Signed-off-by: Kunal Kushwaha [email protected]

    opened by kunalkushwaha 34
  • CNI network for workers

    CNI network for workers

    This PR enables networking for buildkit workers using CNI plugins. This implementation uses default CNI conf files from standard directories.

    Would like to know feedback on this.

    NOTE: Options for providing custom folders (CNI binaries & conf ) are not yet supported in CLI.

    opened by kunalkushwaha 30
  • Add IncludePatterns and ExcludePatterns options for Copy

    Add IncludePatterns and ExcludePatterns options for Copy

    Allow include and exclude patterns to be specified for the "copy" op, similarly to "local".

    Depends on https://github.com/tonistiigi/fsutil/pull/101

    cc @hinshun

    opened by aaronlehmann 29
  • Add hostname specifying for building

    Add hostname specifying for building

    Fix: #1301

    Signed-off-by: l00397676 [email protected]

    I'm trying to send this hostname arg with build-arg:HOSTNAME=<value>

    Added a hostname to message Meta in solver/pb/ops.proto.

    When executor generates the hosts file (in executor/oci/hosts.go), replace the default hostname buildkitsandbox with the value user specified.

    Test with this Dockerfile:

    FROM docker.io/centos:7
    RUN echo "Env variable HOSTNAME: $HOSTNAME" && \
        echo "hostname: $(hostname)" && \
        echo "kernel value: $(cat /proc/sys/kernel/hostname)"  && \
        echo "/etc/hosts: $(cat /etc/hosts)"
    

    Test result:

    # buildctl build --frontend=dockerfile.v0 --local dockerfile=. --local context=. --output type=docker,dest=docker.tar --opt build-arg:HOSTNAME=testtest --no-cache --progress=plain
    #1 [internal] load .dockerignore
    #1 transferring context: 2B done
    #1 DONE 0.0s
    
    #2 [internal] load build definition from Dockerfile
    #2 transferring dockerfile: 256B done
    #2 DONE 0.0s
    
    #3 [internal] load metadata for docker.io/library/centos:7
    #3 DONE 4.5s
    
    #5 [1/2] FROM docker.io/library/centos:[email protected]:4a701376d03f6b39b8c2a8f4a8e...
    #5 resolve docker.io/library/centos:[email protected]:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c done
    #5 CACHED
    
    #4 [2/2] RUN echo "Env variable HOSTNAME: $HOSTNAME" &&     echo "hostname:...
    #4 0.172 Env variable HOSTNAME: testtest
    #4 0.182 hostname: testtest
    #4 0.183 kernel value: testtest
    #4 0.185 /etc/hosts:
    #4 0.185 127.0.0.1      localhost testtest
    #4 0.185 ::1    localhost ip6-localhost ip6-loopback
    #4 DONE 0.2s
    
    #6 exporting to oci image format
    #6 exporting layers
    #6 exporting layers 2.0s done
    #6 exporting manifest sha256:88d227d6e517608fc265e77967ac10a8b0ab47d3153af424aca66a5b72671d57 0.0s done
    #6 exporting config sha256:cd2b4c557e9c29670f541df579edb3f59e6541aad4e9a4c17aa2b4ef7c9f88ea
    #6 exporting config sha256:cd2b4c557e9c29670f541df579edb3f59e6541aad4e9a4c17aa2b4ef7c9f88ea 0.0s done
    #6 sending tarball
    #6 sending tarball 0.9s done
    #6 DONE 3.0s
    
    opened by jingxiaolu 28
  • ADD from oauth+url fails with 'invalid not-modified ETag'

    ADD from oauth+url fails with 'invalid not-modified ETag'

    It looks issue with ETag is reproducible for url with user:password parts.

    Docker version 20.10.8, build 3967b7d #905 #1159

    Here is a minimal reproducible Dockerfile (*don't too worry about the token: it shouldn't have any access except public entities and will expired in 90 days)

    FROM alpine/git
    ADD https://shatilov-diman:[email protected]/repos/shatilov-diman/loggerpp/commits/master/status version.json
    

    When you execute docker build . twice it became failed permanently with next error: failed to load cache key: invalid not-modified ETag: "4651ed3647ddfd32fbcc47f63b5d4247894622a3e234b81e44115eb3b489a331" oauth2.log

    When I remove user:password parts the issue is gone. without_oauth2.log

    docker system prune also fix the issue for the very next time

    Just in case I've collected headers by curl (but I'm sure docker has another request): curl_headers_oauth2.log curl_headers_without_oauth2.log

    opened by shatilov-diman 2
  • Leaked processes in rootless mode

    Leaked processes in rootless mode

    When running rootless with --oci-worker-no-process-sandbox we have noticed that builds can end leaking processes and create issues for following builds.

    Here is an example Dockerfile:

    FROM ubuntu
    ADD script.sh /
    RUN /script.sh
    RUN echo "Dockerfile done"
    

    and script.sh

    #!/bin/bash
    set -x
    apt-get update
    apt-get install -y netcat
    nc -l 5432 &
    echo "Script done"
    

    When running rootless with --oci-worker-no-process-sandbox here is what happens:

    • buildx hangs at echo "Script done" and never gets to the following step in the Dockerfile
    • if the build times out or is interrupted the processes are leaked and the port is still bound
      652 user      0:00 buildkit-runc --log /home/user/.local/share/buildkit/runc-overlayfs/executor/runc-log.json --log-format json ru
      956 user      0:00 nc -l 5432
    

    In addition, killing the leaked process leaves a zombie

    ubuntu      4282  0.0  0.0 709668  6504 ?        Sl   16:35   0:00      |   \_ /proc/self/exe buildkitd --config /etc/buildkit/buildk
    itd.toml
    ubuntu      4297  0.8  0.1 734252 35676 ?        Sl   16:35   0:04      |   |   \_ buildkitd --config /etc/buildkit/buildkitd.toml
    ubuntu     12340  0.0  0.0      0     0 ?        Z    16:39   0:00      |   \_ [nc] <defunct>
    

    If we run rootless but privileged and without --oci-worker-no-process-sandbox everything works as expected: image builds and we don't leak processes (as expected because they run in a different pid namespace)

    Of course, this example is just a reproduction but we have seen the problem with some Dockerfiles where installing a package will start a daemon.

    I'm wondering if we could track processes started in the background.

    In addition, in our setup we use buildkitd to run concurrent builds and sharing the pid and network namespace is likely to create problems from time to time. Have you considered a buildx kubernetes driver that would start a buildkitd pod (directly or with a job) and delete it when the build is over?

    area/rootless 
    opened by lbernail 4
  • Build progress does not include metadata-only changes

    Build progress does not include metadata-only changes

    I noticed this when I started writing a reply / example for https://github.com/moby/moby/issues/42937.

    Unlike the classic builder, BuildKit build output does not show any output for metadata-only changes;

    • ENV
    • ARG
    • LABEL
    • EXPOSE
    • VOLUME

    I expect this is because BuildKit handles these "smarter", and does not require a container to be started for these, so can perform these changes more optimized. However, from a user-perspective this is confusing, and makes it more difficult to debug builds (and to verify all expected steps are executed).

    For example, taking the following Dockerfile:

    FROM alpine
    MAINTAINER yes I am deprecated
    EXPOSE 80 90
    ENV FOO=hello
    ARG BAR=world
    ENV FOOBAR=$FOO-$BAR
    VOLUME /one/$BAR/three
    WORKDIR /somewhere/$FOO
    LABEL somelabel="FOO is $FOO and BAR is $BAR, but HOME is $HOME"
    

    Building this Dockerfile with BuildKit:

    docker build --no-cache --progress=plain .
    
    #1 [internal] load build definition from Dockerfile
    #1 sha256:436619e75e9fa7ed53747ab1188761bc64b9a1834ad10d28dc65ca17043154ed
    #1 transferring dockerfile: 261B done
    #1 DONE 0.0s
    
    #2 [internal] load .dockerignore
    #2 sha256:f44e2c27f29a7c7301b3218dcb8e1f752a22512742bea0813da9155da126a6c9
    #2 transferring context: 2B done
    #2 DONE 0.0s
    
    #3 [internal] load metadata for docker.io/library/alpine:latest
    #3 sha256:d4fb25f5b5c00defc20ce26f2efc4e288de8834ed5aa59dff877b495ba88fda6
    #3 DONE 0.0s
    
    #4 [1/2] FROM docker.io/library/alpine
    #4 sha256:665ba8b2cdc0cb0200e2a42a6b3c0f8f684089f4cd1b81494fbb9805879120f7
    #4 CACHED
    
    #5 [2/2] WORKDIR /somewhere/hello
    #5 sha256:3699b9e02ffe38815da2767b51abeaee9fe5abfd87b08f47c4658cb6d0838ae5
    #5 DONE 0.0s
    
    #6 exporting to image
    #6 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
    #6 exporting layers 0.0s done
    #6 writing image sha256:dfd5c7c3c622b55721c69e8e70647f0f870ee6da925542fccb990727158a582b done
    #6 DONE 0.0s
    

    Notice that none of the ENV, ARG, LABEL, EXPOSE or VOLUME instructions show in the output. The WORKDIR does show (but likely because that command also implicitly creates the directory if missing). On the other hand, other (likely less useful to the user) output, such as the [internal] steps are included, which still makes the output quite verbose, but lots of output that may not be (directly) relevant to the user.

    Comparing this to the classic builder:

    DOCKER_BUILDKIT=0 docker build --no-cache .
    
    Sending build context to Docker daemon  2.048kB
    Step 1/9 : FROM alpine
     ---> 14119a10abf4
    Step 2/9 : MAINTAINER yes I am deprecated
     ---> Running in 34422aa7e83d
    Removing intermediate container 34422aa7e83d
     ---> e2f22c2f2fc3
    Step 3/9 : EXPOSE 80 90
     ---> Running in ee00569c4d37
    Removing intermediate container ee00569c4d37
     ---> a09b77eb4dbb
    Step 4/9 : ENV FOO=hello
     ---> Running in d387403883cd
    Removing intermediate container d387403883cd
     ---> c806ff2c6920
    Step 5/9 : ARG BAR=world
     ---> Running in 8cc3d39cf99f
    Removing intermediate container 8cc3d39cf99f
     ---> 75870b6ba084
    Step 6/9 : ENV FOOBAR=$FOO-$BAR
     ---> Running in e73ad8cb00d0
    Removing intermediate container e73ad8cb00d0
     ---> 5489911812aa
    Step 7/9 : VOLUME /one/$BAR/three
     ---> Running in 79e0d937320c
    Removing intermediate container 79e0d937320c
     ---> ff3e7bd59ade
    Step 8/9 : WORKDIR /somewhere/$FOO
     ---> Running in 685e825c2686
    Removing intermediate container 685e825c2686
     ---> 2f01b4058473
    Step 9/9 : LABEL somelabel="FOO is $FOO and BAR is $BAR, but HOME is $HOME"
     ---> Running in 3096d64d69bb
    Removing intermediate container 3096d64d69bb
     ---> 2ec2eb49140d
    Successfully built 2ec2eb49140d
    

    While the classic builder output is not "ideal" (e.g., the WORKDIR is shown before variable substitution), at least it gives a better insight into what steps are executed.

    If verbosity is a concern, perhaps the metadata changes could be printed as a "combined" step (depending on "where" / "at what time" they're actually applied), although this may be confusing for the user (why is this step executed in a different place than I put it in the Dockerfile?)

    enhancement area/feature-parity 
    opened by thaJeztah 1
  • Bug: progress output substitutes environment variables where it should not

    Bug: progress output substitutes environment variables where it should not

    I noticed this when I started writing a reply / example for https://github.com/moby/moby/issues/42937.

    Docker itself (docker build) does not perform environment variable substitution in CMD, ENTRYPOINT and RUN commands (see https://docs.docker.com/engine/reference/builder/#environment-replacement). Environment variables in those commands are handled by the shell (unless the JSON / "exec form" syntax is used), which means that those variables are evaluated the moment the shell is executed.

    • For RUN, this means: the moment when the RUN command is executed as part of the build
    • For CMD and ENTRYPOINT this means: after the image has been built, and when the container is started

    However, BuildKit progress looks to be substituting these variables, which makes the output confusing as it "appears" the variables are substituted before executing.

    For example, the following Dockerfile:

    FROM alpine
    ENV FOO=hello
    ARG BAR=world
    RUN echo FOO is $FOO and BAR is $BAR
    RUN ["/bin/sh", "-c", "echo FOO is $FOO and BAR is $BAR"]
    ENV SH_ENV=/bin/sh
    RUN ["$SH_ENV", "-c", "echo FOO is $FOO and BAR is $BAR"]
    

    In the above, no variable substitution takes place in any of the RUN commands, but progress looks like below:

    docker build --no-cache -<<'EOF'
    FROM alpine
    ENV FOO=hello
    ARG BAR=world
    RUN echo FOO is $FOO and BAR is $BAR
    RUN ["/bin/sh", "-c", "echo FOO is $FOO and BAR is $BAR"]
    ENV SH_ENV=/bin/sh
    RUN ["$SH_ENV", "-c", "echo FOO is $FOO and BAR is $BAR"]
    EOF
    [+] Building 0.9s (7/7) FINISHED
     => [internal] load build definition from Dockerfile                         0.0s
     => => transferring dockerfile: 256B                                         0.0s
     => [internal] load .dockerignore                                            0.0s
     => => transferring context: 2B                                              0.0s
     => [internal] load metadata for docker.io/library/alpine:latest             0.0s
     => CACHED [1/4] FROM docker.io/library/alpine                               0.0s
     => [2/4] RUN echo FOO is hello and BAR is world                             0.2s
     => [3/4] RUN ["/bin/sh", "-c", "echo FOO is hello and BAR is world"]        0.3s
     => ERROR [4/4] RUN ["/bin/sh", "-c", "echo FOO is hello and BAR is world"]  0.3s
    ------
     > [4/4] RUN ["/bin/sh", "-c", "echo FOO is hello and BAR is world"]:
    #7 0.243 container_linux.go:380: starting container process caused: exec: "$SH_ENV": executable file not found in $PATH
    ------
    executor failed running [$SH_ENV -c echo FOO is $FOO and BAR is $BAR]: exit code: 1
    

    The correct output for the above would be something like:

     => [2/4] RUN echo FOO is $FOO and BAR is $BAR                               0.2s
     => [3/4] RUN ["/bin/sh", "-c", "echo FOO $FOO and BAR is $BAR"]             0.3s
     => ERROR [4/4] RUN ["$SH_ENV", "-c", "echo FOO $FOO and BAR is $BAR"]       0.3s
    ------
     > [4/4] RUN ["$SH_ENV", "-c", "echo FOO $FOO and BAR is $BAR"]:
    #7 0.243 container_linux.go:380: starting container process caused: exec: "$SH_ENV": executable file not found in $PATH
    ------
    executor failed running [$SH_ENV -c echo FOO is $FOO and BAR is $BAR]: exit code: 1
    

    Nit: probably the executor failed running should also print the JSON format of what's executed (instead of the string representation);

    - executor failed running [$SH_ENV -c echo FOO is $FOO and BAR is $BAR]: exit code: 1
    + executor failed running ["$SH_ENV", "-c", "echo FOO is $FOO and BAR is $BAR"]: exit code: 1
    

    Note that this is a regression (compared to the classic builder), which does not show this problem:

    DOCKER_BUILDKIT=0 docker build --no-cache -<<'EOF'
    FROM alpine
    ENV FOO=hello
    ARG BAR=world
    RUN echo FOO is $FOO and BAR is $BAR
    RUN ["/bin/sh", "-c", "echo FOO is $FOO and BAR is $BAR"]
    ENV SH_ENV=/bin/sh
    RUN ["$SH_ENV", "-c", "echo FOO is $FOO and BAR is $BAR"]
    EOF
    
    Sending build context to Docker daemon  2.048kB
    Step 1/7 : FROM alpine
     ---> 14119a10abf4
    Step 2/7 : ENV FOO=hello
     ---> Running in 272281384b45
    Removing intermediate container 272281384b45
     ---> ce9edafb60b8
    Step 3/7 : ARG BAR=world
     ---> Running in e0aa96c678ca
    Removing intermediate container e0aa96c678ca
     ---> f978170c97cc
    Step 4/7 : RUN echo FOO is $FOO and BAR is $BAR
     ---> Running in 414467a8ac72
    FOO is hello and BAR is world
    Removing intermediate container 414467a8ac72
     ---> 77160e213f97
    Step 5/7 : RUN ["/bin/sh", "-c", "echo FOO is $FOO and BAR is $BAR"]
     ---> Running in 171b2d974c05
    FOO is hello and BAR is world
    Removing intermediate container 171b2d974c05
     ---> 986101102b89
    Step 6/7 : ENV SH_ENV=/bin/sh
     ---> Running in c37dfdaf39ec
    Removing intermediate container c37dfdaf39ec
     ---> e0fa28d9c936
    Step 7/7 : RUN ["$SH_ENV", "-c", "echo FOO is $FOO and BAR is $BAR"]
     ---> Running in af9088799576
    OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "$SH_ENV": executable file not found in $PATH: unknown
    

    I should also note that the same would NOT apply to other instructions. For example, docker build DOES perform variable substitution in LABEL commands, however, LABEL instructions are not currently printed in build progress (perhaps they should?)

    bug area/dockerfile 
    opened by thaJeztah 4
  • dockerfile: eliminate dependency on dest directory for COPY

    dockerfile: eliminate dependency on dest directory for COPY

    In MergeOp #2335 we are adding capability that COPY layers can be rebased and reused via --cache-from even if cache for previous layers gets invalidated. All this works remotely with blobs in the registry. You can rebase an image on top of another image without the layers ever being downloaded or uploaded.

    In dockerfile frontend every copy(src, dest) will change to merge(dest, copy(src, scratch()).

    In order for a copy to work on remote objects only, it can not access any individual paths from the destination directory.

    The problem with this is the behavior in the case when the destination directory does not exist. In that case, new dir is created currently with new properties but if it exists then nothing is changed about the directory.

    Eg. when we have a Dockerfile

    FROM alpine
    COPY foo a/b/c/foo
    

    and after change a new Dockerfile

    FROM alpine
    RUN mkdir -p a/b/c && chmod -Rf a/b/c 0600
    COPY foo a/b/c/foo
    

    If we rebase the copy layer blob directly it would be wrong as the layer already contains directory a/b/c with perm 0755 that would overwrite the previous layer. While if the second file runs directly then a/b/c would remain 0600.

    Cases where we can solve this problem

    When USER is root and no --chown/chmod is set we can fix this by never putting records for the implied parent dirs in the tarball that COPY created. The tarball will only contain one record a/b/c/foo. When the image is pulled, a container runtime like docker will fill in the missing directories for a/b/c with default configuration when they do not exist.

    In order to make this work, we need to log the actual changes COPY made so we can exclude the implied parent directories when making a tarball. Started with that in https://github.com/tonistiigi/fsutil/pull/113

    Cases that can't be solved

    When COPY contains --chown=username there is no way this copy can be rebased with remote objects only. The username to uid mapping is in the parent image and the only way to check if it has changed is to extract the image and read /etc/passwd. This is unfortunate as this mapping pretty much never changes but don't see any solutions.

    Cases that could be solved with some additional syntax

    COPY --chown=uid and COPY --chmod=non-default-perms would not work by default. We can't just exclude the implied parents as docker would only create these parents with default perms/user. While in Dockerfile, unfortunately, the rule is that implied parents also get these chown/chmod values (what doesn't really make any sense but we can't just break it and I don't want to create v2 just for this).

    We could allow rebases with these COPY instructions if there would be some additional (opt-in) syntax(eg. new flag) where the user either confirms that COPY should not create implied parent directories or that it should always create them(up to a point). We need to eliminate the need to stat the destination directory in order to determine what the resulting state should be. From user's standpoint they almost always already know if the directory already exists or should be created. Ideally, it would be something that we could at least write a linter rule and suggest all users to always use this syntax.

    Suggestions?

    @sipsma @thaJeztah @crazy-max @AkihiroSuda @aaronlehmann

    enhancement area/dockerfile 
    opened by tonistiigi 7
  • Debugging buildkit builds

    Debugging buildkit builds

    Hello,

    I have encountered a few cases where a build fails without anything (obvious) in logs. When this happens with traditional Docker builds, I can simply use docker run -it <base image> and execute RUN commands while looking at files, processes, ... in the build environment to debug.

    Is there a way to reproduce the buildkit sandbox environments to perform a similar debugging?

    I tried to exec into a running buildkitd container or nsenter the build container using a long running process without success.

    opened by lbernail 0
  • Support copying from an image defined with an ARG

    Support copying from an image defined with an ARG

    It would be desirable to support copying content from an image using a variable/argument for specifying the image name. For instance:

    ARG REGISTRY='docker.io/hdlc'
    
    FROM $REGISTRY/build:build AS build
    
    COPY --from=$REGISTRY/pkg:icestorm /icestorm/usr/local/share/icebox /usr/local/share/icebox
    

    or

    # syntax=docker/dockerfile:1.2
    
    ARG REGISTRY='docker.io/hdlc'
    
    FROM $REGISTRY/build:build AS build
    
    RUN --mount=type=cache,from=$REGISTRY/pkg:icestorm,src=/icestorm/usr/local/share/icebox,target=/usr/local/share/icebox \
    ...
    

    Refs:

    • #2034
    • #1167
    • #815 #2089. from is explicitly not supported by #2089: "from expansion is not supported".
    area/dockerfile 
    opened by umarcor 3
  • when export cache,ERROR: error writing layer blob: failed commit on ref

    when export cache,ERROR: error writing layer blob: failed commit on ref "layer-sha256

    version:0.9.0 buildctl build --frontend dockerfile.v0 --local 'context=.' --local 'dockerfile=.' --opt 'platform=linux/aarch64,linux/amd64' --output 'type=image,"name=test:1.0",push=true' --export-cache 'type=local,mode=max,dest=/var/lib/buildkit/cache' --import-cache 'type=local,mode=max,src=/var/lib/buildkit/cache'

    image

    ERROR: error writing layer blob: failed commit on ref "layer-sha256:0457ba013307be42bb9ab7d5ab544a1439da74d6ec8cfd210a71b5e2a2022a8d": commit failed: rename /var/lib/buildkit/cache/ingest/bc00829aa31d0ce0012e31d98ca5fd3210bacb00cbccb9e5c7b83a716c2540f3/data /var/lib/buildkit/cache/blobs/sha256/0457ba013307be42bb9ab7d5ab544a1439da74d6ec8cfd210a71b5e2a2022a8d: no such file or directory: unknown [2021-10-08 20:41:04.259] ------ [2021-10-08 20:41:04.259] > exporting cache: [2021-10-08 20:41:04.259] ------ [2021-10-08 20:41:04.259] error: failed to solve: error writing layer blob: failed commit on ref "layer-sha256:0457ba013307be42bb9ab7d5ab544a1439da74d6ec8cfd210a71b5e2a2022a8d": commit failed: rename /var/lib/buildkit/cache/ingest/bc00829aa31d0ce0012e31d98ca5fd3210bacb00cbccb9e5c7b83a716c2540f3/data /var/lib/buildkit/cache/blobs/sha256/0457ba013307be42bb9ab7d5ab544a1439da74d6ec8cfd210a71b5e2a2022a8d: no such file or directory: unknown

    area/containerd needs-investigation 
    opened by gitfxx 1
  • Creating additional layers on top of an image with SELinux attributes fails on Buildkit rootless

    Creating additional layers on top of an image with SELinux attributes fails on Buildkit rootless

    Hello, We are currently trying to build images with buildkit in rootless mod, on an unprivileged Kubernetes pod (apparmor and seccomp are disabled). We recently experienced difficulties building some images, and the error message was not very explicit about the error.

    This is a simplified payload of one of these images:

    FROM gitlab/gitlab-runner-helper:x86_64-v14.0.0
    
    RUN echo "test"
    

    Here is the output of the buildkit command:

    #~ docker buildx build --progress plain .
    
    WARN[0000] No output specified for kubernetes driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
    #1 [internal] load build definition from Dockerfile
    #1 transferring dockerfile: 29B 0.1s
    #1 ...
    
    #2 [internal] load .dockerignore
    #2 transferring context: 2B 0.2s done
    #2 DONE 0.2s
    
    #1 [internal] load build definition from Dockerfile
    #1 transferring dockerfile: 103B 0.3s done
    #1 DONE 0.3s
    
    #3 [internal] load metadata for docker.io/gitlab/gitlab-runner-helper:x86_64-v13.12.0
    #3 DONE 0.1s
    
    #4 [1/2] FROM docker.io/gitlab/gitlab-runner-helper:[email protected]:c96ecb7474c62872b7675915daaa216208ec98b5bae45dc9841607d6a2cc2e25
    #4 resolve docker.io/gitlab/gitlab-runner-helper:[email protected]:c96ecb7474c62872b7675915daaa216208ec98b5bae45dc9841607d6a2cc2e25 done
    #4 CACHED
    
    #5 [2/2] RUN echo "test"
    #5 ERROR: mount callback failed on /run/user/1000/containerd-mount718980256: operation not permitted
    ------
     > [2/2] RUN echo "test":
    ------
    Dockerfile:3
    --------------------
       1 |     FROM gitlab/gitlab-runner-helper:x86_64-v13.12.0
       2 |
       3 | >>> RUN echo "test"
       4 |
    --------------------
    error: failed to solve: failed to compute cache key: mount callback failed on /run/user/1000/containerd-mount718980256: operation not permitted
    

    Investigations

    However, when we try to build a more recent version of the image, (starting version x86_64-v14.1.0), without any major change to the Dockerfile, it builds fine on our non-privileged rootless buildkit pods. Thus, we have done some investigations to understand the nature of the error better.

    Testing setup

    Generate strace reports

    1. Run docker run -it --name buildkitd-rootless --cap-add=SYS_PTRACE --security-opt apparmor=unconfined --security-opt seccomp=unconfined --rm moby/buildkit:v0.9.0-rootless --oci-worker-no-process-sandbox in a shell
    2. In another shell, exec into the pod docker exec -it -u root buildkitd-rootless sh and run:
    apk update && apk add strace
    strace -e trace=file -f -p $(pgrep buildkitd) -o error.log 2>/dev/null &
    cd && echo -e "FROM gitlab/gitlab-runner-helper:ubuntu-x86_64-v14.0.0\n RUN echo \"test\"" > Dockerfile.error
    buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=. --opt filename=Dockerfile.error
    pkill strace
    
    strace -e trace=file -f -p $(pgrep buildkitd) -o noerror.log 2>/dev/null &
    cd && echo -e "FROM gitlab/gitlab-runner-helper:ubuntu-x86_64-v14.1.0\n RUN echo \"test\"" > Dockerfile.noerror
    buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=. --opt filename=Dockerfile.noerror
    pkill strace
    

    Get results

    1. docker cp buildkitd-rootless:/root/error.log .
    2. docker cp buildkitd-rootless:/root/noerror.log .

    error.log noerror.log

    Results

    When we analyse the file error.log, we can see the following lines just before the process gets killed:

    57    mkdirat(AT_FDCWD, "/run/user/1000/containerd-mount719339716", 0700) = 0
    57    mount("/home/user/.local/share/buildkit/runc-native/snapshots/snapshots/7", "/run/user/1000/containerd-mount719339716", 0xc000685cfa, MS_BIND|MS_REC, NULL) = 0
    241   newfstatat(AT_FDCWD, "/run/user/1000/containerd-mount719339716/bin", 0xc0008a2788, AT_SYMLINK_NOFOLLOW) = -1 ENOENT (No such file or directory)
    241   symlinkat("usr/bin", AT_FDCWD, "/run/user/1000/containerd-mount719339716/bin") = 0
    241   fchownat(AT_FDCWD, "/run/user/1000/containerd-mount719339716/bin", 0, 0, AT_SYMLINK_NOFOLLOW) = 0
    241   lsetxattr("/run/user/1000/containerd-mount719339716/bin", "security.selinux", "system_u:object_r:unlabeled_t:s0"..., 33, 0) = -1 EPERM (Operation not permitted)
    241   newfstatat(AT_FDCWD, "/run/user/1000/containerd-mount719339716", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
    241   statfs("/run/user/1000/containerd-mount719339716", {f_type=EXT2_SUPER_MAGIC, f_bsize=4096, f_blocks=16448139, f_bfree=3547337, f_bavail=2704394, f_files=4194304, f_ffree=2655908, f_fsid={val=[0x89845361, 0x49eaed1c]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0
    241   umount2("/run/user/1000/containerd-mount719339716", 0) = 0
    241   statfs("/run/user/1000/containerd-mount719339716", {f_type=OVERLAYFS_SUPER_MAGIC, f_bsize=4096, f_blocks=16448139, f_bfree=3547337, f_bavail=2704394, f_files=4194304, f_ffree=2655908, f_fsid={val=[0x89845361, 0x49eaed1c]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_RELATIME}) = 0
    241   umount2("/run/user/1000/containerd-mount719339716", 0) = -1 EINVAL (Invalid argument)
    241   unlinkat(AT_FDCWD, "/run/user/1000/containerd-mount719339716", 0) = -1 EISDIR (Is a directory)
    241   unlinkat(AT_FDCWD, "/run/user/1000/containerd-mount719339716", AT_REMOVEDIR) = 0
    1052  +++ killed by SIGKILL +++
    

    These lines cannot be found in noerror.log. Following these log lines, SELinux attributes seem to be the cause of the issue. If we look at the layers, we can see that these attributes only exist in the image that creates the error (to verify this, you need to save the layer tarball and run tar --verbose --verbose --xattrs-include='*' -tf layer.tar).

    Our current hypothesis is that the image that creates the error was built on a host with SELinux enabled. When buildkit tries to un-tar the layer before creating the new one, it fails because it lack permissions to set SELinux attributes on files.

    Is there a way to prevent builds from failing when buildkit is not able to set the extended attributes when un-taring a layer?

    Of course, we are more than happy to perform additional tests to gather more data.

    bug area/rootless 
    opened by zaymat 0
  • Propagate compression options to the inline cache export

    Propagate compression options to the inline cache export

    This is a following-up patch for #2350 to move that forward.

    Currently, compression options aren't propagated to the inline cache export and it always uses gzip compressor. This leads to an issue that the compression option is ignored when --export-cache type=inline is specified.

    For example, a build something like the following ignores compression=uncompressed option and creates gzip images.

    buildctl build --progress=plain --frontend=dockerfile.v0 --local context=/tmp/tmp.XAIR6qwH5h --local dockerfile=/tmp/tmp.XAIR6qwH5h \
                   --output type=image,name=registry:5000/image:1,push=true,compression=uncompressed \
                   --export-cache type=inline
    

    This patch solves this issue by propagating compression options to the inline cache export as well. This also adds an option to solver.(*exporter).ExportTo() to avoid unexpected contents are recorded when inline export.

    This adds @tonistiigi as a co-author because this patch is based on a commit of #2350.

    opened by ktock 9
Releases(dockerfile/1.3.1-labs)
  • dockerfile/1.3.1-labs(Oct 4, 2021)

  • dockerfile/1.3.1(Oct 4, 2021)

    Usage

    This release is currently in staging.

    # syntax=docker/dockerfile-upstream:1.3.1
    

    Notable changes

    • Fix parsing "required" mount key without a value #2304
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Oct 4, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Builtin Dockerfile frontend has been updated to v1.3.1 changelog

    • Seccomp profile has been updated to properly handle clone3 syscall support. #2379

    • Fix possible panic on ARM32 due to struct alignment #2321

    • Fix occasional "no active session" and "no such job" errors on concurrent builds #2369

    • Fix flakiness during import of a cache with empty layers removed #2372

    • Handle "edge not found" error in build instead of panicking #2382 #2385

    • Fix problems with experimental Github Actions cache backend:

      • Add retry logic when API rate limits are reached https://github.com/tonistiigi/go-actions-cache/pull/8
      • Remove BOM from error messages returned by API https://github.com/tonistiigi/go-actions-cache/pull/6
      • Gracefully handle downloading blobs that have been removed from the cache https://github.com/moby/buildkit/pull/2387
      • Fix "cache already exists" errors in certain conditions https://github.com/moby/buildkit/pull/2387
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.1.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.1.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.1.linux-amd64.tar.gz(45.23 MB)
    buildkit-v0.9.1.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.1.linux-arm64.tar.gz(41.60 MB)
    buildkit-v0.9.1.linux-ppc64le.tar.gz(43.18 MB)
    buildkit-v0.9.1.linux-riscv64.tar.gz(42.16 MB)
    buildkit-v0.9.1.linux-s390x.tar.gz(44.55 MB)
    buildkit-v0.9.1.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.1.windows-arm64.tar.gz(10.55 MB)
  • dockerfile/1.3.0-labs(Jul 16, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile:1.3.0-labs
    

    Notable changes

      • RUN and COPY commands now support Here-document syntax allowing writing multiline inline scripts and files #2132 #2201 #2209 #2213 Documentation
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.3.0(Jul 16, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile:1.3.0
    

    Notable changes

    • RUN command allows --network flag for requesting a specific type of network conditions. --network=host requires allowing network.host entitlement. This feature was previously only available on labs channel. Documentation

    • ADD command with a remote URL input now correctly handles the --chmod flag. #2171

    • Values for RUN --mount flag now support variable expansion, except for the from field #2089

    • Allow BUILDKIT_MULTI_PLATFORM build arg to force always creating multi-platform image, even if only contains single platform #1985

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 16, 2021)

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.3.0 including support for RUN --network and Here-documents in labs channel. Dockerfile changelog Dockerfile labs changelog

    • Experimental support for Github Actions remote cache backend via type=gha #1974 Docs

    • Add support for subdirectories when building from Git source #2116

    • Outgoing TCP connections are now limited to 4 per registry. Metadata requests get one extra connection not used by layer pulls and pushes. #2259 #2242 #2247

    • Buildkitd config allows max-parallelism for limiting the parallelism of the BuildKit solver for low-powered machines #2049

    • OpenTracing providers have been replaced with support for OpenTelemetry #2152 #2192 #2238

    • Errors have been improved by removing gRPC wrapping and providing suggestions for typos #2218 #2047 #2215 #2183

    • New OpenTelemetry support allows forwarding traces with control API from the client or from user programs in llb.Exec container #2163

    • Git: Default branch name is now detected correctly from remote #2228

    • Allow forcing specific compression on exported layers even if another blob exists #2057

    • Plain progress mode now prints last logs of failed command in error summary #2214

    • Plain progress mode does not print to LLB vertex digest anymore to avoid confusion #2126

    • Buildctl allows --metadata-file flag to output build metadata #2095

    • This is the first release supporting Risc-V (experimental) #2222

    • This is the first release supporting MacOS ARM64 and Windows ARM64 for buildctl #2037 #2187

    • Runc has been updated to v1.0.0 GA #2143

    • RootlessKit has been updated to v0.14.2 #2102

    • Embedded QEMU emulators have been updated to v6.0.0 #2225

    • LLB: Root directory can now be copied from empty references without causing an error or panic #2197

    • LLB: Ensure image metadata resolver uses platform constraints set by marshaler #2196

    • LLB: Copy operation now allows include and exclude filters to limit the copied files #2082

    • Stargz snapshotter now supports authentication from Docker config #1733 #2165

    • Support Windows OpenSSH agent forwarding #2127

    • Handle expired tokens errors better #2062 #1957

    • LLB: Support ALL_PROXY as a proxy environment variable that does not stay in build cache #2086

    • Buildkit release images now contain OpenSSH #2135

    • Enable containerd labels in buildctl debug workers and make their order deterministic #2070 #2071

    • Gracefully handle client-side token seed errors #2050

    • Pushing multi-platform images will not try to overwrite image tag internally multiple times. This is important for registries that support immutable tags. #2020

    • Missing /etc/passwd and /etc/group file is now handled gracefully. Important when cross-compiling Windows images. #2249

    • Fix logs clipping behavior and double the limits #1934

    • Support socket activation with --addr fd:// #1924

    • Daemon logs now show traceID and spanID if OpenTelemetry is enabled #2235

    • Fix issue where Dockerfile with the same metadata and timestamps as another build could pick up its build cache by introducing a "none" differ for transferring local files that ignores metadata matches. #2081

    • Fix rare "retry timeout exceeded" errors in job synchronization #2195

    • Fix gracefully handling permission errors on accessing tokens from client-side #2234

    • Ensure containerd executor always waits for IO to complete before returning from Exec/Run. #2205

    • Fix pulling layers that have already been pulled with different compression. #2226

    • Fix some syncronisation errors #2178 #2177 #2156 #2052 #2051

    • Fix goroutine leak from progress writing #2203

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Aaron Lehmann
    • Sebastiaan van Stijn
    • CrazyMax
    • Tibor Vass
    • Alex Couture-Beil
    • Justin Chadwell
    • Kohei Tokunaga
    • Cory Bennett
    • Erik Sipsma
    • Siebe Schaap
    • Vlad A. Ionescu
    • Levi Harrison
    • Brian Goff
    • Edgar Lee
    • Anders F Björklund
    • Charles Korn
    • Claudiu Belu
    • Corey Larson
    • Jesse Rittner
    • Justin Garrison
    • Marko Kohtala
    • Omer Mizrahi
    • Pierre Fenoll
    • Rob Taylor
    • Yamazaki Masashi
    • zhangwenlong
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0.darwin-amd64.tar.gz(11.53 MB)
    buildkit-v0.9.0.darwin-arm64.tar.gz(11.10 MB)
    buildkit-v0.9.0.linux-amd64.tar.gz(45.21 MB)
    buildkit-v0.9.0.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.0.linux-arm64.tar.gz(41.57 MB)
    buildkit-v0.9.0.linux-ppc64le.tar.gz(43.15 MB)
    buildkit-v0.9.0.linux-riscv64.tar.gz(42.15 MB)
    buildkit-v0.9.0.linux-s390x.tar.gz(44.58 MB)
    buildkit-v0.9.0.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.0.windows-arm64.tar.gz(10.55 MB)
  • v0.9.0-rc2(Jul 16, 2021)

    Welcome to the 0.9.0-rc2 release of buildkit! This is a pre-release of buildkit

    Notable Changes

    • Fix progress regression in v0.9.0-rc1`#2254
    • Experimental support for Github Actions remote cache backend via type=gha #1974
    • Outgoing TCP connections are now limited to 4 per registry. Metadata requests get one extra connection not used by layer pulls and pushes. #2259 #2242 #2247
    • Missing /etc/passwd and /etc/group file is now handled gracefully. Important when cross-compiling Windows images. #2249
    • Daemon logs now show traceID and spanID is OpenTelemetry is enabled #2235
    • OpenTelemetry traces now show extra information about outgoing HTTP requests #2238
    • Fix gracefully handling permission errors on accessing tokens from client-side #2234
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0-rc2.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.0-rc2.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.0-rc2.linux-amd64.tar.gz(45.21 MB)
    buildkit-v0.9.0-rc2.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.0-rc2.linux-arm64.tar.gz(41.57 MB)
    buildkit-v0.9.0-rc2.linux-ppc64le.tar.gz(43.15 MB)
    buildkit-v0.9.0-rc2.linux-riscv64.tar.gz(42.15 MB)
    buildkit-v0.9.0-rc2.linux-s390x.tar.gz(44.58 MB)
    buildkit-v0.9.0-rc2.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.0-rc2.windows-arm64.tar.gz(10.55 MB)
  • dockerfile/1.3.0-rc1-labs(Jul 7, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.3.0-rc1-labs
    

    Notable changes

      • RUN and COPY commands now support Here-document syntax allowing writing multiline inline scripts and files #2132 #2201 #2209 #2213 Documentation
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.3.0-rc1(Jul 7, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.3.0-rc1
    

    Notable changes

    • RUN command allows --network flag for requesting a specific type of network conditions. --network=host requires allowing network.host entitlement. This feature was previously only available on labs channel. Documentation

    • ADD command with a remote URL input now correctly handles the --chmod flag. #2171

    • Values for RUN --mount flag now support variable expansion, except for the from field #2089

    • Allow BUILDKIT_MULTI_PLATFORM build arg to force always creating multi-platform image, even if only contains single platform #1985

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0-rc1(Jul 7, 2021)

    Welcome to the 0.9.0-rc1 release of buildkit! This is a pre-release of buildkit

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.3.0-rc1 including support for RUN --network and Here-documents in labs channel. Dockerfile changelog Dockerfile labs changelog

    • Errors have been improved by removing gRPC wrapping and providing suggestions for typos #2218 #2047 #2215 #2183

    • Add support for subdirectories when building from Git source #2116

    • Buildkitd config allows max-parallelism for limiting the parallelism of the BuildKit solver for low-powered machines #2049

    • OpenTracing providers have been replaced with support for OpenTelemetry #2152 #2192

    • New OpenTelemetry support allows forwarding traces with control API from the client or from user programs in llb.Exec container #2163

    • Git: Default branch name is now detected correctly from remote #2228

    • Allow forcing specific compression on exported layers even if another blob exists #2057

    • Plain progress mode now prints last logs of failed command in error summary #2214

    • Plain progress mode does not print to LLB vertex digest anymore to avoid confusion #2126

    • Buildctl allows --metadata-file flag to output build metadata #2095

    • This is the first release supporting Risc-V (experimental) #2222

    • This is the first release supporting MacOS ARM64 and Windows ARM64 for buildctl #2037 #2187

    • Runc has been updated to v1.0.0 GA #2143

    • RootlessKit has been updated to v0.14.2 #2102

    • Embedded QEMU emulators have been updated to v6.0.0 #2225

    • LLB: Root directory can now be copied from empty references without causing an error or panic #2197

    • LLB: Ensure image metadata resolver uses platform constraints set by marshaler #2196

    • LLB: Copy operation now allows include and exclude filters to limit the copied files #2082

    • Stargz snapshotter now supports authentication from Docker config #1733 #2165

    • Support Windows OpenSSH agent forwarding #2127

    • Handle expired tokens errors better #2062 #1957

    • LLB: Support ALL_PROXY as a proxy environment variable that does not stay in build cache #2086

    • Buildkit release images now contain OpenSSH #2135

    • Enable containerd labels in buildctl debug workers and make their order deterministic #2070 #2071

    • Gracefully handle client-side token seed errors #2050

    • Pushing multi-platform images will not try to overwrite image tag internally multiple times. This is important for registries that support immutable tags. #2020

    • Fix logs clipping behavior and double the limits #1934

    • Support socket activation with --addr fd:// #1924

    • Fix issue where Dockerfile with the same metadata and timestamps as another build could pick up its build cache by introducing a "none" differ for transferring local files that ignores metadata matches. #2081

    • Fix rare "retry timeout exceeded" errors in job synchronization #2195

    • Ensure containerd executor always waits for IO to complete before returning from Exec/Run. #2205

    • Fix pulling layers that have already been pulled with different compression. #2226

    • Fix some syncronisation errors #2178 #2177 #2156 #2052 #2051

    • Fix goroutine leak from progress writing #2203

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Aaron Lehmann
    • Sebastiaan van Stijn
    • CrazyMax
    • Tibor Vass
    • Alex Couture-Beil
    • Justin Chadwell
    • Kohei Tokunaga
    • Cory Bennett
    • Erik Sipsma
    • Siebe Schaap
    • Vlad A. Ionescu
    • Levi Harrison
    • Brian Goff
    • Edgar Lee
    • Anders F Björklund
    • Charles Korn
    • Corey Larson
    • Jesse Rittner
    • Justin Garrison
    • Marko Kohtala
    • Omer Mizrahi
    • Pierre Fenoll
    • Rob Taylor
    • Yamazaki Masashi
    • zhangwenlong
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0-rc1.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.0-rc1.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.0-rc1.linux-amd64.tar.gz(45.00 MB)
    buildkit-v0.9.0-rc1.linux-arm-v7.tar.gz(40.54 MB)
    buildkit-v0.9.0-rc1.linux-arm64.tar.gz(41.38 MB)
    buildkit-v0.9.0-rc1.linux-ppc64le.tar.gz(42.94 MB)
    buildkit-v0.9.0-rc1.linux-riscv64.tar.gz(41.92 MB)
    buildkit-v0.9.0-rc1.linux-s390x.tar.gz(44.32 MB)
    buildkit-v0.9.0-rc1.windows-amd64.tar.gz(11.49 MB)
    buildkit-v0.9.0-rc1.windows-arm64.tar.gz(10.55 MB)
  • v0.8.3(Apr 30, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Update containerd with fixes to rootless overlay on kernel 5.11 and push panic #2014

    • Add retry on 5xx push errors https://github.com/moby/buildkit/pull/2043

    • Include basename in content checksum for wildcards #2018

    • Fix missing mounts in execOp cache map #2076

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.3.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.3.linux-amd64.tar.gz(42.16 MB)
    buildkit-v0.8.3.linux-arm-v7.tar.gz(38.02 MB)
    buildkit-v0.8.3.linux-arm64.tar.gz(38.82 MB)
    buildkit-v0.8.3.linux-ppc64le.tar.gz(40.40 MB)
    buildkit-v0.8.3.linux-s390x.tar.gz(41.89 MB)
    buildkit-v0.8.3.windows-amd64.tar.gz(11.02 MB)
  • v0.8.2(Feb 25, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Apparmor profile can be set in the buildkitd config #1966

    • Seccomp updated to 2.4.2 to fix time64 syscall compatibility issues on 32-bit architectures #1955

    • Update builtin QEMU emulators to fix issues with script handling and add i386 emulator #1953

    • Avoid caching token fetch errors #1957

    • Fix possible invalid cache match on specific copy operation #1965

    • Fix reference count issues when returning typed errors from cache mounts #1963

    • Avoid reusing credentials when checking out git submodules #1987

    • Update Runc to v1.0.0-rc93 #1998

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.2.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.2.linux-amd64.tar.gz(42.16 MB)
    buildkit-v0.8.2.linux-arm-v7.tar.gz(38.01 MB)
    buildkit-v0.8.2.linux-arm64.tar.gz(38.81 MB)
    buildkit-v0.8.2.linux-ppc64le.tar.gz(40.39 MB)
    buildkit-v0.8.2.linux-s390x.tar.gz(41.88 MB)
    buildkit-v0.8.2.windows-amd64.tar.gz(11.02 MB)
  • v0.8.1(Dec 15, 2020)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Builtin Dockerfile frontend updated to v1.2.1

    • LLB client sets platform based on parent state to avoid inefficient lookups in cross-compilation cases #1889

    • Fix building from Git URL without specifying protocol #1886

    • Fix possible race in flightcontrol package #1891

    • Fix/optimize scheduler preprocessor logic #1871

    • Fix pushing foreign mediatype, eg. layers pulled from common Windows base images #1879

    • Fix possible panic from a frontend component returning nil values #1898

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.1.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.1.linux-amd64.tar.gz(40.94 MB)
    buildkit-v0.8.1.linux-arm-v7.tar.gz(36.62 MB)
    buildkit-v0.8.1.linux-arm64.tar.gz(37.40 MB)
    buildkit-v0.8.1.linux-ppc64le.tar.gz(38.65 MB)
    buildkit-v0.8.1.linux-s390x.tar.gz(40.29 MB)
    buildkit-v0.8.1.windows-amd64.tar.gz(11.02 MB)
  • dockerfile/1.2.1-labs(Dec 12, 2020)

  • dockerfile/1.2.1(Dec 12, 2020)

    https://hub.docker.com/r/docker/dockerfile

    Notable changes

    • Revert "Ensure ENTRYPOINT command has at least one argument" #1874

    • Optimize processing COPY calls on multi-platform cross-compilation builds #1889

    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.2.0-labs(Dec 3, 2020)

  • dockerfile/1.2.0(Dec 3, 2020)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.2.0
    

    Notable changes

    • RUN --mount syntax for creating secret, ssh, bind, and cache mounts have been moved to mainline channel #1717

    • Metadata load errors are now handled as fatal to avoid incorrect build results #1395

    • ARG command now supports defining multiple build args on the same line similarly to ENV #1692

    • --chown flag in ADD now allows parameter expansion #1473

    • Allow lowercase Dockerfile name #1816

    • ENTRYPOINT requires at least one argument to avoid creating broken images #1862

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Dec 3, 2020)

    Welcome to the 0.8.0 release of buildkit!

    Important

    • This release changes images pull mode in a way that image layers are only pulled from a registry when their contents is needed locally. If your build does not export the build result or does not need to run new containers on top of the image, the build will succeed without pulling the image. This allows you to make metadata modifications to remote images without pulling them or check that the remote cache is still valid for your build without actually pulling the cache layers.

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.2.0 including support for RUN --mount among other features. Dockerfile changelog

    • Gateway API now allows running interactive container processes that can mount previous build results #1627 #1731

    • API: Build errors now contain state for debugging the failure location including the snapshots' data when the error happened #1732

    • Image layers used by the build are now only pulled when their content is being used by subsequent build steps or exporter. BuildKit can now make cache decisions about the data while it remains in the remote registry. #1475

    • Fetching authorization tokens has been moved to client-side (if the client supports it). Passwords do not leak into the build daemon anymore and users can see from build output when credentials or tokens are accessed. #1660

    • Support stargz/eStargz for pulling image layers incrementally based on what files are accessed https://github.com/moby/buildkit/blob/master/docs/stargz-estargz.md #1402

    • Buildkit can now build for multiple architectures with QEMU without binfmt_misc handlers loaded to the kernel. moby/buildkit image comes with the emulator images. #1516

    • Build errors now track the error location in the original source files #1494

    • Frontend API now supports subrequests for implementing supplementary tasks like describing build stages or arguments. #1724

    • Connection errors while communicating with the registry for push and pull now trigger a retry #1791

    • Git source now supports token authentication via build secrets #1533

    • Building from git source now supports forwarding SSH socket for authentication #1782

    • Allow passing secrets to the build with environment variables #1534

    • Increase registry communication performance and stability with custom connection pool and authenticator #1636

    • Running commands do not leak empty stub files to image layers anymore (for example for mounted secrets) #1739

    • Allow better handling client sessions dropping while it is being shared by multiple builds #1551

    • Allow (and default to) using OCI mediatypes on exporting manifests for remote cache #1746

    • Only add manifest descriptor annotations to OCI type manifests and not Docker manifests. This fixes an issue with GCR validation. #1730

    • Avoid builds that generate excessive logs to cause a crash or slow down the build. Clipping is performed if needed. #1754

    • Fix race on creating CNI sandboxes for containers #1775

    • Execution steps now allow overriding the hostname for the build container #1339

    • Always use correct mediatypes on exporting objects, not considering the object's original mediatype #1541

    • Content-based checksums are now calculated in parallel for the build step with multiple mounts #1744

    • Reenable setting insecure-registry config while exporting to a registry #1601

    • Fix synchronization issues on pushing multi-platform images that share layers #1548

    • Cache load errors are now handled gracefully #1498

    • Disable truncating by default when using --progress=plain #1435

    • Official image moby/buildkit now contains pigz for better extraction performance #1799

    • Support for exposing SSH agent socket on Windows has been improved #1695

    • LLB client library now supports using asynchronous callbacks when building the LLB graph #1426

    • Change default Seccomp profile to the one provided by Docker #1807

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Cory Bennett
    • Paul "TBBle" Hampson
    • Sebastiaan van Stijn
    • Edgar Lee
    • Tibor Vass
    • Erik Sipsma
    • Kohei Tokunaga
    • Alex Couture-Beil
    • Vlad A. Ionescu
    • Lu Jingxiao
    • Simon Ferquel
    • CrazyMax
    • Anders F Björklund
    • Andrea Bolognani
    • Andrea Luzzardi
    • Andrew Chang
    • Andrey Smirnov
    • Anurag Goel
    • Chanhun Jeong
    • Chen Bin
    • Ilya Dmitrichenko
    • Jon Zeolla
    • Jonathan Azoff
    • Jörg Franke
    • Kees Cook
    • Miguel Ángel Jimeno
    • Nick Santos
    • Sam Whited
    • Shingo Omura
    • Wang Yumu
    • Wei Fu
    • Xiaofan Zhang
    • Ximo Guanter
    • 岁丰
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.0.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.0.linux-amd64.tar.gz(40.94 MB)
    buildkit-v0.8.0.linux-arm-v7.tar.gz(36.62 MB)
    buildkit-v0.8.0.linux-arm64.tar.gz(37.39 MB)
    buildkit-v0.8.0.linux-ppc64le.tar.gz(38.63 MB)
    buildkit-v0.8.0.linux-s390x.tar.gz(40.29 MB)
    buildkit-v0.8.0.windows-amd64.tar.gz(11.02 MB)
  • v0.8.0-rc3(Nov 26, 2020)

  • dockerfile/1.2.0-rc1-labs(Nov 18, 2020)

  • dockerfile/1.2.0-rc1(Nov 18, 2020)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.2.0-rc1
    

    Notable changes

    • RUN --mount syntax for creating secret, ssh, bind, and cache mounts have been moved to mainline channel #1717

    • Metadata load errors are now handled as fatal to avoid incorrect build results #1395

    • ARG command now supports defining multiple build args on the same line similarly to ENV #1692

    • --chown flag in ADD now allows parameter expansion #1473

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0-rc1(Nov 18, 2020)

    Welcome to the 0.8.0-rc1 release of buildkit! This is a pre-release of buildkit

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.2.0 including support for RUN --mount among other features. Dockerfile changelog

    • Gateway API now allows running interactive container processes that can mount previous build results #1627 #1731

    • API: Build errors now contain state for debugging the failure location including the snapshots' data when the error happened #1732

    • Image layers used by the build are now only pulled when their content is being used by subsequent build steps or exporter. BuildKit can now make cache decisions about the data while it remains in the remote registry. #1475

    • Fetching authorization tokens has been moved to client-side (if the client supports it). Passwords do not leak into the build daemon anymore and users can see from build output when credentials or tokens are accessed. #1660

    • Support stargz/eStargz for pulling image layers incrementally based on what files are accessed https://github.com/moby/buildkit/blob/master/docs/stargz-estargz.md #1402

    • Buildkit can now build for multiple architectures with QEMU without binfmt_misc handlers loaded to the kernel. moby/buildkit image comes with the emulator images. #1516

    • Build errors now track the error location in the original source files #1494

    • Frontend API now supports subrequests for implementing supplementary tasks like describing build stages or arguments. #1724

    • Connection errors while communicating with the registry for push and pull now trigger a retry #1791

    • Git source now supports token authentication via build secrets #1533

    • Building from git source now supports forwarding SSH socket for authentication #1782

    • Allow passing secrets to the build with environmental variables #1534

    • Increase registry communication performance and stability with custom connection pool and authenticator #1636

    • Running commands do not leak empty stub files to image layers anymore (for example for mounted secrets) #1739

    • Allow better handling client sessions dropping while it is being shared by multiple builds #1551

    • Allow (and default to) using OCI mediatypes on exporting manifests for remote cache #1746

    • Only add manifest descriptor annotations to OCI type manifests and not Docker manifests. This fixes an issue with GCR validation. #1730

    • Avoid builds that generate excessive logs to cause a crash or slow down the build. Clipping is performed if needed. #1754

    • Fix race on creating CNI sandboxes for containers #1775

    • Execution steps now allow overriding the hostname for the build container #1339

    • Always use correct mediatypes on exporting objects, not considering the object's original mediatype #1541

    • Content-based checksums are now calculated in parallel for the build step with multiple mounts #1744

    • Reenable setting insecure-registry config while exporting to a registry #1601

    • Fix synchronization issues on pushing multi-platform images that share layers #1548

    • Cache load errors are now handled gracefully #1498

    • Disable truncating by default when using --progress=plain #1435

    • Official image moby/buildkit now contains pigz for better extraction performance #1799

    • Support for exposing SSH agent socket on Windows has been improved #1695

    • LLB client library now supports using asynchronous callbacks when building the LLB graph #1426

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Cory Bennett
    • Paul "TBBle" Hampson
    • Sebastiaan van Stijn
    • Edgar Lee
    • Tibor Vass
    • Erik Sipsma
    • Kohei Tokunaga
    • Alex Couture-Beil
    • Vlad A. Ionescu
    • Lu Jingxiao
    • Simon Ferquel
    • Anders F Björklund
    • Andrea Luzzardi
    • Andrey Smirnov
    • Anurag Goel
    • Chanhun Jeong
    • Chen Bin
    • Ilya Dmitrichenko
    • Jon Zeolla
    • Jonathan Azoff
    • Jörg Franke
    • Kees Cook
    • Miguel Ángel Jimeno
    • Nick Santos
    • Sam Whited
    • Shingo Omura
    • Wang Yumu
    • Wei Fu
    • Xiaofan Zhang
    • Ximo Guanter
    • 岁丰
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.0-rc1.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.0-rc1.linux-amd64.tar.gz(43.42 MB)
    buildkit-v0.8.0-rc1.linux-arm-v7.tar.gz(38.94 MB)
    buildkit-v0.8.0-rc1.linux-arm64.tar.gz(39.73 MB)
    buildkit-v0.8.0-rc1.linux-ppc64le.tar.gz(40.97 MB)
    buildkit-v0.8.0-rc1.linux-s390x.tar.gz(42.79 MB)
    buildkit-v0.8.0-rc1.windows-amd64.tar.gz(11.02 MB)
  • v0.7.2(Jul 28, 2020)

    Fixes:

    • solver: gracefully handle cache loading errors #1498
    • remotecache: only visit each item once when walking results #1577
    • cache: avoid possible nil dereference on error handling #1511
    • contenthash: allow security.capability in cache checksum #1526
    • contenthash: treat unix sockets as regular files #1581
    • push: fix race condition on pushing the same layers in parallel #1548
    • inline cache: fix handling of duplicate blobs in same image #1568
    • gateway: fix metadata getting lost on subsolve in external frontend #1449
    • filesync: avoid ignoring close error #1478
    • runc: update runc binary to v1.0.0-rc91 #1553
    • buildctl-daemonless: allow max retries on socket connect for buildctl #1493
    • buildctl-daemonless: fix shell args expansion #1504
    • buildctl-daemonless: show log on startup timeout #1565
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.7.2.darwin-amd64.tar.gz(10.38 MB)
    buildkit-v0.7.2.linux-amd64.tar.gz(23.75 MB)
    buildkit-v0.7.2.linux-arm-v7.tar.gz(21.25 MB)
    buildkit-v0.7.2.linux-arm64.tar.gz(21.31 MB)
    buildkit-v0.7.2.linux-ppc64le.tar.gz(21.23 MB)
    buildkit-v0.7.2.linux-s390x.tar.gz(23.02 MB)
    buildkit-v0.7.2.windows-amd64.tar.gz(10.50 MB)
  • dockerfile/1.1.7(Apr 17, 2020)

  • v0.7.0(Mar 25, 2020)

    Images

    https://hub.docker.com/r/moby/buildkit/tags/

    • docker.io/moby/buildkit:v0.7.0 sha256:68f03dba7fe0fa40d43ce62bc292ae42a11efe4f2dee64c196e1ff266a5ea507

    • docker.io/moby/buildkit:v0.7.0-rootless sha256:6e7687bc3409812f7e6c6cee87166a0df376ca172e7a24e7f1d477ab1b3116b3

    Important

    • This release breaks compatibility with containerd 1.2 when containerd worker is used. This is to support the lease based resource tracking in containerd 1.3. Note that default configurations of buildkit use the OCI worker and are not affected. If you use containerd worker make sure to upgrade to containerd 1.3 . https://github.com/moby/buildkit/pull/1176

    • This release migrates the internal state files used for resource tracking to containerd leases on the first start and doesn't support downgrades back to v0.6.0 . https://github.com/moby/buildkit/pull/1176

    • Solve requests made from frontends or gateway API are now nonblocking and return a promise of a result. The change should be functionally invisible to old frontends but may change the timing of different requests https://github.com/moby/buildkit/pull/1356

    Notable Changes

    • LLB: Previous solve results can now be reused in new requests https://github.com/moby/buildkit/pull/1286

    • Allow frontends to take LLB states or previous results as inputs https://github.com/moby/buildkit/pull/1361

    • Support for insecure TLS registries and custom TLS config https://github.com/moby/buildkit/pull/1397 https://github.com/moby/buildkit/pull/1410

    • Support for fallbacks to origin server when mirror doesn't have requested repository https://github.com/moby/buildkit/pull/1397

    • Resource tracking has been moved to new containerd leases API from previously used root labels, fixing possible races. https://github.com/moby/buildkit/pull/1176

    • Support for cross-repo pushes for images and remote cache https://github.com/moby/buildkit/pull/1147

    • SSH sockets do not hold FD open until the end of the build https://github.com/moby/buildkit/pull/1150

    • Handle missing Etags in http responses https://github.com/moby/buildkit/pull/1159

    • LLB FileOp now supports wildcards https://github.com/moby/buildkit/pull/1233

    • Support for choosing compression for layer data https://github.com/moby/buildkit/pull/1277

    • Rootless mode supports fuse-overlayfs snapshotter https://github.com/moby/buildkit/pull/1384

    • Updates to supported platforms (eg. enabling binfmt) do not require BuildKit restart anymore https://github.com/moby/buildkit/pull/1381

    • Insecure security mode now supports access to common devices like fuse and loopback https://github.com/moby/buildkit/pull/1351

    • Rootless mode is now out of experimental https://github.com/moby/buildkit/pull/1400

    • Many bugfixes

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Edgar Lee
    • Tibor Vass
    • Andy Caldwell
    • Paul "TBBle" Hampson
    • Nikhil Pandeti
    • Sebastiaan van Stijn
    • Sam Whited
    • Wei Fu
    • Derek McGowan
    • Jeffrey Huang
    • Robert Estelle
    • Tomohiro Kusumoto
    • Troels Liebe Bentsen
    • Zach Badgett
    • Anca Iordache
    • ChaosGramer
    • Cory Bennett
    • Darren Shepherd
    • HowJMay
    • Michael Crosby
    • Oliver Bristow
    • Pablo Chico de Guzman
    • Pratik Raj
    • Lu Jingxiao
    • 岁丰
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.7.0.darwin-amd64.tar.gz(10.38 MB)
    buildkit-v0.7.0.linux-amd64.tar.gz(23.01 MB)
    buildkit-v0.7.0.linux-arm-v7.tar.gz(20.57 MB)
    buildkit-v0.7.0.linux-arm64.tar.gz(20.64 MB)
    buildkit-v0.7.0.linux-ppc64le.tar.gz(20.57 MB)
    buildkit-v0.7.0.linux-s390x.tar.gz(22.30 MB)
    buildkit-v0.7.0.windows-amd64.tar.gz(10.50 MB)
  • dockerfile/1.1.6-rc1-experimental(Mar 12, 2020)

    https://hub.docker.com/layers/docker/dockerfile-upstream/1.1.6-rc1-experimental/images/sha256-5ff27715035510f267d4665cf6847e4742866c2c88159f29c430b19a68fffcf9

    Notable Changes

    • RUN --network=none|default|host allows specifying network mode for a process https://github.com/moby/buildkit/pull/1141
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.1.6-rc1(Mar 12, 2020)

    https://hub.docker.com/layers/docker/dockerfile-upstream/1.1.6-rc1/images/sha256-152891737c5df182757065fec0768428b96fcfd1836427c196640460032ce34e

    Notable Changes

    • Dockerfile frontend now allows inputs to be passed from LLB states/results https://github.com/moby/buildkit/pull/1361

    • Update LLB client to BuildKit v0.7.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0-rc1(Mar 12, 2020)

    Images

    https://hub.docker.com/r/moby/buildkit/tags/

    • docker.io/moby/buildkit:v0.7.0-rc1 sha256:12cb8cb0d426df5ceea53c7be7542b38b389edbe07b9fcd29ed152ea5d12c6f9

    • docker.io/moby/buildkit:v0.7.0-rc1-rootless sha256:7f3ef1607dfccc95fadeebfbb9ed74dd5572eda8d656413ffb1146e6af397acc

    Important

    • This release breaks compatibility with containerd 1.2 when containerd worker is used. This is to support the lease based resource tracking in containerd 1.3. Note that default configurations of buildkit use the OCI worker and are not affected. If you use containerd worker make sure to upgrade to containerd 1.3 . https://github.com/moby/buildkit/pull/1176

    • This release migrates the internal state files used for resource tracking to containerd leases on the first start and doesn't support downgrades back to v0.6.0 . https://github.com/moby/buildkit/pull/1176

    • Solve requests made from frontends or gateway API are now nonblocking and return a promise of a result. The change should be functionally invisible to old frontends but may change the timing of different requests https://github.com/moby/buildkit/pull/1356

    Notable Changes

    • LLB: Previous solve results can now be reused in new requests https://github.com/moby/buildkit/pull/1286

    • Allow frontends to take LLB states or previous results as inputs https://github.com/moby/buildkit/pull/1361

    • Support for insecure TLS registries https://github.com/moby/buildkit/pull/1397

    • Support for fallbacks to origin server when mirror doesn't have requested repository https://github.com/moby/buildkit/pull/1397

    • Resource tracking has been moved to new containerd leases API from previously used root labels, fixing possible races. https://github.com/moby/buildkit/pull/1176

    • Support for cross-repo pushes for images and remote cache https://github.com/moby/buildkit/pull/1147

    • SSH sockets do not hold FD open until the end of the build https://github.com/moby/buildkit/pull/1150

    • Handle missing Etags in http responses https://github.com/moby/buildkit/pull/1159

    • LLB FileOp now supports wildcards https://github.com/moby/buildkit/pull/1233

    • Support for choosing compression for layer data https://github.com/moby/buildkit/pull/1277

    • Rootless mode supports fuse-overlayfs snapshotter https://github.com/moby/buildkit/pull/1384

    • Updates to supported platforms (eg. enabling binfmt) do not require BuildKit restart anymore https://github.com/moby/buildkit/pull/1381

    • Insecure security mode now supports access to common devices like fuse and loopback https://github.com/moby/buildkit/pull/1351

    • Rootless mode is now out of experimental https://github.com/moby/buildkit/pull/1400

    • Many bugfixes

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Edgar Lee
    • Tibor Vass
    • Andy Caldwell
    • Paul "TBBle" Hampson
    • Nikhil Pandeti
    • Sebastiaan van Stijn
    • Sam Whited
    • Wei Fu
    • Derek McGowan
    • Jeffrey Huang
    • Robert Estelle
    • Tomohiro Kusumoto
    • Troels Liebe Bentsen
    • Zach Badgett
    • Anca Iordache
    • ChaosGramer
    • Cory Bennett
    • Darren Shepherd
    • HowJMay
    • Michael Crosby
    • Oliver Bristow
    • Pablo Chico de Guzman
    • Pratik Raj
    • Lu Jingxiao
    • 岁丰
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.7.0-rc1.darwin-amd64.tar.gz(10.38 MB)
    buildkit-v0.7.0-rc1.linux-amd64.tar.gz(23.01 MB)
    buildkit-v0.7.0-rc1.linux-arm-v7.tar.gz(20.57 MB)
    buildkit-v0.7.0-rc1.linux-arm64.tar.gz(20.64 MB)
    buildkit-v0.7.0-rc1.linux-ppc64le.tar.gz(20.57 MB)
    buildkit-v0.7.0-rc1.linux-s390x.tar.gz(22.30 MB)
    buildkit-v0.7.0-rc1.windows-amd64.tar.gz(10.50 MB)
Owner
Moby
An open framework to assemble specialized container systems without reinventing the wheel.
Moby
A Simple and Comprehensive Vulnerability Scanner for Container Images, Git Repositories and Filesystems. Suitable for CI

A Simple and Comprehensive Vulnerability Scanner for Containers and other Artifacts, Suitable for CI. Table of Contents Abstract Features Installation

Aqua Security 8.9k Oct 23, 2021
GitHub中文排行榜,帮助你发现高分优秀中文项目、更高效地吸收国人的优秀经验成果;榜单每周更新一次,敬请关注!

榜单设立目的 ???? GitHub中文排行榜,帮助你发现高分优秀中文项目; 各位开发者伙伴可以更高效地吸收国人的优秀经验、成果; 中文项目只能满足阶段性的需求,想要有进一步提升,还请多花时间学习高分神级英文项目; 榜单设立范围 设立1个总榜(所有语言项目汇总排名)、18个分榜(单个语言项目排名);

kon9chunkit 38.9k Oct 24, 2021
Docker App Development & Command Practice

Docker App Development & Command Practice What is Docker? A platform for building,running and shipping applications. in a consistent manner so if your

MD MOSTAIN BILLAH 33 Aug 22, 2021
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.

Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.

Flux project 2.2k Oct 16, 2021
:rocket: Modern cross-platform HTTP load-testing tool written in Go

English | 中文 Cassowary is a modern HTTP/S, intuitive & cross-platform load testing tool built in Go for developers, testers and sysadmins. Cassowary d

Roger Welin 534 Oct 18, 2021
Power-ups for the daily DevOps life

DevOps Loop Power-Ups Requirements Connected Kubernetes cluster. Some features need support for LoadBalancer services Permission to list, create and d

Adrian Liechti 4 Oct 19, 2021
Ignite a Firecracker microVM

Weave Ignite Weave Ignite is an open source Virtual Machine (VM) manager with a container UX and built-in GitOps management. Combines Firecracker Micr

Weaveworks 1.9k Oct 21, 2021
Fast docker image distribution plugin for containerd, based on CRFS/stargz

[ ⬇️ Download] [ ?? Browse images] [ ☸ Quick Start (Kubernetes)] [ ?? Quick Start (nerdctl)] Stargz Snapshotter Read also introductory blog: Startup C

containerd 426 Oct 25, 2021
⚔️ Web Hacker's Weapons / A collection of cool tools used by Web hackers. Happy hacking , Happy bug-hunting

A collection of cool tools used by Web hackers. Happy hacking , Happy bug-hunting Family project Table of Contents WHW-Tools Weapons Awesome Bookmarkl

HAHWUL 1.6k Oct 18, 2021
A penetration toolkit for container environment

ctrsploit: A penetration toolkit for container environment 中文文档 Pre-Built Release https://github.com/ctrsploit/ctrsploit/releases Usage Quick-Start wg

null 17 Oct 14, 2021
Flexible HTTP command line stress tester for websites and web services

Pewpew Pewpew is a flexible command line HTTP stress tester. Unlike other stress testers, it can hit multiple targets with multiple configurations, si

Ben Gadbois 296 Sep 22, 2021
In ur kubernetes, buildin ur imagez

kim - The Kubernetes Image Manager STATUS: EXPERIMENT - Let us know what you think This project is a continuation of the experiment started with k3c,

Rancher 253 Oct 18, 2021
A simple and powerful SSH keys manager

SKM is a simple and powerful SSH Keys Manager. It helps you to manage your multiple SSH keys easily! Features Create, List, Delete your SSH key(s) Man

Timothy 710 Oct 13, 2021
Gohalt 👮‍♀🛑: Fast; Simple; Powerful; Go Throttler library

Gohalt ??‍♀ ?? : Fast; Simple; Powerful; Go Throttler library go get -u github.com/1pkg/gohalt Introduction Gohalt is simple and convenient yet powerf

Kostiantyn Masliuk 257 Oct 19, 2021
A fluxcd controller for managing remote manifests with kubecfg

kubecfg-operator A fluxcd controller for managing remote manifests with kubecfg This project is in very early stages proof-of-concept. Only latest ima

Pelotech 32 Oct 17, 2021
Hassle-free minimal CI/CD for git repositories with docker or docker-compose projects.

GIT-PIPE Hassle-free minimal CI/CD for git repos for docker-based projects. Features: zero configuration for repos by default automatic encrypted back

Aleksandr Baryshnikov 51 Oct 8, 2021
Google Compute Engine (GCE) VM takeover via DHCP flood - gain root access by getting SSH keys added by google_guest_agent

Abstract This is an advisory about an unpatched vulnerability (at time of publishing this repo, 2021-06-25) affecting virtual machines in Google's Com

null 501 Oct 20, 2021
Example used to try a compose application with Docker Dev Environments

compose-dev-env Example used to try a Compose application with Docker Dev Environments. This example is based on the nginx-golang-mysql sample of awes

Docker Samples 27 Oct 17, 2021
The Elastalert Operator is an implementation of a Kubernetes Operator, to easily integrate elastalert with gitops.

Elastalert Operator for Kubernetes The Elastalert Operator is an implementation of a Kubernetes Operator. Getting started Firstly, learn How to use el

null 13 Sep 23, 2021