concurrent, cache-efficient, and Dockerfile-agnostic builder toolkit

Overview

asciicinema example

BuildKit

GoDoc Build Status Go Report Card codecov

BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner.

Key features:

  • Automatic garbage collection
  • Extendable frontend formats
  • Concurrent dependency resolution
  • Efficient instruction caching
  • Build cache import/export
  • Nested build job invocations
  • Distributable workers
  • Multiple output formats
  • Pluggable architecture
  • Execution without root privileges

Read the proposal from https://github.com/moby/moby/issues/32925

Introductory blog post https://blog.mobyproject.org/introducing-buildkit-17e056cc5317

Join #buildkit channel on Docker Community Slack

ℹ️ If you are visiting this repo for the usage of BuildKit-only Dockerfile features like RUN --mount=type=(bind|cache|tmpfs|secret|ssh), please refer to frontend/dockerfile/docs/syntax.md.

ℹ️ BuildKit has been integrated to docker build since Docker 18.06 . You don't need to read this document unless you want to use the full-featured standalone version of BuildKit.

Used by

BuildKit is used by the following projects:

Quick start

ℹ️ For Kubernetes deployments, see examples/kubernetes.

BuildKit is composed of the buildkitd daemon and the buildctl client. While the buildctl client is available for Linux, macOS, and Windows, the buildkitd daemon is only available for Linux currently.

The buildkitd daemon requires the following components to be installed:

The latest binaries of BuildKit are available here for Linux, macOS, and Windows.

Homebrew package (unofficial) is available for macOS.

$ brew install buildkit

To build BuildKit from source, see .github/CONTRIBUTING.md.

Starting the buildkitd daemon:

You need to run buildkitd as the root user on the host.

$ sudo buildkitd

To run buildkitd as a non-root user, see docs/rootless.md.

The buildkitd daemon supports two worker backends: OCI (runc) and containerd.

By default, the OCI (runc) worker is used. You can set --oci-worker=false --containerd-worker=true to use the containerd worker.

We are open to adding more backends.

To start the buildkitd daemon using systemd socket activiation, you can install the buildkit systemd unit files. See Systemd socket activation

The buildkitd daemon listens gRPC API on /run/buildkit/buildkitd.sock by default, but you can also use TCP sockets. See Expose BuildKit as a TCP service.

Exploring LLB

BuildKit builds are based on a binary intermediate format called LLB that is used for defining the dependency graph for processes running part of your build. tl;dr: LLB is to Dockerfile what LLVM IR is to C.

  • Marshaled as Protobuf messages
  • Concurrently executable
  • Efficiently cacheable
  • Vendor-neutral (i.e. non-Dockerfile languages can be easily implemented)

See solver/pb/ops.proto for the format definition, and see ./examples/README.md for example LLB applications.

Currently, the following high-level languages has been implemented for LLB:

Exploring Dockerfiles

Frontends are components that run inside BuildKit and convert any build definition to LLB. There is a special frontend called gateway (gateway.v0) that allows using any image as a frontend.

During development, Dockerfile frontend (dockerfile.v0) is also part of the BuildKit repo. In the future, this will be moved out, and Dockerfiles can be built using an external image.

Building a Dockerfile with buildctl

buildctl build \
    --frontend=dockerfile.v0 \
    --local context=. \
    --local dockerfile=.
# or
buildctl build \
    --frontend=dockerfile.v0 \
    --local context=. \
    --local dockerfile=. \
    --opt target=foo \
    --opt build-arg:foo=bar

--local exposes local source files from client to the builder. context and dockerfile are the names Dockerfile frontend looks for build context and Dockerfile location.

Building a Dockerfile using external frontend:

External versions of the Dockerfile frontend are pushed to https://hub.docker.com/r/docker/dockerfile-upstream and https://hub.docker.com/r/docker/dockerfile and can be used with the gateway frontend. The source for the external frontend is currently located in ./frontend/dockerfile/cmd/dockerfile-frontend but will move out of this repository in the future (#163). For automatic build from master branch of this repository docker/dockerfile-upstream:master or docker/dockerfile-upstream:master-labs image can be used.

buildctl build \
    --frontend gateway.v0 \
    --opt source=docker/dockerfile \
    --local context=. \
    --local dockerfile=.
buildctl build \
    --frontend gateway.v0 \
    --opt source=docker/dockerfile \
    --opt context=git://github.com/moby/moby \
    --opt build-arg:APT_MIRROR=cdn-fastly.deb.debian.org

Building a Dockerfile with experimental features like RUN --mount=type=(bind|cache|tmpfs|secret|ssh)

See frontend/dockerfile/docs/experimental.md.

Output

By default, the build result and intermediate cache will only remain internally in BuildKit. An output needs to be specified to retrieve the result.

Image/Registry

buildctl build ... --output type=image,name=docker.io/username/image,push=true

To export the cache embed with the image and pushing them to registry together, type registry is required to import the cache, you should specify --export-cache type=inline and --import-cache type=registry,ref=.... To export the cache to a local directy, you should specify --export-cache type=local. Details in Export cache.

buildctl build ...\
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=inline \
  --import-cache type=registry,ref=docker.io/username/image

Keys supported by image output:

  • name=[value]: image name
  • push=true: push after creating the image
  • push-by-digest=true: push unnamed image
  • registry.insecure=true: push to insecure HTTP registry
  • oci-mediatypes=true: use OCI mediatypes in configuration JSON instead of Docker's
  • unpack=true: unpack image after creation (for use with containerd)
  • dangling-name-prefix=[value]: name image with [email protected]<digest> , used for anonymous images
  • name-canonical=true: add additional canonical name [email protected]<digest>
  • compression=[uncompressed,gzip,estargz,zstd]: choose compression type for layers newly created and cached, gzip is default value. estargz should be used with oci-mediatypes=true.
  • force-compression=true: forcefully apply compression option to all layers (including already existing layers).
  • buildinfo=[all,imageconfig,metadata,none]: choose build dependency version to export (default all).

If credentials are required, buildctl will attempt to read Docker configuration file $DOCKER_CONFIG/config.json. $DOCKER_CONFIG defaults to ~/.docker.

Local directory

The local client will copy the files directly to the client. This is useful if BuildKit is being used for building something else than container images.

buildctl build ... --output type=local,dest=path/to/output-dir

To export specific files use multi-stage builds with a scratch stage and copy the needed files into that stage with COPY --from.

...
FROM scratch as testresult

COPY --from=builder /usr/src/app/testresult.xml .
...
buildctl build ... --opt target=testresult --output type=local,dest=path/to/output-dir

Tar exporter is similar to local exporter but transfers the files through a tarball.

buildctl build ... --output type=tar,dest=out.tar
buildctl build ... --output type=tar > out.tar

Docker tarball

# exported tarball is also compatible with OCI spec
buildctl build ... --output type=docker,name=myimage | docker load

OCI tarball

buildctl build ... --output type=oci,dest=path/to/output.tar
buildctl build ... --output type=oci > output.tar

containerd image store

The containerd worker needs to be used

buildctl build ... --output type=image,name=docker.io/username/image
ctr --namespace=buildkit images ls

To change the containerd namespace, you need to change worker.containerd.namespace in /etc/buildkit/buildkitd.toml.

Cache

To show local build cache (/var/lib/buildkit):

buildctl du -v

To prune local build cache:

buildctl prune

Garbage collection

See ./docs/buildkitd.toml.md.

Export cache

BuildKit supports the following cache exporters:

  • inline: embed the cache into the image, and push them to the registry together
  • registry: push the image and the cache separately
  • local: export to a local directory
  • gha: export to GitHub Actions cache

In most case you want to use the inline cache exporter. However, note that the inline cache exporter only supports min cache mode. To enable max cache mode, push the image and the cache separately by using registry cache exporter.

Inline (push image and cache together)

buildctl build ... \
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=inline \
  --import-cache type=registry,ref=docker.io/username/image

Note that the inline cache is not imported unless --import-cache type=registry,ref=... is provided.

ℹ️ Docker-integrated BuildKit (DOCKER_BUILDKIT=1 docker build) and docker buildxrequires --build-arg BUILDKIT_INLINE_CACHE=1 to be specified to enable the inline cache exporter. However, the standalone buildctl does NOT require --opt build-arg:BUILDKIT_INLINE_CACHE=1 and the build-arg is simply ignored.

Registry (push image and cache separately)

buildctl build ... \
  --output type=image,name=localhost:5000/myrepo:image,push=true \
  --export-cache type=registry,ref=localhost:5000/myrepo:buildcache \
  --import-cache type=registry,ref=localhost:5000/myrepo:buildcache

--export-cache options:

  • type=registry
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • ref=docker.io/user/image:tag: reference
  • oci-mediatypes=true|false: whether to use OCI mediatypes in exported manifests. Since BuildKit v0.8 defaults to true.

--import-cache options:

  • type=registry
  • ref=docker.io/user/image:tag: reference

Local directory

buildctl build ... --export-cache type=local,dest=path/to/output-dir
buildctl build ... --import-cache type=local,src=path/to/input-dir

The directory layout conforms to OCI Image Spec v1.0.

--export-cache options:

  • type=local
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • dest=path/to/output-dir: destination directory for cache exporter
  • oci-mediatypes=true|false: whether to use OCI mediatypes in exported manifests. Since BuildKit v0.8 defaults to true.

--import-cache options:

  • type=local
  • src=path/to/input-dir: source directory for cache importer
  • digest=sha256:deadbeef: digest of the manifest list to import.
  • tag=customtag: custom tag of image. Defaults "latest" tag digest in index.json is for digest, not for tag

GitHub Actions cache (experimental)

buildctl build ... \
  --output type=image,name=docker.io/username/image,push=true \
  --export-cache type=gha \
  --import-cache type=gha

Following attributes are required to authenticate against the Github Actions Cache service API:

  • url: Cache server URL (default $ACTIONS_CACHE_URL)
  • token: Access token (default $ACTIONS_RUNTIME_TOKEN)

ℹ️ This type of cache can be used with Docker Build Push Action where url and token will be automatically set. To use this backend in a inline run step, you have to include crazy-max/ghaction-github-runtime in your workflow to expose the runtime.

--export-cache options:

  • type=gha
  • mode=min (default): only export layers for the resulting image
  • mode=max: export all the layers of all intermediate steps.
  • scope=buildkit: which scope cache object belongs to (default buildkit)

--import-cache options:

  • type=gha
  • scope=buildkit: which scope cache object belongs to (default buildkit)

Consistent hashing

If you have multiple BuildKit daemon instances but you don't want to use registry for sharing cache across the cluster, consider client-side load balancing using consistent hashing.

See ./examples/kubernetes/consistenthash.

Metadata

To output build metadata such as the image digest, pass the --metadata-file flag. The metadata will be written as a JSON object to the specified file. The directory of the specified file must already exist and be writable.

buildctl build ... --metadata-file metadata.json
{"containerimage.digest": "sha256:ea0cfb27fd41ea0405d3095880c1efa45710f5bcdddb7d7d5a7317ad4825ae14",...}

Systemd socket activation

On Systemd based systems, you can communicate with the daemon via Systemd socket activation, use buildkitd --addr fd://. You can find examples of using Systemd socket activation with BuildKit and Systemd in ./examples/systemd.

Expose BuildKit as a TCP service

The buildkitd daemon can listen the gRPC API on a TCP socket.

It is highly recommended to create TLS certificates for both the daemon and the client (mTLS). Enabling TCP without mTLS is dangerous because the executor containers (aka Dockerfile RUN containers) can call BuildKit API as well.

buildkitd \
  --addr tcp://0.0.0.0:1234 \
  --tlscacert /path/to/ca.pem \
  --tlscert /path/to/cert.pem \
  --tlskey /path/to/key.pem
buildctl \
  --addr tcp://example.com:1234 \
  --tlscacert /path/to/ca.pem \
  --tlscert /path/to/clientcert.pem \
  --tlskey /path/to/clientkey.pem \
  build ...

Load balancing

buildctl build can be called against randomly load balanced the buildkitd daemon.

See also Consistent hashing for client-side load balancing.

Containerizing BuildKit

BuildKit can also be used by running the buildkitd daemon inside a Docker container and accessing it remotely.

We provide the container images as moby/buildkit:

  • moby/buildkit:latest: built from the latest regular release
  • moby/buildkit:rootless: same as latest but runs as an unprivileged user, see docs/rootless.md
  • moby/buildkit:master: built from the master branch
  • moby/buildkit:master-rootless: same as master but runs as an unprivileged user, see docs/rootless.md

To run daemon in a container:

docker run -d --name buildkitd --privileged moby/buildkit:latest
export BUILDKIT_HOST=docker-container://buildkitd
buildctl build --help

Podman

To connect to a BuildKit daemon running in a Podman container, use podman-container:// instead of docker-container:// .

podman run -d --name buildkitd --privileged moby/buildkit:latest
buildctl --addr=podman-container://buildkitd build --frontend dockerfile.v0 --local context=. --local dockerfile=. --output type=oci | podman load foo

sudo is not required.

Kubernetes

For Kubernetes deployments, see examples/kubernetes.

Daemonless

To run the client and an ephemeral daemon in a single container ("daemonless mode"):

docker run \
    -it \
    --rm \
    --privileged \
    -v /path/to/dir:/tmp/work \
    --entrypoint buildctl-daemonless.sh \
    moby/buildkit:master \
        build \
        --frontend dockerfile.v0 \
        --local context=/tmp/work \
        --local dockerfile=/tmp/work

or

docker run \
    -it \
    --rm \
    --security-opt seccomp=unconfined \
    --security-opt apparmor=unconfined \
    -e BUILDKITD_FLAGS=--oci-worker-no-process-sandbox \
    -v /path/to/dir:/tmp/work \
    --entrypoint buildctl-daemonless.sh \
    moby/buildkit:master-rootless \
        build \
        --frontend \
        dockerfile.v0 \
        --local context=/tmp/work \
        --local dockerfile=/tmp/work

Opentracing support

BuildKit supports opentracing for buildkitd gRPC API and buildctl commands. To capture the trace to Jaeger, set JAEGER_TRACE environment variable to the collection address.

docker run -d -p6831:6831/udp -p16686:16686 jaegertracing/all-in-one:latest
export JAEGER_TRACE=0.0.0.0:6831
# restart buildkitd and buildctl so they know JAEGER_TRACE
# any buildctl command should be traced to http://127.0.0.1:16686/

Running BuildKit without root privileges

Please refer to docs/rootless.md.

Building multi-platform images

Please refer to docs/multi-platform.md.

Contributing

Want to contribute to BuildKit? Awesome! You can find information about contributing to this project in the CONTRIBUTING.md

Comments
  • Failed to compute cache key in newer version

    Failed to compute cache key in newer version

    This is a docker issue but it seems to be related to BuildKit only. this is something that was still working in docker ~19.03.10 but stopped functioning in 20.10.0+. I managed to bring down my DockerFile to a minimal repro:

    This works (A.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer
    

    This also works (B.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
    

    This no longer works (C.DockerFile):

    FROM php:7.4.13-cli
    
    COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/
    COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer
    

    Output from running A and C after eachother:

    C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f A.Dockerfile .
    [+] Building 3.6s (7/7) FINISHED
     => [internal] load build definition from A.Dockerfile                                                                                                                                                           0.0s
     => => transferring dockerfile: 132B                                                                                                                                                                             0.0s
     => [internal] load .dockerignore                                                                                                                                                                                0.0s
     => => transferring context: 2B                                                                                                                                                                                  0.0s
     => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                2.9s
     => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
     => => resolve docker.io/library/composer:2.0.8                                                                                                                                                                  0.5s
     => CACHED [stage-0 1/2] FROM docker.io/library/php:[email protected]:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                           0.0s
     => [stage-0 2/2] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                           0.1s
     => exporting to image                                                                                                                                                                                           0.1s
     => => exporting layers                                                                                                                                                                                          0.0s
     => => writing image sha256:ea6d75bc9ad24e800c8083e9ea6b7774f2bd9610cb0e61b3640058c9c7fe34c6                                                                                                                     0.0s
    
    C:\Users\Test>set "DOCKER_BUILDKIT=1" & docker build -f C.Dockerfile .
    [+] Building 1.0s (8/8) FINISHED
     => [internal] load build definition from C.Dockerfile                                                                                                                                                           0.0s
     => => transferring dockerfile: 221B                                                                                                                                                                             0.0s
     => [internal] load .dockerignore                                                                                                                                                                                0.0s
     => => transferring context: 2B                                                                                                                                                                                  0.0s
     => [internal] load metadata for docker.io/library/php:7.4.13-cli                                                                                                                                                0.2s
     => FROM docker.io/mlocati/php-extension-installer:latest                                                                                                                                                        0.0s
     => => resolve docker.io/mlocati/php-extension-installer:latest                                                                                                                                                  0.0s
     => => sha256:ccf3a05d8241580ad9d2a6c884a735bb248e90942ab23e0f8197f851a999ddac 526B / 526B                                                                                                                       0.0s
     => CACHED FROM docker.io/library/composer:2.0.8                                                                                                                                                                 0.0s
     => [stage-0 1/3] FROM docker.io/library/php:[email protected]:c099060944167d20100140434ee13b7c134bc53ae8c0a72e81b8f01c07a1f49d                                                                                  0.0s
     => CACHED [stage-0 2/3] COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/bin/                                                                                                   0.0s
     => ERROR [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer                                                                                                                     0.0s
    ------
     > [stage-0 3/3] COPY --from=composer:2.0.8 /usr/bin/composer /usr/local/bin/composer:
    ------
    failed to compute cache key: "/usr/bin/composer" not found: not found
    

    This doesn't happen consistently in my build, sometimes everything builds fine and there are no issues. I'm using windows 10 (20H2) and the latest version of Docker Desktop that includes Docker version 20.10.2, build 2291f61, but I have also seen this happen on Linux with the same version

    status/needs-investigation 
    opened by DRoet 61
  • Dockerfile heredocs

    Dockerfile heredocs

    relates to https://github.com/moby/moby/issues/34423

    As mentioned in #2121, I've been making progress towards implementing heredocs in Dockerfiles, and thought it might be time to open a PR for it :tada:

    I've essentially got all the functionality I think we'd need before wanting to merge, though I'm sure there's some fixes/tests to write before that.

    Things that definitely need resolving before a merge is really possible:

    • [x] Gate the feature behind a build tag, as suggested by @tonistiigi
    • [x] Warn/error/do something if a heredoc is used in a place it's not expected (e.g. an ENV command)
    • [x] Handle RUN heredocs in Windows more elegantly (doesn't look particularly doable with cmd so the current hacky approach, might be the best?)
    • [x] Tests! Currently only the parsing stages are tested, so we some some more complex integration tests.

    I'd really appreciate any feedback anyone has on the current design and implementation!

    opened by jedevc 57
  • Add OCI source

    Add OCI source

    This is an early draft. As discussed with @tonistiigi and @sipsma , this extends the source/containerimage/ so it can support "pulling" from 2 distinct places: an actual registry (as now) and an OCI layout. In addition to the docker-image: scheme, it also recognizes and oci-layout: scheme.

    It uses the same source/containerimage/ directory, just adding a typed constant so that it can choose if it is oci-layout from a directory, or from a registry. It then just selects which Pool.Resolver to use.

    Push() is disabled.

    Some things I haven't yet addressed:

    • buildkit often runs inside a container, so how is the provided directory mapped? This already had to be dealt with for local: scheme, so I will look there as well.
    • it is a little "conflicted" in that it expects a @sha256:<hash> but also knows how to read the OCI layout index.json. I will need to pick supporting image names and having hashes optional, or requiring hashes and eliminating name resolution.

    At this point, just ready for overall directional input.

    opened by deitch 38
  • buildkit + gcr.io private repos (credHelpers) do not stack

    buildkit + gcr.io private repos (credHelpers) do not stack

    Docker 18.09-ce here.

    I have FROM directive in my dockerfile pointing to a private registry:

    FROM gcr.io/...
    

    Running DOCKER_BUILDKIT=1 docker build . with this Dockerfile never finishes (after 5 minutes I hit CTRL-C). Without buildkit it builds fine in seconds.

    My ~/.docker/config.json is as follows:

    {
      "credHelpers": {
        "us.gcr.io": "gcloud",
        "staging-k8s.gcr.io": "gcloud",
        "asia.gcr.io": "gcloud",
        "gcr.io": "gcloud",
        "marketplace.gcr.io": "gcloud",
        "eu.gcr.io": "gcloud"
      }
    }
    

    After waiting long time and pressing CTRL-C, the following error is printed (exact image names scrambled with ...):

    ------
     > [stage-1 1/4] FROM gcr.io/...:
    ------
    failed to copy: httpReaderSeeker: failed open: unexpected status code https://gcr.io/v2/...: 403 Forbidden
    

    Bug?

    opened by haizaar 38
  • Support schema1 push for quay?

    Support schema1 push for quay?

    Astonishingly Quay.io still does not support schema2: https://github.com/bazelbuild/rules_docker/issues/102

    DEBU[0011] do request                                    digest=sha256:eb300a827decea6de23bda3e4ec5a60dcb3fb59bd01792fe3b54c08c10f68214 mediatype="application/vnd.docker.distribution.manifest.v2+json" request.headers=map[Content-Type:[application/vnd.docker.distribution.m
    anifest.v2+json]] request.method=PUT size=1245 url="https://quay.io/v2/****/****/manifests/latest"
    DEBU[0012] fetch response received                       digest=sha256:eb300a827decea6de23bda3e4ec5a60dcb3fb59bd01792fe3b54c08c10f68214 mediatype="application/vnd.docker.distribution.manifest.v2+json" response.headers=map[Server:[nginx/1.13.12] Date:[Thu, 24 May 2018 03:1
    1:16 GMT] Content-Type:[application/json] Content-Length:[131]] size=1245 status="415 Unsupported Media Type" url="https://quay.io/v2/****/****/manifests/latest"
    ERRO[0012] /moby.buildkit.v1.Control/Solve returned error: unexpected status: 415 Unsupported Media Type
    

    Do we want to support pushing as schema1?

    I hesitate to add support for such deprecated format, but probably we should do if there are also other registry implementations that lack support for schema2.

    cc @alexellis cc @dmcgowan @stevvooe

    kind/enhancement area/containerd 
    opened by AkihiroSuda 37
  • always display image hashes

    always display image hashes

    It's tough to debug docker building when I can't just get into the previously successful intermediate build image and run the next command manually...

    docker run -it --rm hash_id bash
    # execute the next RUN line here manually.
    

    I would therefore argue that image hashes should always display, just like they do in the current docker.

    kind/question 
    opened by TrentonAdams 36
  • Bridge network

    Bridge network

    adds the support for bridge networking for containerd & runc workers. fixes #28 Needs a review/suggestion Temporary Interface naming.

    NOTE: Still "docker0" is hard-coded, need to provide user input.

    Signed-off-by: Kunal Kushwaha [email protected]

    opened by kunalkushwaha 34
  • Cache manifest lists can't be exported to gcr

    Cache manifest lists can't be exported to gcr

    Related to https://github.com/moby/buildkit/issues/720 that currently has a gcr specific workaround in the pull code.

    When exporting a manifest list that contains the cache metadata for a build (eg. with mode max), the upload fails in gcr with 400 error.

    There was a report in slack that it also failed with 404 https://dockercommunity.slack.com/archives/C7S7A40MP/p1566224768282100 (hence similarities with #720) but couldn't repro that.

    #7 ERROR: error writing manifest blob: failed commit on ref "sha256:813b455d58cf597f96c8f20d04ae670127a94cd4786f14da09fef88e97bab090": unexpected status: 400 Bad Request
    ------
     > exporting cache:
    ------
    error: failed to solve: rpc error: code = Unknown desc = error writing manifest blob: failed commit on ref "sha256:813b455d58cf597f96c8f20d04ae670127a94cd4786f14da09fef88e97bab090": unexpected status: 400 Bad Request
    

    After pushing all the blobs manifest list push fails with:

    request:
    
    PUT https://gcr.io/v2/.../hello/manifests/cache HTTP/2.0
                            ← 400 application/json 190b 197ms
    
    :authority:       gcr.io
    content-type:     application/vnd.docker.distribution.manifest.list.v2+json
    content-length:   935
    accept-encoding:  gzip
    user-agent:       Go-http-client/2.0
    authorization:    Bearer ....
    
    {"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.list.v2+json","manifests":[{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:0503825856099e6adb39c8297af09547f69684b7016b7f3680ed801aa310baaa","size":2789742,"annotations":{"buildkit/createdat":"2019-08-14T17:52:26.789506223-07:00","containerd.io/uncompressed":"sha256:1bfeebd65323b8ddf5bd6a51cc7097b72788bc982e9ab3280d53d3c613adffa7"}},{"mediaType":"application/vnd.docker.image.rootfs.diff.tar.gzip","digest":"sha256:a8ce4bba72fb43cbaeb05eca5d6d682696b915c53a3519d0f00f5aec063c0ae9","size":219,"annotations":{"buildkit/createdat":"2019-08-19T14:39:20.018002388-07:00","containerd.io/uncompressed":"sha256:72e39bafb519d4c8b9d12597a538667085627a6b8c3c3f085d319d3ef6955b4f"}},{"mediaType":"application/vnd.buildkit.cacheconfig.v0","digest":"sha256:02656190e9941165d11c1ac96d683c3da0cec10714e01ed4de4b6ded7e8b7c49","size":565}]}
    
    
    response:
    
    docker-distribution-api-versio  registry/2.0
    n:
    content-type:                   application/json
    content-length:                 190
    content-encoding:               gzip
    date:                           Mon, 19 Aug 2019 21:39:36 GMT
    server:                         Docker Registry
    cache-control:                  private
    x-xss-protection:               0
    x-frame-options:                SAMEORIGIN
    alt-svc:                        quic=":443"; ma=2592000; v="46,43,39"
    [decoded gzip] JSON
    {
        "errors": [
            {
                "code": "MANIFEST_INVALID",
                "message": "Failed to parse manifest for request \"/v2/.../hello/manifests/cache\": Failed to deserialize application/vnd.docker.distribution.manifest.list.v2+json."
            }
        ]
    }
    

    @mattmoor @dmcgowan

    opened by tonistiigi 30
  • CNI network for workers

    CNI network for workers

    This PR enables networking for buildkit workers using CNI plugins. This implementation uses default CNI conf files from standard directories.

    Would like to know feedback on this.

    NOTE: Options for providing custom folders (CNI binaries & conf ) are not yet supported in CLI.

    opened by kunalkushwaha 30
  • Add IncludePatterns and ExcludePatterns options for Copy

    Add IncludePatterns and ExcludePatterns options for Copy

    Allow include and exclude patterns to be specified for the "copy" op, similarly to "local".

    Depends on https://github.com/tonistiigi/fsutil/pull/101

    cc @hinshun

    opened by aaronlehmann 29
  • Add hostname specifying for building

    Add hostname specifying for building

    Fix: #1301

    Signed-off-by: l00397676 [email protected]

    I'm trying to send this hostname arg with build-arg:HOSTNAME=<value>

    Added a hostname to message Meta in solver/pb/ops.proto.

    When executor generates the hosts file (in executor/oci/hosts.go), replace the default hostname buildkitsandbox with the value user specified.

    Test with this Dockerfile:

    FROM docker.io/centos:7
    RUN echo "Env variable HOSTNAME: $HOSTNAME" && \
        echo "hostname: $(hostname)" && \
        echo "kernel value: $(cat /proc/sys/kernel/hostname)"  && \
        echo "/etc/hosts: $(cat /etc/hosts)"
    

    Test result:

    # buildctl build --frontend=dockerfile.v0 --local dockerfile=. --local context=. --output type=docker,dest=docker.tar --opt build-arg:HOSTNAME=testtest --no-cache --progress=plain
    #1 [internal] load .dockerignore
    #1 transferring context: 2B done
    #1 DONE 0.0s
    
    #2 [internal] load build definition from Dockerfile
    #2 transferring dockerfile: 256B done
    #2 DONE 0.0s
    
    #3 [internal] load metadata for docker.io/library/centos:7
    #3 DONE 4.5s
    
    #5 [1/2] FROM docker.io/library/centos:[email protected]:4a701376d03f6b39b8c2a8f4a8e...
    #5 resolve docker.io/library/centos:[email protected]:4a701376d03f6b39b8c2a8f4a8e499441b0d567f9ab9d58e4991de4472fb813c done
    #5 CACHED
    
    #4 [2/2] RUN echo "Env variable HOSTNAME: $HOSTNAME" &&     echo "hostname:...
    #4 0.172 Env variable HOSTNAME: testtest
    #4 0.182 hostname: testtest
    #4 0.183 kernel value: testtest
    #4 0.185 /etc/hosts:
    #4 0.185 127.0.0.1      localhost testtest
    #4 0.185 ::1    localhost ip6-localhost ip6-loopback
    #4 DONE 0.2s
    
    #6 exporting to oci image format
    #6 exporting layers
    #6 exporting layers 2.0s done
    #6 exporting manifest sha256:88d227d6e517608fc265e77967ac10a8b0ab47d3153af424aca66a5b72671d57 0.0s done
    #6 exporting config sha256:cd2b4c557e9c29670f541df579edb3f59e6541aad4e9a4c17aa2b4ef7c9f88ea
    #6 exporting config sha256:cd2b4c557e9c29670f541df579edb3f59e6541aad4e9a4c17aa2b4ef7c9f88ea 0.0s done
    #6 sending tarball
    #6 sending tarball 0.9s done
    #6 DONE 3.0s
    
    opened by jingxiaolu 28
  • build fails on buildkit '#syntax' directive in Fedora guest (mount /tmp/buildkit-metadata1419535027:/run/config/buildkit/metadata (via /proc/self/fd/6), flags: 0x1021: operation not permitted) #1401

    build fails on buildkit '#syntax' directive in Fedora guest (mount /tmp/buildkit-metadata1419535027:/run/config/buildkit/metadata (via /proc/self/fd/6), flags: 0x1021: operation not permitted) #1401

    I am trying to build an image using nerdctl on Fedora guest and it stumbles on #buildkit syntax directive. The build however succeeds on Ubuntu guest.

    Steps to reproduce.

    [[email protected] syntaxtests]$ cat Dockerfile
    # syntax=docker/dockerfile:1
    FROM public.ecr.aws/docker/library/golang:1.18 AS builder
    WORKDIR /build
    COPY . .
    
    ARG TARGET_OS
    ARG TARGET_ARCH
    RUN go env -w GO111MODULE=off
    RUN CGO_ENABLED=0 GOOS=${TARGET_OS} GOARCH=${TARGET_ARCH} go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o hello .
    
    FROM scratch
    EXPOSE 8080
    COPY --from=builder /build/hello /app/
    WORKDIR /app
    ENTRYPOINT ["./hello"]
    

    runc version

    [[email protected] /]$ runc --version 
    runc version 1.1.4
    commit: v1.1.4-0-g5fd4c4d1
    spec: 1.0.2-dev
    go: go1.19.1
    libseccomp: 2.5.1
    

    BuildKit version

    [[email protected] /]$ buildctl --version 
    buildctl github.com/moby/buildkit v0.10.4 a2ba6869363812a210fcc3ded6926757ab780b5f
    

    Logs from build

    [[email protected] /]$ buildctl --addr=unix:///run/user/504/buildkit-default/buildkitd.sock build  --progress=auto  --frontend=dockerfile.v0 --local=context=/Users/siravara/vishwas-tests/syntaxtests/. --output=type=image,unpack=true --local=dockerfile=/Users/siravara/vishwas-tests/syntaxtests/ --opt=filename=Dockerfile
    [+] Building 1.5s (4/4) FINISHED                                                
     => [internal] load .dockerignore                                          0.0s
     => => transferring context: 2B                                            0.0s
     => [internal] load build definition from Dockerfile                       0.0s
     => => transferring dockerfile: 445B                                       0.0s
     => resolve image config for docker.io/docker/dockerfile:1                 1.3s
     => CACHED docker-image://docker.io/docker/dockerfile:[email protected]:9ba7531bd8  0.0s
     => => resolve docker.io/docker/dockerfile:[email protected]:9ba7531bd80fb0a858632  0.0s
    Dockerfile:1
    --------------------
       1 | >>> # syntax=docker/dockerfile:1
       2 |     FROM public.ecr.aws/docker/library/golang:1.18 AS builder
       3 |     WORKDIR /build
    --------------------
    error: failed to solve: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/tmp/buildkit-metadata655966680" to rootfs at "/run/config/buildkit/metadata": mount /tmp/buildkit-metadata655966680:/run/config/buildkit/metadata (via /proc/self/fd/6), flags: 0x1021: operation not permitted: unknown
    

    Wonder why the metadata is handled differently by BuildKit on Fedora.

    opened by vsiravar 2
  • Add Evaluate(Ref) Gateway API

    Add Evaluate(Ref) Gateway API

    This PR is a replacement for #2947 (cherry-picking that commit for consistency, so that calling Evaluate over each ref in a result is the same as setting Evaluate: true in the SolveRequest).

    This can be invoked on a ref using ref.Evaluate() in a frontend - this allows for on-demand unlazying a reference, instead of needing to wait until export. Previously, as a hack, this could be done using StatFile, however this has the disadvantage of needing to pull the image to be able to mount it, which isn't required to ensure that the image is unlazied.

    Additionally, the previous Evaluate field in the SolveRequest was not sufficient, as this causes the Solve to block - we may want to perform other operations, before then determining whether to Evaluate, as in https://github.com/docker/buildx/pull/1197.

    opened by jedevc 0
  • compression: introduce compression.Compression interface

    compression: introduce compression.Compression interface

    Introduce a new compression.Compression interface, which needs to be implemented for each compression type, by that we can reduce the number of switch case statements and ensure that we don't miss the handle of any compression types, and also make more easily for supporting new compression types.

    This is a commit for code improvement, so no logical changes.

    A successful test can be found here.

    opened by imeoer 0
  • Buildkit doesn't detect the architecture of base image

    Buildkit doesn't detect the architecture of base image

    Issue

    Base image doesn't support arm architecture but buildkit builds multi-arch image without any error.

    Expectation

    Image build should fail in this case.

    opened by josedev-union 0
  • nil pointer exception when discarding llbBridgeForwarder

    nil pointer exception when discarding llbBridgeForwarder

    Getting this exception:

    time="2022-09-26T16:03:40Z" level=error msg="/moby.buildkit.v1.frontend.LLBBridge/Solve returned error: rpc error: code = Unknown desc = process \"/_shim uname -m\" did not complete successfully: exit code: 1\n"
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0xb0 pc=0x93778c]
    
    goroutine 249258 [running]:
    github.com/moby/buildkit/frontend/gateway.(*llbBridgeForwarder).Discard(0x4000644f20)
            /src/frontend/gateway/gateway.go:356 +0x1ac
    github.com/moby/buildkit/solver/llbsolver.(*Solver).Solve(0x400054c4e0, {0x112d028, 0x40019c4330}, {0x4001384000, 0x19}, {0x4001384040, 0x19}, {0x0, 0x0, {0x0, ...}, ...}, ...)
            /src/solver/llbsolver/solver.go:324 +0x1380
    github.com/moby/buildkit/control.(*Controller).Solve(0x400055ec80, {0x112d028, 0x40019c4330}, 0x400101c120)
            /src/control/control.go:302 +0x910
    github.com/moby/buildkit/api/services/control._Control_Solve_Handler.func1({0x112d028, 0x40019c4330}, {0xf22500, 0x400101c120})
            /src/api/services/control/control.pb.go:1611 +0x7c
    github.com/moby/buildkit/util/grpcerrors.UnaryServerInterceptor({0x112d028, 0x40019c4330}, {0xf22500, 0x400101c120}, 0x4001180000, 0x4004a56048)
            /src/util/grpcerrors/intercept.go:14 +0x48
    github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x112d028, 0x40019c4330}, {0xf22500, 0x400101c120})
            /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25 +0x58
    go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc.UnaryServerInterceptor.func1({0x112cf80, 0x4003a60140}, {0xf22500, 0x400101c120}, 0x4001180000, 0x4001180020)
            /src/vendor/go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc/interceptor.go:325 +0x578
    main.unaryInterceptor.func1({0x112d028, 0x40019c4120}, {0xf22500, 0x400101c120}, 0x4001180000, 0x4001180020)
            /src/cmd/buildkitd/main.go:566 +0x134
    github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x112d028, 0x40019c4120}, {0xf22500, 0x400101c120})
            /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25 +0x58
    github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x112d028, 0x40019c4120}, {0xf22500, 0x400101c120}, 0x4001180000, 0x4004a56048)
            /src/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34 +0xc4
    github.com/moby/buildkit/api/services/control._Control_Solve_Handler({0xeb7a40, 0x400055ec80}, {0x112d028, 0x40019c4120}, 0x4000c5a000, 0x4000a00a20)
            /src/api/services/control/control.pb.go:1613 +0x14c
    google.golang.org/grpc.(*Server).processUnaryRPC(0x400060c8c0, {0x11433b0, 0x400338e9c0}, 0x4000f78240, 0x400055b1a0, 0x1a2d078, 0x0)
            /src/vendor/google.golang.org/grpc/server.go:1282 +0xc38
    google.golang.org/grpc.(*Server).handleStream(0x400060c8c0, {0x11433b0, 0x400338e9c0}, 0x4000f78240, 0x0)
            /src/vendor/google.golang.org/grpc/server.go:1619 +0xa54
    google.golang.org/grpc.(*Server).serveStreams.func1.2(0x40028c04d0, 0x400060c8c0, {0x11433b0, 0x400338e9c0}, 0x4000f78240)
            /src/vendor/google.golang.org/grpc/server.go:921 +0x94
    created by google.golang.org/grpc.(*Server).serveStreams.func1
            /src/vendor/google.golang.org/grpc/server.go:919 +0x1f0
    

    When doing the following:

    1. Pull alpine image for a few platforms
    2. Run the same execop on each of the platform images
    3. Build and push a multiplatform image with the results of those execops
    4. Pull that new multiplatform image
    5. Run the same execops on each pulled platform image

    I consistently get that nil pointer exception on step 5. This is all in the context of experimental Dagger code, so I don't have a pure LLB reproducer yet, but can update one when I have a a free min.

    Not immediately sure what's happening, it seems like this result must be a typed nil: https://github.com/sipsma/buildkit/blob/61307b8f2e73d681721b51856c10cb8606e5ea70/frontend/gateway/gateway.go#L601-L601

    But not sure if that's an expected but unhandled case or a totally unexpected case.

    kind/bug 
    opened by sipsma 0
  • Add support for JSON formatted logs

    Add support for JSON formatted logs

    It would be great to have support for JSON formatted logs, perhaps with a new option in the buildkitd.toml to switch between the default TextFormatter in here and JSONFormatter (docs).

    opened by tavlima 0
Releases(v0.10.4)
  • v0.10.4(Aug 22, 2022)

    https://hub.docker.com/r/moby/buildkit

    Notable changes:

    • Default Dockerfile frontend has been updated to v1.4.3 with fixes to handling platforms and timestamps for named image contexts. changelog
    • Fix cancellation error not being detected and erroneously cached #2926
    • Fix interactive containers not releasing resources when client doesn't gracefully disconnect them https://github.com/moby/buildkit/pull/3025
    • Fix possible panic on handling nil results https://github.com/moby/buildkit/pull/3043
    • Add logging to healthcheck monitoring and mitigate possibility of healthcheck failing under load2998
    • Add fallback when rootless buildkitd cannot access containerd socket #2968
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.10.4.darwin-amd64.tar.gz(11.42 MB)
    buildkit-v0.10.4.darwin-arm64.tar.gz(11.06 MB)
    buildkit-v0.10.4.linux-amd64.tar.gz(45.27 MB)
    buildkit-v0.10.4.linux-arm-v7.tar.gz(42.26 MB)
    buildkit-v0.10.4.linux-arm64.tar.gz(40.94 MB)
    buildkit-v0.10.4.linux-ppc64le.tar.gz(41.64 MB)
    buildkit-v0.10.4.linux-riscv64.tar.gz(43.16 MB)
    buildkit-v0.10.4.linux-s390x.tar.gz(46.05 MB)
    buildkit-v0.10.4.windows-amd64.tar.gz(11.64 MB)
    buildkit-v0.10.4.windows-arm64.tar.gz(10.78 MB)
  • dockerfile/1.4.3-labs(Aug 22, 2022)

  • dockerfile/1.4.3(Aug 22, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.4.3
    

    Notable changes

    • Fix creation timestamp not getting reset when building image from docker-image:// named context #2920
    • Fix passing --platform flag of FROM command when loading docker-image:// named context #2990
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.4.2-labs(May 6, 2022)

  • dockerfile/1.4.2(May 6, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile:1.4.2
    

    Notable changes

    • Fix loading certain environment variables from an image passed with built context #2847
    Source code(tar.gz)
    Source code(zip)
  • v0.10.1(Apr 7, 2022)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Builtin Dockerfile frontend updated to 1.4.1
    • Fix buildinfo still including attributes for passed named contexts even if no opt-in #2716
    • Fix possible panic on printing progress groups #2763
    • Fix exporting/extracting certain multi-platform images to Containerd image store #2755
    • Fix sending correct User-Agent #2776
    • Update Continuity and Containerd to v1.6.2. Fixes regression on pulling certain images with native snapshotter and possible deadlocks on error handling. #2764
    • Update Stargz snapshotter to v0.11.3. Fixes possible missing file issue #2732
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.10.1.darwin-amd64.tar.gz(11.42 MB)
    buildkit-v0.10.1.darwin-arm64.tar.gz(11.05 MB)
    buildkit-v0.10.1.linux-amd64.tar.gz(45.24 MB)
    buildkit-v0.10.1.linux-arm-v7.tar.gz(42.25 MB)
    buildkit-v0.10.1.linux-arm64.tar.gz(40.91 MB)
    buildkit-v0.10.1.linux-ppc64le.tar.gz(41.62 MB)
    buildkit-v0.10.1.linux-riscv64.tar.gz(43.15 MB)
    buildkit-v0.10.1.linux-s390x.tar.gz(46.04 MB)
    buildkit-v0.10.1.windows-amd64.tar.gz(11.64 MB)
    buildkit-v0.10.1.windows-arm64.tar.gz(10.78 MB)
  • dockerfile/1.4.1-labs(Apr 7, 2022)

  • dockerfile/1.4.1(Apr 7, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile:1.4.1
    

    Notable changes

    • Fix named context resolution for cross-compilation cases from input when input is built for a different platform #2742
    Source code(tar.gz)
    Source code(zip)
  • v0.10.0(Mar 9, 2022)

    Welcome to the v0.10.0 release of buildkit!

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Notable changes:

    • New LLB operation MergeOp allows efficiently chaining groups of layers together without the need to access the individual files. This allows to build layers separately and merge them together later without making them depend on each other. MergeOp can work with remote references, for example, it can be used to rebase an image on top of a newer base image without ever pulling or pushing any layers. #2335 #2614

    • New LLB operation DiffOp allows computing a difference between two points in LLB graph containing the files that have been added and whiteouts for files that have been removed. When DiffOp inputs are based on the same layer chain DiffOp can work directly with remote layer references and doesn't need to pull layers to access individual files. Files of the DiffOp result can be accessed directly or used as input to a MergeOp. #2517 #2434 #2563

    • New build information structures are generated with build metadata that allows you to see all the sources (images, git repositories) that were used by the build with their exact versions and also the configuration that was passed to the build. This information can also be embedded into the image configuration if one is generated. Build sources are added to the image config by default. The build configuration is not currently embedded by default to avoid credential leaks in poorly written Dockerfiles but the intention is to enable it in the future. #2311 #2476 #2654 #2657 #2645 #2684

    • Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache import #2651

    • When possible, blobs are now created with native OverlayFS differ with increased performance. This Differ can directly use files in OverlayFS upper directory instead of scanning for differences between snapshots. #2181 #2491 #2480 #2388 #2390 #2662

    • Frontends can now send warning messages that are shown on the progressbar. Warnings can be associated with specific LLB vertex and contain additional information like URL to documentation or location in original source code. #2482 #2498

    • Layer blobs can now be exported in Zstd compression format with -o compression=zstd. Zstd provides smaller files and faster decompression than gzip. #2344

    • Layers can now be exported with eStargz compression type -o compression=estargz #2246 #2603 #2352 #2674

    • A compression level can be set with -o compression-level=N to configure the compressor when new layer blobs are created. This can be used to create smaller blobs by spending more time on the compression step. #2591

    • Remote cache inlined in image configuration now supports arbitrary configuration of image layers as cache sources #2501

    • Enable eStargz-based lazy pulling on registry cache importer #2648

    • Support exporting non-distributable blob descriptors. -o prefer-nondist-layers=true exports layer with a non-distributable mediatype and external URL if such URL was provided when blob was pulled from the remote registry. Descriptors with non-distributable mediatypes are skipped on push. #2561

    • Build metadata now provides access to OCI descriptor of the result if one was generated. Previously only digest of the descriptor could be accessed. #2610

    • Builder now understands AMD64 Microarchitecture levels, e.g. linux/amd64/v2, linux/amd64/v4. The default variant remains v1 and is normalized to linux/amd64. These variants allow creating more optimized versions of your images that run when the container is running on a more modern CPU. #2588

    • LLB now supports progress groups for grouping multiple steps together so they appear as the same row in build progress #2513

    • LLB ExecOp now supports build secrets that are exposed as environment variables #2579

    • Interactive container API now supports sending signals to processes from the client #2590

    • Logs now use a rolling buffer to show the last logs for a process that ended with an error even if regular logs have been clipped because they have reached the max-logs limit. #2577

    • Remote cache exporting now support setting the same compression options available to image outputs #2685

    • Frontends can now access their own definition and call additional processes on their own image #2443

    • Tmpfs mounts can now set a size limit #2411 #2422

    • Custom Cgroup parent can now be set the LLB ExecOp #2430

    • Add support for Ulimit resource limits in LLB ExecOp #2398

    • Add extra hosts supports to Gateway Exec API #2294

    • Fix security mode config being dropped on Gateway Exec start #2290

    • Git source can now correctly clone annotated tag references #2570

    • OpenTelemetry tracing propagation environment variables have been updated to TRACEPARENT and TRACESTATE to match the changes in upstream. Old variables are deprecated and will be removed in the next release. #2572

    • Make sure supplementary groups are loaded for the default user configuration #2428

    • Allow exporting inline cache when blobs exist in multiple compressions #2405

    • Github cache backend retry logic on hitting rate limits has been improved #2506

    • Color schema on TTY progressbar has been enhanced on Windows for better readability #2368

    • Build status stream now supports ProgressGroup object to group multiple LLB steps into a single progress item #2668

    • Fixes to progress stream handling of multiple "complete" events during cache import #2675 #2641

    • Fix possible out of order indexes in plain progress mode #2688

    • Extra progress step has been added to the step where blobs are prepared for inline cache #2658

    • Allow insecure security mode to work on environments where all capabilities are not available #2394

    • Use standard user umask for Git processes #2356

    • Fix tracing indicators showing up in logs even when tracing is not enabled #2351

    • Handling of doublestar (**) pattern has been improved on transporting local sources. #2319 #2486

    • QEMU embedded emulators have been updated to v6.2.0 #2634

    • Alpine has been updated to 3.15 on release image #2582

    • External registry requests now show BuildKit major version in User-Agent header (previously Containerd) #2593

    • Fix caching of weak ETag references when pulling HTTP URLs #2629

    • Avoid gRPC size limits when transferring lots of logs #2456

    • Correct FileOp.Rm to not follow symlinks to the target #2474

    • Validate manifest blobs mediatypes against their content #2469

    • Make Git checkouts more deterministic for better cache reuse #2397

    • Containerd worker now supports rootless mode #2660

    • Fix handling tokens with multiple scopes #2431

    • Fix possible leaking processes when using external decompressor #2620

    • Fix possible issues when converting blobs to another compression #2600

    • Fix symlink handling on doing copy with includePath filters #2318

    • Performance of creating layer blobs has been improved in some cases #2601

    • Request token first with a POST request and fall back to GET request if needed #2553

    • Gracefully handle the case where a crash happens after snapshot commit #2564

    • Improve Authority pseudo-header handling with new gRPC #2518

    • Clean up temporary mounts on a restart that might have leaked after crash #2652 #2670

    • Fix possible panic on deduplicating mounts #2519

    • Fix shared cache mounts resulting in overlay corruption #2637

    • Fix remote cache imports when invoking builds through gateway API #2659

    • Fix possible panic on loading invalid config #2650

    • Fixes for session handling for parallel builds from local sources #2585

    • Fixes for scheduler inconsistency detector #2556

    • Allow listening buildkitd on multiple sockets with --addr #2649

    • Systemd definitions have been updated with Rootless and notify support #2473

    Contributors

    • Tõnis Tiigi
    • CrazyMax
    • Erik Sipsma
    • Akihiro Suda
    • Kohei Tokunaga
    • Sebastiaan van Stijn
    • Aaron Lehmann
    • Koichi Shiraishi
    • Alex Couture-Beil
    • Brian Goff
    • Justin Chadwell
    • Cory Bennett
    • Anders F Björklund
    • Davis Schirmer
    • Edgar Lee
    • Jacob MacElroy
    • Maxime Lagresle
    • Andrey Smirnov
    • Christian Weichel
    • Csaba Apagyi
    • Edgar Lee
    • Hans van den Bogert
    • Jonathan Giannuzzi
    • Morlay
    • Shingo Omura
    • Shiwei Zhang
    • Solomon Hykes
    • Vlad A. Ionescu
    • lugeng
    • sunchunming

    Dependency Changes

    • github.com/Azure/go-ansiterm d6e3b3328b78 -> d185dfc1b5a1
    • github.com/Microsoft/go-winio v0.4.17 -> v0.5.1
    • github.com/Microsoft/hcsshim v0.8.18 -> v0.9.2
    • github.com/armon/circbuf 5111143e8da2 new
    • github.com/containerd/cgroups v1.0.1 -> v1.0.3
    • github.com/containerd/console v1.0.2 -> v1.0.3
    • github.com/containerd/containerd v1.5.3 -> v1.6.1
    • github.com/containerd/continuity v0.1.0 -> v0.2.2
    • github.com/containerd/go-cni v1.0.2 -> v1.1.3
    • github.com/containerd/stargz-snapshotter v0.6.4 -> v0.11.2
    • github.com/containerd/ttrpc v1.0.2 -> v1.1.0
    • github.com/containernetworking/cni v0.8.1 -> v1.0.1
    • github.com/dimchansky/utfbom v1.1.1 new
    • github.com/docker/cli v20.10.7 -> v20.10.12
    • github.com/docker/distribution v2.7.1 -> v2.8.0
    • github.com/docker/docker ef4d47340142 -> c78f6963a1c0
    • github.com/docker/docker-credential-helpers v0.6.3 -> v0.6.4
    • github.com/go-logr/logr v1.2.2 new
    • github.com/go-logr/stdr v1.2.2 new
    • github.com/gogo/googleapis v1.4.0 -> v1.4.1
    • github.com/golang-jwt/jwt/v4 v4.1.0 new
    • github.com/golang/groupcache 8c9f03a8e57e -> 41bb18bfe9da
    • github.com/google/go-cmp v0.5.6 -> v0.5.7
    • github.com/google/uuid v1.2.0 -> v1.3.0
    • github.com/grpc-ecosystem/go-grpc-middleware v1.2.0 -> v1.3.0
    • github.com/hanwen/go-fuse/v2 v2.1.0 -> f57e95bda82d
    • github.com/hashicorp/errwrap v1.0.0 -> v1.1.0
    • github.com/hashicorp/go-cleanhttp v0.5.1 new
    • github.com/hashicorp/go-retryablehttp v0.7.0 new
    • github.com/klauspost/compress v1.12.3 -> v1.15.0
    • github.com/mitchellh/hashstructure/v2 v2.0.2 new
    • github.com/moby/sys/signal v0.6.0 new
    • github.com/moby/term bea5bbe245bf -> 3f7ff695adc6
    • github.com/opencontainers/image-spec v1.0.1 -> 693428a734f5
    • github.com/opencontainers/runc v1.0.0-rc93 -> v1.1.0
    • github.com/opencontainers/runtime-spec e6143ca7d51d -> 1c3f411f0417
    • github.com/opencontainers/selinux v1.8.0 -> v1.10.0
    • github.com/pelletier/go-toml v1.9.4 new
    • github.com/prometheus/client_golang v1.7.1 -> v1.12.1
    • github.com/prometheus/common v0.10.0 -> v0.32.1
    • github.com/prometheus/procfs v0.6.0 -> v0.7.3
    • github.com/tonistiigi/fsutil d72af97c0eaf -> b19f7f9cb274
    • github.com/tonistiigi/go-actions-cache b93d7f1b2e70 -> 9642704158ff
    • github.com/tonistiigi/go-archvariant v1.0.0 new
    • github.com/urfave/cli v1.22.2 -> v1.22.4
    • github.com/vbatts/tar-split v0.11.2 new
    • go.etcd.io/bbolt v1.3.5 -> v1.3.6
    • go.opencensus.io v0.22.3 -> v0.23.0
    • go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc d010b05b4939 -> v0.29.0
    • go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace d010b05b4939 -> v0.29.0
    • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp d010b05b4939 -> v0.29.0
    • go.opentelemetry.io/otel v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/jaeger v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.4.1 new
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/internal/metric v0.21.0 -> v0.27.0
    • go.opentelemetry.io/otel/metric v0.21.0 -> v0.27.0
    • go.opentelemetry.io/otel/sdk v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/otel/trace v1.0.0-RC1 -> v1.4.1
    • go.opentelemetry.io/proto/otlp v0.9.0 -> v0.12.0
    • golang.org/x/crypto 0c34fe9e7dc2 -> 5770296d904e
    • golang.org/x/net e18ecbb05110 -> fe4d6282115f
    • golang.org/x/sys 5e06dd20ab57 -> da31bd327af9
    • golang.org/x/time 3af7569d3a1e -> 1f47c861a9ac
    • google.golang.org/genproto 8816d57aaa9a -> 3a66f561d7aa
    • google.golang.org/grpc v1.38.0 -> v1.44.0
    • gopkg.in/yaml.v3 9f266ea9e77c -> 496545a6307b

    Previous release can be found at v0.9.0

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.10.0.darwin-amd64.tar.gz(11.42 MB)
    buildkit-v0.10.0.darwin-arm64.tar.gz(11.05 MB)
    buildkit-v0.10.0.linux-amd64.tar.gz(45.23 MB)
    buildkit-v0.10.0.linux-arm-v7.tar.gz(42.24 MB)
    buildkit-v0.10.0.linux-arm64.tar.gz(40.90 MB)
    buildkit-v0.10.0.linux-ppc64le.tar.gz(41.62 MB)
    buildkit-v0.10.0.linux-riscv64.tar.gz(43.15 MB)
    buildkit-v0.10.0.linux-s390x.tar.gz(46.03 MB)
    buildkit-v0.10.0.windows-amd64.tar.gz(11.64 MB)
    buildkit-v0.10.0.windows-arm64.tar.gz(10.77 MB)
  • dockerfile/1.4.0-labs(Mar 9, 2022)

  • dockerfile/1.4.0(Mar 9, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile:1.4.0
    

    Notable changes

    • COPY --link and ADD --link allow copying files with increased cache efficiency and rebase images without requiring them to be rebuilt. --link copies files to a separate layer and then uses new LLB MergeOp implementation to chain independent layers together. #2596 #2672 Documentation

    • Heredocs support have been promoted from labs channel to stable. This feature allows writing multiline inline scripts and files #2589 Documentation

    • Additional named build contexts can be passed to build to add or overwrite a stage or an image inside the build. A source for the context can be a local source, image, Git, or HTTP URL. #2521 #2550 #2549 #2693 Buildx Documentation

    • When using a cross-compilation stage, the target platform for a step is now seen on progress output #2576

    • BUILDKIT_SANDBOX_HOSTNAME build-arg can be used to set the default hostname for the RUN steps. #2373

    • Fixes for some cases where Heredocs incorrectly removed quotes from content #2442

    Source code(tar.gz)
    Source code(zip)
  • v0.10.0-rc2(Feb 25, 2022)

    Welcome to the 0.10.0-rc2 release of buildkit! This is a pre-release of buildkit

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Notable changes

    • Empty layer removal feature on exporting images has been removed because it caused history to change after remote cache import #2651
    • Build status stream now supports ProgressGroup object to group multiple LLB steps into a single progress item #2668
    • Fixes to tracking references to blobs with multiple compressions #2674
    • Fixes to progress stream handling of multiple "complete" events #2675 #2641
    • Extra progress step has been added to the step where blobs are prepared for inline cache #2658
    • Buildinfo attributes now contain information about the additional named contexts #2654
    • Fix possible panic with eStargz on restart #2670
    • Clean up temporary mounts on a restart that might have leaked after crash #2652
    • Fix remote cache imports when invoking builds through gateway API #2659
    • Containerd worker now supports rootless mode #2660
    • Enable eStargz-based lazy pulling on registry cache importer #2648
    • Fix possible panic on buildinfo generation #2657
    • Buildinfo can now be exported independently from exporting an image #2645
    • Fix possible panic on loading invalid config #2650
    • Allow listening buildkitd on multiple sockets with --addr #2649
    • Fix shared cache mounts resulting in overlay corruption #2637
    • Update CNI to avoid possible panic in previous release #2640

    Dependency Changes

    • github.com/containerd/containerd v1.6.0-rc.2 -> v1.6.0
    • github.com/containerd/go-cni v1.1.2 -> v1.1.3
    • github.com/containerd/stargz-snapshotter v0.11.0 -> b1ce4c8d8294
    • github.com/containerd/stargz-snapshotter/estargz v0.11.0 -> b1ce4c8d8294
    • github.com/docker/docker 40bb9831756f -> c78f6963a1c0
    • github.com/google/go-cmp v0.5.6 -> v0.5.7
    • github.com/klauspost/compress v1.14.2 -> v1.14.3
    • github.com/prometheus/client_golang v1.12.0 -> v1.12.1
    • go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.28.0 -> v0.29.0
    • go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace 2bb24f926b80 -> v0.29.0
    • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp 7e31ebe04030 -> v0.29.0
    • go.opentelemetry.io/otel v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/jaeger v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/internal/metric v0.26.0 -> v0.27.0
    • go.opentelemetry.io/otel/metric v0.26.0 -> v0.27.0
    • go.opentelemetry.io/otel/sdk v1.3.0 -> v1.4.1
    • go.opentelemetry.io/otel/trace v1.3.0 -> v1.4.1
    • go.opentelemetry.io/proto/otlp v0.11.0 -> v0.12.0

    Previous release can be found at v0.10.0-rc1

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.10.0-rc2.darwin-amd64.tar.gz(11.41 MB)
    buildkit-v0.10.0-rc2.darwin-arm64.tar.gz(11.05 MB)
    buildkit-v0.10.0-rc2.linux-amd64.tar.gz(45.17 MB)
    buildkit-v0.10.0-rc2.linux-arm-v7.tar.gz(42.20 MB)
    buildkit-v0.10.0-rc2.linux-arm64.tar.gz(40.86 MB)
    buildkit-v0.10.0-rc2.linux-ppc64le.tar.gz(41.56 MB)
    buildkit-v0.10.0-rc2.linux-riscv64.tar.gz(43.09 MB)
    buildkit-v0.10.0-rc2.linux-s390x.tar.gz(45.99 MB)
    buildkit-v0.10.0-rc2.windows-amd64.tar.gz(11.63 MB)
    buildkit-v0.10.0-rc2.windows-arm64.tar.gz(10.76 MB)
  • dockerfile/1.4.0-rc2(Feb 25, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.4.0-rc2
    

    Notable changes

    • Fix COPY --link resetting previous platform/environment information in some cases #2672
    Source code(tar.gz)
    Source code(zip)
  • v0.10.0-rc1(Feb 15, 2022)

    Welcome to the 0.10.0-rc1 release of buildkit! This is a pre-release of buildkit

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Notable changes:

    • New LLB operation MergeOp allows efficiently chaining groups of layers together without the need to access the individual files. This allows to build layers separately and merge them together later without making them depend on each other. MergeOp can work with remote references, for example, it can be used to rebase an image on top of a newer base image without ever pulling or pushing any layers. #2335 #2614

    • New LLB operation DiffOp allows computing a difference between two points in LLB graph containing the files that have been added and whiteouts for files that have been removed. When DiffOp inputs are based on the same layer chain DiffOp can work directly with remote layer references and doesn't need to pull layers to access individual files. Files of the DiffOp result can be accessed directly or used as input to a MergeOp. #2517 #2434 #2563

    • New build information structures are generated with build metadata that allows you to see all the sources (images, git repositories) that were used by the build with their exact versions and also the configuration that was passed to the build. This information can also be embedded into the image configuration if one is generated. Build sources are added to the image config by default. The build configuration is not currently embedded by default to avoid credential leaks in poorly written Dockerfiles but the intention is to enable it in the future. #2311 #2476

    • When possible, blobs are now created with native OverlayFS differ with increased performance. This Differ can directly use files in OverlayFS upper directory instead of scanning for differences between snapshots. #2181 #2491 #2480 #2388 #2390

    • Frontends can now send warning messages that are shown on the progressbar. Warnings can be associated with specific LLB vertex and contain additional information like URL to documentation or location in original source code. #2482 #2498

    • Layer blobs can now be exported in Zstd compression format with -o compression=zstd. Zstd provides smaller files and faster decompression than gzip. #2344

    • Layers can now be exported with eStargz compression type -o compression=estargz #2246 #2603 #2352

    • A compression level can be set with -o compression-level=N to configure the compressor when new layer blobs are created. This can be used to create smaller blobs by spending more time on the compression step. #2591

    • Remote cache inlined in image configuration now supports arbitrary configuration of image layers as cache sources #2501

    • Support exporting non-distributable blob descriptors. -o prefer-nondist-layers=true exports layer with a non-distributable mediatype and external URL if such URL was provided when blob was pulled from the remote registry. Descriptors with non-distributable mediatypes are skipped on push. #2561

    • Build metadata now provides access to OCI descriptor of the result if one was generated. Previously only digest of the descriptor could be accessed. #2610

    • Builder now understands AMD64 Microarchitecture levels, e.g. linux/amd64/v2, linux/amd64/v4. The default variant remains v1 and is normalized to linux/amd64. These variants allow creating more optimized versions of your images that run when the container is running on a more modern CPU. #2588

    • LLB now supports progress groups for grouping multiple steps together so they appear as the same row in build progress #2513

    • LLB ExecOp now supports build secrets that are exposed as environment variables #2579

    • Interactive container API now supports sending signals to processes from the client #2590

    • Logs now use a rolling buffer to show the last logs for a process that ended with an error even if regular logs have been clipped because they have reached the max-logs limit. #2577

    • Frontends can now access their own definition and call additional processes on their own image #2443

    • Tmpfs mounts can now set a size limit #2411 #2422

    • Custom Cgroup parent can now be set the LLB ExecOp #2430

    • Add support for Ulimit resource limits in LLB ExecOp #2398

    • Add extra hosts supports to Gateway Exec API #2294

    • Fix security mode config being dropped on Gateway Exec start #2290

    • Git source can now correctly clone annotated tag references #2570

    • OpenTelemetry tracing propagation environment variables have been updated to TRACEPARENT and TRACESTATE to match the changes in upstream. Old variables are deprecated and will be removed in the next release. #2572

    • Make sure supplementary groups are loaded for the default user configuration #2428

    • Allow exporting inline cache when blobs exist in multiple compressions #2405

    • Github cache backend retry logic on hitting rate limits has been improved #2506

    • Color schema on TTY progressbar has been enhanced on Windows for better readability #2368

    • Allow insecure security mode to work on environments where all capabilities are not available #2394

    • Use standard user umask for Git processes #2356

    • Fix tracing indicators showing up in logs even when tracing is not enabled #2351

    • Handling of doublestar (**) pattern has been improved on transporting local sources. #2319 #2486

    • QEMU embedded emulators have been updated to v6.2.0 #2634

    • Alpine has been updated to 3.15 on release image #2582

    • External registry requests now show BuildKit major version in User-Agent header (previously Containerd) #2593

    • Fix caching of weak ETag references when pulling HTTP URLs #2629

    • Avoid gRPC size limits when transferring lots of logs #2456

    • Correct FileOp.Rm to not follow symlinks to the target #2474

    • Validate manifest blobs mediatypes against their content #2469

    • Make Git checkouts more deterministic for better cache reuse #2397

    • Fix cross-repo push fallback when credentials for source repository become invalid #2630

    • Fix handling tokens with multiple scopes #2431

    • Fix possible leaking processes when using external decompressor #2620

    • Fix possible issues when converting blobs to another compression #2600

    • Fix symlink handling on doing copy with includePath filters #2318

    • Performance of creating layer blobs has been improved in some cases #2601

    • Request token first with a POST request and fall back to GET request if needed #2553

    • Gracefully handle the case where a crash happens after snapshot commit #2564

    • Improve Authority pseudo-header handling with new gRPC #2518

    • Fix possible panic on deduplicating mounts #2519

    • Fixes for session handling for parallel builds from local sources #2585

    • Fixes for scheduler inconsistency detector #2556

    • Systemd definitions have been updated with Rootless and notify support #2473

    Contributors

    • Tõnis Tiigi
    • CrazyMax
    • Erik Sipsma
    • Akihiro Suda
    • Kohei Tokunaga
    • Sebastiaan van Stijn
    • Aaron Lehmann
    • Koichi Shiraishi
    • Alex Couture-Beil
    • Brian Goff
    • Justin Chadwell
    • Cory Bennett
    • Anders F Björklund
    • Davis Schirmer
    • Jacob MacElroy
    • Maxime Lagresle
    • Andrey Smirnov
    • Christian Weichel
    • Csaba Apagyi
    • Edgar Lee
    • Hans van den Bogert
    • Jonathan Giannuzzi
    • Morlay
    • Shingo Omura
    • Shiwei Zhang
    • Solomon Hykes
    • lugeng
    • sunchunming

    Dependency Changes

    • github.com/Microsoft/go-winio v0.4.17 -> v0.5.1
    • github.com/Microsoft/hcsshim v0.8.18 -> v0.9.2
    • github.com/containerd/cgroups v1.0.1 -> v1.0.3
    • github.com/containerd/console v1.0.2 -> v1.0.3
    • github.com/containerd/containerd v1.5.3 -> v1.6.0-rc.2
    • github.com/containerd/continuity v0.1.0 -> v0.2.2
    • github.com/containerd/go-cni v1.0.2 -> v1.1.2
    • github.com/containerd/stargz-snapshotter v0.6.4 -> v0.11.0
    • github.com/containerd/ttrpc v1.0.2 -> v1.1.0
    • github.com/containernetworking/cni v0.8.1 -> v1.0.1
    • github.com/dimchansky/utfbom v1.1.1 new
    • github.com/docker/cli v20.10.7 -> v20.10.12
    • github.com/docker/distribution v2.7.1 -> v2.8.0
    • github.com/docker/docker ef4d47340142 -> 40bb9831756f
    • github.com/docker/docker-credential-helpers v0.6.3 -> v0.6.4
    • github.com/gogo/googleapis v1.4.0 -> v1.4.1
    • github.com/google/uuid v1.2.0 -> v1.3.0
    • github.com/grpc-ecosystem/go-grpc-middleware v1.2.0 -> v1.3.0
    • github.com/hanwen/go-fuse/v2 v2.1.0 -> f57e95bda82d
    • github.com/klauspost/compress v1.12.3 -> v1.14.2
    • github.com/moby/sys/signal v0.6.0 new
    • github.com/moby/term bea5bbe245bf -> 3f7ff695adc6
    • github.com/opencontainers/image-spec v1.0.1 -> 693428a734f5
    • github.com/opencontainers/runc v1.0.0-rc93 -> v1.1.0
    • github.com/opencontainers/runtime-spec e6143ca7d51d -> 1c3f411f0417
    • github.com/opencontainers/selinux v1.8.0 -> v1.10.0
    • github.com/pelletier/go-toml v1.9.4 new
    • github.com/tonistiigi/fsutil d72af97c0eaf -> b19f7f9cb274
    • github.com/tonistiigi/go-actions-cache b93d7f1b2e70 -> 9642704158ff
    • github.com/tonistiigi/go-archvariant v1.0.0 new
    • github.com/urfave/cli v1.22.2 -> v1.22.4
    • go.etcd.io/bbolt v1.3.5 -> v1.3.6
    • go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc d010b05b4939 -> v0.28.0
    • go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace d010b05b4939 -> 2bb24f926b80
    • go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp d010b05b4939 -> 7e31ebe04030
    • go.opentelemetry.io/otel v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/exporters/jaeger v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.3.0 new
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/internal/metric v0.21.0 -> v0.26.0
    • go.opentelemetry.io/otel/metric v0.21.0 -> v0.26.0
    • go.opentelemetry.io/otel/sdk v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/otel/trace v1.0.0-RC1 -> v1.3.0
    • go.opentelemetry.io/proto/otlp v0.9.0 -> v0.11.0
    • golang.org/x/crypto 0c34fe9e7dc2 -> 5770296d904e
    • golang.org/x/net e18ecbb05110 -> fe4d6282115f
    • golang.org/x/sys 5e06dd20ab57 -> da31bd327af9
    • golang.org/x/time 3af7569d3a1e -> 1f47c861a9ac
    • google.golang.org/genproto 8816d57aaa9a -> 3a66f561d7aa
    • google.golang.org/grpc v1.38.0 -> v1.44.0
    • gopkg.in/yaml.v3 9f266ea9e77c -> 496545a6307b
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.10.0-rc1.darwin-amd64.tar.gz(11.40 MB)
    buildkit-v0.10.0-rc1.darwin-arm64.tar.gz(11.04 MB)
    buildkit-v0.10.0-rc1.linux-amd64.tar.gz(45.13 MB)
    buildkit-v0.10.0-rc1.linux-arm-v7.tar.gz(42.16 MB)
    buildkit-v0.10.0-rc1.linux-arm64.tar.gz(40.83 MB)
    buildkit-v0.10.0-rc1.linux-ppc64le.tar.gz(41.53 MB)
    buildkit-v0.10.0-rc1.linux-riscv64.tar.gz(43.05 MB)
    buildkit-v0.10.0-rc1.linux-s390x.tar.gz(45.95 MB)
    buildkit-v0.10.0-rc1.windows-amd64.tar.gz(11.62 MB)
    buildkit-v0.10.0-rc1.windows-arm64.tar.gz(10.76 MB)
  • dockerfile/1.4.0-rc1(Feb 15, 2022)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.4.0-rc1
    

    Notable changes

    • COPY --link and ADD --link allow copying files with increased cache efficiency and rebase images without requiring them to be rebuilt. --link copies files to a separate layer and then uses new LLB MergeOp implementation to chain independent layers together. #2596 Documentation

    • Heredocs support have been promoted from labs channel to stable. This feature allows writing multiline inline scripts and files #2589 Documentation

    • Additional named build contexts can be passed to build to add or overwrite a stage or an image inside the build. A source for the context can be a local source, image, Git, or HTTP URL. #2521 #2550 #2549 Buildx Documentation

    • When using a cross-compilation stage, the target platform for a step is now seen on progress output #2576

    • BUILDKIT_SANDBOX_HOSTNAME build-arg can be used to set the default hostname for the RUN steps. #2373

    • Fixes for some cases where Heredocs incorrectly removed quotes from content #2442

    Source code(tar.gz)
    Source code(zip)
  • v0.9.2(Nov 2, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Fix handling authentication requests with multiple scopes #2431
    • Fix a possible deadlock when pushing items from one registry to another #2403
    • Fix issues with concurrent cache export requests #2410
    • Fix handling external blob deletions when reexporting cache with Github Actions backend #2433
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.2.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.2.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.2.linux-amd64.tar.gz(45.23 MB)
    buildkit-v0.9.2.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.2.linux-arm64.tar.gz(41.62 MB)
    buildkit-v0.9.2.linux-ppc64le.tar.gz(43.18 MB)
    buildkit-v0.9.2.linux-riscv64.tar.gz(42.15 MB)
    buildkit-v0.9.2.linux-s390x.tar.gz(44.55 MB)
    buildkit-v0.9.2.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.2.windows-arm64.tar.gz(10.55 MB)
  • v0.9.1(Oct 4, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Builtin Dockerfile frontend has been updated to v1.3.1 changelog

    • Seccomp profile has been updated to properly handle clone3 syscall support. #2379

    • Fix possible panic on ARM32 due to struct alignment #2321

    • Fix occasional "no active session" and "no such job" errors on concurrent builds #2369

    • Fix flakiness during import of a cache with empty layers removed #2372

    • Handle "edge not found" error in build instead of panicking #2382 #2385

    • Fix problems with experimental Github Actions cache backend:

      • Add retry logic when API rate limits are reached https://github.com/tonistiigi/go-actions-cache/pull/8
      • Remove BOM from error messages returned by API https://github.com/tonistiigi/go-actions-cache/pull/6
      • Gracefully handle downloading blobs that have been removed from the cache https://github.com/moby/buildkit/pull/2387
      • Fix "cache already exists" errors in certain conditions https://github.com/moby/buildkit/pull/2387
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.1.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.1.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.1.linux-amd64.tar.gz(45.23 MB)
    buildkit-v0.9.1.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.1.linux-arm64.tar.gz(41.60 MB)
    buildkit-v0.9.1.linux-ppc64le.tar.gz(43.18 MB)
    buildkit-v0.9.1.linux-riscv64.tar.gz(42.16 MB)
    buildkit-v0.9.1.linux-s390x.tar.gz(44.55 MB)
    buildkit-v0.9.1.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.1.windows-arm64.tar.gz(10.55 MB)
  • dockerfile/1.3.1-labs(Oct 4, 2021)

  • dockerfile/1.3.1(Oct 4, 2021)

    Usage

    This release is currently in staging.

    # syntax=docker/dockerfile-upstream:1.3.1
    

    Notable changes

    • Fix parsing "required" mount key without a value #2304
    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jul 16, 2021)

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.3.0 including support for RUN --network and Here-documents in labs channel. Dockerfile changelog Dockerfile labs changelog

    • Experimental support for Github Actions remote cache backend via type=gha #1974 Docs

    • Add support for subdirectories when building from Git source #2116

    • Outgoing TCP connections are now limited to 4 per registry. Metadata requests get one extra connection not used by layer pulls and pushes. #2259 #2242 #2247

    • Buildkitd config allows max-parallelism for limiting the parallelism of the BuildKit solver for low-powered machines #2049

    • OpenTracing providers have been replaced with support for OpenTelemetry #2152 #2192 #2238

    • Errors have been improved by removing gRPC wrapping and providing suggestions for typos #2218 #2047 #2215 #2183

    • New OpenTelemetry support allows forwarding traces with control API from the client or from user programs in llb.Exec container #2163

    • Git: Default branch name is now detected correctly from remote #2228

    • Allow forcing specific compression on exported layers even if another blob exists #2057

    • Plain progress mode now prints last logs of failed command in error summary #2214

    • Plain progress mode does not print to LLB vertex digest anymore to avoid confusion #2126

    • Buildctl allows --metadata-file flag to output build metadata #2095

    • This is the first release supporting Risc-V (experimental) #2222

    • This is the first release supporting MacOS ARM64 and Windows ARM64 for buildctl #2037 #2187

    • Runc has been updated to v1.0.0 GA #2143

    • RootlessKit has been updated to v0.14.2 #2102

    • Embedded QEMU emulators have been updated to v6.0.0 #2225

    • LLB: Root directory can now be copied from empty references without causing an error or panic #2197

    • LLB: Ensure image metadata resolver uses platform constraints set by marshaler #2196

    • LLB: Copy operation now allows include and exclude filters to limit the copied files #2082

    • Stargz snapshotter now supports authentication from Docker config #1733 #2165

    • Support Windows OpenSSH agent forwarding #2127

    • Handle expired tokens errors better #2062 #1957

    • LLB: Support ALL_PROXY as a proxy environment variable that does not stay in build cache #2086

    • Buildkit release images now contain OpenSSH #2135

    • Enable containerd labels in buildctl debug workers and make their order deterministic #2070 #2071

    • Gracefully handle client-side token seed errors #2050

    • Pushing multi-platform images will not try to overwrite image tag internally multiple times. This is important for registries that support immutable tags. #2020

    • Missing /etc/passwd and /etc/group file is now handled gracefully. Important when cross-compiling Windows images. #2249

    • Fix logs clipping behavior and double the limits #1934

    • Support socket activation with --addr fd:// #1924

    • Daemon logs now show traceID and spanID if OpenTelemetry is enabled #2235

    • Fix issue where Dockerfile with the same metadata and timestamps as another build could pick up its build cache by introducing a "none" differ for transferring local files that ignores metadata matches. #2081

    • Fix rare "retry timeout exceeded" errors in job synchronization #2195

    • Fix gracefully handling permission errors on accessing tokens from client-side #2234

    • Ensure containerd executor always waits for IO to complete before returning from Exec/Run. #2205

    • Fix pulling layers that have already been pulled with different compression. #2226

    • Fix some syncronisation errors #2178 #2177 #2156 #2052 #2051

    • Fix goroutine leak from progress writing #2203

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Aaron Lehmann
    • Sebastiaan van Stijn
    • CrazyMax
    • Tibor Vass
    • Alex Couture-Beil
    • Justin Chadwell
    • Kohei Tokunaga
    • Cory Bennett
    • Erik Sipsma
    • Siebe Schaap
    • Vlad A. Ionescu
    • Levi Harrison
    • Brian Goff
    • Edgar Lee
    • Anders F Björklund
    • Charles Korn
    • Claudiu Belu
    • Corey Larson
    • Jesse Rittner
    • Justin Garrison
    • Marko Kohtala
    • Omer Mizrahi
    • Pierre Fenoll
    • Rob Taylor
    • Yamazaki Masashi
    • zhangwenlong
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0.darwin-amd64.tar.gz(11.53 MB)
    buildkit-v0.9.0.darwin-arm64.tar.gz(11.10 MB)
    buildkit-v0.9.0.linux-amd64.tar.gz(45.21 MB)
    buildkit-v0.9.0.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.0.linux-arm64.tar.gz(41.57 MB)
    buildkit-v0.9.0.linux-ppc64le.tar.gz(43.15 MB)
    buildkit-v0.9.0.linux-riscv64.tar.gz(42.15 MB)
    buildkit-v0.9.0.linux-s390x.tar.gz(44.58 MB)
    buildkit-v0.9.0.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.0.windows-arm64.tar.gz(10.55 MB)
  • v0.9.0-rc2(Jul 16, 2021)

    Welcome to the 0.9.0-rc2 release of buildkit! This is a pre-release of buildkit

    Notable Changes

    • Fix progress regression in v0.9.0-rc1`#2254
    • Experimental support for Github Actions remote cache backend via type=gha #1974
    • Outgoing TCP connections are now limited to 4 per registry. Metadata requests get one extra connection not used by layer pulls and pushes. #2259 #2242 #2247
    • Missing /etc/passwd and /etc/group file is now handled gracefully. Important when cross-compiling Windows images. #2249
    • Daemon logs now show traceID and spanID is OpenTelemetry is enabled #2235
    • OpenTelemetry traces now show extra information about outgoing HTTP requests #2238
    • Fix gracefully handling permission errors on accessing tokens from client-side #2234
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0-rc2.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.0-rc2.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.0-rc2.linux-amd64.tar.gz(45.21 MB)
    buildkit-v0.9.0-rc2.linux-arm-v7.tar.gz(40.76 MB)
    buildkit-v0.9.0-rc2.linux-arm64.tar.gz(41.57 MB)
    buildkit-v0.9.0-rc2.linux-ppc64le.tar.gz(43.15 MB)
    buildkit-v0.9.0-rc2.linux-riscv64.tar.gz(42.15 MB)
    buildkit-v0.9.0-rc2.linux-s390x.tar.gz(44.58 MB)
    buildkit-v0.9.0-rc2.windows-amd64.tar.gz(11.50 MB)
    buildkit-v0.9.0-rc2.windows-arm64.tar.gz(10.55 MB)
  • dockerfile/1.3.0-labs(Jul 16, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile:1.3.0-labs
    

    Notable changes

      • RUN and COPY commands now support Here-document syntax allowing writing multiline inline scripts and files #2132 #2201 #2209 #2213 Documentation
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.3.0(Jul 16, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile:1.3.0
    

    Notable changes

    • RUN command allows --network flag for requesting a specific type of network conditions. --network=host requires allowing network.host entitlement. This feature was previously only available on labs channel. Documentation

    • ADD command with a remote URL input now correctly handles the --chmod flag. #2171

    • Values for RUN --mount flag now support variable expansion, except for the from field #2089

    • Allow BUILDKIT_MULTI_PLATFORM build arg to force always creating multi-platform image, even if only contains single platform #1985

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0-rc1(Jul 7, 2021)

    Welcome to the 0.9.0-rc1 release of buildkit! This is a pre-release of buildkit

    Notable Changes

    • Builtin Dockerfile frontend defaults to v1.3.0-rc1 including support for RUN --network and Here-documents in labs channel. Dockerfile changelog Dockerfile labs changelog

    • Errors have been improved by removing gRPC wrapping and providing suggestions for typos #2218 #2047 #2215 #2183

    • Add support for subdirectories when building from Git source #2116

    • Buildkitd config allows max-parallelism for limiting the parallelism of the BuildKit solver for low-powered machines #2049

    • OpenTracing providers have been replaced with support for OpenTelemetry #2152 #2192

    • New OpenTelemetry support allows forwarding traces with control API from the client or from user programs in llb.Exec container #2163

    • Git: Default branch name is now detected correctly from remote #2228

    • Allow forcing specific compression on exported layers even if another blob exists #2057

    • Plain progress mode now prints last logs of failed command in error summary #2214

    • Plain progress mode does not print to LLB vertex digest anymore to avoid confusion #2126

    • Buildctl allows --metadata-file flag to output build metadata #2095

    • This is the first release supporting Risc-V (experimental) #2222

    • This is the first release supporting MacOS ARM64 and Windows ARM64 for buildctl #2037 #2187

    • Runc has been updated to v1.0.0 GA #2143

    • RootlessKit has been updated to v0.14.2 #2102

    • Embedded QEMU emulators have been updated to v6.0.0 #2225

    • LLB: Root directory can now be copied from empty references without causing an error or panic #2197

    • LLB: Ensure image metadata resolver uses platform constraints set by marshaler #2196

    • LLB: Copy operation now allows include and exclude filters to limit the copied files #2082

    • Stargz snapshotter now supports authentication from Docker config #1733 #2165

    • Support Windows OpenSSH agent forwarding #2127

    • Handle expired tokens errors better #2062 #1957

    • LLB: Support ALL_PROXY as a proxy environment variable that does not stay in build cache #2086

    • Buildkit release images now contain OpenSSH #2135

    • Enable containerd labels in buildctl debug workers and make their order deterministic #2070 #2071

    • Gracefully handle client-side token seed errors #2050

    • Pushing multi-platform images will not try to overwrite image tag internally multiple times. This is important for registries that support immutable tags. #2020

    • Fix logs clipping behavior and double the limits #1934

    • Support socket activation with --addr fd:// #1924

    • Fix issue where Dockerfile with the same metadata and timestamps as another build could pick up its build cache by introducing a "none" differ for transferring local files that ignores metadata matches. #2081

    • Fix rare "retry timeout exceeded" errors in job synchronization #2195

    • Ensure containerd executor always waits for IO to complete before returning from Exec/Run. #2205

    • Fix pulling layers that have already been pulled with different compression. #2226

    • Fix some syncronisation errors #2178 #2177 #2156 #2052 #2051

    • Fix goroutine leak from progress writing #2203

    Please try out the release binaries and report any issues at https://github.com/moby/buildkit/issues.

    Contributors

    • Tõnis Tiigi
    • Akihiro Suda
    • Aaron Lehmann
    • Sebastiaan van Stijn
    • CrazyMax
    • Tibor Vass
    • Alex Couture-Beil
    • Justin Chadwell
    • Kohei Tokunaga
    • Cory Bennett
    • Erik Sipsma
    • Siebe Schaap
    • Vlad A. Ionescu
    • Levi Harrison
    • Brian Goff
    • Edgar Lee
    • Anders F Björklund
    • Charles Korn
    • Corey Larson
    • Jesse Rittner
    • Justin Garrison
    • Marko Kohtala
    • Omer Mizrahi
    • Pierre Fenoll
    • Rob Taylor
    • Yamazaki Masashi
    • zhangwenlong
    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.9.0-rc1.darwin-amd64.tar.gz(11.23 MB)
    buildkit-v0.9.0-rc1.darwin-arm64.tar.gz(11.00 MB)
    buildkit-v0.9.0-rc1.linux-amd64.tar.gz(45.00 MB)
    buildkit-v0.9.0-rc1.linux-arm-v7.tar.gz(40.54 MB)
    buildkit-v0.9.0-rc1.linux-arm64.tar.gz(41.38 MB)
    buildkit-v0.9.0-rc1.linux-ppc64le.tar.gz(42.94 MB)
    buildkit-v0.9.0-rc1.linux-riscv64.tar.gz(41.92 MB)
    buildkit-v0.9.0-rc1.linux-s390x.tar.gz(44.32 MB)
    buildkit-v0.9.0-rc1.windows-amd64.tar.gz(11.49 MB)
    buildkit-v0.9.0-rc1.windows-arm64.tar.gz(10.55 MB)
  • dockerfile/1.3.0-rc1-labs(Jul 7, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.3.0-rc1-labs
    

    Notable changes

      • RUN and COPY commands now support Here-document syntax allowing writing multiline inline scripts and files #2132 #2201 #2209 #2213 Documentation
    Source code(tar.gz)
    Source code(zip)
  • dockerfile/1.3.0-rc1(Jul 7, 2021)

    Usage

    # syntax=docker.io/docker/dockerfile-upstream:1.3.0-rc1
    

    Notable changes

    • RUN command allows --network flag for requesting a specific type of network conditions. --network=host requires allowing network.host entitlement. This feature was previously only available on labs channel. Documentation

    • ADD command with a remote URL input now correctly handles the --chmod flag. #2171

    • Values for RUN --mount flag now support variable expansion, except for the from field #2089

    • Allow BUILDKIT_MULTI_PLATFORM build arg to force always creating multi-platform image, even if only contains single platform #1985

    Source code(tar.gz)
    Source code(zip)
  • v0.8.3(Apr 30, 2021)

    https://hub.docker.com/r/moby/buildkit

    Notable changes

    • Update containerd with fixes to rootless overlay on kernel 5.11 and push panic #2014

    • Add retry on 5xx push errors https://github.com/moby/buildkit/pull/2043

    • Include basename in content checksum for wildcards #2018

    • Fix missing mounts in execOp cache map #2076

    Source code(tar.gz)
    Source code(zip)
    buildkit-v0.8.3.darwin-amd64.tar.gz(10.89 MB)
    buildkit-v0.8.3.linux-amd64.tar.gz(42.16 MB)
    buildkit-v0.8.3.linux-arm-v7.tar.gz(38.02 MB)
    buildkit-v0.8.3.linux-arm64.tar.gz(38.82 MB)
    buildkit-v0.8.3.linux-ppc64le.tar.gz(40.40 MB)
    buildkit-v0.8.3.linux-s390x.tar.gz(41.89 MB)
    buildkit-v0.8.3.windows-amd64.tar.gz(11.02 MB)
Owner
Moby
An open framework to assemble specialized container systems without reinventing the wheel.
Moby
Conjur Kubernetes All-in-One Dockerfile

conjur-authn-k8s-aio Conjur Kubernetes All-in-One Dockerfile Supported Authenticators Usage Build Secretless Broker Build Conjur Authn-K8s Client Buil

Joe Garcia 1 Sep 14, 2022
Buildg: A tool to interactively debug Dockerfile

buildg: A tool to interactively debug Dockerfile buildg is a tool to interactively debug Dockerfile based on BuildKit. Source-level inspection Breakpo

Kohei Tokunaga 780 Oct 5, 2022
BuildKit - A toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner

BuildKit BuildKit is a toolkit for converting source code to build artifacts in an efficient, expressive and repeatable manner. Key features: Automati

CrazyMax 5 Feb 19, 2022
Tpf2-tpnetmap-toolkit - A toolkit to create svg map images from TransportFever2 world data

tpf2-tpnetmap-toolkit TransportFever2 のワールドデータから svg のマップ画像を作成するツールキットです。 1. 導入方

Nosrith 1 Feb 17, 2022
Highly configurable prompt builder for Bash, ZSH and PowerShell written in Go.

Go Bullet Train (GBT) Highly configurable prompt builder for Bash, ZSH and PowerShell written in Go. It's inspired by the Oh My ZSH Bullet Train theme

Jiri Tyr 517 Sep 16, 2022
Simple, rootless, "FROM scratch" OCI image builder

zeroimage zeroimage some-program is like building the following Docker image: FROM scratch COPY some-program /some-program ENTRYPOINT ["/some-program"

Alex Hamlin 2 Jun 26, 2022
Kevi: a kubernetes appliance builder/deployer

kevi kevi is a kubernetes appliance builder/deployer (syskeví = Greek for "appli

Josh Wolf 1 May 3, 2022
Demo of skaffold's port-forwarding with ko builder (does not work)

skaffold port-forwarding : Ko builder vs docker builder When using ko builder (see folder ko/), port forwarding does not work (skaffold debug or skaff

null 0 Jan 6, 2022
Fast, concurrent, streaming access to Amazon S3, including gof3r, a CLI. http://godoc.org/github.com/rlmcpherson/s3gof3r

s3gof3r s3gof3r provides fast, parallelized, pipelined streaming access to Amazon S3. It includes a command-line interface: gof3r. It is optimized for

Randall McPherson 1.1k Sep 27, 2022
concurrent map implementation using bucket list like a skip list.

Skip List Map in Golang Skip List Map is an ordered and concurrent map. this Map is goroutine safety for reading/updating/deleting, no-require locking

Kazuhisa TAKEI 7 Apr 23, 2022
Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics

kepler Kepler (Kubernetes Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics Architectur

Sustainable Computing 156 Sep 27, 2022
Harbormaster - Toolkit for automating the creation & mgmt of Docker components and tools

My development environment is MacOS with an M1 chip and I mostly develop for lin

Gabe Susman 0 Feb 17, 2022
APKrash is an Android APK security analysis toolkit focused on comparing APKs to detect tampering and repackaging.

APKrash APKrash is an Android APK security analysis toolkit focused on comparing APKs to detect tampering and repackaging. Features Able to analyze pu

Henrique Goncalves 10 Sep 27, 2022
Substation is a cloud native toolkit for building modular ingest, transform, and load (ITL) data pipelines

Substation Substation is a cloud native data pipeline toolkit. What is Substation? Substation is a modular ingest, transform, load (ITL) application f

Brex 20 Sep 14, 2022
Walker's alias method is an efficient algorithm to sample from a discrete probability distribution.

walker-alias Walker's alias method is an efficient algorithm to sample from a discrete probability distribution. This means given an arbitrary probabi

null 3 Jun 14, 2022
A penetration toolkit for container environment

ctrsploit: A penetration toolkit for container environment 中文文档 Pre-Built Release https://github.com/ctrsploit/ctrsploit/releases Usage Quick-Start wg

null 42 Aug 29, 2022
CDK - Zero Dependency Container Penetration Toolkit

CDK is an open-sourced container penetration toolkit, offering stable exploitation in different slimmed containers without any OS dependency. It comes with penetration tools and many powerful PoCs/EXPs helps you to escape container and takeover K8s cluster easily.

null 2.6k Sep 27, 2022
IndieAuth Toolkit for Go.

IndieAuth Toolkit for Go This repository contains a set of tools to help you implement IndieAuth, both server and client, in Go. The documentation can

Henrique Dias 18 Sep 4, 2022
JOY5 AV Toolkit.

JOY5 AV Toolkit.

null 333 Sep 13, 2022