🌍 Earthly is a build automation tool for the container era

Overview

Earthly

GitHub Actions CI Join the chat on Slack Docs Website Install Earthly Docker Hub License BSL

🐳 Build anything via containers - build images or standalone artifacts (binaries, packages, arbitrary files)

πŸ›  Programming language agnostic - allows use of language-specific build tooling

πŸ” Repeatable builds - does not depend on user's local installation: runs the same locally, as in CI

β›“ Parallelism that just works - build in parallel without special considerations

🏘 Mono and Poly-repo friendly - ability to split the build definitions across vast project hierarchies

πŸ’Ύ Shared caching - share build cache between CI runners

πŸ”€ Multi-platform - build for multiple platforms in parallel


🌍 Earthly is a build automation tool for the container era. It allows you to execute all your builds in containers. This makes them self-contained, repeatable, portable and parallel. You can use Earthly to create Docker images and artifacts (eg binaries, packages, arbitrary files).


Get Earthly




Table of Contents



Why Use Earthly?

πŸ” Reproduce CI failures

Earthly builds are self-contained, isolated and repeatable. Regardless of whether Earthly runs in your CI or on your laptop, there is a degree of guarantee that the build will run the same way. This allows for faster iteration on the build scripts and easier debugging when something goes wrong. No more git commit -m "try again".

🀲 Builds that run the same for everyone

Repeatable builds also mean that your build will run the same on your colleagues' laptop without any additional project-specific or language-specific setup. This fosters better developer collaboration and mitigates works-for-me type of issues.

πŸš€ From zero to working build in minutes

Jump from project to project with ease, regardless of the language they are written in. Running the project's test suites is simply a matter of running an Earthly target (without fiddling with project configuration to make it compile and run on your system). Contribute across teams with confidence.

πŸ“¦ Reusability

A simple, yet powerful import system allows for reusability of builds across directories or even across repositories. Importing other builds does not have hidden environment-specific implications - it just works.

❀️ It's like Makefile and Dockerfile had a baby

Taking some of the best ideas from Makefiles and Dockerfiles, Earthly combines two build specifications into one.



Where Does Earthly Fit?

Earthly fits between language-specific tooling and the CI

Earthly is meant to be used both on your development machine and in CI. It can run on top of popular CI systems (like Jenkins, Circle, GitHub Actions). It is typically the layer between language-specific tooling (like maven, gradle, npm, pip, go build) and the CI build spec.



How Does It Work?

In short: containers, layer caching and complex build graphs!

Earthly executes builds in containers, where execution is isolated. The dependencies of the build are explicitly specified in the build definition, thus making the build self-sufficient.

We use a target-based system to help users break-up complex builds into reusable parts. Nothing is shared between targets, other than clearly declared dependencies. Nothing shared means no unexpected race conditions. In fact, the build is executed in parallel whenever possible, without any need for the user to take care of any locking or unexpected environment interactions.

ℹ️ Note

Earthfiles might seem very similar to Dockerfile multi-stage builds. In fact, the same technology is used underneath. However, a key difference is that Earthly is designed to be a general purpose build system, not just a Docker image specification. Read more about how Earthly is different from Dockerfiles.


Installation

See installation instructions.

To build from source, check the contributing page.



Quick Start

Here are some resources to get you started with Earthly

See also the full documentation.

Reference pages

A simple example (for Go)

# Earthfile
FROM golang:1.15-alpine3.13
RUN apk --update --no-cache add git
WORKDIR /go-example

all:
  BUILD +lint
  BUILD +docker

build:
  COPY main.go .
  RUN go build -o build/go-example main.go
  SAVE ARTIFACT build/go-example AS LOCAL build/go-example

lint:
  RUN go get golang.org/x/lint/golint
  COPY main.go .
  RUN golint -set_exit_status ./...

docker:
  COPY +build/go-example .
  ENTRYPOINT ["/go-example/go-example"]
  SAVE IMAGE go-example:latest
// main.go
package main

import "fmt"

func main() {
  fmt.Println("hello world")
}

Invoke the build using earthly +all.

Demonstration of a simple Earthly build

Examples for other languages are available on the examples page.



Features

πŸ“¦ Modern import system

Earthly can be used to reference and build targets from other directories or even other repositories. For example, if we wanted to build an example target from the github.com/earthly/earthly repository, we could issue

# Try it yourself! No need to clone.
earthly github.com/earthly/earthly/examples/go:main+docker
# Run the resulting image.
docker run --rm go-example:latest

πŸ”¨ Reference other targets using +

Use + to reference other targets and create complex build inter-dependencies.

Target and artifact reference syntax

Examples

  • Same directory (same Earthfile)

    BUILD +some-target
    FROM +some-target
    COPY +some-target/my-artifact ./
  • Other directories

    BUILD ./some/local/path+some-target
    FROM ./some/local/path+some-target
    COPY ./some/local/path+some-target/my-artifact ./
  • Other repositories

    BUILD github.com/someone/someproject:v1.2.3+some-target
    FROM github.com/someone/someproject:v1.2.3+some-target
    COPY github.com/someone/someproject:v1.2.3+some-target/my-artifact ./

πŸ’Ύ Caching that works the same as Docker builds

Demonstration of Earthly's caching

Cut down build times in CI through Shared Caching.

πŸ›  Multi-platform support

Build for multiple platforms in parallel.

all:
    BUILD \
        --platform=linux/amd64 \
        --platform=linux/arm64 \
        --platform=linux/arm/v7 \
        --platform=linux/arm/v6 \
        +build

build:
    FROM alpine:3.13
    CMD ["uname", "-m"]
    SAVE IMAGE multiplatform-image

β›“ Parallelization that just works

Whenever possible, Earthly automatically executes targets in parallel.

Demonstration of Earthly's parallelization

🀲 Make use of build tools that work everywhere

No need to ask your team to install protoc, a specific version of Python, Java 1.6 or the .NET Core ecosystem. You only install once, in your Earthfile, and it works for everyone. Or even better, you can just make use of the rich Docker Hub ecosystem.

FROM golang:1.15-alpine3.13
WORKDIR /proto-example

proto:
  FROM namely/protoc-all:1.29_4
  COPY api.proto /defs
  RUN --entrypoint -- -f api.proto -l go
  SAVE ARTIFACT ./gen/pb-go /pb AS LOCAL pb

build:
  COPY go.mod go.sum .
  RUN go mod download
  COPY +proto/pb pb
  COPY main.go ./
  RUN go build -o build/proto-example main.go
  SAVE ARTIFACT build/proto-example

See full example code.

πŸ”‘ Cloud secrets support built-in

Secrets are never stored within an image's layers and they are only available to the commands that need them.

earthly set /user/github/token 'shhh...'
release:
  RUN --push --secret GITHUB_TOKEN=+secrets/user/github/token github-release upload file.bin


FAQ

How is Earthly different from Dockerfiles?

Dockerfiles were designed for specifying the make-up of Docker images and that's where Dockerfiles stop. Earthly takes some key principles of Dockerfiles (like layer caching), but expands on the use-cases. For example, Earthly can output regular artifacts, run unit and integration tests and also create several Docker images at a time - all of which are outside the scope of Dockerfiles.

It is possible to use Dockerfiles in combination with other technologies (eg Makefiles or bash files) in order to solve for such use-cases. However, these combinations are difficult to parallelize, difficult to scale across repositories as they lack a robust import system and also they often vary in style from one team to another. Earthly does not have these limitations as it was designed as a general purpose build system.

As an example, Earthly introduces a richer target, artifact and image referencing system, which allows for better reuse in complex builds spanning a single large repository or multiple repositories. Because Dockerfiles are only meant to describe one image at a time, such features are outside the scope of applicability of Dockerfiles.

How do I tell apart classical Dockerfile commands from Earthly commands?

Check out the Earthfile reference doc page. It has all the commands there and it specifies which commands are the same as Dockerfile commands and which are new.

Can Earthly build Dockerfiles?

Yes! You can use the command FROM DOCKERFILE to inherit the commands in an existing Dockerfile.

build:
  FROM DOCKERFILE .
  SAVE IMAGE some-image:latest

You may also optionally port your Dockerfiles to Earthly entirely. Translating Dockerfiles to Earthfiles is usually a matter of copy-pasting and making small adjustments. See the getting started page for some Earthfile examples.

How is Earthly different from Bazel?

Bazel is a build tool developed by Google for the purpose of optimizing speed, correctness and reproducibility of their internal monorepo codebase. Earthly draws inspiration from some of the principles of Bazel (mainly the idea of repeatable builds), but it is different in a few key ways:

  • Earthly does not replace language-specific tools, like Maven, Gradle, Webpack etc. Instead, it leverages and integrates with them. Adopting Bazel usually means that all build files need to be completely rewritten. This is not the case with Earthly as it mainly acts as the glue between builds.
  • The learning curve of Earthly is more accessible, especially if the user already has experience with Dockerfiles. Bazel, on the other hand, introduces some completely new concepts.
  • Bazel has a purely descriptive specification language. Earthly is a mix of descriptive and imperative language.
  • Bazel uses tight control of compiler toolchain to achieve true hermetic builds, whereas Earthly uses containers and well-defined inputs.

Overall, compared to Bazel, Earthly sacrifices some correctness and reproducibility in favor of significantly better usability and composability with existing open-source technologies.



Contributing

  • Please report bugs as GitHub issues.
  • Join us on Slack!
  • Questions via GitHub issues are welcome!
  • PRs welcome! But please give a heads-up in GitHub issue before starting work. If there is no GitHub issue for what you want to do, please create one.
  • To build from source, check the contributing page.


Licensing

Earthly is licensed under the Business Source License 1.1. See licenses/BSL for more information.

Comments
  • Some DIND scenarios fail when using cgroups v2

    Some DIND scenarios fail when using cgroups v2

    Notable, podman and kind have been documented to fail on systems using cgroups v2. Here is all the relevant information:

    • Docker updated to use cgroups v2 in Docker Desktop at version 4.3.0. I do not know what docker CLI version this correlates to
    • Our unit tests that use podman appear to be trying to use cgroups v1. We may need to update podman for these unit tests since docker is on v2 (and podman is inside docker here, for our unit tests), or add compatibility flags
    • To turn off/on cgroups v2, I use the following kernel param: systemd.unified_cgroup_hierarchy=0 (Pop!_OS 21.10)
    • v2 is a unified, single-root hierarchy; vs the v1 multi-root approach
    • You can examine currently running cgroups via systemd-cgtop
    • kind also does not run when using v2, inside a WITH DOCKER. I haven't dug too much into regular docker, but I assume its only partially functional in this case
    • You can check what cgroups versions you have by grep cgroup /proc/filesystems
    type:bug 
    opened by dchw 24
  • SSL/TLS trust issues: unable to add custom certificates or disable verification

    SSL/TLS trust issues: unable to add custom certificates or disable verification

    Hi, I'm working in a corporate environment where I need to specify the certificates which should be used for verification of remote connections. Unfortunately, right now I can't pull from internal registries since I get the following error:

    ...snip...
         r/r/ubi8:latest | --> Load metadata linux/amd64
         r/r/ubi8:latest | WARN: (Load metadata linux/amd64) failed to do request: Head https://REGISTRY/PATH: x509: certificate signed by unknown authority
    Error: failed to do request: Head https://REGISTRY/PATH: x509: certificate signed by unknown authority
    

    Since the registry is contacted every time regardless of local image presence (outlined further here in #345), it appears that I have a blocker to adopting Earthly.

    How can I provide more information to troubleshooting this? I would be happy to try some things out, as well. I really appreciate all of your time and efforts.

    Thanks, +Jonathan

    type:enhancement 
    opened by yonkeltron 20
  • SAVE IMAGE is slow, even when there's no work to be done

    SAVE IMAGE is slow, even when there's no work to be done

    I was observing that for highly optimized builds the slowest part can be saving images. For example, here’s a repo where earth +all takes 12s if everything is cached (like I run earth +all twice). Yet if I comment the SAVE IMAGE lines out the total time drops to 2s. This implies that SAVE IMAGE is doing a lot of work, even when nothing has changed.

    Is there anything I can do speed up SAVE IMAGE in instances like this? I’m surprised that SAVE IMAGE does anything if the image hasn’t changed, is it possible for it do some more sophisticated content negotiation with the layers?

    After talking with @agbell in Slack I hypothesized that it might not be possible for Earthly to what images/layers the host has. This is all conjecture on my part, but:

    If Earth is running in a container then it doesn’t know the state of the registry on the host machine, and what layers it has. It’s only option is to export the entire image to the host, which on a Mac could be slow because containers on a mac are actually running in a VM.

    Maybe if Earth could mount/be aware of the host docker registry it could just do docker push? This reminds me of similar problems that are being solved in the Kubernetes local cluster space https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry

    type:performance 
    opened by jazzdan 20
  • Cache export hangs on target with just `BUILD` commands, and no `FROM` in the base target

    Cache export hangs on target with just `BUILD` commands, and no `FROM` in the base target

    I am using earthly with GitHub Actions and the following earthly command:

    earthly --ci --remote-cache=ghcr.io/notin-dev/cache:cache --push +build-all
    

    The entire build is taking about 8 minutes to complete, but the cache step is taking over 20 minutes to complete and I get the following output

    cache | --> exporting cache
    [653](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:653)
                   cache | [          ]   0% preparing build cache for export
    [654](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:654)
                   cache | [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] 100% preparing build cache for export
    [655](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:655)
                 ongoing | cache (1 minute ago)
    [656](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:656)
                 ongoing | cache (2 minutes ago)
    [657](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:657)
                 ongoing | cache (3 minutes ago)
    [658](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:658)
                 ongoing | cache (4 minutes ago)
    [659](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:659)
                 ongoing | cache (5 minutes ago)
    [660](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:660)
                 ongoing | cache (6 minutes ago)
    [661](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:661)
                 ongoing | cache (7 minutes ago)
    [662](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:662)
                 ongoing | cache (8 minutes ago)
    [663](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:663)
                 ongoing | cache (9 minutes ago)
    [664](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:664)
                 ongoing | cache (10 minutes ago)
    [665](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:665)
                 ongoing | cache (11 minutes ago)
    [666](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:666)
                 ongoing | cache (12 minutes ago)
    [667](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:667)
                 ongoing | cache (13 minutes ago)
    [668](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:668)
                 ongoing | cache (14 minutes ago)
    [669](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:669)
                 ongoing | cache (15 minutes ago)
    [670](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:670)
                 ongoing | cache (16 minutes ago)
    [671](https://github.com/notin-dev/lomp/runs/6957678719?check_suite_focus=true#step:5:671)
                 ongoing | cache (17 minutes ago)
    

    I initially though that the issue was with heavy usage of SAVE IMAGE --cache-hint and so I removed every occurrence of that in the repository but even then the cache step is taking forever to complete.

    I'm not sure if I'm doing something wrong here.

    This is my GitHub Actions file:

    name: Build Project
    on:
      pull_request:
        branches: [main]
      push:
        branches:
          - main
    concurrency:
      group: ${{ github.head_ref || github.ref }}
      cancel-in-progress: true
    jobs:
      build_project:
        runs-on: ubuntu-latest
        permissions:
          contents: read
          packages: write
        steps:
          - name: Checkout repository
            uses: actions/[email protected]
          - name: Logging into Container Registry
            uses: docker/[email protected]
            with:
              registry: ghcr.io
              username: ${{ github.actor }}
              password: ${{ secrets.GITHUB_TOKEN }}
          - name: Installing earthly
            run: sudo /bin/sh -c 'wget https://github.com/earthly/earthly/releases/download/v0.6.14/earthly-linux-amd64 -O /usr/local/bin/earthly && chmod +x /usr/local/bin/earthly'
          - name: Building project
            run: earthly --ci --remote-cache=ghcr.io/notin-dev/cache:cache --push +build-all
    
    type:bug 
    opened by chandraaditya 17
  • Earthly image --push to Artifactory fails with 401

    Earthly image --push to Artifactory fails with 401

    I'm using Earthly with the --push flag to push images to Artifactory and upon doing so Earthly returns the error:

                  output | [          ] pushing layers ... 0%
                 ongoing | output (15 seconds ago)
                  output | [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] pushing layers ... 100%
                  output | WARN: (exporting outputs) failed commit on ref "layer-sha256:b8c8466005032e78723c242ae920cf0eb59f27af010c8646dea7a16f4dda57db": unexpected status: 401 Unauthorized
    ================================ SUCCESS [main] ================================
    Error: failed commit on ref "layer-sha256:b8c8466005032e78723c242ae920cf0eb59f27af010c8646dea7a16f4dda57db": unexpected status: 401 Unauthorized
    

    and the earthly-buildkitd logs show:

    time="2021-05-28T10:12:32Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed commit on ref \"layer-sha256:b8c8466005032e78723c242ae920cf0eb59f27af010c8646dea7a16f4dda57db\": unexpected status: 401 Unauthorized\n"
    err EOF
    

    I issue a docker login before attempting the Earthly build.

    Using docker to push the image works as does docker buildx build --push so Artifactory is accepting image uploads from other sources, I just can't get it to work directly from Earthly.

    I have tried this with some other registries and they all succeed:

    • Nexus
    • Docker v2 registry (running locally)
    • JFrog Container Registry

    Would you have any suggestions for what could be wrong?

    This is a corporate Artifactory server so I don't currently have logs from it but I am trying to get them to see if they reveal anything.

    Thanks!

    type:bug 
    opened by jamesalucas 17
  • bug: docker authentication expires after a few hours causing 401 errors while building

    bug: docker authentication expires after a few hours causing 401 errors while building

    I have an Earthfile that extends a private image. After a few hours of using earthly, I start getting 401 errors. I'm still able to pull the image with docker pull, and running docker login doesn't have any effect.

    I am able to get it working again by restarting the earthly container manually, or by changing the arguments I run with.

    $ earthly -i +test
         buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
         quay.io/<org>/<image>:<tag> | --> Load metadata linux/amd64
         quay.io/<org>/<image>:<tag> | WARN: (Load metadata linux/amd64) unexpected status code [manifests ci]: 401 UNAUTHORIZED
    Error: unexpected status code [manifests ci]: 401 UNAUTHORIZED
    
    $ docker login quay.io                                              
    Authenticating with existing credentials...
    Login Succeeded
    
    $ docker pull quay.io/<org>/<image>:<tag>
    <tag>: Pulling from <org>/<image>
    Digest: sha256:fcacc41495f6a7655032299d0ea83de0a9e577b64eaaaab8e4bf53a8f81bddb3
    Status: Image is up to date for quay.io/<org>/<image>:<tag>
    quay.io/<org>/<image>:<tag>
    
    $ earthly -i +test
         buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
         quay.io/<org>/<image>:<tag> | --> Load metadata linux/amd64
         quay.io/<org>/<image>:<tag> | WARN: (Load metadata linux/amd64) unexpected status code [manifests ci]: 401 UNAUTHORIZED
    Error: unexpected status code [manifests ci]: 401 UNAUTHORIZED
    
    $ earthly +test
                       buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
                       buildkitd | Settings do not match. Restarting buildkit daemon with updated settings...
                       buildkitd | ...Done
     quay.io/<org>/<image>:<tag> | --> Load metadata linux/amd64
                           +base | --> FROM quay.io/<org>/<image>:<tag>
                         context | --> local context .
                           +base | [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] resolve quay.io/<org>/<image>:<tag>@sha256:fcacc41495f6a7655032299d0ea83de0a9e577b64eaaaab8e4bf53a8f81bddb3 ... 100%
                         context | [β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ] transferring .: ... 100%
    

    This documentation suggests that Earthly should be able to inherited from the host. This page doesn't contain anything special for quay.

    Is it possible I'm missing something? Or is this unintended behavior?

    opened by tchupp 17
  • panic: failed to get edge

    panic: failed to get edge

    I've recently upgraded to Earthly 0.5.13 and our CI jobs that run multiple Earthly targets in parallel have started failing with errors such as this:

    # ...other yarn install logs...
    ./p/sometarget+dependencies | [5/5] Building fresh packages...
    ./p/sometarget+dependencies | success Saved lockfile.
    ./p/sometarget+dependencies | Done in 102.14s.
    ./p/sometarget+dependencies | yarn cache v1.22.0
    ./p/sometarget+dependencies | success Cleared cache.
    ./p/sometarget+dependencies | Done in 6.45s.
    =================================Buildkit Logs==================================
    Error: transport is closing
    It seems that buildkitd is shutting down or it has crashed. You can report crashes at https://github.com/earthly/earthly/issues/new.
    starting earthly-buildkit with EARTHLY_GIT_HASH=a7f082f8fade1badbee0a221c9a9da2317449e4c BUILDKIT_BASE_IMAGE=github.com/earthly/buildkit:c1749dff2545b0202fc15f33eaa3278b1aa8803e+build
    BUILDKIT_ROOT_DIR=/tmp/earthly/buildkit
    CACHE_SIZE_MB=150000
    EARTHLY_ADDITIONAL_BUILDKIT_CONFIG=
    CNI_MTU=1500
    ======== CNI config ==========
    {
    	"cniVersion": "0.3.0",
    	"name": "buildkitbuild",
    	"type": "bridge",
    	"bridge": "cni0",
    	"isGateway": true,
    	"ipMasq": true,
    	"mtu": 1500,
    	"ipam": {
    		"type": "host-local",
    		"subnet": "172.30.0.0/16",
    		"routes": [
    			{ "dst": "0.0.0.0/0" }
    		]
    	}
    }
    ======== End CNI config ==========
    ======== Buildkitd config ==========
    debug = false
    root = "/tmp/earthly/buildkit"
    insecure-entitlements = [ "security.insecure" ]
    [worker.oci]
      enabled = true
      snapshotter = "auto"
      gc = true
      networkMode = "cni"
      cniBinaryPath = "/usr/libexec/cni"
      cniConfigPath = "/etc/cni/cni-conf.json"
        # Please note the required indentation to fit in buildkit.toml.template accordingly.
      # 1/100 of total cache size.
      gckeepstorage = 1500000000
      [[worker.oci.gcpolicy]]
        # 1/10 of total cache size.
        keepBytes = 15000000000
        filters = [ "type==source.local", "type==source.git.checkout"]
      [[worker.oci.gcpolicy]]
        all = true
        # Cache size MB with 6 zeros, to turn it into bytes.
        keepBytes = 150000000000
    ======== End buildkitd config ==========
    Detected container architecture is x86_64
    starting shellrepeater
    time="2021-05-27T14:06:14Z" level=info msg="auto snapshotter: using overlayfs"
    time="2021-05-27T14:06:29Z" level=info msg="found worker \"7dl0od0h2nmn3awn6fgkr04er\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:9a81ae26219d org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/386]"
    time="2021-05-27T14:06:29Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
    time="2021-05-27T14:06:29Z" level=info msg="found 1 workers, default=\"7dl0od0h2nmn3awn6fgkr04er\""
    time="2021-05-27T14:06:29Z" level=warning msg="currently, only the default worker can be used."
    time="2021-05-27T14:06:29Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
    panic: failed to get edge
    goroutine 16 [running]:
    github.com/moby/buildkit/solver.(*pipeFactory).NewInputRequest(0xc00c90def8, 0x0, 0x1549b20, 0xc00c6d8700, 0xc01bed6400, 0xc001961928, 0xc00453d200)
    	/src/solver/scheduler.go:354 +0x1fc
    github.com/moby/buildkit/solver.(*edge).createInputRequests(0xc001960dc0, 0x2, 0xc00c90def8, 0x0, 0x1dff070)
    	/src/solver/edge.go:809 +0x31d
    github.com/moby/buildkit/solver.(*edge).unpark(0xc001960dc0, 0xc01bed8420, 0x1, 0x1, 0xc00c90de48, 0x0, 0x0, 0x1dff070, 0x0, 0x0, ...)
    	/src/solver/edge.go:360 +0x179
    github.com/moby/buildkit/solver.(*scheduler).dispatch(0xc00023d180, 0xc001960dc0)
    	/src/solver/scheduler.go:136 +0x430
    github.com/moby/buildkit/solver.(*scheduler).loop(0xc00023d180)
    	/src/solver/scheduler.go:104 +0x179
    created by github.com/moby/buildkit/solver.newScheduler
    	/src/solver/scheduler.go:35 +0x1ab
    

    Similar logs in the job that is in progress at the same time...

    ./p/someothertarget+dependencies | [5/5] Building fresh packages...
    ./p/someothertarget+dependencies | success Saved lockfile.
    ./p/someothertarget+dependencies | Done in 85.08s.
    ./p/someothertarget+dependencies | yarn cache v1.22.0
    ./p/someothertarget+dependencies | success Cleared cache.
    ./p/someothertarget+dependencies | Done in 7.32s.
    ./p/someothertarget+dependencies | package=someothertarget
    ./p/someothertarget+dependencies | *cached* --> SAVE ARTIFACT ./ ./packages/someothertarget+dependencies/app
    =================================Buildkit Logs==================================
    Error: transport is closing
    It seems that buildkitd is shutting down or it has crashed. You can report crashes at https://github.com/earthly/earthly/issues/new.
    starting earthly-buildkit with EARTHLY_GIT_HASH=a7f082f8fade1badbee0a221c9a9da2317449e4c BUILDKIT_BASE_IMAGE=github.com/earthly/buildkit:c1749dff2545b0202fc15f33eaa3278b1aa8803e+build
    BUILDKIT_ROOT_DIR=/tmp/earthly/buildkit
    CACHE_SIZE_MB=150000
    EARTHLY_ADDITIONAL_BUILDKIT_CONFIG=
    CNI_MTU=1500
    ======== CNI config ==========
    {
    	"cniVersion": "0.3.0",
    	"name": "buildkitbuild",
    	"type": "bridge",
    	"bridge": "cni0",
    	"isGateway": true,
    	"ipMasq": true,
    	"mtu": 1500,
    	"ipam": {
    		"type": "host-local",
    		"subnet": "172.30.0.0/16",
    		"routes": [
    			{ "dst": "0.0.0.0/0" }
    		]
    	}
    }
    ======== End CNI config ==========
    ======== Buildkitd config ==========
    debug = false
    root = "/tmp/earthly/buildkit"
    insecure-entitlements = [ "security.insecure" ]
    [worker.oci]
      enabled = true
      snapshotter = "auto"
      gc = true
      networkMode = "cni"
      cniBinaryPath = "/usr/libexec/cni"
      cniConfigPath = "/etc/cni/cni-conf.json"
        # Please note the required indentation to fit in buildkit.toml.template accordingly.
      # 1/100 of total cache size.
      gckeepstorage = 1500000000
      [[worker.oci.gcpolicy]]
        # 1/10 of total cache size.
        keepBytes = 15000000000
        filters = [ "type==source.local", "type==source.git.checkout"]
      [[worker.oci.gcpolicy]]
        all = true
        # Cache size MB with 6 zeros, to turn it into bytes.
        keepBytes = 150000000000
    ======== End buildkitd config ==========
    Detected container architecture is x86_64
    starting shellrepeater
    time="2021-05-27T14:06:14Z" level=info msg="auto snapshotter: using overlayfs"
    time="2021-05-27T14:06:29Z" level=info msg="found worker \"7dl0od0h2nmn3awn6fgkr04er\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:9a81ae26219d org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/amd64 linux/386]"
    time="2021-05-27T14:06:29Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
    time="2021-05-27T14:06:29Z" level=info msg="found 1 workers, default=\"7dl0od0h2nmn3awn6fgkr04er\""
    time="2021-05-27T14:06:29Z" level=warning msg="currently, only the default worker can be used."
    time="2021-05-27T14:06:29Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
    panic: failed to get edge
    goroutine 16 [running]:
    github.com/moby/buildkit/solver.(*pipeFactory).NewInputRequest(0xc00c90def8, 0x0, 0x1549b20, 0xc00c6d8700, 0xc01bed6400, 0xc001961928, 0xc00453d200)
    	/src/solver/scheduler.go:354 +0x1fc
    github.com/moby/buildkit/solver.(*edge).createInputRequests(0xc001960dc0, 0x2, 0xc00c90def8, 0x0, 0x1dff070)
    	/src/solver/edge.go:809 +0x31d
    github.com/moby/buildkit/solver.(*edge).unpark(0xc001960dc0, 0xc01bed8420, 0x1, 0x1, 0xc00c90de48, 0x0, 0x0, 0x1dff070, 0x0, 0x0, ...)
    	/src/solver/edge.go:360 +0x179
    github.com/moby/buildkit/solver.(*scheduler).dispatch(0xc00023d180, 0xc001960dc0)
    	/src/solver/scheduler.go:136 +0x430
    github.com/moby/buildkit/solver.(*scheduler).loop(0xc00023d180)
    	/src/solver/scheduler.go:104 +0x179
    created by github.com/moby/buildkit/solver.newScheduler
    	/src/solver/scheduler.go:35 +0x1ab
    

    Should running jobs in parallel on the same machine be OK? They are both run in Gitlab via Docker executors but on the same Docker host so the buildkitd instance is shared.

    type:bug 
    opened by jamesalucas 15
  • Ruby on rails example

    Ruby on rails example

    We need an example rails application added to the examples folder: https://github.com/earthly/earthly/tree/master/examples

    An example Earthfile should also be included which can build, test, and produce a docker file of the example.

    The Go example can serve as a reference point for how to structure an example.

    good first issue hacktoberfest type:example 
    opened by adamgordonbell 15
  • React example

    React example

    We need an example React JS app added to the examples folder: https://github.com/earthly/earthly/tree/master/examples

    An example Earthfile should also be included which can build and run tests.

    The Go example can serve as a reference point for how to structure an example.

    good first issue hacktoberfest type:example 
    opened by adamgordonbell 15
  • expand autocomplete to work with user home directories

    expand autocomplete to work with user home directories

    Currently if you type earth ~<tab><tab> this auto completes to earth ~/, instead it should auto complete to list all available users (e.g. ~/, ~adam, ~alex, ~corey, ~vlad).

    Here's the relevant section of the code that needs to be expanded to list other users:

    https://github.com/earthly/earthly/blob/master/autocomplete/complete.go#L106-L109

    This will also have to be able to do autocompletion so when you type earth ~a<tab><tab>, it'll only display users starting with an a (e.g. ~adam, ~alex)

    good first issue hacktoberfest 
    opened by alexcb 15
  • Support multi-platform builds

    Support multi-platform builds

    Is it possible to support multi-platform (e.g. amd64, arm64, arm v7) image build with docker buildx feature? With docker command, by using compatible builder, we can now provide --platform flag to specify target platform.

    docker [buildx] build --platform linux/amd64,linux/arm64,linux/arm/v7 .

    https://docs.docker.com/buildx/working-with-buildx/

    type:proposal 
    opened by LuaxY 14
  • Don't use stack frames for org/project

    Don't use stack frames for org/project

    Setting org/project on the current stack frame was leading to UDC secrets not having access to the org/project. This change sets the org/project values directly on the var collection.

    opened by mikejholly 0
  • Bump github.com/labstack/echo/v4 from 4.6.3 to 4.9.0 in /examples/go-monorepo/services/one

    Bump github.com/labstack/echo/v4 from 4.6.3 to 4.9.0 in /examples/go-monorepo/services/one

    Bumps github.com/labstack/echo/v4 from 4.6.3 to 4.9.0.

    Release notes

    Sourced from github.com/labstack/echo/v4's releases.

    v4.9.0

    Security

    • Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

    Enhancements

    • Allow configuring ErrorHandler in CSRF middleware #2257
    • Replace HTTP method constants in tests with stdlib constants #2247

    v4.8.0

    Most notable things

    You can now add any arbitrary HTTP method type as a route #2237

    e.Add("COPY", "/*", func(c echo.Context) error 
      return c.String(http.StatusOK, "OK COPY")
    })
    

    You can add custom 404 handler for specific paths #2217

    e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
    

    g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

    Enhancements

    • Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127
    • Refactor: body_limit middleware unit test #2145
    • Refactor: Timeout mw: rework how test waits for timeout. #2187
    • BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191
    • Refactor: duplicated findStaticChild process at findChildWithLabel #2176
    • Allow different param names in different methods with same path scheme #2209
    • Add support for registering handlers for different 404 routes #2217
    • Middlewares should use errors.As() instead of type assertion on HTTPError #2227
    • Allow arbitrary HTTP method types to be added as routes #2237

    v4.7.2

    Fixes

    • Fix nil pointer exception when calling Start again after address binding error #2131
    • Fix CSRF middleware not being able to extract token from multipart/form-data form #2136
    • Fix Timeout middleware write race #2126

    Enhancements

    ... (truncated)

    Changelog

    Sourced from github.com/labstack/echo/v4's changelog.

    v4.9.0 - 2022-09-04

    Security

    • Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

    Enhancements

    • Allow configuring ErrorHandler in CSRF middleware #2257
    • Replace HTTP method constants in tests with stdlib constants #2247

    v4.8.0 - 2022-08-10

    Most notable things

    You can now add any arbitrary HTTP method type as a route #2237

    e.Add("COPY", "/*", func(c echo.Context) error 
      return c.String(http.StatusOK, "OK COPY")
    })
    

    You can add custom 404 handler for specific paths #2217

    e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
    

    g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

    Enhancements

    • Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127
    • Refactor: body_limit middleware unit test #2145
    • Refactor: Timeout mw: rework how test waits for timeout. #2187
    • BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191
    • Refactor: duplicated findStaticChild process at findChildWithLabel #2176
    • Allow different param names in different methods with same path scheme #2209
    • Add support for registering handlers for different 404 routes #2217
    • Middlewares should use errors.As() instead of type assertion on HTTPError #2227
    • Allow arbitrary HTTP method types to be added as routes #2237

    v4.7.2 - 2022-03-16

    Fixes

    • Fix nil pointer exception when calling Start again after address binding error #2131
    • Fix CSRF middleware not being able to extract token from multipart/form-data form #2136
    • Fix Timeout middleware write race #2126

    ... (truncated)

    Commits
    • 16d3b65 Changelog for 4.9.0
    • 0ac4d74 Fix #2259 open redirect vulnerability in echo.StaticDirectoryHandler (used by...
    • d77e8c0 Added ErrorHandler and ErrorHandlerWithContext in CSRF middleware (#2257)
    • 534bbb8 replace POST constance with stdlib constance
    • fb57d96 replace GET constance with stdlib constance
    • d48197d Changelog for 4.8.0
    • cba12a5 Allow arbitrary HTTP method types to be added as routes
    • a327884 add:README.md-Third-party middlewares-github.com/go-woo/protoc-gen-echo
    • 61422dd Update CI-flow (Go 1.19 +deps)
    • a9879ff Middlewares should use errors.As() instead of type assertion on HTTPError
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies go 
    opened by dependabot[bot] 1
  • Bump github.com/labstack/echo/v4 from 4.6.3 to 4.9.0 in /examples/go-monorepo/services/two

    Bump github.com/labstack/echo/v4 from 4.6.3 to 4.9.0 in /examples/go-monorepo/services/two

    Bumps github.com/labstack/echo/v4 from 4.6.3 to 4.9.0.

    Release notes

    Sourced from github.com/labstack/echo/v4's releases.

    v4.9.0

    Security

    • Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

    Enhancements

    • Allow configuring ErrorHandler in CSRF middleware #2257
    • Replace HTTP method constants in tests with stdlib constants #2247

    v4.8.0

    Most notable things

    You can now add any arbitrary HTTP method type as a route #2237

    e.Add("COPY", "/*", func(c echo.Context) error 
      return c.String(http.StatusOK, "OK COPY")
    })
    

    You can add custom 404 handler for specific paths #2217

    e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
    

    g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

    Enhancements

    • Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127
    • Refactor: body_limit middleware unit test #2145
    • Refactor: Timeout mw: rework how test waits for timeout. #2187
    • BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191
    • Refactor: duplicated findStaticChild process at findChildWithLabel #2176
    • Allow different param names in different methods with same path scheme #2209
    • Add support for registering handlers for different 404 routes #2217
    • Middlewares should use errors.As() instead of type assertion on HTTPError #2227
    • Allow arbitrary HTTP method types to be added as routes #2237

    v4.7.2

    Fixes

    • Fix nil pointer exception when calling Start again after address binding error #2131
    • Fix CSRF middleware not being able to extract token from multipart/form-data form #2136
    • Fix Timeout middleware write race #2126

    Enhancements

    ... (truncated)

    Changelog

    Sourced from github.com/labstack/echo/v4's changelog.

    v4.9.0 - 2022-09-04

    Security

    • Fix open redirect vulnerability in handlers serving static directories (e.Static, e.StaticFs, echo.StaticDirectoryHandler) #2260

    Enhancements

    • Allow configuring ErrorHandler in CSRF middleware #2257
    • Replace HTTP method constants in tests with stdlib constants #2247

    v4.8.0 - 2022-08-10

    Most notable things

    You can now add any arbitrary HTTP method type as a route #2237

    e.Add("COPY", "/*", func(c echo.Context) error 
      return c.String(http.StatusOK, "OK COPY")
    })
    

    You can add custom 404 handler for specific paths #2217

    e.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })
    

    g := e.Group("/images") g.RouteNotFound("/*", func(c echo.Context) error { return c.NoContent(http.StatusNotFound) })

    Enhancements

    • Add new value binding methods (UnixTimeMilli,TextUnmarshaler,JSONUnmarshaler) to Valuebinder #2127
    • Refactor: body_limit middleware unit test #2145
    • Refactor: Timeout mw: rework how test waits for timeout. #2187
    • BasicAuth middleware returns 500 InternalServerError on invalid base64 strings but should return 400 #2191
    • Refactor: duplicated findStaticChild process at findChildWithLabel #2176
    • Allow different param names in different methods with same path scheme #2209
    • Add support for registering handlers for different 404 routes #2217
    • Middlewares should use errors.As() instead of type assertion on HTTPError #2227
    • Allow arbitrary HTTP method types to be added as routes #2237

    v4.7.2 - 2022-03-16

    Fixes

    • Fix nil pointer exception when calling Start again after address binding error #2131
    • Fix CSRF middleware not being able to extract token from multipart/form-data form #2136
    • Fix Timeout middleware write race #2126

    ... (truncated)

    Commits
    • 16d3b65 Changelog for 4.9.0
    • 0ac4d74 Fix #2259 open redirect vulnerability in echo.StaticDirectoryHandler (used by...
    • d77e8c0 Added ErrorHandler and ErrorHandlerWithContext in CSRF middleware (#2257)
    • 534bbb8 replace POST constance with stdlib constance
    • fb57d96 replace GET constance with stdlib constance
    • d48197d Changelog for 4.8.0
    • cba12a5 Allow arbitrary HTTP method types to be added as routes
    • a327884 add:README.md-Third-party middlewares-github.com/go-woo/protoc-gen-echo
    • 61422dd Update CI-flow (Go 1.19 +deps)
    • a9879ff Middlewares should use errors.As() instead of type assertion on HTTPError
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies go 
    opened by dependabot[bot] 1
  • `earthly ls` produces a misleading error message for invalid Earthfile

    `earthly ls` produces a misleading error message for invalid Earthfile

    Running earthly ls against some invalid Earthfile produces a misleading error message that does not indicates parsing errors.

    $ earthly -v
    earthly version v0.6.24 20339818277375d1addc9cc20066a9890483761e linux/amd64; Ubuntu 22.04.1 LTS (Jammy Jellyfish)
    
    $ echo "VERSION 0.6\nx" > Earthfile
    
    $ earthly ls
    Error: unable to locate Earthfile under current directory
    
    opened by mitre-tm 0
Releases(v0.6.24)
  • v0.6.24(Sep 22, 2022)

    Added

    • The new earthly org invite command now has the ability to invite multiple email addresses at once.
    • Experimental support for TRY/FINALLY, which allows saving artifacts upon failure. #988, #587. Not that this is only a partial implementation, and only accepts a single RUN command in the TRY, and only SAVE ARTIFACT commands in the FINALLY block.
    • Ability to enable specific satellite features via cli flags, e.g. the new experimental sleep feature can be enabled with earthly satellite launch --feature-flags satellite-sleep my-satellite.

    Changed

    • Bootstraping zsh autocompletion will first attempt to install under /usr/local/share/zsh/site-functions, and will now fallback to /usr/share/zsh/site-functions.
    • The earthly preview org command has been promoted to GA, and is now available under earthly org.
    • earthly sat select with no arguments now prints the current satellite and the usage text.
    • The interactive debugger now connects over the buildkit session connection rather than an unencrypted tcp connection; this makes it possible to use the interactive debugger with remote buildkit instances.

    Fixed

    • Fixed Earthly failing when using a remote docker host from a machine with an incompatible architecture. #1895
    • Earthly will no longer race with itself when starting up buildkit. #2194
    • The error reported when failing to initiate a connection to buildkit has been reworded to account for the remote buildkit/satellite case too.
    • Errors related to parsing VERSION feature flags will no longer be displayed during auto-completion.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(34.14 MB)
    earthly-darwin-arm64(34.74 MB)
    earthly-linux-amd64(35.07 MB)
    earthly-linux-arm64(34.03 MB)
    earthly-windows-amd64.exe(35.35 MB)
  • v0.6.23(Sep 6, 2022)

    Fixed

    • Using --remote-cache on a target that contains only BUILD instructions caused a hang. #1945
    • Fixed WAIT/END related bug which prevent WITH DOCKER --load from building referenced target.
    • Images and artifacts which are output (or pushed), are now displayed in the final earthly output.
    • ssh: parse error in message type 27 error when using OpenSSH 8.9; fixed by upstream in golang/go#51689.

    Changed

    • Removed warning stating that WAIT/END code is experimental and may be incomplete -- it is still experimental; however, it now has a higher degree of test-coverage. It can be enabled with VERSION --wait-block 0.6.
    • A warning is now displayed during exporting a multi-platform image to the local host if no platform is found that matches the host's platform type.
    • Reduced verbosity of To enable pushing use earthly --push message.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(33.41 MB)
    earthly-darwin-arm64(34.01 MB)
    earthly-linux-amd64(34.35 MB)
    earthly-linux-arm64(33.34 MB)
    earthly-windows-amd64.exe(34.60 MB)
  • v0.6.22(Aug 19, 2022)

    Added

    • --cache-from earthly flag, which allows defining multiple ordered caches. #1693
    • WAIT/END support for saving artifacts to local host.
    • WAIT/END support for RUN --push commands.

    Fixed

    • Updated EXPOSE parsing to accept (and ignore) host IP prefix, as well as expose udp ports; this should be fully-compatible with dockerfile's format. #1986
    • The earthly-buildkit container is now only initialized when required.

    Changed

    • The earthly-buildkit container is now only initialized when required.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(33.35 MB)
    earthly-darwin-arm64(33.94 MB)
    earthly-linux-amd64(34.27 MB)
    earthly-linux-arm64(33.32 MB)
    earthly-windows-amd64.exe(34.53 MB)
  • v0.6.21(Aug 4, 2022)

    Added

    • EARTHLY_LOCALLY builtin arg which is set to true or false when executing locally or within a container, respectively. This ARG must be enabled with the VERSION --earthly-locally-arg feature flag.

    Fixed

    • Fixed an incompatibility with older versions of remote BuildKits and Satellites, which was resulting in Earthly crashing.
    • Fixed WITH DOCKER not loading correctly when the image name contained a port number under VERSION --use-registry-for-with-docker. #2071
    • Race condition in WAIT / END block, which prevented waiting on some BUILD commands.

    Changed

    • Added a deprecation warning for secrets using a +secrets/ prefix. Support for this prefix will be removed in a future release.
    • per-file stat transfers are now logged when running under --debug mode.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(33.27 MB)
    earthly-darwin-arm64(33.86 MB)
    earthly-linux-amd64(34.20 MB)
    earthly-linux-arm64(33.30 MB)
    earthly-windows-amd64.exe(34.46 MB)
  • v0.6.20(Jul 18, 2022)

    Changed

    • Updated buildkit to include changes up to 12cfc87450c8d4fc31c8c0a09981e4c3fb3e4d9f

    Added

    • Adding support for saving artifact from --interactive-keep. #1980
    • New EARTHLY_PUSH builtin arg, which is set to true when earthly is run with the --push flag, and the argument is referenced under the direct target, or a target which is indirectly referenced via a BUILD command; otherwise it will be set to false. The value mimics when a RUN --push command is executed. This feature must be enabled with VERSION --wait-block 0.6.

    Fixed

    • Fixed context.Canceled being reported as the error in some builds instead of the root cause. #1991
    • Improved cache use of WITH DOCKER command.
    • The earthly/earthly docker image is now also built for arm64 (in addition to amd64).
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(33.18 MB)
    earthly-darwin-arm64(33.74 MB)
    earthly-linux-amd64(34.11 MB)
    earthly-linux-arm64(33.15 MB)
    earthly-windows-amd64.exe(34.36 MB)
  • v0.6.19(Jun 29, 2022)

  • v0.6.18(Jun 27, 2022)

  • v0.6.16(Jun 17, 2022)

  • v0.6.15(Jun 2, 2022)

    Changed

    Added

    • Experimental support for Docker registry based image creation and transfer WITH DOCKER loads and pulls. Enable with the VERSION --use-registry-for-with-docker flag.
    • Git config options for non-standard port and path prefix; these options are incompatible with a custom git substition regex.
    • Experimental WAIT / END blocks, to allow for finer grain of control between pushing images and running commands.
    • Improved ARG error messages to include the ARG name associated with the error.

    Fixed

    • Panic when running earthly --version under some versions of windows
    • Removed duplicate git commit hash from earthly --version output string (when running dev versions of earthly)
    • Garbled auto-completion when using Earthfiles without a VERSION command (or with other warnings) #1837.
    • Masking of cgroups for podman support.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(32.13 MB)
    earthly-darwin-arm64(32.68 MB)
    earthly-linux-amd64(32.22 MB)
    earthly-linux-arm64(31.26 MB)
    earthly-linux-arm7(28.26 MB)
    earthly-windows-amd64.exe(32.91 MB)
  • v0.6.14(Apr 11, 2022)

    Added

    • Experimental support for SAVE IMAGE --no-manifest-list. This option disables creating a multi-platform manifest list for the image, even if the image is created with a non-default platform. This allows the user to create non-native images (e.g. amd64 image on an M1 laptop) that are still compatible with AWS lambda. To enable this feature, please use VERSION --use-no-manifest-list 0.6. #1802
    • Introduced Experimental support for --chmod flag in COPY. To enable this feature, please use VERSION --use-chmod 0.6. #1817
    • Experimental secret_provider config option allows users to provide a script which returns secrets. #1808
    • /etc/ssh/ssh_known_hosts are now passed to buildkit. #1769

    Fixed

    • Targets with the same CACHE commands incorrectly shared cached contents. #1805
    • Sometimes local outputs and pushes are skipped mistakenly when a target is referenced both via FROM and via BUILD #1823
    • GIT CLONE failure (makeCloneURL does not support gitMatcher substitution) when used with a self-hosted git repo that was configured under ~/.earthly/config.yml #1757
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.79 MB)
    earthly-darwin-arm64(32.33 MB)
    earthly-linux-amd64(31.88 MB)
    earthly-linux-arm64(30.99 MB)
    earthly-linux-arm7(27.99 MB)
    earthly-windows-amd64.exe(32.57 MB)
  • v0.6.13(Mar 30, 2022)

    Added

    • Earthly now warns when encountering Earthfiles with no VERSION specified. In the future, the VERSION command will be mandatory. #1775

    Changed

    • WITH DOCKER now merges changes into /etc/docker/daemon.json rather than overwriting the entire file; this change introduces jq as a dependency, which will be auto-installed if missing.

    Fixed

    • The COPY command, when used with LOCALLY was incorrectly ignoring the WORKDIR value. #1792
    • The --shell-out-anywhere feature introduced a bug which interfered with asynchronous builds. #1785
    • EARTHLY_GIT_SHORT_HASH was not set when building a remotely-referenced target. #1787
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.77 MB)
    earthly-darwin-arm64(32.30 MB)
    earthly-linux-amd64(31.86 MB)
    earthly-linux-arm64(30.91 MB)
    earthly-linux-arm7(27.92 MB)
    earthly-windows-amd64.exe(32.55 MB)
  • v0.6.12(Mar 23, 2022)

    Changed

    • A more obvious error is printed if WITH DOCKER starts non-natively. This is not supported and it wasn't obvious before.

    Added

    • The feature flag --exec-after-build has been enabled retroactively for VERSION 0.5. This speeds up largs builds by 15-20%.
    • The feature flag --parallel-load has been enabled for every VERSION. This speeds up by parallelizing targets built for loading via WITH DOCKER --load.
    • VERSION 0.0 is now permitted, however it is only meant for Earthly internal debugging purposes. VERSION 0.0 disables all feature flags.
    • A new experimental mode in which --platform operates. To enable these features in your builds, set VERSION --new-platform 0.6:
      • There is now a distinction between user platform and native platform. The user platform is the platform of the user running Earthly, while the native platform is the platform of the build worker (these can be different when using a remote buildkit)
      • New platform shorthands are provided: --platform=native, --platform=user.
      • New builtin args are available: NATIVEPLATFORM, NATIVEOS, NATIVEARCH, NATIVEVARIANT (these are the equivalent of the USER* and TARGET* platform args).
      • When no platform is provided, earthly will default to the native platform
      • Additionally, earthly now default to native platform for internal operations too (copy operations, git clones etc)
      • Earthly now allows changing the platform in the middle of a target (FROM --platform is not a contradiction anymore). There is a distinction between the "input" platform of a target (the platform the caller passes in) vs the "output" platform (the platform that ends up being the final platform of the image). These can be different if the caller passes BUILD --platform=something +target, but the target starts with FROM --platform=otherthing ....
    • Ability to shell-out in any Earthly command, (e.g. SAVE IMAGE myimage:$(cat version)), as well as in the middle of ARG strings. To enable this feature, use VERSION --shell-out-anywhere 0.6.

    Fixed

    • An experimental fix for duplicate output when building images that are loaded via WITH DOCKER --load. This can be enabled via VERSION --no-tar-build-output 0.6.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.77 MB)
    earthly-darwin-arm64(32.30 MB)
    earthly-linux-amd64(31.86 MB)
    earthly-linux-arm64(30.84 MB)
    earthly-linux-arm7(27.92 MB)
    earthly-windows-amd64.exe(32.54 MB)
  • v0.6.11(Mar 17, 2022)

    Added

    • An experimental feature whereby WITH DOCKER parallelizes building of the images to be loaded has been added. To enable this feature use VERSION --parallel-load 0.6. #1725
    • Added cache_size_pct config option to allow specifying cache size as a percentage of disk space.

    Fixed

    • Fixed a duplicate build issue when using IF together with WITH DOCKER #1724
    • Fixed a bug where BUILD --platform=$ARG did not expand correctly
    • Fixed issue preventing use of WITH DOCKER with systemd-based images such as kind, when used under hosts with cgroups v2.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.67 MB)
    earthly-darwin-arm64(32.21 MB)
    earthly-linux-amd64(31.76 MB)
    earthly-linux-arm64(30.82 MB)
    earthly-linux-arm7(27.89 MB)
    earthly-windows-amd64.exe(32.45 MB)
  • v0.6.10(Mar 3, 2022)

  • v0.6.9(Mar 2, 2022)

    Changed

    • Log sharing is enabled by default for logged in users, it can be disabled with earthly config global.disable_log_sharing true.
    • SAVE ARTIFACT ... AS LOCAL now sets mtime of output artifacts to the current time.

    Added

    • Earthly is now 15-30% faster when executing large builds #1589
    • Experimental HOST command, which can be used like this: HOST <domain> <ip> to add additional hosts during the execution of your build. To enable this feature, use VERSION --use-host-command 0.6. #1168

    Fixed

    • Errors when using inline caching indicating invalid layer index #1635
    • Podman can now use credentials from the default location #1644
    • Podman can now use the local registry cache without modifying registries.conf #1675
    • Podman can now use WITH DOCKER --load inside a target marked as LOCALLY #1675
    • Interactive sessions should now work with rootless configurations that have no apparent external IP address #1573, #1689
    • On native Windows installations, Earthly properly detects the local git path when it's available #1663
    • On native Windows installations, Earthly will properly identify targets in Earthfiles outside of the current directory using the \ file separator #1663
    • On native Windows installations, Earthly will save local artifacts to directories using the \ file separator #1663
    • A parsing error, when using WITH DOCKER --load in conjunction with new-style build args. #1696
    • ENTRYPOINT and CMD were not properly expanding args when used in shell mode.
    • A race condition sometimes caused a Canceled error to be reported, instead of the real error that caused the build to fail
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.63 MB)
    earthly-darwin-arm64(32.16 MB)
    earthly-linux-amd64(31.72 MB)
    earthly-linux-arm64(30.81 MB)
    earthly-linux-arm7(27.88 MB)
    earthly-windows-amd64.exe(32.40 MB)
  • v0.6.8(Feb 16, 2022)

  • v0.6.7(Feb 9, 2022)

    Log Sharing (experimental)

    This version of Earthly includes an experimental log-sharing feature which will upload build-logs to the cloud when enabled.

    To enable this experimental feature, you must first sign up for an earthly account by using the earthly account register command, or by visiting https://ci.earthly.dev/

    Once logged in, you must explicitly enable log-sharing by running:

    earthly config global.disable_log_sharing false
    

    In a future version, log-sharing will be enabled by default for logged-in users; however, you will still be able to disable it, if needed.

    When log-sharing is enabled, you will see a message such as

    Share your build log with this link: https://ci.earthly.dev/logs?logId=dc622821-9fe4-4a13-a1db-12680d73c442
    

    as the last line of earthly output.

    Fixed

    • GIT CLONE now works with annotated git tags. #1571
    • CACHE command was not working for versions of earthly installed via homebrew.
    • Autocompletion bug when directory has both an Earthfile and subdir containing an earthfile.
    • Autocompletion bug when directory has two subdirectories where one is a prefix of the other.

    Changed

    • earthly account logout raises an error when EARTHLY_TOKEN is set.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.55 MB)
    earthly-darwin-arm64(32.09 MB)
    earthly-linux-amd64(31.64 MB)
    earthly-linux-arm64(30.67 MB)
    earthly-linux-arm7(27.74 MB)
    earthly-windows-amd64.exe(32.32 MB)
  • v0.6.6(Jan 26, 2022)

    Added

    • Ability to change mounted secret file mode. fixes #1434

    Changed

    • Permission errors related to reading ~/.earthly/config.yml and .env files are now treated as errors rather than silently ignored (and assuming the file does not exist).
    • Speedup from pre-emptive execution of build steps prior to them being referenced in the build graph.

    Fixed

    • earthly panic when running with SUDO_USER pointing to a user the current user did not have read/write permission; notably encountered when running under circleci.

    Removed

    • Removed --git-url-instead-of flag, which has been replaced by earthly config git ...
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.44 MB)
    earthly-darwin-arm64(32.00 MB)
    earthly-linux-amd64(31.53 MB)
    earthly-linux-arm64(30.58 MB)
    earthly-linux-arm7(27.71 MB)
    earthly-windows-amd64.exe(32.18 MB)
  • v0.6.5(Jan 24, 2022)

    Added

    • Ability to load a different .env file via the --env-file flag.
    • Added experimental feature than changes the ARGs defined in the +base target to be local, unless defined with a --global flag; To enable this feature use VERSION --explicit-global 0.6.

    Changed

    • Updated buildkit to include changes up to 17c237d69a46d61653746c03bcbe6953014b41a5

    Fixed

    • failed to solve: image is defined multiple times for the same default platform errors. #1594, #1582
    • failed to solve: image rmi after pull and retag: command failed: docker image rm ...: exit status 1: Error: No such image errors. #1590
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.43 MB)
    earthly-darwin-arm64(31.98 MB)
    earthly-linux-amd64(31.51 MB)
    earthly-linux-arm64(30.58 MB)
    earthly-linux-arm7(27.71 MB)
    earthly-windows-amd64.exe(32.18 MB)
  • v0.6.4(Jan 17, 2022)

  • v0.6.3(Jan 12, 2022)

    Changed

    • Updated buildkit to contain changes up to 15fb1145afa48bf81fbce41634bdd36c02454f99 from moby/master.

    Added

    • Expirmental CACHE command can be used in Earthfiles to optimize the cache in projects that perform better with incremental changes. For example, a Maven project where SNAPSHOT dependencies are added frequently, an NPM project where node_modules change frequently, or programming languages using incremental compilers. #1399
    • Config file entries can be deleted using a --delete flag (for example earthly config global.conversion_parallelism --delete). #1449
    • Earthly now provides the following builtin ARGs: EARTHLY_VERSION and EARTHLY_BUILD_SHA. These will be generally available in Earthly version 0.7+, however, they can be enabled earlier by using the --earthly-version-arg. feature flag #1452
    • Config option to disable known_host checking for specific git hosts by setting strict_host_key_checking to false under the git section of earthly/config.yml (defaults to true).
    • Error check for using both --interactive and --buildkit-host (which are not currently supported together). #1492
    • earthly ls [<project-ref>] to list Earthfile targets.

    Fixed

    • Gracefully handle empty string "" being provided as a value to earthly config commands. #1449
    • known_host entries were being ignored when custom pattern and substituted git config options were used (commonly used for self-hosted git repos)
    • Unable to connect to ssh server when known_hosts doesn't contain ssh-rsa host scan, but contains a different key-scan (e.g. ecdsa-sha2-nistp256, ssh-ed25519, etc).
    • When git auth is set to ssh but no user is given, default to current user (similar to calling ssh example.com vs ssh [email protected]).
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.39 MB)
    earthly-darwin-arm64(31.94 MB)
    earthly-linux-amd64(31.47 MB)
    earthly-linux-arm64(30.56 MB)
    earthly-linux-arm7(27.63 MB)
    earthly-windows-amd64.exe(32.14 MB)
  • v0.6.2(Dec 1, 2021)

  • v0.6.1(Nov 29, 2021)

    Fixed

    • BUILD arguments containing a subshell ($(...)) were executed twice, and when +base target was empty would result errors such as the first command has to be FROM, FROM DOCKERFILE, LOCALLY, ARG, BUILD or IMPORT #1448
    • TLS error (transport: authentication handshake failed: remote error: tls: no application protocol) when enabling buildkit mTLS #1439
    • Unable to save artifacts to local directory (.) #1422
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.01 MB)
    earthly-darwin-arm64(31.53 MB)
    earthly-linux-amd64(31.10 MB)
    earthly-linux-arm64(30.14 MB)
    earthly-linux-arm7(27.35 MB)
    earthly-windows-amd64.exe(31.73 MB)
  • v0.6.0(Nov 24, 2021)

    This version promotes a number of features that have been previously in Experimental and Beta status. To make use of the features in this version you need to declare VERSION 0.6 at the top of your Earthfile. If a version is not declared, then Earthly's interpreter will assume VERSION 0.5.

    If you are not ready to update your scripts to take advantage of VERSION 0.6, then you may upgrade Earthly anyway and your scripts should continue to work as before, provided that they either declare VERSION 0.5 or they don't declare a version at all.

    Declaring VERSION 0.6 is equivalent to

    VERSION \
      --use-copy-include-patterns \
      --referenced-save-only \
      --for-in \
      --require-force-for-unsafe-saves \
      --no-implicit-ignore \
      0.5
    

    It is recommended to use VERSION 0.6 instead as individual feature flags don't guarantee proper forwards-backwards compatibility. Note, however, that Earthly 0.5.* is not able to run a VERSION 0.6 Earthfile and will return an error.

    For more information on the individual Earthfile feature flags see the Earthfile version-specific features page.

    Changed

    • What Earthly outputs locally has changed in a way that is not backwards compatible. For an artifact or an image to be produced locally it needs to be part of a BUILD chain (or be part of the target being directly built). Artifacts and images introduced through FROM or COPY are no longer output locally.

      To update existing scripts, you may issue a duplicate BUILD in addition to a FROM (or a COPY), should you wish for the referenced target to perform output.

      For example, the following script

      FROM +some-target
      COPY +another-target/my-artifact ./
      

      could become

      FROM +some-target
      BUILD +some-target
      COPY +another-target/my-artifact ./
      BUILD +another-target
      

      in order to produce the same outputs.

      For more details see #896.

    • The syntax for passing build args has been changed.

      Earthly v0.5 (old way)

      FROM --build-arg NAME=john +some-target
      COPY --build-arg NAME=john +something/my-artifact ./
      WITH DOCKER --build-arg NAME=john --load +another-target
        ...
      END
      

      Earthly v0.6 (new way)

      FROM +some-target --NAME=john
      COPY (+something/my-artifact --NAME=john) ./
      WITH DOCKER --load (+another-target --NAME=john)
        ...
      END
      

      Passing build args on the command-line has also changed similarly:

      Earthly v0.5 (old way)

      earthly --build-arg NAME=john +some-target
      

      Earthly v0.6 (new way)

      earthly +some-target --NAME=john
      

      This change is part of the UDC proposal #581. The old way of passing args is deprecated and will be removed in a future version (however, it still works in 0.6).

    • If a SAVE ARTIFACT is unsafe (writing to a directory outside of the Earthfile directory), it'll require the --force flag.

    • .earthlyignore no longer includes any implicit entries like Earthfile or .earthlyignore. These will need to be specified explicitly. #1294

    • Buildkit was updated to d429b0b32606b5ea52e6be4a99b69d67b7c722b2. This includes a number of bug fixes, including eliminating crashes due to panic failed to get edge.

    Added

    • Earthly now performs local image outputs to the local Docker daemon through a built-in registry. This speeds up the process drastically as common layers no longer need to be transferred over #500.
    • Earthly now enables additional parallelism to speed up certain operations that were previously serialized #888. Note that this setting was previously controlled by --conversion-parallelism flag or the EARTHLY_CONVERSION_PARALLELISM environment variable while in experimental stage. It has now been moved as part of the Earthly config and has been promoted to GA.
    • COPY transfers are sped up as only the necessary files are sent over to BuildKit #1062.
    • WITH DOCKER has been promoted to GA #576.
    • FROM DOCKERFILE has been promoted to GA.
    • LOCALLY has been promoted to GA #580.
    • RUN --interactive and RUN --interactive-keep have been promoted to GA #693.
    • IF and FOR have been promoted to GA #779.
    • Support for Apple Silicon M1 has been promoted to GA #722.
    • Multi-platform builds have been promoted to GA #536.
    • Mounting secrets as files have been promoted as GA #579.
    • VERSION has been promoted to GA #991
    • User-defined commands (UDCs) have been promoted to GA #581.
    • Allow running SAVE ARTIFACT after RUN --push is now GA #586.
    • SAVE ARTIFACT --if-exists and COPY --if-exists have been promoted to GA #588.
    • Shared cache and --ci mode are now GA #11.
    • New builtin args USERPLATFORM, USEROS, USERARCH, and USERVARIANT which represent the platform, OS, architecture, and processor variant of the system Earthly is being called from #1251. Thanks to @akrantz01 for the contribution!
    • Config option for buildkit's max_parallelism configuration. Use this to increase parallelism for faster builds or decrease parallelism when resources are constraint. The default is 20. #1308
    • Support for required ARGs (ARG --required foo) #904. Thanks to @camerondurham for the contribution!
    • Extended auto-completion to be build-arg aware. Typing earthly +my-target --<tab><tab> now prints possible build-args specific to +my-target. #1330.
    • The console output now has an improved structure #1226.

    Fixed

    • Eliminated some spurious warnings (ReadDataPacket failed, Failed to connect to terminal, failed to read from stdin and others) #1241.
    • Minor fixes related to the experimental Podman support #1239.
    • Improved some error messages related to frontend detection #1250.
    • Fixed Podman's ability to load OCI images #1287.
    • Fixed homebrew installation on macOS 12. #1370, homebrew/earthly#13
    • failed due to failed to autodetect a supported frontend errors will now include underlying reason for failure
    • Cache export was not honoring EARTHLY_MAX_REMOTE_CACHE setting.
    • Buildkit logs were not being sent to earthly-buildkitd container's output.
    • kind required permissions were not available in earthly-buildkitd.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.01 MB)
    earthly-darwin-arm64(31.51 MB)
    earthly-linux-amd64(31.09 MB)
    earthly-linux-arm64(30.14 MB)
    earthly-linux-arm7(27.29 MB)
    earthly-windows-amd64.exe(31.72 MB)
  • v0.6.0-rc3(Nov 15, 2021)

    Fixed

    • cache export was not honoring EARTHLY_MAX_REMOTE_CACHE setting
    • buildkit logs were not being sent to earthly-buildkitd container's output.
    • kind required permissions were not available in earthly-buildkitd.

    Changed

    • docker and fsutils versions were set to match versions defined in earthly's buildkit fork.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.09 MB)
    earthly-darwin-arm64(31.62 MB)
    earthly-linux-amd64(31.18 MB)
    earthly-linux-arm64(30.29 MB)
    earthly-linux-arm7(27.37 MB)
    earthly-windows-amd64.exe(31.81 MB)
  • v0.6.0-rc2(Nov 1, 2021)

  • v0.6.0-rc1(Oct 28, 2021)

    This version promotes a number of features that have been previously in Experimental and Beta status. To make use of the features in this version you need to declare VERSION 0.6 at the top of your Earthfile. If a version is not declared, then Earthly's interpreter will assume VERSION 0.5.

    If you are not ready to update your scripts to take advantage of VERSION 0.6, then you may upgrade Earthly anyway and your scripts should continue to work as before, provided that they either declare VERSION 0.5 or they don't declare a version at all.

    Declaring VERSION 0.6 is equivalent to

    VERSION \
      --use-copy-include-patterns \
      --referenced-save-only \
      --for-in \
      --require-force-for-unsafe-saves \
      --no-implicit-ignore \
      0.5
    

    It is recommended to use VERSION 0.6 instead as individual feature flags don't guarantee proper forwards-backwards compatibility. Note, however, that Earthly 0.5.* is not able to run a VERSION 0.6 Earthfile and will return an error.

    For more information on the individual Earthfile feature flags see the Earthfile version-specific features page.

    Added

    • Earthly now performs local image outputs to the local Docker daemon through a built-in registry. This speeds up the process drastically as common layers no longer need to be transferred over #500.
    • Earthly now enables additional parallelism to speed up certain operations that were previously serialized #888. Note that this setting was previously controlled by --conversion-parallelism flag or the EARTHLY_CONVERSION_PARALLELISM environment variable while in experimental stage. It has now been moved as part of the Earthly config and has been promoted to GA.
    • COPY transfers are sped up as only the necessary files are sent over to BuildKit #1062.
    • WITH DOCKER has been promoted to GA #576.
    • FROM DOCKERFILE has been promoted to GA.
    • Support for Apple Silicon M1 has been promoted to GA #722.
    • Multi-platform builds have been promoted to GA #536.
    • Mounting secrets as files have been promoted as GA #579.
    • VERSION has been promoted to GA #991
    • User-defined commands (UDCs) have been promoted to GA #581.
    • Allow running SAVE ARTIFACT after RUN --push is now GA #586.
    • SAVE ARTIFACT --if-exists and COPY --if-exists have been promoted to GA #588.
    • Shared cache and --ci mode are now GA #11.
    • LOCALLY has been promoted to GA #580.
    • RUN --interactive and RUN --interactive-keep have been promoted to GA #693.
    • IF and FOR have been promoted to GA #779.
    • If a SAVE ARTIFACT is unsafe (writing to a directory outside of the Earthfile directory), it'll require the --force flag.
    • .earthlyignore no longer includes any implicit entries like Earthfile or .earthlyignore. These will need to be specified explicitly. #1294
    • The console output now has an improved structure #1226.
    • Fixed homebrew installation on macOS 12. #1370, homebrew/earthly#13

    Changed

    • What Earthly outputs locally has changed in a way that is not backwards compatible. For an artifact or an image to be produced locally it needs to be part of a BUILD chain (or be part of the target being directly built). Artifacts and images introduced through FROM or COPY are no longer output locally.

      To update existing scripts, you may issue a duplicate BUILD in addition to a FROM (or a COPY), should you wish for the referenced target to perform output.

      For example, the following script

      FROM +some-target
      COPY +another-target/my-artifact ./
      

      could become

      FROM +some-target
      BUILD +some-target
      COPY +another-target/my-artifact ./
      BUILD +another-target
      

      in order to produce the same outputs.

      For more details see #896.

    • The syntax for passing build args has been changed.

      Earthly v0.5 (old way)

      FROM --build-arg NAME=john +some-target
      COPY --build-arg NAME=john +something/my-artifact ./
      WITH DOCKER --build-arg NAME=john --load +another-target
        ...
      END
      

      Earthly v0.6 (new way)

      FROM +some-target --NAME=john
      COPY (+something/my-artifact --NAME=john) ./
      WITH DOCKER --load (+another-target --NAME=john)
        ...
      END
      

      Passing build args on the command-line has also changed similarly:

      Earthly v0.5 (old way)

      earthly --build-arg NAME=john +some-target
      

      Earthly v0.6 (new way)

      earthly +some-target --NAME=john
      

      This change is part of the UDC proposal #581. The old way of passing args is deprecated and will be removed in a future version (however, it still works in 0.6).

    • Add builtin args USERPLATFORM, USEROS, USERARCH, and USERVARIANT which represent the platform, OS, architecture, and processor variant of the system Earthly is being called from #1251. Thanks to @akrantz01 for the contribution!

    • Support for required ARGs (ARG --required foo) #904. Thanks to @camerondurham for the contribution!

    • Add a config item for buildkit's max_parallelism configuration. Use this to increase parallelism for faster builds or decrease parallelism when resources are constraint. The default is 20. #1308

    • Extend auto-completion to be build-arg aware. Typing earthly +my-target --<tab><tab> now prints possible build-args specific to +my-target. #1330.

    • Buildkit was updated to d429b0b32606b5ea52e6be4a99b69d67b7c722b2. This includes a number of bug fixes, including eliminating crashes due to panic failed to get edge.

    Fixed

    • Eliminated some spurious warnings (ReadDataPacket failed, Failed to connect to terminal, failed to read from stdin and others) #1241.
    • Minor fixes related to the experimental Podman support #1239.
    • Improved some error messages related to frontend detection #1250.
    • Fixed Podman's ability to load OCI images #1287.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(30.77 MB)
    earthly-darwin-arm64(31.30 MB)
    earthly-linux-amd64(31.07 MB)
    earthly-linux-arm64(30.13 MB)
    earthly-linux-arm7(27.28 MB)
    earthly-windows-amd64.exe(31.69 MB)
  • v0.5.24(Sep 30, 2021)

    Added

    • New --output flag, which forces earthly to enable outputs, even when running under --ci mode #1200.
    • Experimental support for Podman #760.
    • Automatically adds compatibility arguments for cases where docker is running under user namespaces.

    Fixed

    • Removed spurious "Buildkit and Local Registry URLs are pointed at different hosts (earthly-buildkitd vs. 127.0.0.1)" warning.
    • Scrub git credentials when running under --debug mode.
    • "FROM DOCKERFILE" command was ignoring the path (when run on a remote target), which prevented including dockerfiles which were named something else.
    • Removed the creation of a temporary output directory when run in --no-output mode, or when building targets that don't output artifacts, the temporary directory is now created just before it is needed.
    • Fixed race condition involving WITH DOCKER and IF statements, which resulted in failed to solve: NotFound: no access allowed to dir errors.
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(31.02 MB)
    earthly-darwin-arm64(30.64 MB)
    earthly-linux-amd64(31.35 MB)
    earthly-linux-arm64(29.56 MB)
    earthly-linux-arm7(26.82 MB)
    earthly-windows-amd64.exe(32.10 MB)
  • v0.5.23(Aug 24, 2021)

    • introduced COPY --if-exists which allows users to ignore errors which would have occurred if the file did not exist.
    • introduced new ip_tables config option for controlling which iptables binary is used; fixes #1160
    • introduced warning message when saving to paths above the current directory of the current Earthfile; these warnings will eventually become errors unless the --force flag is passed to SAVE ARTIFACT.
    • bugfix for remote buildkit configuration options being ignored; fixes #1177
    • suppressed erroneous internal-term error messages which occurred when running under non-interactive ( e.g. ci ) modes; fixes #1108
    • changed help text for --artifact mode
    • deb and yum packages no longer clear the earthly cache on upgrades
    Source code(tar.gz)
    Source code(zip)
    earthly-darwin-amd64(30.91 MB)
    earthly-darwin-arm64(30.53 MB)
    earthly-linux-amd64(31.23 MB)
    earthly-linux-arm64(29.40 MB)
    earthly-linux-arm7(26.79 MB)
    earthly-windows-amd64.exe(31.98 MB)
Owner
Earthly
Build automation for the post-container era
Earthly
CBuild build system - A tiny build system for C

cuild - CBuild A build system for C Building $ go build . Usage Similar to GNU Make, a file named "Cuildfile" is required. You have a few flags to us

Laurentino Luna 0 Jan 17, 2022
a build tool for Go, with a focus on cross-compiling, packaging and deployment

goxc NOTE: goxc has long been in maintenance mode. Ever since Go1.5 supported simple cross-compilation, this tool lost much of its value. There are st

Am Laher 1.7k Sep 19, 2022
Create build pipelines in Go

taskflow Create build pipelines in Go This package aims to simplify the creation of build pipelines in Go instead of using scripts or Make. taskflow A

Robert PajΔ…k 308 Sep 26, 2022
Colorize (highlight) `go build` command output

colorgo colorgo is a wrapper to go command that colorizes output from go build and go test. Installation go get -u github.com/songgao/colorgo Usage c

Song Gao 110 Sep 13, 2022
EGo lets you build, debug und run Go apps on Intel SGX - as simple as conventional Go programming!

EGo is a framework for building confidential apps in Go. Confidential apps run in always-encrypted and verifiable enclaves on Intel SGX-enabled ha

Edgeless Systems GmbH 339 Sep 21, 2022
Build system and task runner for Go projects

Gilbert is task runner that aims to provide declarative way to define and run tasks like in other projects like Gradle, Maven and etc.

Gilbert 104 Sep 26, 2022
Build and (re)start go web apps after saving/creating/deleting source files.

unmaintained Fresh Fresh is a command line tool that builds and (re)starts your web application everytime you save a Go or template file. If the web f

Andrea Franz 3.4k Sep 22, 2022
KintoHub is an open source build and deployment platform designed with a developer-friendly interface for Kubernetes.

What is Kintohub? KintoHub is an open source build and deployment platform designed with a developer-friendly interface for Kubernetes. Build your cod

KintoHub 30 Jun 7, 2022
Build systems with Go examples

What is this? This is a repository containing all the examples from the book BUILD SYSTEMS with GO (and save the world). This book is written to help

Juan M. Tirado 92 Sep 27, 2022
An experimental way to apply patches to the Go runtime at build time.

go-patch-overlay An experimental way to apply patches to the Go runtime at build time. Assuming you have a directory of patches to apply to the Go sou

Felix GeisendΓΆrfer 16 Feb 9, 2022
Please is a cross-language high-performance extensible build system for reproducible multi-language builds.

Please is a cross-language build system with an emphasis on high performance, extensibility and reproducibility. It supports a number of popular languages and can automate nearly any aspect of your build process.

Thought Machine 2.1k Sep 23, 2022
Blueprint Build System For Golang

Blueprint Build System Blueprint is being archived on 2021 May 3. On 2021 May 3, we will be archiving the Blueprint project. This means it will not be

GengKapak WIP 0 Nov 20, 2021
Various tools for usage with Golang like installer, github tool and cloud features.

Gopei2 (Go Programming Environment Installer) Gopei shell install Go compiler, LiteIDE and configure for you the entire environment, variables, paths,

George Calianu 99 Sep 20, 2022
Tool to check for dependency confusion vulnerabilities in multiple package management systems

Confused A tool for checking for lingering free namespaces for private package names referenced in dependency configuration for Python (pypi) requirem

Visma Product Security 498 Sep 30, 2022
An extremely opinionated TypeScript monorepo tool.

Unirepo is an extremely opinionated TypeScript build tool. Typical monorepo management tools in the Node.js ecosystem provide automation aroun

Brandon Bloom 222 Sep 19, 2022
a Make/rake-like dev tool using Go

About Mage is a make-like build tool using Go. You write plain-old go functions, and Mage automatically uses them as Makefile-like runnable targets. I

Mage 3.2k Sep 30, 2022
πŸš€ gowatch is a command line tool that builds and (re)starts your go project everytime you save a Go or template file.

gowatch δΈ­ζ–‡ζ–‡ζ‘£ gowatch is a command line tool that builds and (re)starts your go project everytime you save a Go or template file. Installation To insta

silenceper 722 Sep 21, 2022
A simple tool to help WoW repack administrators manipulate the repack database(s)

WoW Repack Manipulator This tool makes it easier for an administrator of a WoW Repack (private WoW server, basically) to manipulate the database that

Michael Crilly 2 Feb 7, 2022
Build automation for the container era

?? Build anything via containers - build images or standalone artifacts (binaries, packages, arbitrary files) ?? Programming language agnostic - allow

Earthly 8k Sep 25, 2022
Kubernetes built with Earthly

Kubernetes built with Earthly Earthly is a very pragmatic iteration from the existing ecosystem of container build tools available today. It combines

Andrew Sy Kim 2 Nov 3, 2021