Please is a cross-language high-performance extensible build system for reproducible multi-language builds.

Overview

Please Build Status Build Status Go Report Card Gitter chat

Please is a cross-language build system with an emphasis on high performance, extensibility and reproducibility. It supports a number of popular languages and can automate nearly any aspect of your build process.

See http://please.build for more information.

Currently Linux (tested on Ubuntu), macOS and FreeBSD are actively supported.

If you're a fan of Please, don't forget to add yourself to the adopters file.

Getting Started

The easiest way to install it on your own machine is to run:

curl -s https://get.please.build | bash

Or, if you prefer, grab one of the tarballs off our releases page and extract it yourself; it typically lives in ~/.please.

You can also install using Homebrew:

brew tap thought-machine/please
brew install please

Then you simply run plz init at the root of your project to set up a default config and you're ready to go. The best way to get to grips with Please is through the codelabs! There's also the getting started guide that explains the core concepts.

How is it so fast?

Please has a robust and correct caching model that enables us to aggressively cache artifacts. Caching is based on the hashes of inputs (both files, and environment variables) to each rule rather than last modified timestamps. Builds are hermetic so don't have access to anything they haven't explicitly defined as inputs. This means that if anything changes, we know exactly what might've been affected, so the minimal set of targets get built and tested.

Because each task is hermetic, they can be run in parallel without any chance of interfering with each-other. Combine these two concepts with shared remote caches, and it makes for a blazing fast build system for any language or technology.

Please is also written in Go and every effort has been made to make it as fast as possible. There's no startup time waiting to bring up VMs, interpreting code or communicating with remote JVM processes. The code itself takes full advantage of Go's concurrency and asynchronicity. The end result is a snappy command line tool that gets to work immediately and feels great to use.

Why Please, and not Maven, pip, or go build?

A build system is more than just a mechanism for invoking the compiler. If you're working on just one language, don't have any code generation, and don't need to publish any artifacts anywhere, you might not need Please. Chances are this is not the case.

Building software often involves more than just compiling code. There's deployment config to template, code to generate, and quite often, there's more than one language involved. Please provides a powerful, comprehensive, and understandable framework that you can use to craft a truly holistic build process.

Please does this through a consistent and seamless command line interface; there's no need to learn new build systems and technologies for different languages. Build any target with plz build, test any target with plz test, no matter what's going on under the hood.

The Docker & Kubernetes codelab covers building a Kubernetes based application with Please, including reliably deploying code to a local cluster, and pushing it to a remote registry.

The genrule() codelab covers extending Please with custom build definitions to truly automate any part of your deployment process.

Why Please, and not make?

Make is a great tool for running tasks. It's easy enough to understand because it leaves you very close to the shell. The problem is, it has limited capability to build out complexity. There have been attempts to generate make files from higher level tools like cmake and ninja, but they fall short of what Please sets out to achieve.

The Please build language is a full programming language. There are a high level set of build rules that make up a declarative DSL to define build targets, however you can drop into an imperative language that resembles python when necessary:

subinclude("//my_custom_defs:markdown_page")

pages = []
for page in glob(include = ["*.md"]):
    pages += markdown_page(
        name = page.removesuffix(".md"),
        srcs = [page],
        visibility = ["//website/..."],
    )   

go_binary (
    name = "webserver",
    srcs = ["main.go"],
    deps = ["//third_party/go:protobuf"],
    data = pages,
    visibility = ["//services/foo/..."],
)

This is distinctively operating at a higher level when compared to make:

protobuf:
    go install google.golang.org/protobuf

webserver: protobuf
    go tool compile --pack foo.go -o foo.a

pages: ???

Additionally, make builds are not hermetic. The above make example installs protobuf into the host machines Go path. Please builds only have access to files and environment variables they have explicitly been given access to. You can play around in the environment targets are built in with plz build //some/target --shell. Additionally, on linux systems, Please can take this a step further with Linux namespaces to improve sandboxing especially of the network. Please also has built in task parallelism so can take full advantage of multi-core machines which were not a consideration 40 years ago when make was designed.

Finally, Please has a robust caching mechanism base on hashing the inputs of each rule. Makes cache invalidation is based on the last modified timestamp which can change unexpectedly forwards and backwards in time. Combine this with hermetic builds, and Please caching is never incorrect.

Why Please, not Bazel, Buck or Pants?

These build systems are all very similar so choosing between them can be hard. Please orginally replaced buck implementing the subset of features we used. We found that buck (and the competition) worked great when using the supported languages but fell flat when breaking new ground.

The biggest difference between Please and the competition is that Please is designed from the ground up to be extensible. The built-in languages are all defined in the same build language as you use to define your targets, demonstrating that there's nothing special about them. This puts the build definitions where they should be: in your domain. You have all the same tools Please has to expand your build definitions to satisfy your needs.

Please does all this with a focus on simplicity and transparency. There are a limited amount of concepts that are needed to get started and once they are grocked, the possibilities are endless. Please relies on these concepts rather than requiring lots of magic and incantation. Configuration is simple and largely optional so getting going is easy, and there's no single WORKSPACE file nobody really owns, with lines of configuration that nobody really understands.

The command line interface is designed with similar considerations in mind. Subcommands can be added to Please though aliases and tie into the Please tab-completions. Not only can flags and arguments be completed, but they can also leverage the build graph to complete labels enabling you to truly craft your developer experience the way you want it.

Building Please

If you're looking to get involved, check out the contributor guidance to help you get started. If you're a fan of Please, don't forget to add yourself to the adopters file.

To build Please yourself, run ./bootstrap.sh in the repo root. This will bootstrap a minimal version of Please using Go and then rebuild it using itself.

You'll need to have Go 1.16+ installed to build Please although once built it can target any version from 1.8+ onwards.

Optional dependencies for various tests include Python, Java, clang, gold and docker - none of those are required to build components so their tests will be excluded if they aren't available.

If you'd rather not worry about installing the dependencies, we provide a prebuilt Docker image based on Ubuntu which is capable of building the whole thing for you: docker run -it thoughtmachine/please_ubuntu

You can install a locally built copy of Please using plz install, or if it's the first time, by running ./install.sh after it's built. This will overwrite whatever you currently have in ~/.please with your local version, although you can get back to a released version again by running plz update --force.

To automatically fix linting and code generation issues, run plz autofix.

Documentation

Status

Please is released & we consider it stable; we follow semver for releases, so major versions indicate potentially breaking changes to the BUILD language, command line or other behaviour. We try to minimise this where possible.

We're very happy to accept pull requests, feature requests, and bugs if it's not working for you. We don't always have time for everything but Please is under active development.

Issues
  • Please diff

    Please diff

    First thanks for the amazing tool,

    Now I have this #1592 TL;DR I'm building multiple docker images with please in our monorepo where all docker files are put in the root of repo because of the dependencies stuff. now, my question is I want to build docker image based on changes in repo so you know when CI runs not all docker images will push/build and only changed one push/build.

    something like this:

    subinclude("//pleasing//docker")
    
    docker_image(
        name = "abc",
        dockerfile = "abc.Dockerfile",
        srcs = ["abc","efg","abc.go"],
        visibility = ["PUBLIC"],
        version = f'$(echo "$(git rev-parse --abbrev-ref HEAD)-$(git rev-parse --short HEAD)")'
    )
    docker_image(
        name = "xyz",
        dockerfile = "xyz.Dockerfile",
        srcs = ["xyz","efg","xyz.go"],
        visibility = ["PUBLIC"],
        version = f'$(echo "$(git rev-parse --abbrev-ref HEAD)-$(git rev-parse --short HEAD)")'
    )
    

    and in CI I'm running

    ./pleasew run //:abc_push
    ./pleasew run //:xyz_push
    

    so now if there is a change in xyz directory, target abc is also running and building docker image. which is not appropriate because abc image/srcs has no change since it's getting list of srcs!!. so am I missing something or please is not doing git diffstuff

    wontfix 
    opened by pratikbin 14
  • Is there a way to bootstrap without network access?

    Is there a way to bootstrap without network access?

    Is there a way to vendor all 3rd party dependencies to avoid requiring network access during bootstrapping? We are working on FreeBSD port and the build currently fails with

    1 error occurred:
            * Get https://github.com/kevinburke/go-bindata/archive/46eb4c183bfc1ebb527d9d19bcded39476302eb8.zip: dial tcp: lookup github.com on 1.0.0.1:53: write udp 127.0.0.1:61403->1.0.0.1:53: write: permission denied
    
    Build stopped after 40ms. 1 target failed:
        //third_party/go:_go-bindata-go-bindata#download
    1 error occurred:
            * Get https://github.com/kevinburke/go-bindata/archive/46eb4c183bfc1ebb527d9d19bcded39476302eb8.zip: dial tcp: lookup github.com on 1.0.0.1:53: write udp 127.0.0.1:61403->1.0.0.1:53: write: permission denied
    

    because network access is not allowed during build phase.

    question 
    opened by dmgk 14
  • Go module support

    Go module support

    Module support is now included in Go 1.11 and will be on by default with Go 1.12 (now in beta). How will this effect please, do we need to support it?

    On a side note, Bazels Go rules support an importpath variable. If we had something similar, we could move away from building everything in a single "virtual" gopath and decouple the filesystem layout from the import paths.

    enhancement help wanted not-stale 
    opened by nekinie 14
  • Platform specfic targets

    Platform specfic targets

    Certain targets are only required (and can only built on) for certain platforms. For example: certain dependencies might only be required by linux.

    My current approach is an if, combined with a custom select_config rule:

    if select_config([{
        "os": "linux",
        "cpu": "amd64",
    }]):
        go_library(name = "logrus", srcs = [":_logrus#go_source"], cover = False, import_path = "github.com/sirupsen/logrus", visibility = ["PUBLIC"], deps = select({"//example/third_party/go/__config:darwin_amd64": [], "//example/third_party/go/__config:linux_amd64": ["//example/third_party/go/golang.org/x/sys/unix"], "default": []}))
    

    The problem is: it's more generated code, and the buildifier package formats the target itself on a single line.

    Is there a better way to represent platform specific targets? (It's not the end of the world, but it's hard to read and more changes in a VCS)

    wontfix 
    opened by sagikazarmark 12
  • Add gocloud to allow Blob Storage as a cache

    Add gocloud to allow Blob Storage as a cache

    A follow-up from #1140: It might be beneficial to add gocloud into please, in order to allow blob storage to be used as a cache backend (so backends like AWS S3, Google Cloud Storage, or Azure Blob Storage could be used).

    This would allow users who run please builds in different CI providers to connect to a remote blob storage for their cache (or run their own one! there's always the S3-compatible flavour of the week version).

    Gocloud provides a common interface, so configuration is universal; however, it seems to call each services' specific SDKs below, so it might be a heavy dependency.

    enhancement not-stale 
    opened by robxu9 12
  • Multiple versions can't coexist when installing please via Homebrew

    Multiple versions can't coexist when installing please via Homebrew

    When please is installed with Homebrew and there different Please versions in use, Please always downloads a new copy differing from the current version.

    I guess it's related to Homebrew.

    opened by sagikazarmark 11
  • FYI: there's a Skylark interpreter in Go

    FYI: there's a Skylark interpreter in Go

    I just learned of 'please' today, and I notice you're calling CPython 2.x in process. You might be interested in a pure-Go implementation of Skylark here: github.com/google/skylark. It has certain benefits relative to Python:

    • no GIL, so you can execute multiple threads in parallel, which can make the loading phase (BUILD file execution) and dependency analysis (rule.implementation function execution) much faster.
    • restricted import, no OS access, etc
    • control over mutability, so no possibility of a data race, despite familiar imperative paradigm.
    • compatibility with Bazel.

    I gave a talk on it: https://www.youtube.com/watch?v=9P_YKVhncWI

    cheers alan

    opened by alandonovan 11
  • Fix go_binary linking of third party dependencies outside of root #1061

    Fix go_binary linking of third party dependencies outside of root #1061

    For reference I am using MacOS.

    This issue only exists for the go_binary rule and third party dependencies which are added as library paths through command substitution. i.e. third party dependencies defined outside of the root third_party/go/BUILD.plz. The go_library rule works only because it's not actually quoting the file paths due to a subtle bug.

    $(for i in $(find "$TMP_DIR" -name pkg -type d); do echo -n " -I \"$i\"/{CONFIG.GOOS}_{CONFIG.GOARCH} "; done)
    

    The snippet above in the python go_library rule becomes the following in the actual shell.

    $(for i in $(find "$TMP_DIR" -name pkg -type d); do echo -n " -I "$i"/darwin_amd64 "; done)
    

    The escaped quote is lost and instead we get string concatenation which is probably not what was originally intended. In regards to the go_binary rule it is actually quoting the file paths. However, because it is being injected via command substitution it does not work properly. There are a couple issues here.

    Issue 1) The output of command substitution is split on whitespace regardless of quotes within the output. So, for example, a file path which includes spaces printargs $(echo '"foo bar\baz.txt"') would become the following arguments.

    arg 0: "foo
    arg 1: bar\baz.txt"
    

    Issue 2) The quotes within the command substitution output are preserved. So, if we have -L "foo/bar/pkg/darwin_amd64" it's going to attempt to look in a directory named "foo, which begins with a quote, and not the correct directory foo.

    This fix removes the need for quoting by using null separated arguments instead and providing them to the link/compile command via xargs instead of command substitution. Once I began manually testing with file paths which included spaces I noticed that the _LINK_PKGS_CMD also didn't account for spaces so I fixed it in a similar manner.

    opened by brian-bice 10
  • [Go] - use full url on package imports

    [Go] - use full url on package imports

    I see here https://github.com/thought-machine/please/blob/master/src/please.go#L27 the import of package hashes instead of github.com/thought-machine/src/hashes as usual on Go

    is there any way of importing the package using the full URL?

    opened by mishudark 10
  • feat: add asm support to please_go_install

    feat: add asm support to please_go_install

    This is the first working version after a couple hours of hacking.

    I think we should reuse some parts of the Go build tool: https://github.com/golang/go/blob/e125ccd10ea191101dbc31f0dd39a98f9d3ab929/src/cmd/go/internal/work/gc.go#L325

    As you can see, it handles a bunch of edge cases.

    A couple thins I noticed while I made this work:

    • please_go_install uses the gopath as a workspace (instead of its tmpdir) which causes some trouble for packages (eg. github.com/golang/snappy uses pkg/darwin_amd64/github.com/golang/ as a workspace which could overlap with other packages in github.com/golang.) Using $TMP_DIR and moving the final archive at the end would be better IMHO. That's actually what the go build tool does.
    opened by sagikazarmark 9
  • ARM support

    ARM support

    Now that Apple is committing itself to ARM, I guess it makes sense to open an issue for ARM support.

    This entails:

    • Building Please for ARM arch
    • Building applications with Please for ARM arch

    I guess in many cases it will be the same, but in case of Go for example, things are a little bit different: https://github.com/golang/go/wiki/GoArm

    enhancement help wanted 
    opened by sagikazarmark 9
  • `java_library`: output JNI headers generated by `javac`

    `java_library`: output JNI headers generated by `javac`

    The Java compiler outputs header files for Java sources that use the Java Native Interface (JNI), but there's currently no way for them to be consumed by the native-code components that depend on them, preventing them from being built.

    Provide the JNI headers as outputs of java_library when cc_hdrs is required. This allows native-code JNI components written in C/C++ to be built by including the java_library target as a dependency.

    enhancement 
    opened by chrisnovakovic 0
  • REX: Remove old specific path code

    REX: Remove old specific path code

    Implementing the TODO to remove this stuff. It was useful for compat but we don't support a rex server without non-specific output paths support; in practice many things will fail before we get to tests running.

    opened by peterebden 0
  • Feature Request: PHP build rules

    Feature Request: PHP build rules

    I'm considering using Please as a monorepos build tool , but I cannot figure out if it support php or not , I've taken a look into documentation but I couldn't figure it out. Please can you clarify?

    enhancement 
    opened by kawtharmahboubi 1
  • feat: allow adding additional optional outs in c and c++ rules

    feat: allow adding additional optional outs in c and c++ rules

    C and C++ rules have optional outs for coverage.

    This pr allow to add more optional outs for specific usage. For example c/c++ plugin of sonarqube produce additional outputs that can be kept using the new optional_outs c/c++ rules.

    opened by sfirmery 0
Releases(v16.21.3)
Builds and restarts a Go project when it crashes or some watched file changes

gaper Used to build and restart a Go project when it crashes or some watched file changes Aimed to be used in development only. Changelog See Releases

Max Claus Nunes 55 Jun 21, 2022
GoReleaser builds Go binaries as fast and easily as possible

GoReleaser builds Go binaries for several platforms, creates a GitHub release and then pushes a Homebrew formula to a tap repository. All that wrapped in your favorite CI.

GoReleaser 10.4k Aug 7, 2022
🚀 gowatch is a command line tool that builds and (re)starts your go project everytime you save a Go or template file.

gowatch 中文文档 gowatch is a command line tool that builds and (re)starts your go project everytime you save a Go or template file. Installation To insta

silenceper 716 Aug 4, 2022
a build tool for Go, with a focus on cross-compiling, packaging and deployment

goxc NOTE: goxc has long been in maintenance mode. Ever since Go1.5 supported simple cross-compilation, this tool lost much of its value. There are st

Am Laher 1.7k Jul 28, 2022
Build system and task runner for Go projects

Gilbert is task runner that aims to provide declarative way to define and run tasks like in other projects like Gradle, Maven and etc.

Gilbert 101 Jun 28, 2022
Blueprint Build System For Golang

Blueprint Build System Blueprint is being archived on 2021 May 3. On 2021 May 3, we will be archiving the Blueprint project. This means it will not be

GengKapak WIP 0 Nov 20, 2021
go language generics system

Gotgo This document describes the third iteration of my attempt at a reasonable implementation of generics for go based on the idea of template packag

David Roundy 119 Dec 4, 2021
Create build pipelines in Go

taskflow Create build pipelines in Go This package aims to simplify the creation of build pipelines in Go instead of using scripts or Make. taskflow A

Robert Pająk 303 Aug 5, 2022
Colorize (highlight) `go build` command output

colorgo colorgo is a wrapper to go command that colorizes output from go build and go test. Installation go get -u github.com/songgao/colorgo Usage c

Song Gao 110 Jun 27, 2022
EGo lets you build, debug und run Go apps on Intel SGX - as simple as conventional Go programming!

EGo is a framework for building confidential apps in Go. Confidential apps run in always-encrypted and verifiable enclaves on Intel SGX-enabled ha

Edgeless Systems GmbH 321 Jul 29, 2022
Build and (re)start go web apps after saving/creating/deleting source files.

unmaintained Fresh Fresh is a command line tool that builds and (re)starts your web application everytime you save a Go or template file. If the web f

Andrea Franz 3.4k Aug 1, 2022
KintoHub is an open source build and deployment platform designed with a developer-friendly interface for Kubernetes.

What is Kintohub? KintoHub is an open source build and deployment platform designed with a developer-friendly interface for Kubernetes. Build your cod

KintoHub 30 Jun 7, 2022
Build systems with Go examples

What is this? This is a repository containing all the examples from the book BUILD SYSTEMS with GO (and save the world). This book is written to help

Juan M. Tirado 82 Jul 24, 2022
🌍 Earthly is a build automation tool for the container era

?? Earthly is a build automation tool for the container era. It allows you to execute all your builds in containers. This makes them self-contained, repeatable, portable and parallel. You can use Earthly to create Docker images and artifacts (eg binaries, packages, arbitrary files).

Earthly 7.6k Jul 31, 2022
An experimental way to apply patches to the Go runtime at build time.

go-patch-overlay An experimental way to apply patches to the Go runtime at build time. Assuming you have a directory of patches to apply to the Go sou

Felix Geisendörfer 16 Feb 9, 2022
A Distributed Continuous Integration System from MongoDB

Evergreen Evergreen is a distributed continuous integration system built by MongoDB. It dynamically allocates hosts to run tasks in parallel across ma

Evergreen 356 Jul 28, 2022
Golang binaries compiled on-demand for your system

Go Binaries Go Binaries is an on-demand binary server, allowing non-Go users to quickly install tools written in Go without installing go itself, and

TJ Holowaychuk 746 Jul 27, 2022
A reproducible example for an issue I'm encoutering when deploying my gunicorn app to nginx

Ngnix Issue This is a minimum, reproducible example for an issue I'm encoutering when deploying my gunicorn app to nginx. Installation (Linux) Webserv

Space Simulation Services of Canada Corporation 1 May 29, 2022
A command line tool that builds and (re)starts your web application everytime you save a Go or template fileA command line tool that builds and (re)starts your web application everytime you save a Go or template file

# Fresh Fresh is a command line tool that builds and (re)starts your web application everytime you save a Go or template file. If the web framework yo

null 0 Nov 22, 2021
high performance distributed task scheduling system, Support multi protocol scheduling tasks

high performance distributed task scheduling system, Support multi protocol scheduling tasks

null 58 Aug 4, 2022
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

James Mills 7 Apr 15, 2022
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

mobus 2 Sep 19, 2021
Docker go multi stage builds

Сборка go mod download go build Запуск ./go-sample-app Примеры использования: ht

Aleksey Zverev 1 Jan 12, 2022
The high-performance build system for JavaScript & TypeScript codebases

Documentation Visit https://turborepo.org to view the full documentation. Community To chat with other community members, you can join the Turborepo D

Vercel 8.6k Aug 8, 2022
go-fastdfs 是一个简单的分布式文件系统(私有云存储),具有无中心、高性能,高可靠,免维护等优点,支持断点续传,分块上传,小文件合并,自动同步,自动修复。Go-fastdfs is a simple distributed file system (private cloud storage), with no center, high performance, high reliability, maintenance free and other advantages, support breakpoint continuation, block upload, small file merge, automatic synchronization, automatic repair.(similar fastdfs).

中文 English 愿景:为用户提供最简单、可靠、高效的分布式文件系统。 go-fastdfs是一个基于http协议的分布式文件系统,它基于大道至简的设计理念,一切从简设计,使得它的运维及扩展变得更加简单,它具有高性能、高可靠、无中心、免维护等优点。 大家担心的是这么简单的文件系统,靠不靠谱,可不

小张 3.2k Aug 1, 2022
CBuild build system - A tiny build system for C

cuild - CBuild A build system for C Building $ go build . Usage Similar to GNU Make, a file named "Cuildfile" is required. You have a few flags to us

Laurentino Luna 0 Jan 17, 2022
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

lni 4.3k Jul 27, 2022
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

lni 4.3k Jul 30, 2022
High-performance Redis-Server multi-threaded framework, based on rawepoll model.

RedHub High-performance Redis-Server multi-threaded framework, based on RawEpoll model. Ultra high performance Fully multi-threaded support Low CPU re

IceFireDB 59 Jun 8, 2022