goyek is used to create build pipelines in Go

Overview

goyek

Create build pipelines in Go

Go Reference Keep a Changelog GitHub Release go.mod LICENSE

Build Status Go Report Card codecov Mentioned in Awesome Go

Table of Contents:

Description

goyek is used to create build pipelines in Go. As opposed to many other tools, it is just a Go library.

Here are some good parts:

  • No binary installation is needed. Simply add it to go.mod like any other Go module.
    • You can be sure that everyone uses the same version of goyek.
  • It has low learning curve, thanks to the minimal API surface, documentation, and examples.
  • One can reuse code like in any Go application. It may be helpful to use packages like:
  • It is easy to debug, like a regular Go application.
  • The API is based on testing. The task's command look like a unit test. It is even possible to use testify for asserting.
  • Tasks and helpers can be easily tested. See exec_test.go.

goyek API is mainly inspired by the http, testing, and flag packages.

Please Star this repository if you find it valuable and worth maintaining.

Quick start

Copy and paste the following code into build/build.go:

package main

import "github.com/goyek/goyek"

func main() {
	flow := &goyek.Taskflow{}

	hello := flow.Register(taskHello())
	fmt := flow.Register(taskFmt())

	flow.Register(goyek.Task{
		Name:  "all",
		Usage: "build pipeline",
		Deps: goyek.Deps{
			hello,
			fmt,
		},
	})

	flow.Main()
}

func taskHello() goyek.Task {
	return goyek.Task{
		Name:  "hello",
		Usage: "demonstration",
		Command: func(tf *goyek.TF) {
			tf.Log("Hello world!")
		},
	}
}

func taskFmt() goyek.Task {
	return goyek.Task{
		Name:    "fmt",
		Usage:   "go fmt",
		Command: goyek.Exec("go", "fmt", "./..."),
	}
}

Run:

go mod tidy

Sample usage:

$ go run ./build -h
Usage: [flag(s) | task(s)]...
Flags:
  -v    Default: false    Verbose output: log all tasks as they are run. Also print all text from Log and Logf calls even if the task succeeds.
Tasks:
  all      build pipeline
  fmt      go fmt
  hello    demonstration
$ go run ./build all
ok     0.167s
$ go run ./build all -v
===== TASK  hello
Hello world!
----- PASS: hello (0.00s)
===== TASK  fmt
Cmd: go fmt ./...
----- PASS: fmt (0.18s)
ok      0.183s

Tired of writing go run ./build each time? Just add an alias to your shell. For example, add the line below to ~/.bash_aliases:

alias goyek='go run ./build'

Examples

Features

Task registration

The registered tasks are required to have a non-empty name, matching the regular expression ^[a-zA-Z0-9_][a-zA-Z0-9_-]*$, available as TaskNamePattern. This means the following are acceptable:

  • letters (a-z and A-Z)
  • digits (0-9)
  • underscore (_)
  • hyphens (-) - except at the beginning

A task with a given name can be only registered once.

A task without description is not listed in CLI usage.

Task command

Task command is a function which is executed when a task is executed. It is not required to to set a command. Not having a command is very handy when registering "pipelines".

Task dependencies

During task registration it is possible to add a dependency to an already registered task. When taskflow is processed, it makes sure that the dependency is executed before the current task is run. Take note that each task will be executed at most once.

Helpers for running programs

Use func Exec(name string, args ...string) func(*TF) to create a task's command which only runs a single program.

Use func (tf *TF) Cmd(name string, args ...string) *exec.Cmd if within a task's command function when you want to execute more programs or you need more granular control.

Verbose mode

Enable verbose output using the -v CLI flag. It works similar to go test -v. Verbose mode streams all logs to the output. If it is disabled, only logs from failed task are send to the output.

Use func (f *Taskflow) VerboseParam() BoolParam if you need to check if verbose mode was set within a task's command.

Default task

Default task can be assigned via the Taskflow.DefaultTask field.

When the default task is set, then it is run if no task is provided via CLI.

Parameters

The parameters can be set via CLI using the flag syntax.

On the CLI, flags can be set in the following ways:

  • -param simple - for simple single-word values
  • -param "value with blanks"
  • -param="value with blanks"
  • -param - setting boolean parameters implicitly to true

For example, go run ./build test -v -pkg ./... would run the test task with v bool parameter (verbose mode) set to true, and pkg string parameter set to "./...".

Parameters must first be registered via func (f *Taskflow) RegisterValueParam(newValue func() ParamValue, info ParamInfo) ValueParam, or one of the provided methods like RegisterStringParam.

The registered parameters are required to have a non-empty name, matching the regular expression ^[a-zA-Z0-9][a-zA-Z0-9_-]*$, available as ParamNamePattern. This means the following are acceptable:

  • letters (a-z and A-Z)
  • digits (0-9)
  • underscore (_) - except at the beginning
  • hyphens (-) - except at the beginning

After registration, tasks need to specify which parameters they will read. Do this by assigning the RegisteredParam instance from the registration result to the Task.Params field. If a task tries to retrieve the value from an unregistered parameter, the task will fail.

When registration is done, the task's command can retrieve the parameter value using the Get(*TF) method from the registration result instance during the task's Command execution.

See examples/parameters/main.go for a detailed example.

Taskflow will fail execution if there are unused parameters.

Supported Go versions

Minimal supported Go version is 1.11.

Alternatives

Make

While Make is currently the de facto standard, it has some pitfalls:

  • Requires to learn Make, which is not so easy.
  • It is hard to develop a Makefile which is truly cross-platform.
  • Debugging and testing Make targets is not fun.

However, if you know Make and are happy with it, do not change it. Make is very powerful and a lot of stuff can be made faster, if you know how to use it.

goyek is intended to be simpler and easier to learn, while still being able to handle most use cases.

Mage

Mage is a framework/tool which magically discovers the targets from magefiles, which results in some drawbacks:

  • Requires using build tags.
  • Reusing tasks is hacky.
  • Requires installation or using zero install option which is slow.
  • Debugging would be extermly complex.
  • Magical by design (of course one may like it).

goyek is intended to be a non-magical alternative for Mage. Write regular Go code. No build tags, special names for functions, tricky imports.

Task

While Task is simpler and easier to use than Make it still has similar problems:

  • Requires to learn Task's YAML structure and the minimalistic, cross-platform interpreter which it uses.
  • Debugging and testing tasks is not fun.
  • Hard to make reusable tasks.
  • Requires to "install" the tool.

Bazel

Bazel is a very sophisticated tool which is created to efficiently handle complex and long-running build pipelines. It requires the build target inputs and outputs to be fully specified.

goyek is just a simple library that is mainly supposed to create a build pipeline consisting of commands like go vet, go test, go build. However, take notice that goyek is a library. Nothing prevents you from, for example, using Mage's target package to make your build pipeline more efficient.

Presentations

Date Presentation Description
2021-05-05 goyek - Create build pipelines in Go goyek v0.3.0 demo
2021-03-10 taskflow - Create build pipelines in Go taskflow v0.1.1 demo
2020-12-14 Build pipeline for a Go project build pipeline using Make, Mage, and taskflow v0.1.0

Note: goyek was named taskflow before v0.3.0.

Contributing

We are open to any feedback and contribution.

You can find us on Gophers Slack in #goyek channel.

Feel free to create an issue, or a pull request.

You may use Gitpod to experiment with the codebase.

Comments
  • Rework output handling

    Rework output handling

    Why

    This PR is a proposal to better separate stdout/stderr usage. Primary motivation for this was the recommendation from clig.dev to send messages to stderr and have stdout only for net information from the tasks/tools. Since the change is an incompatible one, further, related incompatibilities are included, see below.

    What

    1. Expand Taskflow.Output as a separate type having two writer, one for Primary output, one for Message output. (incompatible API change)
    2. Goyek code was modified to only use Message output. (incompatible user-interface change) As a side-effect, any fmt.Fprintf error "handling" has been centralized and explicitly skipped.
    3. Removed the DefaultOutput global variable. It seems as if this was only used for the tests. Tests should override this explicitly anyway, to avoid side-effects. Furthermore, such global defaults are tricky; I refer to the potential problems as seen with http.Client and the transport defaults. (incompatible API change)

    Documentation/Changelog has not yet been updated, would do so if proposed changes are discussed and settled.

    Checklist

    • [ ] CHANGELOG.md is updated.
    • [ ] README.md is updated.
    • [x] The code changes follow Effective Go.
    opened by dertseha 12
  • Feature: registered parameters

    Feature: registered parameters

    This PR is the result of #37 .

    Thinking further, I figured that the registered parameters could be tracked by Go's built-in flags system. This handles usage output, as well as invalid parameter handling out of the box; As well as having less code to handle within this library.

    Example output of go run build\build.go -h (output shortened to highlight diff):

    Usage: [flag(s)] task(s)
    Flags:
      -ci string
            Whether CI is calling the build script (default "false")
      -v    Verbose output: log all tasks as they are run. Also print all text from Log and Logf calls even if the task succeeds.
    Tasks:
      ...
      diff        git diff; -ci
      ...
    

    This is a breaking change as Taskflow.Params is dropped and tasks need to specify which parameters they read (a change as intended).

    More thoughts on these parameters:

    • I wish the type information could be saved. As it is now, implementation of TFParams mirrors all the parsing what flag package has internally for the typed parameters. As it stands, the current setup does not support keeping the type information. Unless the type information of the parameter (string, int, ...) is carried over to a RegisteredParamString, RegisteredParamInt, and so forth - though this generates even more code.
    • Is there big of a need of a built-in support for text/json parsing of parameters? This sounds a bit overengineered to me.

    Most of the tests could carry over or needed only slight modification. Configure and MustConfigure aren't covered yet.

    How does this look?

    opened by dertseha 12
  • Add possibility to change the OOTB parameters

    Add possibility to change the OOTB parameters

    Why

    I wanted a method to set the default values for OOTB params which wasn't possible, because if you tried to register a verbose parameter again, goyek would panic because it was already registered.

    What

    I added a new method to create a flow (NewFlow) which accepts an Options-struct in which the defaults can be set.

    Checklist

    • [x] CHANGELOG.md is updated.
    • [x] README.md is updated.
    • [x] The code changes follow Effective Go.
    opened by arnovanliere 11
  • Use HTML comment syntax in the PR template

    Use HTML comment syntax in the PR template

    Why

    Why this pull request was created? Explain the problem that this pull request is trying to solve.

    The current pull request template:

    • requires to add new lines manually (if not the explanation is be added to the section comments):
      • write: write
      • preview: preview
    • doesn't inform what to do with the checklist items when the mentioned points are not applicable to the introduced changes

    What

    What does this pull request contain? Explain what is this pull request changing.

    Add comments to the pull request template so filling it is easier. The comment in the checklist informs that when no changes are required, the items should be checked (better visibility). The comments are not visible in the preview mode, so all new PRs will look the same as the older one.

    Checklist

    • [x] CHANGELOG.md is updated.
    • [x] README.md is updated.
    • [x] The code changes follow Effective Go.
    opened by agabrys 9
  • Feature: Registered parameters

    Feature: Registered parameters

    Continuation of #39 .

    Main points:

    • Parameters need to be registered in order to be used
    • Registered parameters are described in usage output
    • Parameters are provided uniformly via flags (either as --longform or, optionally, -l) -- see https://clig.dev for inspiration

    Note: This is an incompatible change, for both the CLI and the Go interface, in line with current v0 of Taskflow.

    opened by dertseha 9
  • Draft: Option to mark params as required

    Draft: Option to mark params as required

    Why

    Params could not be marked as required, so for every parameter you had to check whether it had a value or not inside the task handler. It seems like basic functionality for a task runner to be able to mark params as required.

    What

    It is a draft pull request to get feedback on the way I've set the validation up. I've made it so only the params which are used in the task which is being run, are checked whether they are required. All params which are required, but not set, are printed and the task fails.

    If a maintainer or other contributor thinks this is the right direction for such a feature, I will write tests, update the CHANGELOG etc, but I wanted to have feedback first.

    Checklist

    • [ ] CHANGELOG.md is updated.
    • [ ] README.md is updated.
    • [x] The code changes follow Effective Go.
    • [ ] The code changes are covered by tests.
    opened by arnovanliere 8
  • Alternative method signature for Exec / Simplified Error handling

    Alternative method signature for Exec / Simplified Error handling

    Hi,

    I became aware of this by the Golang Weekly newsletter and I like the idea of goyek. I started to play around with it and I would like to start a discussion about two potential changes:

    Signature change for goyek.Exec

    Change the signature for goyek.Exec from

    func Exec(name string, args ...string) func(*TF)
    

    to

    func Exec(cmdLine string) func(*TF)
    

    This is inspired by github.com/bitfield/script (see https://github.com/bitfield/script/blob/9847d19e69d72d73b191e1f3c65d5dba94605a1c/filters.go#L123).

    The main advantage of this change to me is, that it increases readability and it simplifies the process of copying back and forth of the shell commands between goyek and an actual shell (or a Makefile/Shell script one is migrating). By using the backticks for the string in Go, in most cases there would be no need to modify the command while copying back and forth.

    Error handling

    I could imagine, that a task often has a pattern like this:

    res, err := someFunc()
    if err != nil {
    	tf.Error(err)
    }
    ...
    

    Since goyek is somewhat inspired by testing.T (and its relatives), I though about how I nowadays handle errors in tests. I highly prefere the very simple https://github.com/matryer/is package, which provides me a method for error handling like this (https://pkg.go.dev/github.com/matryer/is#I.NoErr):

    is.NoErr(err)
    

    If the error is nil, the normal flow continues. If the error is non nil, the tests are ended with t.Fail().

    For goyek I could imagine two methods with the following signature:

    func (*goyek.TF).IfErr(err error, args ...interface{})
    func (*goyek.TF).IfErrf(err error, format string, args ...interface{})
    

    For the exact naming there are several options:

    • tf.IfErr
    • tf.IfError
    • tf.IsErr
    • tf.IsNoErr
    • ...

    Thanks again for your initiative, keep up the good work.

    documentation 
    opened by breml 8
  • Add TF.Run for running sub-tasks

    Add TF.Run for running sub-tasks

    Why

    Like testing package it would be very helpful to create something like sub-tasks.

    Example use case: run go test for each Go Module within a repository.

    How

    Implement TF.Run() method which works similarly to the one in the testing package.

    We can also dogfood it e.g. to run golangci-lint for all Go modules within this repository.

    We could also support running a concrete subtask e.g ./goyek Task/Sub1 (separate PR).

    We could also try to support running tasks and subtasks using regex/wildcards (separate PR). go test -run regex takes regex . If we decide to take the same approach we wrap the regex with ^ $.

    enhancement wontfix 
    opened by pellared 6
  • TF Log(f), Error(f), Fatal(f), Skip(f), methods add file and line information

    TF Log(f), Error(f), Fatal(f), Skip(f), methods add file and line information

    Why

    Like testing package it would be very helpful if the "printing" methods add file and line information.

    For example, it will help to find where exactly a task is failing.

    How

    1. The TF's Log(f), Error(f), Fatal(f), Skip(f) add file and line information
    2. Add TF.Helper() method which works like the one for testing package.

      Helper marks the calling function as a test helper function. When printing file and line information, that function will be skipped. Helper may be called simultaneously from multiple goroutines.

    Remarks:

    1. Printing via tf.Output() MUST NOT print file and line information
    2. Reference implementation: https://github.com/golang/go/blob/master/src/testing/testing.go

    We can consider also adding a field like Taskflow.DisableCodeline to provide a possibility to disable such functionality. However, I would rather not add it until someone really needs it. One can always use tf.Output() to print without file and line information.

    enhancement 
    opened by pellared 6
  • Change API

    Change API

    Why

    Partially addresses https://github.com/goyek/goyek/issues/62

    What

    • Rename Task.Command field to Action to avoid confusion with exec.Command and TF.Cmd.
    • Rename Taskflow type to Flow to make the name simpler.
    • Rename TF type to Progress to name the type more meaningful.
    • Remove DefaultOutput global variable to reduce the API surface.
    • Remove TF.Exec method to reduce the API surface.

    Checklist

    • [x] CHANGELOG.md is updated.
    • [x] README.md is updated.
    • [x] The code changes follow Effective Go.
    opened by pellared 6
  • Update wrapper scripts in README.md

    Update wrapper scripts in README.md

    Why

    Why this pull request was created? Explain the problem that this pull request is trying to solve.

    README.md has not up-to-date wrapper scripts.

    What

    What does this pull request contain? Explain what is this pull request changing.

    Update wrapper scripts in README.md.

    Checklist

    • [x] CHANGELOG.md is updated.
    • [x] README.md is updated.
    • [x] The code changes follow Effective Go.
    opened by pellared 6
  • Allow running tasks concurrently

    Allow running tasks concurrently

    Why

    I'm looking at goyek as an alternative to Mage - thanks for open sourcing this, so far I am loving the API as it feels very similar to the built in testing library and to libraries like urfav/cli.

    Something that I liked in Mage is that it was very easy to run tasks concurrently. For example, if task all depends on tasks a and b, tasks a and b would be executed concurrently in separate Goroutines. I tried modelling this in goyek as follows:

    package main
    
    import (
    	"time"
    
    	"github.com/goyek/goyek"
    )
    
    func main() {
    	flow := &goyek.Flow{}
    
    	a := flow.Register(goyek.Task{
    		Name: "a",
    		Action: func(tf *goyek.TF) {
    			time.Sleep(time.Second * 1)
    			tf.Log("a")
    			time.Sleep(time.Second * 1)
    			tf.Log("a")
    			time.Sleep(time.Second * 1)
    			tf.Log("a")
    		},
    	})
    
    	b := flow.Register(goyek.Task{
    		Name: "b",
    		Action: func(tf *goyek.TF) {
    			time.Sleep(time.Second * 1)
    			tf.Log("b")
    			time.Sleep(time.Second * 1)
    			tf.Log("b")
    			time.Sleep(time.Second * 1)
    			tf.Log("b")
    		},
    	})
    
    	flow.Register(goyek.Task{
    		Name: "all",
    		Deps: goyek.Deps{a, b},
    	})
    
    	flow.Main()
    }
    

    It appears that the tasks run sequentially, even though a and b don't depend on one another.

    go run ./build all -v
    ===== TASK  a
    a
    a
    a
    ----- PASS: a (3.00s)
    ===== TASK  b
    b
    b
    b
    ----- PASS: b (3.00s)
    ok      6.006s
    

    What

    Would you be open to a contribution to support running tasks concurrently?

    Some approaches to the API could be to add a flag at the Flow level:

    flow := &goyek.Flow{
    	Concurrent: true,
    }
    
    or to allow concurrency to be specified at a task level:
    
    ```go
    flow.Register(goyek.Task{
    	Name: "all",
    	Deps: goyek.ConcurrentDeps{a, b},
    })
    
    enhancement help wanted 
    opened by chrnorm 6
Releases(v2.0.0-rc.12)
  • v2.0.0-rc.12(Nov 24, 2022)

  • v2.0.0-rc.11(Nov 14, 2022)

  • v2.0.0-rc.10(Nov 7, 2022)

    This release adds helpers methods for type A that are commonly used in the testing package.

    Added

    • Add A.Cleanup method that registers an action's cleanup function.
    • Add A.Setenv method that sets the environment variable and reverts the previous value during cleanup.
    • Add A.TempDir method that creates a temporary directory and removes it during cleanup.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.9(Nov 6, 2022)

    This release introduces a breaking change as it renames the heavily used TF type to A. This follows the convention used in testing package to name the parameter type as the first letter of the "function type".

    Added

    • Add Status.String method for printing Status.

    Changed

    • Rename TF type to A as this is the Action parameter.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.8(Oct 29, 2022)

    Added

    • Make logging more customizable. If Logger implements Error, Errorf, Fatal, Fatalf, Skip, Skipf then they will be used by the TF equivalent methods.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.7(Oct 29, 2022)

    This release has all the features and changes planned for v2. The v2.0.0 is to be released in two months to give the users some time for feedback.

    Added

    • Add TF.Helper and CodeLineLogger.Helper methods that work like the equivalent method in the testing package.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.6(Oct 27, 2022)

    Added

    • Add NoDeps option to skip processing of all dependencies.
    • Add Skip option to skip processing of given tasks.

    Changed

    • Report NOOP status if the task's action was nil.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.5(Oct 25, 2022)

  • v2.0.0-rc.4(Oct 25, 2022)

    This release focuses on improving the API to make creating and customizing reusable build pipelines easier.

    Added

    • Add DefinedTask.SetName, DefinedTask.SetUsage, DefinedTask.Action, DefinedTask.SetAction, DefinedTask.SetAction, DefinedTask.SetDeps to enable modifying the task after the initial definition.
    • Add Flow.Undefine to unregister a task.
    • Passing nil to Flow.SetDefault unsets the default task.
    • Add DryRun, ReportLongRun middlewares.

    Changed

    • DefinedTask.Deps returns Deps to facilitate reusing defined task's dependencies when creating a new one or redefining existing one.
    • Rename Reporter middleware to ReportStatus.
    • Change Flow.Execute to accept []string instead of ...string to make the API forward compatible.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.3(Oct 19, 2022)

    This release focuses on improving usability, extensibility and customization.

    Added

    • Add Flow.SetLogger for setting a custom log decorator that is used by TF logging methods.
    • Add Flow.Logger for getting the log decorator (CodeLineLogger by default).
    • Add CodeLineLogger which is the default for Flow.Logger.
    • Add FmtLogger which is the default when using NewRunner.
    • Add NOOP status report for tasks that were intentionally not run to differentiate from being skipped during execution.
    • Add Flow.Use method o support task run interception using middlewares.
    • Add middleware package with Reporter and SilentNonFailed middlewares.
    • TF.Error, TF.Errorf, TF.Fail may be called simultaneously from multiple goroutines.
    • Add DefaultFlow that is the default flow.
    • Add the top-level functions such as Define, Main, and so on which are wrappers for the methods of Flow called for DefaultFlow.

    Changed

    • Usually, the task does not call panic directly. panic failure message no longer contains a prefix with file and line information. The stack trace is printed instead. The behavior is based on testing package.
    • Flow.Main changes the working directory to parent.
    • Rename Flow.Run to Flow.Execute to reduce possible confusion with Runner.
    • Report PASS for a task without an action.
    • Task status reporting is disabled by default. It can be enabled by calling Flow.Use(middleware.Reporter).
    • Flow.Print output format is similar to flag.PrintDefaults. Moreover, it does not print tasks with empty Task.Usage.
    • Change Flow.Execute to return an error instead of returning the exit code and printing to output.
    • Change Flow.Output field to Flow.SetOutput setter and Flow.Output getter.
    • Change Flow.Usage field to Flow.SetUsage setter and Flow.Usage getter.

    Removed

    • Flow.Verbose is removed. To be non-verbose use Flow.Use(middleware.SilentNonFailed) instead.

    Fixed

    • Fix panic handling so that panic(nil) and runtime.Goexit() now cause task failure.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.2(Oct 14, 2022)

    This release focuses on improving usability and encapsulation.

    Added

    • Add Flow.Usage for setting a custom parsing tasks error handler.

    Changed

    • Flow.Usage (or Flow.Print if it is nil) is called when Flow.Run returns CodeInvalidArgs.
    • Rename Flow.Register to Flow.Define.
    • Change Flow.RegisteredTask to sealed interface Flow.DefinedTask.
    • Change the Flow.DefaultTask field to Flow.SetDefault and Flow.Default methods.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-rc.1(Oct 14, 2022)

    This release contains breaking changes in the Go API. It focuses on improving customization mainly by removing the parameters API. It gives the user the possibility of parsing the input.

    Added

    • Add Flow.Verbose for controlling the verbose mode.
    • Add Flow.Print for printing the flow usage.
    • Flow.Main now exits on receiving the second SIGINT.

    Changed

    • RegisteredTask.Deps returns []string (dependency names) for easier introspection.

    Removed

    • Remove parameters API and out-of-the-box flags (-v, -wd).
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Oct 12, 2022)

    This release focuses on improving output printing. There are no API changes.

    Added

    • The TF methods Log[f], Error[f], Fatal[f], Skip[f], panic prints indented text with a prefix containing file and line information.

    Fixed

    • Remove TF.Cmd undocumented behavior (printing command name and arguments).
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Sep 8, 2022)

  • v0.6.3(Feb 23, 2022)

    Added

    • TaskNamePattern and ParamNamePattern has been changed so that the plus (+) and colon (:) characters can be used in the task and parameter name (but not as the first character).
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Jan 25, 2022)

  • v0.6.1(Dec 27, 2021)

    This release adds the possibility to change the defaults of the global parameters.

    Added

    • Add RegisterVerboseParam and RegisterWorkDirParam to overwrite default values for verbose and work dir parameters.
    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(Aug 2, 2021)

    This release contains multiple breaking changes in the Go API. It is supposed to make it cleaner.

    Added

    • Tf.Cmd also sets Stdin to os.Stdin.
    • Add Flow.Tasks and Flow.Params to allow introspection of the registered tasks and parameters.

    Changed

    • Rename Taskflow type to Flow to simplify the naming.
    • Rename Task.Command field to Action to avoid confusion with exec.Command and TF.Cmd.

    Removed

    • Remove DefaultOutput global variable.
    • Remove TF.Exec method.
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Jun 21, 2021)

    Added

    • Add the stack trace when a task panics.

    Changed

    • Names of parameters now have to follow goyek.ParamNamePattern. They were allowed to start with an underscore (_), and now no longer are.
    • The PowerShell wrapper scripts goyek.ps1 better handles stderr redirection.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 26, 2021)

    Added

    • Add Bash and PowerShell wrapper scripts.
    • Add -wd global parameter allowing to change the working directory. The new Taskflow.WorkDirParam method can be used to get its value in a task's command.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(May 3, 2021)

    The repository has migrated to https://github.com/goyek/goyek.

    This release contains multiple breaking changes for both the CLI and the Go API.

    The biggest change is a redesign of the parameters API, so they have to be explicitly registered. It makes the usage of the parameters more controlled and provides a better help output. Moreover, the parameters are set via CLI using the flag syntax.

    Added

    • Help is printed when -h, --help or help is passed.
    • Help contains parameters' information.
    • The tasks and parameters can be passed via CLI in any order.
    • Taskflow.Run handles nil passed as context.Context argument.
    • Taskflow.Run panics when a registered parameter is not assigned to any task.

    Changed

    • Module path changed to github.com/goyek/goyek.
    • Rename package task to goyek.
    • Use the flag syntax for setting parameters via CLI.
    • Rename Task.Description field to Usage.
    • Rename Task.Dependencies field to Deps.
    • Rename CodeFailure constant to CodeFail.
    • Rename Taskflow.MustRegister method to Register and remove previous Taskflow.Register implementation.
    • Remove Taskflow.Params field and TF.Params method, add Taskflow.Register*Param methods and Task.Params field instead.
    • Remove TF.Verbose, add Taskflow.VerboseParam instead.
    • Remove New function, create instance using &Taskflow{} instead.
    • Unexport Runner type, use Taskflow in tests instead.
    • Enforce patterns for task names (TaskNamePattern) and parameter names (ParamNamePattern).
    • Drop official support for Go 1.10.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Mar 14, 2021)

  • v0.1.1(Feb 28, 2021)

  • v0.1.0(Jan 14, 2021)

Owner
null
A library to help you create pipelines in Golang

pipeline Pipeline is a go library that helps you build pipelines without worrying about channel management and concurrency. It contains common fan-in

Delivery Hero SE 142 Dec 13, 2022
Build powerful pipelines in any programming language.

Gaia is an open source automation platform which makes it easy and fun to build powerful pipelines in any programming language. Based on HashiCorp's g

Gaia 4.9k Jan 3, 2023
Pipelines using goroutines

pipeline This package provides a simplistic implementation of Go pipelines as outlined in Go Concurrency Patterns: Pipelines and cancellation. Docs Go

Nikhil Mungel 45 Oct 5, 2022
elPrep: a high-performance tool for analyzing sequence alignment/map files in sequencing pipelines.

Overview elPrep is a high-performance tool for analyzing .sam/.bam files (up to and including variant calling) in sequencing pipelines. The key advant

null 261 Nov 2, 2022
Robust, flexible and resource-efficient pipelines using Go and the commandline

Robust, flexible and resource-efficient pipelines using Go and the commandline Project links: Documentation & Main Website | Issue Tracker | Chat Why

SciPipe 972 Dec 25, 2022
🐺 Deploy Databases and Services Easily for Development and Testing Pipelines.

Peanut provides an API and a command line tool to deploy and configure the commonly used services like databases, message brokers, graphing tools ... etc. It perfectly suited for development, manual testing, automated testing pipelines where mocking is not possible and test drives.

Ahmed 690 Jan 3, 2023
🏯 Monitor your (gitlab/github) CI/CD pipelines via command line interface with fortress

__ _ / _| | | | |_ ___ _ __| |_ _ __ ___ ___ ___ | _/ _ \| '__| __| '__/ _ \/ __/ _

MrJosh 6 Mar 31, 2022
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Optimus Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality

Open Data Platform 680 Jan 6, 2023
Dud is a lightweight tool for versioning data alongside source code and building data pipelines.

Dud Website | Install | Getting Started | Source Code Dud is a lightweight tool for versioning data alongside source code and building data pipelines.

Kevin Hanselman 121 Jan 1, 2023
GitOops is a tool to help attackers and defenders identify lateral movement and privilege escalation paths in GitHub organizations by abusing CI/CD pipelines and GitHub access controls.

GitOops is a tool to help attackers and defenders identify lateral movement and privilege escalation paths in GitHub organizations by abusing CI/CD pipelines and GitHub access controls.

OVO Technology 601 Jan 2, 2023
A simple CLI tool to use the _simulate API of elasticsearch to quickly test pipelines

elasticsearch-pipeline-tester A simple CLI tool to use the _simulate API of elasticsearch to quickly test pipelines usage: pipelinetester [<flags>] <p

Mario Castro 1 Oct 19, 2021
Drone plugin to skip pipelines based on changed files

drone-skip-pipeline Drone plugin to skip pipelines based on changed files. Build Build the binary with the following command: export GOOS=linux export

ownCloud CI 1 Aug 7, 2022
tfa is a 2fa cli tool that aims to help you to generate 2fa code on CI/CD pipelines.

tfa tfa is 2fa cli tool that aim to help you to generate 2fa code on CI/CD pipelines. You can provide secret with stdin or flag. Install brew install

Kaan Karakaya 28 Nov 27, 2022
Application to learn and demo Tekton pipelines

Tekton sample Application to learn and demo Tekton pipelines Building $ go test ./pkg/api && go build Running it locally $ podman-compose up --force-r

Sergi Jimenez 1 Oct 28, 2021
A small web dashboard with stats for all pipelines of Buildkite organization.

Buildkite Stats A small Buildkite dashboard useful to prioritize which pipelines a Buildkite organization is waiting the most on. Noteworthy details:

Jens Rantil 1 Apr 25, 2022
Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines.

Dataflow Summary Dataflow is a Kubernetes-native platform for executing large parallel data-processing pipelines. Each pipeline is specified as a Kube

argoproj-labs 267 Jan 4, 2023
Run Buildkite Pipelines locally

bkl - Run Buildkite Pipelines locally Run buildkite pipelines locally (no buildkite.com required). Installing Either download a release binary or if y

Lachlan Donald 16 Aug 23, 2022
K3ai Executor is the runner pod to execute the "one-click" pipelines

Welcome to K3ai Project K3ai is a lightweight tool to get an AI Infrastructure Stack up in minutes not days. NOTE on the K3ai origins Original K3ai Pr

k3ai 0 Nov 11, 2021
🤖 DroneCI plugin to skip pipelines based on files changes

DroneCI Skip Pipeline ?? DroneCI plugin to skip pipelines based on files changes Motivations This DroneCI plugin enables you skip (or short-circuit) a

Josh Komoroske 4 Dec 14, 2022
Just a playground with some interesting concepts like pipelines aka middleware, handleFuncs, request validations etc. Check it out.

Pipeline a.k.a middleware in Go Just a playground with some interesting concepts like pipelines aka middleware, handleFuncs, request validations etc.

null 0 Dec 9, 2021