Hi,

Request

Add the possibility to use import or includes to imports some others tasks. With the default strategy as namespace.

Example

File: docker/v1.0/Taskfile.yaml

version: "2"

task:
  build:
    cmds:
      - docker build...

File: go/v1.0/Taskfile.yaml

version: "2"

task:
  build:
    cmds:
      - go build ...

File (in my project): Taskfile.yaml

imports:
  docker: https://.../docker/v1.0/Taskfile.yaml
  go: file://.../go/v1.0/Taskfile.yaml

— or —

includes:
  docker: https://.../docker/v1.0/Taskfile.yaml
  go: file://.../go/v1.0/Taskfile.yaml

task:
  build:
    - task: "docker:build"
    - task: "go:build"

How to use?

task go:build go:docker

— or —

task build

Finally

What do you think?

UPDATE: This is a general feature request to not run equal dependencies more than once. E.g. if task A and task B both depend on taskC, running both task A and task B in parallel (witin one instance of task), should be able to resolve the dependencies so that taskC is only run once, and this is the interesting port, if it's equal, which we will get to next.

what makes a task equal?

Since you can pass variables to tasks, it's not enough to just look at the name. The best option is probably to rely on a hash of the name + sorted and resolved input variables. A close second could be to rely on unresolved but still sorted input variables, which is not perfect, but could be an acceptable trade-off if it's somehow easier in code.

Original description

I have multiple tasks in my Taskfile that depend on a single task called go-dep that runs the command glide install. When I run task to build multiple targets, I would expect go-dep to be called at most 1 time (zero if it's found to be up-to-date). Instead, I see it's called once for each task that depend on it:

$ task
task: No argument given, trying default task
glide install
glide install
glide install
glide install

PS! go-deps in this case is called with no vars, but the rule should really apply to run a dep at most 1 time when the vars are exactly the same.

This is a general issue to track proposals with small or big breaking changes, that could be released in the next major version.

This list is a work in progress and can change at any time.

Development is happening on the v3 branch.

Cleanup

• [x] Change module name from github.com/go-task/task/v2 to v3
• [x] Remove all code related support of version 1 (#237)
• [x] Remove automatic inclusion of Taskvars.yml and Taskfile_{{OS}}.yml. This was done before we had a generic inclusion of Taskfiles implemented. We should add options to
  • [ ] Don't include the file if it doesn't exist
  • [x] Allow interpolation on the file name to make it possible to include a OS specific file
• [x] Finally delete the vendor directory? (#214)

Breaking changes

• [x] Added global method: option. Default changed from timestamp to checksum (#246)
• [x] Refactor vars and templates to reduce the number of bugs and undesired behavior (#218)
• [ ] Make CLI arguments override everything (#181) (EDIT: What about #336?)
• [ ] Make up-to-date resolution work when only "generates" is given (#187)
• [ ] Don't run the same dependency (with the same args) more than once by default (#53, #193)
• [ ] Allow forcing specific tasks (#189)

New stuff

• [x] Colored output messages (#207)
• [x] Expose .TIMESTAMP and .CHECKSUM to status: (#216)
• [x] Allow shorter syntax for tasks with default configuration (#194)
• [ ] Allow use as a library (#121)

Documentation

• [x] Mention changes related to method: (shouldn't also check the main guide)
• [x] Update "Taskfile versions" article

There are more issues and feature request that could be added to this release.

For some startup tasks or cleanup paths commands could run in parallel while others should be run in order. Add a Parallel property to task to allow switching between modes

Hi everybody 👋

Once in a while, people ask me why the development is slow, questions got unanswered, bug reports are not responded, features not added, PRs not reviewed, etc.

The answer is simple: the project has grown to become bigger than I ever expected. With a full time job and a life outside work, I have limited time to dedicate to this project. More demand is generated from users than I could keep up even if I doubled the time I dedicate to this project.

So first of all I'd like to ask for your patience and understanding.

Secondly, I'd like to hear opinions and ideas on how to keep the project moving.

And lastly, I'ld like to ask for some help. I had two other people with write access to the repo in the past, but they quickly lost interest in the project. Other than that short period, I've being alone. You could help:

1. Answering questions, validating bug reports, giving opinions on feature requests, closing issues/PRs when they do not make sense, etc.
2. Writing code and reviewing pull requests, specially bug fixes and small features. (I'd still like to have the final word on bigger features as sometimes people suggest changes that breaks backward compatibility or add more complexity than desired)
3. Some people have helped the project by contributing to mvdan/sh as well, and I'm thankful for that. Help on mvdan/sh#93 in particular would hugely benefit Windows users if you're interested to help.

Adding a --summary flag for task to get more a summary of the task

See #107

New field in task called summary

version: '2'
tasks:
  release:
    deps: [build]
    summary: |
      Release your project to github

      It will build your project before starting the release it.
      Please make sure that you have set GITHUB_TOKEN before starting.
    cmds:
      - your-release-tool
  build:
    cmds:
      - your-build-tool

Output for task --summary release:

task: release

Release your project to github

It will build your project before starting the release it.
Please make sure that you have set GITHUB_TOKEN before starting.

dependencies:
 - build

commands:
 - your-release-tool

Behaviour:

• --summary will not execute the task
• --summary will display the summary of a task (see above)
• --summary will use description as fallback, if summary does not exist
• --summary will output a warning (or information), if no description/summary is present
• --summary will output the dependencies if there are any
• --summary will output the commands if there are any

Tests:

• Because there are a lot of edge-cases I decided to extract out a package (summary), which is completly unit tested with above named edge cases.
• There is an integration like test in task_test.go

Documentation is present for that feature

Does go-task currently offer the option of "passthrough" arguments? For example, if I have currently have a task to run the AWS CLI out of a Docker container:

tasks:
  aws:
    cmds:
      - docker-compose run --rm aws		

Is it possible to have the ability for a command like task aws -- ls s3://my-bucket to be translated to docker-compose run --rm aws ls s3://my-bucket, meaning task will pass through anything after -- to the underlying shell command?

This is a more convenient way to pass extra variables compared to the current approach: task write-file FILE=file.txt "CONTENT=Hello, World!". Currently, using this approach allows me to write task aws CMD="ls s3://my-bucket" but the extra variable definition and quotes creates a lot of friction that I think can be alleviated with my proposal.

If this is not possible, it would be a really nice capability to have in the future.

This is my first proposal to add includeable Taskfiles over http.

The config looks like the following:

includes: 
  moo: 
    path: https://raw.githubusercontent.com/go-task/task/master/Taskfile.yml
    cache: 30s
  foo: https://raw.githubusercontent.com/tonobo/task/master/Taskfile.yml
  hui: 
    path: ./
    flush_includes: yes

References: #98

• Task version: 2.5.1
• OS: Windows 10 Professional (64-bit)
• Example Taskfile showing the issue:

# https://taskfile.dev

version: '2'

tasks:

  a:
    cmds:
      - echo "Task a"
      - echo $PATH

  b:
    cmds:
      - echo "Task b"
      - echo "{{.PATH}}"

  c:
    env:
      PATH: tools
    cmds:
      - echo "Task c"
      - echo $PATH

  d:
    env:
      PATH: tools
    cmds:
      - echo "Task d"
      - echo "{{.PATH}}"

  e:
    env:
      PATH:
        sh: echo "tools;$PATH"
    cmds:
      - echo "Task e"
      - echo $PATH

  f:
    env:
      PATH:
        sh: echo "./tools;$PATH"
    cmds:
      - echo "Task f"
      - echo $PATH

  g:
    env:
      PATH:
        sh: echo "$(realpath ./tools);$PATH"
    cmds:
      - echo "Task g"
      - echo $PATH

Hello. Firstly, thank-you for Task. It has, overall, made my development life easier and more pleasant.

I don't know if the problem I have is a bug or if I simply don't understand the correct way to achieve my goal. I'd like to be able to modify the environment variable PATH (either prepending or appending a project-specific directory to the existing value), but I can't figure out how to do this using the set of commands available within Taskfiles.

To provide context, my Taskfile is for use in building go applications. Some applications (projects) require project-specific build tools/helpers that I place in a top-level tools subdirectory (i.e., <project name>/tools). Note that my Taskfile resides at <project name>/Taskfile.yml. I wish to use the go generate command, which effectively requires that external generation tools be accessible via PATH, so I must add <project name>/tools to PATH before running go generate in the task.

While I'm working in Windows, I'm making effective use of busybox-w32 to provide a fairly comprehensive set of what are otherwise Linux tools, so things like cp, echo and realpath are available. I've also tried this via WSL such that things are more like a real Linux environment, with the same results.

The output I get from the above Taskfile (when run under sh or bash via WSL) is as follows (results in native Windows are the same):

$ task a; task b; task c; task d; task e; task f; task g                                             
echo "Task a"                                                                                        
Task a                                                                                               
echo $PATH                                                                                           
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games             
echo "Task b"                                                                                        
Task b                                                                                               
echo "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"      
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games             
echo "Task c"                                                                                        
Task c                                                                                               
echo $PATH                                                                                           
tools                                                                                                
echo "Task d"                                                                                        
Task d                                                                                               
echo "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"      
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games             
echo "Task e"                                                                                        
Task e                                                                                               
echo $PATH                                                                                           
tools;/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games       
echo "Task f"                                                                                        
Task f                                                                                               
echo $PATH                                                                                           
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games             
echo "Task g"                                                                                        
Task g                                                                                               
echo $PATH                                                                                           
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

Tasks a, b, c and d are just simple tests; it's tasks e, f and g that illustrate the real issue. Task e works correctly, prepending tools to the current PATH, but neither f nor g have any effect. Task g is the closest to what I need in terms of operations (i.e., adding the absolute path of a project-specific directory).

I'd greatly appreciate any input you could provide on this issue. Thanks in advance.

• Task version: v3.7.0 (h1:8dDFv12/Lxi6PHkYNIIm7N6v6oRGxePpLK67LuMh/Rs=)

• Operating System: Linux 5.13.10-arch1-1 #1 SMP PREEMPT Thu, 12 Aug 2021 21:59:14 +0000 x86_64 GNU/Linux

Example Taskfile showing the issue

version: '3'

silent: false

tasks:
  check:
    desc: Generates proto with prepared docker-image
    cmds:
      - echo $GID
      - echo $UID

check it:

✗ task check
task: [check] echo $GID

task: [check] echo $UID
1000
✗ echo $GID
1000
✗ echo $UID
1000

hello @andreynering, the fix is quite simple :-D, what is complicated and worthwhile to spend some time to understand is the first commit a496a75, that introduces extensive tests to reproduce the behavior.

I realized that to fix this problem, it is not possible to tweak the timeout of mvdan.cc/sh, we must be more drastic and don't pass to it any cancellable context. The problem that I observed is the following (reproduced in the tests):

• As explained in #458, when hitting CTRL-C, the terminal driver sends a SIGINT to the foreground process group, which is composed of Task and of all the subprocesses that Task has spawned via mvdan.cc/sh.
• Task receives the signal and forwards it to mvdan.cc/sh, which immediately sends it to the subprocess.
• The result is that the subprocess receives always 2 signals: one from the process group, and one from Task.
• Depending on how the program of the subprocess deals with interrupts, this can be unnoticeable or cause immediate termination, which is the case of Terraform.
• Terraform on reception of the first signals enters a cleanup phase, but if it receives a second signal, it will terminate immediately the cleanup, leaving a broken state.
• Due to the signal amplification explained above, Terraform always receives two signals when driven from Task...

I suggest to checkout a496a75, run

task install && go test -run '^TestSignalSentToProcessGroup' -count=1 -v=1

and get acquainted with the (expected) failures.

Then, checkout the tip of the branch and run the same tests, to validate that everything is fixed.

There are some things that are still missing: testing what happens on Windows and deciding how far propagate in the Task functions the fact that the context should not even passed. I mean: func RunCommand(ctx context.Context, opts *RunCommandOptions) is now lying, since the context will not be passed to anything.

As a reference, here are the failing tests on Linux:

task install && go test -run '^TestSignalSentToProcessGroup' -count=1 -v=1
task: [install] go install -v -ldflags="-w -s -X main.version=a496a75b05" ./cmd/task
github.com/go-task/task/v3/internal/execext
=== RUN   TestSignalSentToProcessGroup
=== RUN   TestSignalSentToProcessGroup/child_simulates_terraform:_receives_1_sigint_and_does_cleanup
    unix_test.go:174: 
        wanted but not found:
        [sleepit: cleanup done
         task: Failed to run task "default": exit status 3
        ]
    unix_test.go:179: 
        unwanted but found:
        [sleepit: got signal=interrupt count=2
         sleepit: cleanup canceled
         task: Failed to run task "default": exit status 4
        ]
    unix_test.go:183: 
        output:
        [task: [default] /home/mmolteni/opensource/go-task/tmp/sleepit handle -term-after=2 -sleep=10s -cleanup=50ms
         sleepit: ready
         sleepit: PID=3713003 sleep=10s cleanup=50ms
         sleepit: work started
         task: signal received: interrupt
         sleepit: got signal=interrupt count=1
         sleepit: got signal=interrupt count=2
         sleepit: work canceled
         sleepit: cleanup started
         sleepit: cleanup canceled
         task: Failed to run task "default": exit status 4
         ]
=== RUN   TestSignalSentToProcessGroup/child_does_not_handle_sigint:_receives_sigint_and_terminates_immediately
    unix_test.go:174: 
        wanted but not found:
        [task: Failed to run task "default": exit status 130
        ]
    unix_test.go:179: 
        unwanted but found:
        [task: Failed to run task "default": context canceled
        ]
    unix_test.go:183: 
        output:
        [task: [default] /home/mmolteni/opensource/go-task/tmp/sleepit default -sleep=10s
         sleepit: ready
         sleepit: PID=3713018 sleep=10s cleanup=0s
         sleepit: work started
         task: signal received: interrupt
         task: Failed to run task "default": context canceled
         ]
=== RUN   TestSignalSentToProcessGroup/child_intercepts_sigint:_receives_sigint_and_does_cleanup
    unix_test.go:174: 
        wanted but not found:
        [sleepit: cleanup done
         task: Failed to run task "default": exit status 3
        ]
    unix_test.go:179: 
        unwanted but found:
        [sleepit: got signal=interrupt count=2
         task: Failed to run task "default": context canceled
        ]
    unix_test.go:183: 
        output:
        [task: [default] /home/mmolteni/opensource/go-task/tmp/sleepit handle -sleep=10s -cleanup=2s
         sleepit: ready
         sleepit: PID=3713030 sleep=10s cleanup=2s
         sleepit: work started
         sleepit: got signal=interrupt count=1
         task: signal received: interrupt
         sleepit: work canceled
         sleepit: got signal=interrupt count=2
         sleepit: cleanup started
         task: Failed to run task "default": context canceled
         ]
--- FAIL: TestSignalSentToProcessGroup (2.13s)
    --- FAIL: TestSignalSentToProcessGroup/child_simulates_terraform:_receives_1_sigint_and_does_cleanup (0.11s)
    --- FAIL: TestSignalSentToProcessGroup/child_does_not_handle_sigint:_receives_sigint_and_terminates_immediately (0.01s)
    --- FAIL: TestSignalSentToProcessGroup/child_intercepts_sigint:_receives_sigint_and_does_cleanup (2.01s)
FAIL
exit status 1
FAIL    github.com/go-task/task/v3      2.136s

Hi,

The files in the checksum directory are created with user-write-permissions - https://github.com/go-task/task/blob/9eda1629bb72459580a156d52fa5f3dd15d93b7b/internal/status/checksum.go#L47

This prevents other users from run task later in that directory.

Would be helpful if it could be changed - either by default, by parameter, or by '--no-checksum' argument.

Thanks

Bumps github.com/stretchr/testify from 1.7.2 to 1.7.5.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Current there is no way to disable this message

silent: true can't disable this message and also I'd like to keep the output of what command are actually executing.

Just like make (without --debug) wouuld do, only show commands but not which make target is matched.

It took me some time to discover why the output of some commands was very confusing as the order of the console lines were not in order.

That is happening because when run using grouping both stdout and stderr are captures separated and printed separated, one after another. As you can imagine that can cause very confusing console output for script. Easy to see especially when running bash scripts with set -x.

It would be preferable if the subprocess would use line-buffering and print lines in order they were generated, making read of output more human friendly.

• Task version: Task versionv3.12.1
• Operating System: Linux

I have encountered an inconsistency in vars of included taskfiles and need some assistance. The setup I have looks like this. There is a common-include taskfile and a package1 taskfile. The parent taskfile includes both taskfiles. The package1 also includes common-include. The problem arises with the variables that are passed. From parent to common-include everything works fine but from package1 to common-include the passed variable is simply ignored.

Example Taskfile showing the issue

❯ tree .
.
├── include
│   └── Taskfile.include.yaml
├── package1
│   └── Taskfile.package1.yaml
└── Taskfile.yaml

2 directories, 3 files

./Taskfile.yaml

version: '3'

includes:
  included:
    taskfile: ./include/Taskfile.include.yaml
    vars:
      VAR_1: provided-from-parent-var1
  package1:
    taskfile: ./package1/Taskfile.package1.yaml

tasks:
  default:
    cmds:
      - task: included:task1

./package1/Taskfile.package1.yaml

version: '3'

includes:
  included:
    taskfile: ../include/Taskfile.include.yaml
    vars:
      VAR_1: provided-from-package1-var1

tasks:
  default:
    cmds:
      - task: included:task1

./include/Taskfile.include.yaml

version: "3"

vars:
  VAR_1: '{{.VAR_1 | default "included-default-var1"}}'

tasks:
  task1:
    cmds:
      - echo "VAR_1 is {{.VAR_1}}"

❯ go-task
task: [included:task1] echo "VAR_1 is provided-from-parent-var1"
VAR_1 is provided-from-parent-var1

❯ go-task package1:default
task: [package1:included:task1] echo "VAR_1 is included-default-var1"
VAR_1 is included-default-var1

By executing go-task the default task from parent is executed and the VAR_1 has the provided value. By executing go-task package1:default the default task from package1 is executed and the VAR_1 does not have the provided value provided-from-package1-var1 but the default value from the common-include included-default-var1.