Gohack: mutable checkouts of Go module dependencies

Overview

Gohack: mutable checkouts of Go module dependencies

The new Go module system is awesome. It ensures repeatable, deterministic builds of Go code. External module code is cached locally in a read-only directory, which is great for reproducibility. But if you're used to the global mutable namespace that is $GOPATH, there's an obvious question: what if I'm hacking on my program and I want to change one of those external modules?

You might want to put a sneaky log.Printf statement to find out how some internal data structure works, or perhaps try out a bug fix to see if it solves your latest problem. But since all those external modules are in read-only directories, it's hard to change them. And you really don't want to change them anyway, because that will break the integrity checking that the Go tool does when building.

Luckily the modules system provides a way around this: you can add a replace statement to the go.mod file which substitutes the contents of a directory holding a module for the readonly cached copy. You can of course do this manually, but gohack aims to make this process pain-free.

Install gohack with

go get github.com/rogpeppe/gohack

or use gobin:

gobin github.com/rogpeppe/gohack

For quick edits to a module (without version control information)

If the module to edit is example.com/foo/bar, run:

gohack get example.com/foo/bar

This will make a copy of the module into $HOME/gohack/example.com/foo/bar and add replace directives to the local go.mod file:

replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar

Note: This copy will not include version control system information so it is best for quick edits that aren't intended to land back into version control.

To edit the module with full version control

Run:

gohack get -vcs example.com/foo/bar

This will clone the module's repository to $HOME/gohack/example.com/foo/bar, check out the correct version of the source code there and add the replace directive into the local go.mod file.

Undoing replacements

Once you are done hacking and wish to revert to the immutable version, you can remove the replace statement with:

gohack undo example.com/foo/bar

or you can remove all gohack replace statements with:

gohack undo

Note that undoing a replace does not remove the external module's directory - that stays around so your changes are not lost. For example, you might wish to turn that bug fix into an upstream PR.

If you run gohack on a module that already has a directory, gohack will try to check out the current version without recreating the repository, but only if the directory is clean - it won't overwrite your changes until you've committed or undone them.

Comments
  • ability to place gohack files inside module directory instead of $HOME

    ability to place gohack files inside module directory instead of $HOME

    I haven't tried gohack yet, but I am curious about this scenario which I have encountered when doing a similar thing manually:

    I have two separate projects (let's call them myProjectA and myProjectB), each of which happen to use the github.com/jmoiron/sqlx package.

    For project myProjectA, I discover a need to modify function f of sqlx, so I use gohack to get a mutable copy of the repo, and make my changes.

    For project myProjectB, I want to use the original unmodified version of sqlx, but I want to place a log.Printf(...) statement inside function g of sqlx. If I were to use gohack to replace sqlx in this project, my understanding is that project myProjectB would begin using the same copy of sqlx as project myProjectA.

    Is my reasoning correct?

    If so, a way to avoid this seems to be to place gohack'd packages inside, say, myProjectA/.gohack/github.com/jmoiron/sqlx. This may also resolve #5?

    I suppose the main issues with this are:

    • Having a git repo within a git repo - however, the .gohack directory would most likely be included in .gitignore
    • More difficult to use the same gohack'd package across multiple modules
    working as intended 
    opened by b0o 8
  • cannot get module info

    cannot get module info

    I am getting error

    $ gohack get -vcs gopkg.in/danilopolani
    cannot get module info: go list -m: can't compute 'all' using the vendor directory
    	(Use -mod=mod or -mod=readonly to bypass.)
    
    opened by gadelkareem 4
  • do not use go list to find go.mod file

    do not use go list to find go.mod file

    The go list command fails if the go.mod file isn't well formed (like for example one of the replace directives refers to a non-existent target). This can be a problem, so find the go.mod file by looking directly for it instead.

    opened by rogpeppe 3
  • cannot update VCS dir for github.com/labstack/echo: error: pathspec 'v3.2.1+incompatible' did not match any file(s) known to git.

    cannot update VCS dir for github.com/labstack/echo: error: pathspec 'v3.2.1+incompatible' did not match any file(s) known to git.

    when run gohack github.com/labstack/echo

    it show this error message: creating github.com/labstack/[email protected]+incompatible cannot update VCS dir for github.com/labstack/echo: error: pathspec 'v3.2.1+incompatible' did not match any file(s) known to git. all modules failed; not replacing anything

    waiting for info 
    opened by wangyun 3
  • multiple subcommands

    multiple subcommands

    As the amount of functionality grows, it seems like we should consider having subcommands.

    A possible set of commands:

     gohack get [-vcs] [-u] [-f] [module...]
    

    Get gets the modules at the current version and adds replace statements to the go.mod file if they're not already replaced. If the -u flag is provided, the source code will also be updated to the current version if it's clean. If the -f flag is provided with -u, the source code will be updated even if it's not clean. If the -vcs flag is provided, it also checks out VCS information for the modules. If the modules were already gohacked in non-VCS mode, gohack switches them to VCS mode, preserving any changes made (this might result in the directory moving).

    With no module arguments and the -u flag, it will try to update all currently gohacked modules.

    gohack diff module
    

    Diff prints (in git style) changes that have been made to the module since it was checked out.

    gohack rm [-f] module...
    

    Rm removes the gohack directory if it is clean and then runs gohack undo. If the -f flag is provided, the directory is removed even if it's not clean.

     gohack undo [module...]
    

    Undo removes the replace statements for the modules. If no modules are provided, it will undo all gohack replace statements. The gohack module directories are unaffected.

    gohack dir [-vcs] [module...]
    

    Dir prints the gohack module directory names for the given modules. If no modules are given, all the currently gohacked module directories are printed. If the -vcs flag is provided, the directory to be used in VCS mode is printed. Unlike the other subcommands, the modules don't need to be referenced by the current module.

    opened by rogpeppe 3
  • Ability to hack on an already replaced module

    Ability to hack on an already replaced module

    I think this makes sense; based on an actual scenario.

    A module M I was working on had a dependency on D1. But I'd already forked D1 to D1'. Hence my go.mod already had a replace from D1 => D1'.

    So my desire to hack on D1 translates to needing to hack on D1'. And hence gohack should be following replace directives to work out what should ultimately be hacked on. Instead I got:

    "github.com/shurcooL/vfsgen" is already replaced; will not override replace statement in go.mod
    all modules failed; not replacing anything
    error: [
            {/home/myitcv/gostuff/src/gopkg.in/errgo.v2/fmt/errors/alias.go:15: all modules failed; not replacing anything}
    ]
    
    opened by myitcv 3
  • Don't pollute $HOME by default

    Don't pollute $HOME by default

    I know the purpose of ~/gohack, but I'm not a big fan of tools that add stuff to my home directory by default.

    Perhaps we could use $GOPATH/hack or something else instead. Don't have a clear answer to this one.

    opened by mvdan 3
  • Idea: flag for

    Idea: flag for "fork" remote

    So the workflow I just used was the following:

    • use gohack to pull down depedenency I want to add log statements too or changes
    • cd int ~/gohack/github.com/ldelossa/repo
    • added a remote called "fork" which points to my fork of github.com/ldelossa/repo
    • hacked on code till I got what I wanted, pushed branch to my fork
    • open PR of fork with original repo.

    Obviously the fork will need to be setup everytime we run GoHack. Curious if you think it's a good idea to add a "remote" or "fork" cli flag to gohack which setups a remote fork to push your changes too in an automated fashion. If so I'd like to take a stab at this PR.

    opened by ldelossa 2
  • main: fix definition and implementation of relative GOHACK

    main: fix definition and implementation of relative GOHACK

    rogpeppe has a far sharper brain than me and pointed out a major issue with the previous implementation: that we hadn't actually implemented a relative value of GOHACK relative to the main module.

    opened by myitcv 2
  • gohack does not allow relative directories in GOHACK env var.

    gohack does not allow relative directories in GOHACK env var.

    We want to be able to set GOHACK to a relative dir, say ./. But this is not possible at the moment as the initial ./ is removed in the replace directive, creating a broken replacement line for go:

    replacement module without version must be directory path (rooted or starting with ./ or ../)
    

    We can work around the problem by manually prefixing the replacement with ./.

    We like the relative path use-case, because it allows enhanced interop in WSL, when the path is on a shared Windows mount (/mnt/c/...), Windows go tools can interpret the same (relative) path as the WSL go tools. With absolute paths, Windows tools obviously cannot interpret the /mnt/c/... mount.

    This is somewhat related to #27.

    opened by davidovich 2
  • provide way to print directory without doing anything else

    provide way to print directory without doing anything else

    It would be useful to be able to print the gohack directory for a module without actually making any changes.

    Perhaps:

    gohack -p modulepath
    

    would print the directory for modulepath. With no arguments, -p could print the directories for all the modules in the current module that have gohack directories.

    opened by rogpeppe 2
  • supporting workspaces?

    supporting workspaces?

    Would it be reasonable for gohack to support adding a module to a workspace (if a go.work file is present) rather than adding a replace to the go.mod file?

    opened by matloob 0
  • Imply module name from $PWD

    Imply module name from $PWD

    -*- mode: compilation; default-directory: "~/go/pkg/mod/github.com/stapelberg/[email protected]/" -*-
    Compilation started at Fri May 29 21:53:41
    
    goversion -m $(which gohack) && gohack get
    /home/michael/go/bin/gohack go1.14
    	path  github.com/rogpeppe/gohack
    	mod   github.com/rogpeppe/gohack       v1.0.2
    	dep   github.com/rogpeppe/go-internal  v1.0.0
    	dep   golang.org/x/tools               v0.0.0-20180917221912-90fa682c2a6e
    	dep   gopkg.in/errgo.v2                v2.1.0
    cannot determine main module: go list -m: not using modules
    
    Compilation exited abnormally with code 1 at Fri May 29 21:53:41
    

    I have to use gohack get github.com/stapelberg/glog even though it’s obvious from my working directory which package I mean.

    Would a PR for this be accepted?

    opened by stapelberg 5
  • gohack undo leaves trailing newlines

    gohack undo leaves trailing newlines

    This is how we can reproduce the issue:

    $ git clone https://github.com/rogpeppe/go-internal.git
    $ gohack get gopkg.in/errgo.v2
    $ gohack undo
    $ git status
    On branch master
    Your branch is up to date with 'origin/master'.
    
    Changes not staged for commit:
      (use "git add <file>..." to update what will be committed)
      (use "git restore <file>..." to discard changes in working directory)
            modified:   go.mod
    
    no changes added to commit (use "git add" and/or "git commit -a")
    

    git diff produces:

    diff --git a/go.mod b/go.mod
    index 1c11744..e980eb2 100644
    --- a/go.mod
    +++ b/go.mod
    @@ -3,3 +3,5 @@ module github.com/rogpeppe/go-internal
     go 1.11
    
     require gopkg.in/errgo.v2 v2.1.0
    +
    +
    

    What I expected is that it there should be no diff, i.e.

    $ git status
    On branch master
    Your branch is up to date with 'origin/master'.
    
    nothing to commit, working tree clean
    

    Looking at the Go's source code https://github.com/golang/go/blob/26154f31ad6c801d8bad5ef58df1e9263c6beec7/src/cmd/go/internal/modcmd/edit.go#L209

    maybe we could include modf.Cleanup() before modf.Format() in https://github.com/rogpeppe/gohack/blob/03d2ff3646b7ffc380e059413e4302f6cbdeb09b/gomodcmd.go#L91

    Created PR #69 for this. :smile:

    opened by shihanng 3
  • gohack redo

    gohack redo

    When doing undo in the folder <somepath>, gohack should save what was undone to the file $HOME/gohack/<somepath> or to the file .gohack-redo. Command gohack redo should undo the undoing, so basically turn back what was removed (and delete redo file).

    Use case in my mind is the following:

    1. I'm writing the pre-commit hook, which is calling gohack undo
    2. I'm writing post-commit hook, which is calling gohack redo
    3. (optionally) I'm including .gohack-redo file in the .gitignore to not accidentally commit it
    4. I'm working as usual, being more or less sure, that replace directives would not be commited

    That would be particulary convenient for multi-repo applications, which has a shared codebase in a separate repos, because allows working on the cross-repo feature (/service1 /service2 and /lib1 /lib2 folders cloned under one root), while keeping CI builds intact (which only have /service1 cloned).

    opened by strowk 0
Releases(v1.0.2)
Owner
Roger Peppe
Roger Peppe
Fetch license information for all direct and indirect dependencies of your Golang project

gocomply beta Give open source Golang developers the credit they deserve, follow your legal obligations, and save time with gocomply. This tiny little

Tawesoft 16 Nov 1, 2022
depaware makes you aware of your Go dependencies

depaware depaware makes you aware of your Go dependencies. It generates a list of your dependencies which you check in to your repo: https://github.co

Tailscale 414 Nov 19, 2022
Go-htutil - Go HTTP utilities, with no dependencies

snai.pe/go-htutil go get snai.pe/go-htutil Go HTTP utilities with no dependenci

null 2 Jan 26, 2022
Checks if there are any updates for imports in your module.

Go Up goup checks if there are any updates for imports in your module. It parses go.mod files to get dependencies with their version, uses go-git to r

Hervé Gouchet 38 Jul 7, 2022
dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

Coleman Word 266 Nov 11, 2022
A lightweight Vault client module written in Go, with no dependencies, that is intuitive and user-friendly

libvault A lightweight Hashicorp Vault client written in Go, with no dependencies. It aims to provide an intuitive, simple API that is easy to use. Ju

null 74 Sep 18, 2022
PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. You can use PHP like functions in your app, module etc. when you add this module to your project.

PHP Functions for Golang - phpfuncs PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. Y

Serkan Algur 52 Aug 26, 2022
Idiomatic Go input parsing with subcommands, positional values, and flags at any position. No required project or package layout and no external dependencies.

Sensible and fast command-line flag parsing with excellent support for subcommands and positional values. Flags can be at any position. Flaggy has no

Eric Greer 814 Nov 15, 2022
Go package to make lightweight ASCII line graph ╭┈╯ in command line apps with no other dependencies.

asciigraph Go package to make lightweight ASCII line graphs ╭┈╯. Installation go get github.com/guptarohit/asciigraph Usage Basic graph package main

Rohit Gupta 2.1k Nov 18, 2022
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Tom Wright 3.8k Nov 21, 2022
AppsFlyer 499 Nov 7, 2022
BPG decoder for Go (Zero Dependencies).

Go语言QQ群: 102319854, 1055927514 凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa BPG for Go BPG is defined at: http://bellard.o

chai2010 20 Sep 7, 2020
Go bindings for OpenCV1.1 (Dev/Zero Dependencies).

Go语言QQ群: 102319854, 1055927514 凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa Go bindings for OpenCV1.1 PkgDoc: http://godoc

chai2010 116 Aug 24, 2022
Rich TIFF/BigTIFF/GeoTIFF decoder/encoder for Go (Pure Go/Zero Dependencies)

Go语言QQ群: 102319854, 1055927514 凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa TIFF for Go Features: Support BigTiff Support

chai2010 45 Nov 16, 2022
WebP decoder and encoder for Go (Zero Dependencies).

Go语言QQ群: 102319854, 1055927514 凹语言(凹读音“Wa”)(The Wa Programming Language): https://github.com/wa-lang/wa webp ██╗ ██╗███████╗██████╗ ██████╗ ██║

chai2010 369 Nov 26, 2022
Simply way to control goroutines execution order based on dependencies

Goflow Goflow is a simply package to control goroutines execution order based on dependencies. It works similar to async.auto from node.js async packa

Kamil Drazkiewicz 201 Nov 24, 2022
A well tested and comprehensive Golang statistics library package with no dependencies.

Stats - Golang Statistics Package A well tested and comprehensive Golang statistics library / package / module with no dependencies. If you have any s

Montana Flynn 2.6k Nov 24, 2022
Vendor Go dependencies

nut Gophers love nuts. nut is a tool that allows Go projects to declare dependencies, download dependencies, rewrite import paths and ensure that depe

Owen Ou 238 Sep 27, 2022
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Tom Wright 3.8k Nov 20, 2022