A Go package to help write migrations with
go-pg now has Go modules support,
go-pg-migrations also has modules support; it currently depends on v10 of
go-pg. To install it, use the following command in a project with a
$ go get github.com/robinjoseph08/go-pg-migrations/v3
If you are not yet using Go modules, you can still use v1 of this package.
To see how this package is intended to be used, you can look at the example directory. All you need to do is have a
main package (e.g.
migrations.Run with the directory you want the migration files to be saved in (which will be the same directory of the main package, e.g.
example), an instance of
os.Args; and log any potential errors that could be returned.
Once this has been set up, then you can use the
help commands like so:
$ go run example/*.go create create_users_table Creating example/20180812001528_create_users_table.go... $ go run example/*.go migrate Running batch 1 with 1 migration(s)... Finished running "20180812001528_create_users_table" $ go run example/*.go rollback Rolling back batch 1 with 1 migration(s)... Finished rolling back "20180812001528_create_users_table" $ go run example/*.go help Usage: go run example/*.go [command] Commands: create - create a new migration in example with the provided name migrate - run any migrations that haven't been run yet rollback - roll back the previous run batch of migrations help - print this help text Examples: go run example/*.go create create_users_table go run example/*.go migrate go run example/*.go rollback go run example/*.go help
While this works when you have the Go toolchain installed, there might be a scenario where you have to run migrations and you don't have the toolchain available (e.g. in a
alpine Docker image deployed to production). In that case, you should compile another binary (in addition to your actual application) and copy it into the final image. This will include all of your migrations and allow you to run it by overriding the command when running the Docker container.
This would look something like this:
# Dockerfile FROM golang:1.13.3 as build WORKDIR /app COPY go.mod go.mod COPY go.sum go.sum RUN go mod download COPY . . RUN CGO_ENABLED=0 GOOS=linux go build -installsuffix cgo -ldflags '-w -s' -o ./bin/serve ./cmd/serve RUN CGO_ENABLED=0 GOOS=linux go build -installsuffix cgo -ldflags '-w -s' -o ./bin/migrations ./cmd/migrations FROM alpine:3.8 RUN apk --no-cache add ca-certificates COPY --from=build /app/bin /bin CMD ["serve"]
$ docker build -t service:latest . $ docker run --rm service:latest migrations migrate
While go-pg has its own
migrations package, it leaves a bit to be desired. Some additional features that this package supports:
- Complete migration diffing to determine which migrations still need to be run. Since
go-pg/migrationschecks the current version of migrations and runs any migrations after that, some migrations can be missed, especially when several people are working on the same project.
- Timestamp-based prefixes to prevent two people creating a migration with the same version on two separate branches. If the current version is 3, and more than one person branches off and creates a new migration, all of them will be version 4.
- The ability to run migrations in a transaction on a case-by-case basis. Most of the time, running migrations within a transaction is desirable, so that if it errs out within the "up" function, the whole migration is reverted. But since some long-running migrations might have a statement with a relatively exclusive lock, you might opt out of running that specific migration within a transaction.
- A migration locking mechanism. This is to avoid two people (or an automated deployment system) attempting to run migrations at the same time against the same database, which could lead to undesired behavior.
- An expected workflow of how this package should be used within a project. While
go-pg/migrationshas some recommendations and examples, this package takes a more opinionated approach which makes it so you don't have to think about it as much, and there's less code for you to write and maintain.
- Batch-level rollbacks. When there are multiple migration files run during the same migration invocation, they are all grouped together into a "batch". During rollbacks, each batch gets rolled back together. This tends to be more desireable since this usually means the application is reverting back to a previous release, so the database should be in the state expected for that release.
Many of these features and expected behaviors come from using Knex.js migrations in production for many years. This project is heavily inspired by Knex to provide a robust and safe migration experience.
go-pg is a great and performant project, and hopefully, this makes it a little better.
To develop on this project, you'll need to have Postgres running because the tests depend on it.
If you have it running on your machine because it was installed through your package manager (like
apt-get), you just need to run the following to get it set up correctly:
If you don't have it on your laptop, you can run the following to start it within Docker:
That should start the container and keep it running while you develop. Once you're done, you can
^C out of it and it will stop the container.
To run the tests, you should run:
To run the linter, you should run: