overseer
overseer is a package for creating monitorable, gracefully restarting, self-upgrading binaries in Go (golang). The main goal of this project is to facilitate the creation of self-upgrading binaries which play nice with standard process managers, secondly it should expose a small and simple API with reasonable defaults.
Commonly, graceful restarts are performed by the active process (dark blue) closing its listeners and passing these matching listening socket files (green) over to a newly started process. This restart causes any foreground process monitoring to incorrectly detect a program crash. overseer attempts to solve this by using a small process to perform this socket file exchange and proxying signals and exit code from the active process.
Features
- Simple
- Works with process managers (systemd, upstart, supervisor, etc)
- Graceful, zero-down time restarts
- Easy self-upgrading binaries
Install
go get github.com/jpillora/overseer
Quick example
This program works with process managers, supports graceful, zero-down time restarts and self-upgrades its own binary.
package main
import (
"fmt"
"log"
"net/http"
"time"
"github.com/jpillora/overseer"
"github.com/jpillora/overseer/fetcher"
)
//create another main() to run the overseer process
//and then convert your old main() into a 'prog(state)'
func main() {
overseer.Run(overseer.Config{
Program: prog,
Address: ":3000",
Fetcher: &fetcher.HTTP{
URL: "http://localhost:4000/binaries/myapp",
Interval: 1 * time.Second,
},
})
}
//prog(state) runs in a child process
func prog(state overseer.State) {
log.Printf("app (%s) listening...", state.ID)
http.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "app (%s) says hello\n", state.ID)
}))
http.Serve(state.Listener, nil)
}
How it works:
overseeruses the main process to check for and install upgrades and a child process to runProgram.- The main process retrieves the files of the listeners described by
Address/es. - The child process is provided with these files which is converted into a
Listener/sfor theProgramto consume. - All child process pipes are connected back to the main process.
- All signals received on the main process are forwarded through to the child process.
Fetcherruns in a goroutine and checks for updates at preconfigured interval. WhenFetcherreturns a valid binary stream (io.Reader), the master process saves it to a temporary location, verifies it, replaces the current binary and initiates a graceful restart.- The
fetcher.HTTPaccepts aURL, it polls this URL with HEAD requests and until it detects a change. On change, weGETtheURLand stream it back out tooverseer. See alsofetcher.S3. - Once a binary is received, it is run with a simple echo token to confirm it is a
overseerbinary. - Except for scheduled restarts, the active child process exiting will cause the main process to exit with the same code. So,
overseeris not a process manager.
See Configuration options here and the runtime State available to your program here.
More examples
See the example/ directory and run example.sh, you should see the following output:
$ cd example/
$ sh example.sh
BUILT APP (1)
RUNNING APP
app#1 (c7940a5bfc3f0e8633d3bf775f54bb59f50b338e) listening...
app#1 (c7940a5bfc3f0e8633d3bf775f54bb59f50b338e) says hello
app#1 (c7940a5bfc3f0e8633d3bf775f54bb59f50b338e) says hello
BUILT APP (2)
app#2 (3dacb8bc673c1b4d38f8fb4fad5b017671aa8a67) listening...
app#2 (3dacb8bc673c1b4d38f8fb4fad5b017671aa8a67) says hello
app#2 (3dacb8bc673c1b4d38f8fb4fad5b017671aa8a67) says hello
app#1 (c7940a5bfc3f0e8633d3bf775f54bb59f50b338e) says hello
app#1 (c7940a5bfc3f0e8633d3bf775f54bb59f50b338e) exiting...
BUILT APP (3)
app#3 (b7614e7ff42eed8bb334ed35237743b0e4041678) listening...
app#3 (b7614e7ff42eed8bb334ed35237743b0e4041678) says hello
app#3 (b7614e7ff42eed8bb334ed35237743b0e4041678) says hello
app#2 (3dacb8bc673c1b4d38f8fb4fad5b017671aa8a67) says hello
app#2 (3dacb8bc673c1b4d38f8fb4fad5b017671aa8a67) exiting...
app#3 (b7614e7ff42eed8bb334ed35237743b0e4041678) says hello
Note: app#1 stays running until the last request is closed.
Only use graceful restarts
func main() {
overseer.Run(overseer.Config{
Program: prog,
Address: ":3000",
})
}
Send main a SIGUSR2 (Config.RestartSignal) to manually trigger a restart
Only use auto-upgrades, no restarts
func main() {
overseer.Run(overseer.Config{
Program: prog,
NoRestart: true,
Fetcher: &fetcher.HTTP{
URL: "http://localhost:4000/binaries/myapp",
Interval: 1 * time.Second,
},
})
}
Your binary will be upgraded though it will require manual restart from the user, suitable for creating self-upgrading command-line applications.
Multi-platform binaries using a dynamic fetch URL
func main() {
overseer.Run(overseer.Config{
Program: prog,
Fetcher: &fetcher.HTTP{
URL: "http://localhost:4000/binaries/app-"+runtime.GOOS+"-"+runtime.GOARCH,
//e.g.http://localhost:4000/binaries/app-linux-amd64
},
})
}
Known issues
- The master process's
overseer.Configcannot be changed via an upgrade, the master process must be restarted.- Therefore,
Addressescan only be changed by restarting the main process.
- Therefore,
- Currently shells out to
mvfor moving files becausemvhandles cross-partition moves unlikeos.Rename. - Package
init()functions will run twice on start, once in the main process and once in the child process.
More documentation
Third-party Fetchers
- overseer-bindiff A binary diff fetcher and builder
Contributing
See CONTRIBUTING.md
![Lots of output: [overseer master] proxy signal (urgent I/O condition)](https://avatars.githubusercontent.com/u/1061610?v=4)
![[overseer master] proxy signal (broken pipe)](https://avatars.githubusercontent.com/u/52022868?v=4)
92 Dec 30, 2022
101 Dec 15, 2022
215 Jul 2, 2022
883 Dec 27, 2022
208 Nov 21, 2022
7.1k Jan 2, 2023
193 Sep 27, 2022
1 Nov 1, 2021
2 Jun 5, 2022
3 Nov 15, 2022
6 Sep 17, 2022
0 Jan 1, 2022
1 May 2, 2022
0 Jan 2, 2022
3 Oct 5, 2022
1 Jan 7, 2022
2 Aug 2, 2022