Embed files into a Go executable

Overview

statik

Build Status

statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem.

Is this a crazy idea? No, not necessarily. If you're building a tool that has a Web component, you typically want to serve some images, CSS and JavaScript. You like the comfort of distributing a single binary, so you don't want to mess with deploying them elsewhere. If your static files are not large in size and will be browsed by a few people, statik is a solution you are looking for.

Usage

Install the command line tool first.

go get github.com/rakyll/statik

statik is a tiny program that reads a directory and generates a source file that contains its contents. The generated source file registers the directory contents to be used by statik file system.

The command below will walk on the public path and generate a package called statik under the current working directory.

$ statik -src=/path/to/your/project/public

The command below will filter only files on listed extensions.

$ statik -include=*.jpg,*.txt,*.html,*.css,*.js

In your program, all your need to do is to import the generated package, initialize a new statik file system and serve.

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Serve the contents over HTTP.
  http.Handle("/public/", http.StripPrefix("/public/", http.FileServer(statikFS)))
  http.ListenAndServe(":8080", nil)

Visit http://localhost:8080/public/path/to/file to see your file.

You can also read the content of a single file:

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Access individual files by their paths.
  r, err := statikFS.Open("/hello.txt")
  if err != nil {
    log.Fatal(err)
  }    
  defer r.Close()
  contents, err := ioutil.ReadAll(r)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println(string(contents))

There is also a working example under example directory, follow the instructions to build and run it.

Note: The idea and the implementation are hijacked from camlistore. I decided to decouple it from its codebase due to the fact I'm actively in need of a similar solution for many of my projects.

Deterministic output

By default, statik includes the "last modified" (mtime) time on files that it packs. This allows an HTTP FileServer to present the correct file modification times to clients.

However, if you have a continuous integration task that checks that your checked-in static files in a git repository match the code that is generated on your CI system, you'll run into a problem: The mtime on the git checkout does not match what you have locally, causing tests to fail.

You can fix the test in one of two ways:

  1. In CI, manually set the mtime on the freshly checked out tree: here's a stackoverflow answer that provides a shell command to do that; or,
  2. Instruct statik not to store the "last modified" time.

To ignore the last modified time, use the -m to statik, like so:

$ statik -m -include=*.jpg,*.txt,*.html,*.css,*.js

Note that this will cause http.FileServer to consider the file to always have changed & serve it with a "Last-Modified" of the time of the request.

Issues
  • fixed 'invalid cross-device link' problem

    fixed 'invalid cross-device link' problem

    Renaming the generated source file failed for me with the following message: rename /tmp/statik888425530 statik/statik.go: invalid cross-device link

    Try to rename and if that fails copy byte by byte.

    opened by gfrey 8
  • Readdir should treat zero for count value as it was negative to be fully compatible with the interface

    Readdir should treat zero for count value as it was negative to be fully compatible with the interface

    Hi @rakyll

    Thanks for the statik, it's a great lib:)

    I found an issue in the Readdir implementation while I was using another library that called statik's fs.Readdir(0) and it resulted in a problem. No file was found in this case.

    It should treat the zero value and the negatives the same as the GoDoc recommends: https://golang.org/pkg/os/#File.Readdir

    The problematic Readdir implementation: https://github.com/rakyll/statik/blob/3bac566d30cdbeddef402a80f3d6305860e59f12/fs/fs.go#L190

    I fould the issue when I was using this library: https://github.com/shurcooL/httpfs/issues/8

    All best, @shakahl

    opened by shakahl 7
  • support positive count in Readdir()

    support positive count in Readdir()

    follow on #60.

    The Readdir() should support positive count in order to return the specified counts of files.

    For it, I defined statikFS.dirs and httpFile.dirIdx for managing the internal state and use it inside Readdir().

    opened by Songmu 6
  • Multiple static file packages

    Multiple static file packages

    I'd like to produce two (or more) different statik fs's in different import paths. Due to relying on init() and no public API from the package, it's impossible to use both filesystems at the same time; concurrent access is also impossible as fs.Register is package-scoped.

    I'd propose modifying the API:

    fs.New would take zipDatas ...string and fallback on whatever was put in fs.Register (backwards compatible), The generated filesystem package could add Data() string for this purpose in addition to keeping init() behaviour.

    opened by titpetric 6
  • Reproductible output

    Reproductible output

    Hello,

    Is there any way to have reproductible output from statik?

    As a context, this is needed for Debian. One of the packages I manage (InfluxDB) uses statik and I would like for it to be reproductible.

    opened by aviau 6
  • Offical exmple does not work?

    Offical exmple does not work?

    image

    As you see , when i copy the exmple from offical packages, and run go run main.go, access browser just show 404. What is the mistake I got from , please help! Thanks in advance!

    > go env
    set GOARCH=amd64
    set GOBIN=
    set GOCACHE=C:\Users\Vector\AppData\Local\go-build
    set GOEXE=.exe
    set GOFLAGS=
    set GOHOSTARCH=amd64
    set GOHOSTOS=windows
    set GOOS=windows
    set GOPATH=D:\GoProject
    set GOPROXY=
    set GORACE=
    set GOROOT=D:\Dev\go
    set GOTMPDIR=
    set GOTOOLDIR=D:\Dev\go\pkg\tool\windows_amd64
    set GCCGO=gccgo
    set CC=gcc
    set CXX=g++
    set CGO_ENABLED=1
    set GOMOD=
    set CGO_CFLAGS=-g -O2
    set CGO_CPPFLAGS=
    set CGO_CXXFLAGS=-g -O2
    set CGO_FFLAGS=-g -O2
    set CGO_LDFLAGS=-g -O2
    set PKG_CONFIG=pkg-config
    set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Vector\AppData\Local\Temp\go-build291331690=/tmp/go-build -gno-record-
    gcc-switches
    
    opened by carmel 4
  • Document determinism

    Document determinism

    This PR does two documentation-y things:

    • Adds a section in the README on how to achieve deterministic output w.r.t. file contents, with a link to a method that restores file modification times.
    • Adjusts the -help text to point out more clearly -m does and that it helps with deterministic output.

    This addresses #99

    opened by antifuchs 3
  •  undefined: fs.RegisterWithNamespace

    undefined: fs.RegisterWithNamespace

    I am trying to build (GOOS=linux GOARCH=amd64 go build app/main.go) but getting this error: app/statik/statik.go:12:2: undefined: fs.RegisterWithNamespace

    code from auto generated statik: fs.RegisterWithNamespace("default", data)

    how to resolve this?

    opened by semyon-dev 3
  • Fix Readdir() and Name()

    Fix Readdir() and Name()

    Statik's Readdir did not behave in the same way as a http.FileSystem's Readdir. Similarly, the Name function in http.FileSystem returned the base name, and not a full file path.

    This commit fixes both issues, making both Name and Readdir behave in the same way as when running the raw filesystem directly using http.Dir()

    The commit is an extension of #55, which was a first step towards replicating the correct behaviour, but failed to account for the Name() issue, and did not include the necessary extra Trim to get rid of leading slash.

    The corresponding tests ensure that the results are identical for http.Dir() and statik.

    Closes #58

    opened by dkumor 3
  • add custom fs loading

    add custom fs loading

    Closes #56 ; it only takes the first parameter for the FS data, but it would be trivial to support them all, I'm just not sure if it would be expected that they would be loaded one over the other.

    opened by titpetric 3
  • use static files  in html templates

    use static files in html templates

    Hi, i use Statik for css and js files and wondering if it possible to use compiled html files in templates. Like loginfile, _ := statikFS.Open("/tmpl/login.html") tHeader, err := tHeader.ParseFiles(loginfile.data) Thanks.

    opened by Pechenn 3
  • Doc update, mentioning go's native embedding

    Doc update, mentioning go's native embedding

    This has been a very useful tool. But the introduction of //go:embed in Go 1.16 provides roughly the same functionality, but without requiring a go generate pre-compilation step.

    It would be good to add a comment to the beginning of the readme, saying that it is best to use go's native embedding, and to mention any cases which are not handled by go:embed where statik would still be useful.

    opened by amnonbc 1
  • How I replaced this package with go:embed

    How I replaced this package with go:embed

    Hopefully this issue and PR helps those who want to transition from statik to go:embed.

    My motivation?

    2 fold:

    1. This issue - In short, the Walk function in statik did not include files without extension for some reason.
    2. Remove another external dependency

    It wasn't easy, but it's doable. In the end, I managed to get all features working with go:embed

    https://github.com/countertenor/mozart/issues/53

    opened by prashantgupta24 0
  • Does not read files without an extension?

    Does not read files without an extension?

    For some reason, the Walk function on a directory is ignoring files without an extension within that directory. I'm not sure if those files are omitted from the static phase itself, because the code inside the Walk function should ideally return all files, including ones without extension.

    opened by prashantgupta24 0
  • When compling on the CI i get an statik error

    When compling on the CI i get an statik error

    I can compile the app (as usual) locally without issues. After switching to to an azure pipeline, i compile the up under golang:1.15-buster

    I do install statik using

    cd /tmp
    go get github.com/rakyll/statik
    

    Then i call statik as expected

    cd /src
    ${GOPATH}/bin/statik -f -src=./public
    

    And this properly generates the build under ../statik/statik.go in my src folder

    Without any issues. But when i call go build (from the same cwd)

    cd /src
    env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags netgo -ldflags "-X github.com/<redacted>/build.Version=${{ variables.version }}" -o dist/<redacted> <redacted>_static.go
    

    It errors with

    statik/statik.go:6:2: cannot find package "." in:
    	/__w/1/s/vendor/github.com/rakyll/statik/fs
    

    I tried quiet some pieces but yet nothing worked.

    My <redacted>_statik.go looks like

    import (
    	"flag"
    	"github.com/<redacted>/bootstrap"
    	"net/http"
    	// will be generated by statik
    	_ "github.com/<redacted>/statik"
    	"github.com/rakyll/statik/fs"
    	"log"
    )
    
    func main() {...}
    

    Helpful for any clues!

    opened by EugenMayer 0
  • added power support arch ppc64le on yml file.

    added power support arch ppc64le on yml file.

    Added power support for the travis.yml file with ppc64le. This is part of the Ubuntu distribution for ppc64le. This helps us simplify testing later when distributions are re-building and re-releasing. For more info tag @gerrith3.

    opened by srinivas32 0
Releases(v0.1.7)
The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Buffalo - The Go Web Eco-System 3.4k Aug 17, 2022
A tool to be used with 'go generate' to embed external template files into Go code.

templify A tool to be used with 'go generate' to embed external template files into Go code. Scenario An often used scenario in developing go applicat

Michael Wolber 27 Jan 24, 2022
a better customizable tool to embed files in go; also update embedded files remotely without restarting the server

fileb0x What is fileb0x? A better customizable tool to embed files in go. It is an alternative to go-bindata that have better features and organized c

null 613 Jul 21, 2022
Using brotli compression to embed static files in Go.

?? Broccoli go get -u aletheia.icu/broccoli Broccoli uses brotli compression to embed a virtual file system of static files inside Go executables. A f

Aletheia 526 Jul 15, 2022
:file_folder: Embeds static resources into go files for single binary compilation + works with http.FileSystem + symlinks

Package statics Package statics embeds static files into your go applications. It provides helper methods and objects to retrieve embeded files and se

Go Playgound 64 Mar 3, 2022
Generates go code to embed resource files into your library or executable

Deprecating Notice go is now going to officially support embedding files. The go command will support //go:embed tags. Go Embed Generates go code to e

Peter 6.3k Jun 2, 2021
Embed files into a Go executable

statik statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem. Is this a crazy idea? No

Jaana Dogan 3.5k Aug 17, 2022
Embed files into a Go executable

statik statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem. Is this a crazy idea? No

Jaana Dogan 3.5k Aug 9, 2022
Get an embed.FS from inside an embed.FS

embed.FS wrapper providing additional functionality Features Get an embed.FS from an embedded subdirectory Handy Copy(sourcePath, targetPath) method t

Lea Anthony 20 Aug 6, 2022
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

Patrick D'appollonio 177 Aug 10, 2022
Split multiple Kubernetes files into smaller files with ease. Split multi-YAML files into individual files.

kubectl-slice: split Kubernetes YAMLs into files kubectl-slice is a neat tool that allows you to split a single multi-YAML Kubernetes manifest into mu

Patrick D'appollonio 175 Aug 5, 2022
The simple and easy way to embed static files into Go binaries.

NOTICE: Please consider migrating your projects to github.com/markbates/pkger. It has an idiomatic API, minimal dependencies, a stronger test suite (t

Buffalo - The Go Web Eco-System 3.4k Aug 17, 2022
A tool to be used with 'go generate' to embed external template files into Go code.

templify A tool to be used with 'go generate' to embed external template files into Go code. Scenario An often used scenario in developing go applicat

Michael Wolber 27 Jan 24, 2022
a better customizable tool to embed files in go; also update embedded files remotely without restarting the server

fileb0x What is fileb0x? A better customizable tool to embed files in go. It is an alternative to go-bindata that have better features and organized c

null 613 Jul 21, 2022
Ghostinthepdf - This is a small tool that helps to embed a PostScript file into a PDF

This is a small tool that helps to embed a PostScript file into a PDF in a way that GhostScript will run the PostScript code during the

Emil Lerner 124 Aug 10, 2022
This command line converts .html file into .html with images embed.

embed-html This command line converts .html file into .html with images embed. Install > go get github.com/gonejack/embed-html Usage > embed-html *.ht

会有猫的 0 Jan 13, 2022
📦 Package Node.js applications into executable binaries 📦

caxa ?? Package Node.js applications into executable binaries ?? Support Recurring support on Patreon: https://patreon.com/leafac One-time support on

Leandro Facchinetti 562 Aug 5, 2022
donLoader is a shellcode loader creation tool that uses donut to convert executable payloads into shellcode to evade detection on disk.

donLoader WARNING: This is WIP, barely anything was tested properly. Use at your own risk. Description donLoader is a shellcode loader creation tool t

blink3n 13 Jul 9, 2022
Using brotli compression to embed static files in Go.

?? Broccoli go get -u aletheia.icu/broccoli Broccoli uses brotli compression to embed a virtual file system of static files inside Go executables. A f

Aletheia 526 Jul 15, 2022
Embed static files in Go binaries (replacement for gobuffalo/packr)

Pkger github.com/markbates/pkger is a tool for embedding static files into Go binaries. It will, hopefully, be a replacement for github.com/gobuffalo/

Mark Bates 1.2k Aug 13, 2022
Command line tool for adding Windows resources to executable files

go-winres A simple command line tool for embedding usual resources in Windows executables built with Go: A manifest An application icon Version inform

null 140 Aug 15, 2022
erchive is a go program that compresses and encrypts files and entire directories into .zep files (encrypted zip files).

erchive/zep erchive is a go program that compresses and encrypts files and entire directories into .zep files (encrypted zip files). it compresses usi

Christopher Walters 1 May 16, 2022
Go package to embed the Mozilla Included CA Certificate List

rootcerts Package rootcerts provides an embedded copy of the Mozilla Included CA Certificate List, more specifically the PEM of Root Certificates in M

Lucas Bremgartner 84 Jul 24, 2022
Recreate embedded filesystems from embed.FS type in current working directory.

rebed Recreate embedded filesystems from embed.FS type in current working directory. Expose the files you've embedded in your binary so users can see

Patricio Whittingslow 22 Feb 28, 2022
go:embed and the golang-standards project layout

An example of using the golang-standards project layout and the go:embed directive.

Leon Rowland 6 May 17, 2022
This command line converts .webarchive file to resources embed .html file

webarchive-to-singlefile This command line converts Safari's .webarchive file to complete .html. Notice Only tested on MacOS. Google Chrome required.

会有猫的 3 Aug 10, 2022
Little golang app that allows you to download a youtube video as mp3, and optionally embed ID3 tags -Cover Art, Artist ...-

yt2mp3 Little golang app that allows you to download a youtube video as mp3, and optionally embed ID3 tags -Cover Art, Artist ...- Instructions At the

null 0 Dec 25, 2021
Use this program to embed sh scripts in binaries

sh2bin Use this program to embed sh scripts in binaries. Feel free to fork this

Carlos Polop 17 Jul 21, 2022
PoC de embed de arquivos arbitrários em imagens e vídeos

imgcode PoC de embed de arquivos arbitrários em imagens e vídeos Esse projeto de Abril de 2020 eu achei perdido no meu HD externo hoje (Janeiro de 202

Lucas Eduardo 0 Jan 8, 2022