Cogger is a standalone binary and a golang library that reads an internally tiled geotiff

Related tags

cogger
Overview

Cogger

Go Reference License

Cogger is a standalone binary and a golang library that reads an internally tiled geotiff (optionally with overviews and masks) and rewrites it as a Cloud Optimized Geotiff (COG). This process being a reshuffling of the original geotiff's bytes, it should run as fast as the underlying disk or network i/o.

Cogger does not do any pixel manipulation on the provided image, it is up to you to provide an input geotiff which can be suitably transformed to a COG, namely:

  • it must be internally tiled
  • it should be compressed with one of the standard supported tiff compression mechanisms
  • it should contain overviews

Installation

Binaries

We publish the cogger binaries for the major platforms/cpus, which you can grab from our releases

From source

The library version of cogger can be used in go code with:

import "github.com/airbusgeo/cogger"

The cogger binary can be installed directly to your $GOPATH/bin with:

go install github.com/airbusgeo/cogger/cmd/[email protected]

Usage

Binary

gdal_translate -of GTIFF -co BIGTIFF=YES -co TILED=YES -co COMPRESS=ZSTD -co NUM_THREADS=4 input.file geotif.tif
gdaladdo --config GDAL_NUM_THREADS 4 geotif.tif 2 4 8 16 32
cogger -output mycog.tif geotif.tif

Library

The cogger API consists of a single function:

func Rewrite(out io.Writer, readers ...tiff.ReadAtReadSeeker) error

with the reader allowing random read access to the input file, i.e. implementing

Read(buf []byte) (int,error)
ReatAt(buf []byte, offset int64) (int,error)
Seek(off int64, whence int) (int64,error)

The writer is a plain io.Writer which means that the output cog can be directly streamed to http/cloud storage without having to be stored in an intermediate file.

For an full example of library usage, see the main.go file in cmd/cogger.

Advanced

Cogger is able to assemble a single COG from a main tif file and overviews that have been computed in distinct files. This may be useful as gdaladdo is missing some features to fine tune the options of each individual overview.

gdal_translate -of GTIFF -co BIGTIFF=YES -co TILED=YES -co COMPRESS=ZSTD -co NUM_THREADS=4 input.file geotif.tif
# compute first overview
gdal_translate -of GTIFF -outsize 50% 50% -co BLOCKXSIZE=128 -co TILED=YES -co COMPRESS=ZSTD -co NUM_THREADS=4  geotif.tif ovr.tif.1
# compute second overview
gdal_translate -of GTIFF -outsize 50% 50% -co BLOCKXSIZE=256 -co TILED=YES -co COMPRESS=ZSTD -co NUM_THREADS=4  ovr.tif.1 ovr.tif.2
# compute third overview
gdal_translate -of GTIFF -outsize 50% 50% -co BLOCKXSIZE=512 -co TILED=YES -co COMPRESS=ZSTD -co NUM_THREADS=4  ovr.tif.2 ovr.tif.3
# compute COG from geotif.tif and ovr.tif.* overviews
cogger -output mycog.tif geotif.tif ovr.tif.1 ovr.tif.2 ovr.tif.3

Contributing

Contributions are welcome. Please read the contribution guidelines before submitting fixes or enhancements.

Licensing

Cogger is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Issues
  • ENH idea: emit header ghost area and tile header and trailer bytes

    ENH idea: emit header ghost area and tile header and trailer bytes

    For really huge COGs (let's say 1 million pixel x 1 million pixel), the size of the TileByteCount and TileOffsets values can be quite big (several tens of MB). If one knows that the tiles are ordered in a top-to-bottom, left-to-right order in the file, then a smart reader can possibly only use TileOffsets, and that's what GDAL does, to save a GET request. To do so, one must write the header ghost area per https://gdal.org/drivers/raster/cog.html#header-ghost-area and add a 4-byte header before each tile data and a 4-byte trailer after it (https://gdal.org/drivers/raster/cog.html#tile-data-leader-and-trailer), whose purpose is to check that a writer hasn't modified tile content since the COG generation, which would make the optimization no longer valid.

    enhancement 
    opened by rouault 8
  • Overviews: optional or not?

    Overviews: optional or not?

    The read me says both "internally tiled geotiff (optionally with overviews" and "it should contain overviews". The statements are at odds with each other. Which is the more correct meaning?

    opened by maphew 4
  • Add README file

    Add README file

    opened by yjacolin 3
  • implement support for gdal ghost area optimization

    implement support for gdal ghost area optimization

    implements #2

    opened by tbonfort 1
Releases(v0.0.5)
Owner
Airbus DS GEO S.A.
Airbus DS GEO S.A.
Versatile Go code generator.

Generis Versatile Go code generator. Description Generis is a lightweight code preprocessor adding the following features to the Go language : Generic

SenseLogic 28 Jul 30, 2021
Clojure-esque extension language for Go.

let-go Greetings loafers! (λ-gophers haha, get it?) This is supposed to be a compiler and bytecode VM for a language resembling Clojure as close as po

Marcin Gasperowicz 38 Jul 27, 2021
Elf binary infector written in Golang

Elf binary infector written in Golang. It can be used for infecting executables of type ET_DYN and ET_EXEC with a payload of your creation. Utilizing the classic elf text segment padding algorithm by Silvio Cesar, your payload (parasite) will run before native functionality of the binary effectively backooring the binary.

null 113 Jul 23, 2021
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Spiral Scout 1k Jul 26, 2021
Go package to generate and manage color palettes & schemes 🎨

Go package to generate and manage color palettes & schemes

Christian Muehlhaeuser 438 Jul 25, 2021
Perforator is a tool for recording performance metrics over subregions of a program using the Linux "perf" interface.

Perforator Perforator is a tool for recording performance metrics over subregions of a program (e.g., functions) using the Linux "perf" interface.

Zachary Yedidia 19 Jul 31, 2021
A directory of hardware related libs, tools, and tutorials for Go

Go + hardware This repo is a directory of tools, packages and tutorials to let you introduce Go in your hardware projects. Why Go? Go can target platf

Jaana Dogan 1.2k Jul 19, 2021
a thread-safe concurrent map for go

concurrent map As explained here and here, the map type in Go doesn't support concurrent reads and writes. concurrent-map provides a high-performance

Or Hiltch 2.4k Jul 25, 2021
A super simple Lodash like utility library with essential functions that empowers the development in Go

A simple Utility library for Go Go does not provide many essential built in functions when it comes to the data structure such as slice and map. This

Rahul Baruri 81 Jul 17, 2021
Go tool to modify struct field tags

Go tool to modify/update field tags in structs. gomodifytags makes it easy to update, add or delete the tags in a struct field. You can easily add new tags, update existing tags (such as appending a new key, i.e: db, xml, etc..) or remove existing tags

Fatih Arslan 1.5k Jul 28, 2021
A tool and library for using structural regular expressions.

Structural Regular Expressions sregx is a package and tool for using structural regular expressions as described by Rob Pike (link).

Zachary Yedidia 22 Jun 18, 2021
A library for diffing golang structures

Diff A library for diffing golang structures and values. Utilizing field tags and reflection, it is able to compare two structures of the same type an

R3 Labs 340 Jul 23, 2021
sigurls is a reconnaissance tool, it fetches URLs from AlienVault's OTX, Common Crawl, URLScan, Github and the Wayback Machine.

sigurls is a reconnaissance tool, it fetches URLs from AlienVault's OTX, Common Crawl, URLScan, Github and the Wayback Machine. DiSCLAIMER: fe

Alex Munene 128 May 22, 2021
sigbypass4xx is a utility to automate well-know techniques used to bypass access control restrictions.

sigbypass4xx sigbypass4xx is a utility to automate well-know techniques used to bypass access control restrictions. Resources Usage Installation From

Signed Security 3 Jul 12, 2021