Simple and minimal image server capable of storing, resizing, converting and caching images.

Overview

webp-server

Coverage Status Go Report Card Release Github Workflow Status Mentioned in Awesome Go

Simple and minimal image server capable of storing, resizing, converting, and caching images. You can quickly find out how it works by looking at the flowchart below.

Flowchart

Contents

Quickstart

Run a docker container of webp-server.

docker run -d -v webp_server_volume:/var/lib/webp-server --name webp-server -e TOKEN='MY_STRONG_TOKEN' -p 127.0.0.1:8080:8080 ms68/webp-server

Upload an image:

curl -H 'Token: MY_STRONG_TOKEN' -X POST -F '[email protected]/path/to/image.jpg' http://127.0.0.1:8080/upload/

# this api will return an image_id

Open these urls in your browser.

http://127.0.0.1:8080/image/width=500,height=500,fit=contain,quality=100/{image_id}
http://127.0.0.1:8080/image/width=300,height=300,fit=cover,quality=90/{image_id}

For supporting more image sizes and qualities, you should edit the config file which resides in webp_server_volume:

docker volume ls -f name=webp_server_volume --format "{{ .Mountpoint }}"

And then, restart the server:

docker container restart webp-server

FAQ

  • What is webp-server?

    webp-server is a dynamic image resizer and format converter server built on top of libvips, bimg, and fasthttp. Backend developers can run this program on their server machines and upload images to it instead of storing them. It will return an image_id which needs to be saved on a database by the backend application (on a varchar field with a length of at least 12). By using that image_id, web clients can request images from webp-server and get them in the appropriate size and format.

    Here is an example request URL for an image cropped to 500x500 size.

    https://example.com/image/w=500,h=500,fit=cover/(image_id)
    
  • What are the benfits of serving images in WebP format?

    According to Google Developers website:

    WebP is a modern image format that provides superior lossless and lossy compression for images on the web. Using WebP, webmasters and web developers can create smaller, richer images that make the web faster.

    Although nowadays most web browsers support WebP, less than 1% of websites serve their images in this format. That's maybe because converting images to WebP can be complicated and time-consuming or developers don't know what to do with the browsers which don't support WebP.

  • What can webp-server do about the browsers which don't support WebP?

    When browsers request an image, they will send an accept header containing supported image formats. webp-server will lookup that header to see if the browser supports WebP or not. If not, it will send the image in JPEG.

  • Isn't it resource expensive to convert images on each request?

    Yes, it is. For this reason, webp-server will cache each converted image after the first request.

  • What about security topics such as DOS attacks or heavy storage usage?

    It is up to you. You can limit the combinations of widths and heights or qualities that you accept from the client in webp-server configuration file, and by doing that you will narrow down the type of accepted requests for generating images. In case of serving requests from the cache, powered by fasthttp, webp-server can be blazingly fast.

  • Can web clients upload images to webp-server and send the image_id to a web server?

    It is strongly recommended not to do this and also do not share your webp-server token with frontend applications for security reasons. Process should be like this: Frontend uploads the image to the backend, backend uploads it to wepb-server, and stores the returning image_id in database.

  • What is the advantage of using webp-server instead of similar projects?

    It is simple and minimal and has been designed to work along with the backend applications for serving images of websites in WebP format. It does not support all kinds of manipulations that one can do with images. It does a few things and tries to do them perfectly.

Installation

There are two methods for running webp-server. Either use docker or build it yourself:

Docker

docker run -d -v webp_server_volume:/var/lib/webp-server --name webp-server -e TOKEN='MY_STRONG_TOKEN' -p 127.0.0.1:8080:8080 ms68/webp-server

Download Binary

webp-server is depending on libvips=>8.9. On Ubuntu 20.04 You can install it By the command below:

sudo apt install libvips

After installation, you can check your libvips version by running this command:

vips -v

Download the binary from here:

wget https://github.com/mehdipourfar/webp-server/releases/download/v1.0.0/webp-server_1.0.0_linux_amd64.tar.gz

Build From Source

sudo apt install libvips-dev git


## in Case you don't have Golang installed on your system.

wget https://golang.org/dl/go1.15.6.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.15.6.linux-amd64.tar.gz
export GOPATH=$HOME/go
export PATH=$PATH:/usr/local/go/bin

go get -u -v github.com/mehdipourfar/webp-server
sudo cp $HOME/go/bin/webp-server /usr/bin/


# Download and edit `example-config.yml` to your desired config
wget https://raw.githubusercontent.com/mehdipourfar/webp-server/master/example-config.yml

# Run the server:
webp-server -config example-config.yml

Configuration

There is an example configuration file example-config.yml in the code directory. Here is the list of parameters that you can configure:

  • data_dir: Data directory in which images and cached images are stored. Note that in this directory, there will be two separate directories named images and caches. You can remove the caches directory at any point in time if you wanted to free up some disk space.

  • server_address: Combination of ip:port. Default value is 127.0.0.1:8080.

  • token: The token that your backend application should send in the request header for upload and delete operations.

  • default_image_quality: When converting images, webp-server uses this value for conversion quality in case the user omits the quality option in the request. The default value is 95. By decreasing this value, size and quality of the image will be decreased.

  • valid_image_qualities: List of integer values from 10 to 100 which will be accepted from users as the quality option. (Narrow down these values to prevent attackers from creating too many cache files for your images.)

  • valid_image_sizes: List of string values in (width)x(height) format which will be accepted from users as width and height options. In case you want your users to be able to set width=500 without providing height, you can add 500x0 to the values list. (Narrow down these values to prevent attackers from creating too many cache files for your images.)

  • max_uploaded_image_size: Maximum size of accepted uploaded images in Megabytes.

  • debug: When set to true /image/ API does not check if width, height, and quality are included in valid_image_sizes and valid_image_qualities. It can be useful when you are developing your frontend applications and you are not yet sure which sizes and qualities you want. But do not set it to true on production server.

Backend APIs

  • /upload/ [Method: POST]: Accepts image in multipart/form-data format with a field name of image_file. You should also pass the Token previously set in your configuration file as a header. All responses are in JSON format. If request is successful, you will get 200 status code with such body: {"image_id": "lulRDHbMg"} (Note that image_id length can vary from 9 to 12). Otherwise, depending on the error, you will get 4xx or 5xx status code with a body like this: {"error": "reason of error"}.

    Example:

    curl -H 'Token: 456e910f-3d07-470d-a862-1deb1494a38e' -X POST -F '[email protected]/path/to/image.png' http://127.0.0.1:8080/upload/
  • /delete/(image_id) [Method: DELETE]: Accepts image_id as URL parameter. If the image is deleted without a problem, the server will return 204 status code with an empty body. Otherwise, it will return 4xx or 5xx with an error message in JSON format.

    Example:

    curl -H 'Token: 456e910f-3d07-470d-a862-1deb1494a38e' -X DELETE "http://localhost:8080/delete/lulRDHbMg";
  • /health/ [Method: GET]: It returns 200 status code if the server is up and running. It can be used by container managers to check the status of a webp-server container.

Frontend APIs

  • /image/(image_id) [Method: GET]: Returns the image which has been uploaded to webp-server in original size and format.

  • /image/(filter_options)/(image_id) [Method: GET]: Returns the filtered image with content-type based on Accept header of the browser. Filter options can be these parameters:

    • w, width: Width of the requested image.
    • h, height: Height of the requested image.
    • q, quality: Quality of the requested image. The default value should be set in the server config.
    • fit: Accepts cover, contain and scale-down as value.
      • contain: Image will be resized (shrunk or enlarged) to be as large as possible within the given width or height while preserving the aspect ratio. This is the default value for fit.
      • scale-down: Image will be shrunk in size to fully fit within the given width or height, but won’t be enlarged.
      • cover: Image will be resized to exactly fill the entire area specified by width and height, and will cropped if necessary.

Some example image urls:

http://example.com/image/w=500,h=500/lulRDHbMg
http://example.com/image/w=500,h=500,q=95/lulRDHbMg
http://example.com/image/w=500,h=500,fit=cover/lulRDHbMg
http://example.com/image/w=500,h=500,fit=contain/lulRDHbMg
http://example.com/image/w=500,fit=contain/lulRDHbMg

Reverse Proxy

webp-server does not support SSL or domain name validation. It is recommended to use a reverse proxy such as nginx in front of it. It should only cover frontend APIs. Backend APIs should be called locally. Here is a minimal nginx configuration that redirects all the paths which start with /image/ to webp-server.

upstream webp_server {
    server 127.0.0.1:8080 fail_timeout=0;
}

server {
   # ...

   location /image/ {
        proxy_redirect off;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Scheme $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Protocol $scheme;
        proxy_pass http://webp_server;
    }
}

Security Checklist

  • Set debug config to false value in production.
  • Narrow down valid_image_qualities and valid_image_sizes to the values you really want.
  • From the outside of the server, webp-server address should not be accessible, and users should only be able to see the /image/ path through your reverse proxy.
Issues
  • sharex config

    sharex config

    is it compatible with sharex and is there any config for it?

    opened by rafalohaki 1
Owner
Mehdi Pourfar
Mehdi Pourfar
Cloud function + website for resizing, cropping and bordering images for pragalicious.com

Cloud function + website for resizing, cropping and bordering images for pragalicious.com

Ard Scheirlynck 0 Jan 22, 2022
A lightning fast image processing and resizing library for Go

govips A lightning fast image processing and resizing library for Go This package wraps the core functionality of libvips image processing library by

David Byttow 614 Jan 12, 2022
Image resizing in pure Go and SIMD

rez Package rez provides image resizing in pure Go and SIMD. Download: go get github.com/bamiaux/rez Full documentation at http://godoc.org/github.com

Benoît Amiaux 202 Jan 3, 2022
Pure golang image resizing

This package is no longer being updated! Please look for alternatives if that bothers you. Resize Image resizing for the Go programming language with

Jan Schlicht 2.8k Jan 16, 2022
🔍 Go tool for LSB steganography, capable of hiding any file within an image.

stegify Overview stegify is a simple command line tool capable of fully transparent hiding any file within an image or set of images. This technique i

Dimitar Petrov 976 Jan 15, 2022
Image resizing for the Go programming language with common interpolation methods

This package is no longer being updated! Please look for alternatives if that bothers you. Resize Image resizing for the Go programming language with

null 1 Dec 14, 2021
A Go package converting a monochrome 1-bit bitmap image into a set of vector paths.

go-bmppath Overview Package bmppath converts a monochrome 1-bit bitmap image into a set of vector paths. Note that this package is by no means a sophi

tunabay 1 Nov 3, 2021
Image - This repository holds supplementary Go image librariesThis repository holds supplementary Go image libraries

Go Images This repository holds supplementary Go image libraries. Download/Insta

null 0 Jan 5, 2022
Image compression codec for 16 bit medical images

MIC - Medical Image Codec This library introduces a lossless medical image compression codec MIC for 16 bit images which provides compression ratio si

Kuldeep S 0 Dec 26, 2021
darkroom - An image proxy with changeable storage backends and image processing engines with focus on speed and resiliency.

Darkroom - Yet Another Image Proxy Introduction Darkroom combines the storage backend and the image processor and acts as an Image Proxy on your image

Gojek 180 Jan 6, 2022
An API which allows you to upload an image and responds with the same image, stripped of EXIF data

strip-metadata This is an API which allows you to upload an image and responds with the same image, stripped of EXIF data. How to run You need to have

Cristina Simionescu 0 Nov 25, 2021
Easily customizable Social image (or Open graph image) generator

fancycard Easily customizable Social image (or Open graph image) generator Built with Go, Gin, GoQuery and Chromedp Build & Run Simply, Clone this rep

Youngbin Han 4 Jan 14, 2022
Storage and image processing server written in Go

Mort An S3-compatible image processing server written in Go. Still in active development. Features HTTP server Resize, Rotate, SmartCrop Convert (JPEG

Marcin Kaciuba 443 Dec 27, 2021
An image server toolkit in Go (Golang)

Image Server An image server toolkit in Go (Golang) Features HTTP server Resize (GIFT, nfnt resize, Graphicsmagick) Rotate Crop Convert (JPEG, GIF (an

Pierre Durand 1.9k Jan 14, 2022
Pure Golang Library that allows simple LSB steganography on images

Steganography Lib Steganography is a library written in Pure go to allow simple LSB steganography on images. It is capable of both encoding and decodi

Rafael Passos 123 Dec 29, 2021
Fast, simple, scalable, Docker-ready HTTP microservice for high-level image processing

imaginary Fast HTTP microservice written in Go for high-level image processing backed by bimg and libvips. imaginary can be used as private or public

Tom 4.2k Jan 18, 2022
Imaging is a simple image processing package for Go

Imaging Package imaging provides basic image processing functions (resize, rotate, crop, brightness/contrast adjustments, etc.). All the image process

Grigory Dryapak 4.1k Jan 18, 2022
Simple image compression using SVD

SVD image compression An implementation image compression using SVD decomposition on Go Built With Go 1.17 Gonum Compression examples Header Image Ori

null 4 Jan 20, 2022
magicimage is a simple image validation & save with rich feature package for net/http

Installation go get github.com/IndominusByte/magicimage Usage examples A few usage examples can be found below. See the documentation for the full lis

Nyoman Pradipta Dewantara 2 Jan 10, 2022