Content aware image resize library

Overview

Caire Logo

Build Status GoDoc license release homebrew snapcraft

Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.

How does it work

  • An energy map (edge detection) is generated from the provided image.
  • The algorithm tries to find the least important parts of the image taking into account the lowest energy values.
  • Using a dynamic programming approach the algorithm will generate individual seams across the image from top to down, or from left to right (depending on the horizontal or vertical resizing) and will allocate for each seam a custom value, the least important pixels having the lowest energy cost and the most important ones having the highest cost.
  • Traverse the image from the second row to the last row and compute the cumulative minimum energy for all possible connected seams for each entry.
  • The minimum energy level is calculated by summing up the current pixel with the lowest value of the neighboring pixels from the previous row.
  • Traverse the image from top to bottom and compute the minimum energy level. For each pixel in a row we compute the energy of the current pixel plus the energy of one of the three possible pixels above it.
  • Find the lowest cost seam from the energy matrix starting from the last row and remove it.
  • Repeat the process.

The process illustrated:

Original image Energy map Seams applied
original sobel debug

Features

Key features which differentiates this library from the other existing open source solutions:

  • Customizable command line support
  • Support for both shrinking or enlarging the image
  • Resize image both vertically and horizontally
  • Can resize all the images from a directory
  • Does not require any third party library
  • Use of sobel threshold for fine tuning
  • Use of blur filter for increased edge detection
  • Square the image with a single command
  • Support for proportional scaling
  • Face detection to avoid face deformation
  • Support for multiple output image type (jpg, jpeg, png, bmp, gif)

Face detection

The library is capable of detecting human faces prior resizing the images by using the Pigo (https://github.com/esimov/pigo) face detection library, which does not require to have OpenCV installed.

The image below illustrates the application capabilities for human face detection prior resizing. It's clearly visible from the image that with face detection activated the algorithm will avoid cropping pixels inside the detected faces, retaining the face zone unaltered.

Original image With face detection Without face detection
Original With Face Detection Without Face Detection

Sample image source

Install

First, install Go, set your GOPATH, and make sure $GOPATH/bin is on your PATH.

$ export GOPATH="$HOME/go"
$ export PATH="$PATH:$GOPATH/bin"

Next download the project and build the binary file.

$ go get -u -f github.com/esimov/caire/cmd/caire
$ go install

MacOS (Brew) install

The library can also be installed via Homebrew.

$ brew tap esimov/caire
$ brew install caire

Usage

$ caire -in input.jpg -out output.jpg

Supported commands:

$ caire --help

The following flags are supported:

Flag Default Description
in - Input file
out - Output file
width n/a New width
height n/a New height
perc false Reduce image by percentage
square false Reduce image to square dimensions
scale false Proportional scaling
blur 1 Blur radius
sobel 10 Sobel filter threshold
debug false Use debugger
face false Use face detection
angle float Plane rotated faces angle
cc string Cascade classifier

Use the face detection option to avoid face deformation

To detect faces prior rescaling use the -face flag and provide the face classification binary file included into the data folder. The sample code below will rescale the provided image with 20% but will search for human faces prior rescaling.

For the face detection related arguments check the Pigo documentation.

$ caire -in input.jpg -out output.jpg -face=1 -cc="data/facefinder" -perc=1 -width=20

Other options

In case you wish to scale down the image by a specific percentage, it can be used the -perc boolean flag. In this case the values provided for the width and height options are expressed in percentage and not pixel values. For example to reduce the image dimension by 20% both horizontally and vertically you can use the following command:

$ caire -in input/source.jpg -out ./out.jpg -perc=1 -width=20 -height=20 -debug=false

Also the library supports the -square option. When this option is used the image will be resized to a square, based on the shortest edge.

The -scale option will resize the image proportionally. First the image is scaled down preserving the image aspect ratio, then the seam carving algorithm is applied only to the remaining points. Ex. : given an image of dimensions 2048x1536 if we want to resize to the 1024x500, the tool first rescale the image to 1024x768 and will remove only the remaining 268px.

Notice: Using the -scale option will reduce drastically the processing time. Use this option whenever is possible!

The CLI command can process all the images from a specific directory:

$ caire -in ./input-directory -out ./output-directory

You can also use stdin and stdout with -:

$ cat input/source.jpg | caire -in - -out - >out.jpg

in and out default to - so you can also use:

$ cat input/source.jpg | caire >out.jpg
$ caire -out out.jpg < input/source.jpg

Caire integrations

snapcraft caire

Results

Shrunk images

Original Shrunk
broadway_tower_edit broadway_tower_edit
waterfall waterfall
dubai dubai
boat boat

Enlarged images

Original Extended
gasadalur gasadalur
dubai dubai

Useful resources

Author

License

Copyright © 2018 Endre Simo

This project is under the MIT License. See the LICENSE file for the full license text.

Issues
  • "cannot rescale to this size preserving the image aspect ratio"

    When I try to rescale this image: https://wallhaven.cc/w/dgpj33 from its original size 3840x1080 to new size 3046x1050, this happens: obrazek

    "cannot rescale to this size preserving the image aspect ratio" seems to always show up. I was under the impression that the whole purpose of this utility is to change the aspect ratio of the image?

    investigation 
    opened by JanPokorny 11
  • Enlarging an image with face detect panics

    Enlarging an image with face detect panics

    I am attempting to enlarge an image by 200% with face detection. It panics:

    	p := &caire.Processor{
    		NewWidth:   1024,
    		NewHeight:  2048,
    		FaceDetect: true,
    	}
    
    	resized, err := p.Resize(src)
    	if err != nil {
    		return nil, err
    	}
    
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1046efcc8]
    
    goroutine 1 [running, locked to thread]:
    github.com/esimov/pigo/core.(*Pigo).RunCascade(0x208?, {0xaa, 0x400, 0x3fb999999999999a, 0x3ff199999999999a, {{0x14000400000, 0x80000, 0x80000}, 0x200, 0x400, ...}}, ...)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/core/pigo.go:216 +0x68
    github.com/esimov/caire.(*Carver).ComputeSeams(0x14000172240, 0x140001442c0, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/carver.go:102 +0x1cc
    github.com/esimov/caire.(*Processor).enlarge(0x140001442c0, 0x1045c8338?, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:581 +0xbc
    github.com/esimov/caire.(*Processor).Resize.func2(0x14000135d78?, 0x1045c893c?)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:169 +0x7c
    github.com/esimov/caire.(*Processor).Resize(0x140001442c0, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:356 +0x894
    facemask/core.FaceMask({0x1047d4fb8?, 0x1400012c078?}, {0x14000135f40, 0x1, 0x1400015bf28?})
    	/Users/pj/software/facemask/core/facemask.go:86 +0x2b8
    main.main()
    	/Users/pj/software/facemask/main.go:23 +0xec
    exit status 2
    

    When I try the same operation without face-detection, OR, I try to shrink to image (with or without face-detection), it works. (Note: enlarging the image by 200% without face-detection works, but takes 10-15mins on a new Apple M1)

    face

    investigation 
    opened by pjebs 10
  • Enlarge doesn't work

    Enlarge doesn't work

    Describe the bug

    I installed it from the AUR (`yay -S caire).

    When I'm trying this command:

    caire -perc=0 -width 3840 -height 1080 -in everyone.png -out test.png
    

    than I'm getting this:

    ⚡ CAIRE is resizing the image... ✔
    Error resizing the image:
    	Reason: scale option can not be used on image enlargement
    

    How can I expand the image to 3840x1080?

    API related bug

    I don't really know what to write here... (I'm not familiar with go)

    Expected behavior

    That I'm getting the same image just bigger.

    Screenshots

    • [Screenshots, logs or errors]

    image

    Bug with the Desktop version (please complete the following information):

    • Sytem information like OS: [e.g. macOS, Ubuntu]
    • You are using the binary file from the uploaded releases or you are doing a manual build?

    As I said in the beginning I installed it from the AUR and run it via CLI.

    Additional context

    investigation 
    opened by TornaxO7 9
  • imported and not used

    imported and not used "fmt"

    I get this error when I use the lib as described in this tutorial: https://getstream.io/blog/go-1-11-rocket-tutorial/

    # github.com/esimov/caire ../go/pkg/mod/github.com/esimov/[email protected]/process.go:18:2: imported and not used: "fmt"

    Is there something related to the way the tutorial imports CAIRE ? If I delete the "fmt" import myself, then it works fine.

    opened by AdrianTudC 9
  • error while loading shared libraries: libEGL.so.1 (installed caire using snap)

    error while loading shared libraries: libEGL.so.1 (installed caire using snap)

    Installed caire using sudo snap install caire.

    Then I gave caire command. This is the error I'm getting

    /snap/caire/128/bin/caire: error while loading shared libraries: libEGL.so.1: cannot open shared object file: No such file or directory

    opened by NarenZen 8
  • caire never completes

    caire never completes

    Hello, I just gave caire a try against one of my images and it seems to be spinning non-stop.

    ./caire -in ./source.JPG -out ./dest.JPG -width 480 -height 720

    And, I then changed to perc 20 and this worked but produced the image that is almost 3 times the original size.

    Not sure what I am missing. Also, -debug doesn't print anything ...

    Let me know what additional info you need ...

    Thanks!

    opened by foobargeez 8
  • Homebrew Installation is Out of Date

    Homebrew Installation is Out of Date

    Trying to take the library for a spin, but it seems like the Homebrew installation is out of date. After installing and running caire --help I get the following output:

    Usage of caire:
      -blur int
        	Blur radius (default 1)
      -debug
        	Use debugger
      -height int
        	New height
      -in string
        	Source
      -out string
        	Destination
      -perc
        	Reduce image by percentage
      -sobel int
        	Sobel filter threshold (default 10)
      -square
        	Reduce image to square dimensions
      -width int
        	New width
    

    Some notable flags missing are -scale and -face. I've tried to use both of these as my images never seem to finish processing.

    I tried the alternative installation method, but when I run go get -u -f github.com/esimov/caire/cmd/caire I get the following error (could open a separate issue for this if needed):

    # gocv.io/x/gocv
    imgproc.cpp:27:26: error: expected '(' for function-style cast or type construction
    imgproc.cpp:30:19: error: expected '(' for function-style cast or type construction
    imgproc.cpp:133:28: error: expected '(' for function-style cast or type construction
    

    Anyone else experiencing something similar?

    opened by quicklikerabbit 6
  • Free image after processing

    Free image after processing

    Hi,

    thank you for this awesome library. I aspire to use it in my application for "cropping" profile images to squares when they are not uploaded as squares.

    But: The library does not seem to free its used memory when it's done processing which means my server hits memory limit after two or three users uploaded their image. (200MB to 1.5GB, depending on the image)

    The code I used to resize images

    Please remind me if I have missed something and there is an option to cleanup.

    enhancement 
    opened by jeyemwey 5
  • Square option problem

    Square option problem

    Hi, when I try to resize an image with size 2400 * 2400 with setting New_width = 180, scale = true, Square = true, face detection = true, it will not resize the image and output the original image after process, if Square option is removed, then it will resize the image correctly. I wonder if it is config problem or others?

    Thank you

    opened by zluo01 5
  • Leaky ASCII colours

    Leaky ASCII colours

    Hi there, I resized an image and the yellow/green highlight got left on. I guess this may be a bug in terminator code? Is an expression missing?

    screen shot 2018-08-13 at 4 35 02 pm

    Thanks,

    Alex

    opened by alexellis 5
  • Wrap CAIRE up into an app

    Wrap CAIRE up into an app

    Hey! I feel it'd be great having this as tool as a Web App for general public to make use of. Would you be interested in building a backend over this so that a Web App (which I could build) could use it as an API.

    Anyone interested in such a project feel free start a discussion cause I see potential in this!

    -ajayns

    enhancement 
    opened by ajayns 5
  • Detect if the resize operation would deform an image with human faces.

    Detect if the resize operation would deform an image with human faces.

    Is your feature request related to a problem? Please describe.

    Let's suppose that an image is bigger vertically than horizontally, but also contains sensible information like a person face. If we wish to reduce the image vertically by a high order of magnitude this will results in a condensed, ugly and almost unrecognizable person face. We wish to avoid this, so in some way we need to detect if an image contains a person face and the applied resize would deform the resulted image.

    Here is an example:

    | Original image | Resulted image | |:--:|:--:| | winking-teenage-girl-life-in-viewscience-photo-library | winking-teenage-girl-life-in-viewscience-photo-library |

    Describe the solution you'd like

    Since we already using Pigo for face detection we can detect if an image contains human faces prior resizing but also we can analyze if the resize operation could be performed without distorting the generated image. In the other cases we can either skip the image resize operation and leave the image unaltered showing a warning message or we can scale the image to the most appropriate dimension.

    opened by esimov 0
Releases(v1.4.4)
Owner
Endre Simo
Programmer, crafting things and ideas mostly around graphics, image processing, computer vision and machine learning.
Endre Simo
Asset storage and on-the-fly image resize

air - Asset & Image Resize Asset storage and on-the-fly image resize powered by libvips. Uploading an asset $ http -f POST http://127.0.0.1:1323/uploa

Alexandre Ferland 0 Feb 5, 2022
Image - This repository holds supplementary Go image librariesThis repository holds supplementary Go image libraries

Go Images This repository holds supplementary Go image libraries. Download/Insta

null 0 Jan 5, 2022
Resize images and animated GIFs in Go

Lilliput relies on mature, high-performance C libraries to do most of the work of decompressing, resizing and compressing images. It aims to do as little memory allocation as possible and especially not to create garbage in Go. As a result, it is suitable for very high throughput image resizing services.

Discord 1.7k May 6, 2022
Resize upladed images to s3 bucket with given sizes, and uploades new images back to bucket

Features Resize upladed images to s3 bucket with given sizes, and uploades new images back to bucket Environment Variables IMAGE_SIZES - formax 200x20

null 1 Feb 2, 2022
A collection of (ANSI-sequence aware) text reflow operations & algorithms

reflow A collection of ANSI-aware methods and io.Writers helping you to transform blocks of text. This means you can still style your terminal output

Christian Muehlhaeuser 336 May 10, 2022
darkroom - An image proxy with changeable storage backends and image processing engines with focus on speed and resiliency.

Darkroom - Yet Another Image Proxy Introduction Darkroom combines the storage backend and the image processor and acts as an Image Proxy on your image

Gojek 188 Apr 26, 2022
Easily customizable Social image (or Open graph image) generator

fancycard Easily customizable Social image (or Open graph image) generator Built with Go, Gin, GoQuery and Chromedp Build & Run Simply, Clone this rep

Youngbin Han 4 Jan 14, 2022
An API which allows you to upload an image and responds with the same image, stripped of EXIF data

strip-metadata This is an API which allows you to upload an image and responds with the same image, stripped of EXIF data. How to run You need to have

Cristina Simionescu 0 Nov 25, 2021
Imgpreview - Tiny image previews for HTML while the original image is loading

imgpreview This is a Go program that generates tiny blurry previews for images t

Dmitry Chestnykh 7 Mar 12, 2022
Go package for fast high-level image processing powered by libvips C library

bimg Small Go package for fast high-level image processing using libvips via C bindings, providing a simple programmatic API. bimg was designed to be

Tom 1.9k May 12, 2022
Image processing library and rendering toolkit for Go.

blend Image processing library and rendering toolkit for Go. (WIP) Installation: This library is compatible with Go1. go get github.com/phrozen/blend

Guillermo Estrada 59 Dec 24, 2021
A lightning fast image processing and resizing library for Go

govips A lightning fast image processing and resizing library for Go This package wraps the core functionality of libvips image processing library by

David Byttow 678 May 17, 2022
A fast, correct image dithering library in Go.

dither is a library for dithering images in Go. It has many dithering algorithms built-in, and allows you to specify your own. Correctness is a

makeworld 248 May 16, 2022
ColorX is a library to determine the most prominent color in an image written in golang

ColorX is a library to determine the most prominent color in an image. ColorX doesn't use any sort of complex algorithms to calculate the prominent color, it simply loops over the image pixels and returns the color that occurs the most.

Hesham Abourgheba 1 Nov 11, 2021
A library for basic image processing in Go.

Imaging Package imaging provides basic image processing functions (resize, rotate, crop, brightness/contrast adjustments, etc.). All the image process

null 0 Nov 26, 2021
A library for basic image processing in Go.

Imaging Package imaging provides basic image processing functions (resize, rotate, crop, brightness/contrast adjustments, etc.). All the image process

null 0 Nov 26, 2021
A Go-language library for the automatic generation of image collages.

CollageCreator is a Go-language library for the automatic generation of image collages.

August Schwerdfeger 0 Jan 29, 2022
Image processing algorithms in pure Go

bild A collection of parallel image processing algorithms in pure Go. The aim of this project is simplicity in use and development over absolute high

Anthony N. Simon 3.5k May 12, 2022
Decode embedded EXIF meta data from image files.

goexif Provides decoding of basic exif and tiff encoded data. Still in alpha - no guarantees. Suggestions and pull requests are welcome. Functionality

Robert Carlsen 521 May 13, 2022