Content aware image resize library

Overview

Caire Logo

Build Status GoDoc license release homebrew snapcraft

Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.

How does it work

  • An energy map (edge detection) is generated from the provided image.
  • The algorithm tries to find the least important parts of the image taking into account the lowest energy values.
  • Using a dynamic programming approach the algorithm will generate individual seams across the image from top to down, or from left to right (depending on the horizontal or vertical resizing) and will allocate for each seam a custom value, the least important pixels having the lowest energy cost and the most important ones having the highest cost.
  • Traverse the image from the second row to the last row and compute the cumulative minimum energy for all possible connected seams for each entry.
  • The minimum energy level is calculated by summing up the current pixel with the lowest value of the neighboring pixels from the previous row.
  • Traverse the image from top to bottom and compute the minimum energy level. For each pixel in a row we compute the energy of the current pixel plus the energy of one of the three possible pixels above it.
  • Find the lowest cost seam from the energy matrix starting from the last row and remove it.
  • Repeat the process.

The process illustrated:

Original image Energy map Seams applied
original sobel debug

Features

Key features which differentiates this library from the other existing open source solutions:

  • Customizable command line support
  • Support for both shrinking or enlarging the image
  • Resize image both vertically and horizontally
  • Can resize all the images from a directory
  • Does not require any third party library
  • Use of sobel threshold for fine tuning
  • Use of blur filter for increased edge detection
  • Square the image with a single command
  • Support for proportional scaling
  • Face detection to avoid face deformation
  • Support for multiple output image type (jpg, jpeg, png, bmp, gif)

Face detection

The library is capable of detecting human faces prior resizing the images by using the Pigo (https://github.com/esimov/pigo) face detection library, which does not require to have OpenCV installed.

The image below illustrates the application capabilities for human face detection prior resizing. It's clearly visible from the image that with face detection activated the algorithm will avoid cropping pixels inside the detected faces, retaining the face zone unaltered.

Original image With face detection Without face detection
Original With Face Detection Without Face Detection

Sample image source

Install

First, install Go, set your GOPATH, and make sure $GOPATH/bin is on your PATH.

$ export GOPATH="$HOME/go"
$ export PATH="$PATH:$GOPATH/bin"

Next download the project and build the binary file.

$ go get -u -f github.com/esimov/caire/cmd/caire
$ go install

MacOS (Brew) install

The library can also be installed via Homebrew.

$ brew tap esimov/caire
$ brew install caire

Usage

$ caire -in input.jpg -out output.jpg

Supported commands:

$ caire --help

The following flags are supported:

Flag Default Description
in - Input file
out - Output file
width n/a New width
height n/a New height
perc false Reduce image by percentage
square false Reduce image to square dimensions
scale false Proportional scaling
blur 1 Blur radius
sobel 10 Sobel filter threshold
debug false Use debugger
face false Use face detection
angle float Plane rotated faces angle
cc string Cascade classifier

Use the face detection option to avoid face deformation

To detect faces prior rescaling use the -face flag and provide the face classification binary file included into the data folder. The sample code below will rescale the provided image with 20% but will search for human faces prior rescaling.

For the face detection related arguments check the Pigo documentation.

$ caire -in input.jpg -out output.jpg -face=1 -cc="data/facefinder" -perc=1 -width=20

Other options

In case you wish to scale down the image by a specific percentage, it can be used the -perc boolean flag. In this case the values provided for the width and height options are expressed in percentage and not pixel values. For example to reduce the image dimension by 20% both horizontally and vertically you can use the following command:

$ caire -in input/source.jpg -out ./out.jpg -perc=1 -width=20 -height=20 -debug=false

Also the library supports the -square option. When this option is used the image will be resized to a square, based on the shortest edge.

The -scale option will resize the image proportionally. First the image is scaled down preserving the image aspect ratio, then the seam carving algorithm is applied only to the remaining points. Ex. : given an image of dimensions 2048x1536 if we want to resize to the 1024x500, the tool first rescale the image to 1024x768 and will remove only the remaining 268px.

Notice: Using the -scale option will reduce drastically the processing time. Use this option whenever is possible!

The CLI command can process all the images from a specific directory:

$ caire -in ./input-directory -out ./output-directory

You can also use stdin and stdout with -:

$ cat input/source.jpg | caire -in - -out - >out.jpg

in and out default to - so you can also use:

$ cat input/source.jpg | caire >out.jpg
$ caire -out out.jpg < input/source.jpg

Caire integrations

snapcraft caire

Results

Shrunk images

Original Shrunk
broadway_tower_edit broadway_tower_edit
waterfall waterfall
dubai dubai
boat boat

Enlarged images

Original Extended
gasadalur gasadalur
dubai dubai

Useful resources

Author

License

Copyright © 2018 Endre Simo

This project is under the MIT License. See the LICENSE file for the full license text.

Comments
  • Algorithm doesnt consider mask properly

    Algorithm doesnt consider mask properly

    I was playing around with the mask parameter and I feel that the algorithm sometimes distorts the masked objects. For example in this image I marked the girl as important, still in output her face was not preserved. Is there any parameter where I can force the algorithm to not touch the masked object (like completely ignore it)

    example sal_example output

    opened by kartikwar 16
  • Missing sources on Linux (Ubuntu) - maybe instructions could be updated

    Missing sources on Linux (Ubuntu) - maybe instructions could be updated

    Describe the bug

    If I do go install github.com/esimov/caire/cmd/[email protected] I am getting following error:

    In file included from ../../go/pkg/mod/[email protected]/internal/vk/vulkan_x11.go:13:
    /usr/include/vulkan/vulkan.h:59:10: fatal error: X11/Xlib.h: No such file or directory
       59 | #include <X11/Xlib.h>
          |          ^~~~~~~~~~~~
    

    I tried earlier to include in my go file in import ("github.com/esimov/caire") And experienced following errors:

    No such file or directory for:

    • <wayland-client.h>
    • <vulkan/vulkan.h>

    I could fix missing source files by installing dev version of the libs on Linux/Ubuntu with:

    apt install libwayland-dev
    apt install libwayland-dev
    
    

    Bug with the Desktop version (please complete the following information):

    Ubuntu 22.04.1 LTS

    uname -r                                             
    5.15.0-46-generic
    
    

    -

    Ideally the source files are not needed. I assume that would only be possible if you have precompiled binaries.

    opened by kivi 12
  • "cannot rescale to this size preserving the image aspect ratio"

    When I try to rescale this image: https://wallhaven.cc/w/dgpj33 from its original size 3840x1080 to new size 3046x1050, this happens: obrazek

    "cannot rescale to this size preserving the image aspect ratio" seems to always show up. I was under the impression that the whole purpose of this utility is to change the aspect ratio of the image?

    investigation 
    opened by JanPokorny 11
  • Enlarging an image with face detect panics

    Enlarging an image with face detect panics

    I am attempting to enlarge an image by 200% with face detection. It panics:

    	p := &caire.Processor{
    		NewWidth:   1024,
    		NewHeight:  2048,
    		FaceDetect: true,
    	}
    
    	resized, err := p.Resize(src)
    	if err != nil {
    		return nil, err
    	}
    
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1046efcc8]
    
    goroutine 1 [running, locked to thread]:
    github.com/esimov/pigo/core.(*Pigo).RunCascade(0x208?, {0xaa, 0x400, 0x3fb999999999999a, 0x3ff199999999999a, {{0x14000400000, 0x80000, 0x80000}, 0x200, 0x400, ...}}, ...)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/core/pigo.go:216 +0x68
    github.com/esimov/caire.(*Carver).ComputeSeams(0x14000172240, 0x140001442c0, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/carver.go:102 +0x1cc
    github.com/esimov/caire.(*Processor).enlarge(0x140001442c0, 0x1045c8338?, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:581 +0xbc
    github.com/esimov/caire.(*Processor).Resize.func2(0x14000135d78?, 0x1045c893c?)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:169 +0x7c
    github.com/esimov/caire.(*Processor).Resize(0x140001442c0, 0x140001721c0)
    	/Users/pj/go/pkg/mod/github.com/esimov/[email protected]/process.go:356 +0x894
    facemask/core.FaceMask({0x1047d4fb8?, 0x1400012c078?}, {0x14000135f40, 0x1, 0x1400015bf28?})
    	/Users/pj/software/facemask/core/facemask.go:86 +0x2b8
    main.main()
    	/Users/pj/software/facemask/main.go:23 +0xec
    exit status 2
    

    When I try the same operation without face-detection, OR, I try to shrink to image (with or without face-detection), it works. (Note: enlarging the image by 200% without face-detection works, but takes 10-15mins on a new Apple M1)

    face

    investigation 
    opened by pjebs 10
  • Face flag doesnt seem to work

    Face flag doesnt seem to work

    I am using the face flag to prevent faces from getting distorted (as mentioned in the documentation). But there seems to be no difference in output with/without this flag. Am I doing something wrong?

    This is the command I used

    go run github.com/esimov/caire/cmd/[email protected] -in /home/kartik/example.jpg -out output.jpg -face=1 -perc=1 -width=30

    Attaching the output as well

    output0

    opened by kartikwar 9
  • Enlarge doesn't work

    Enlarge doesn't work

    Describe the bug

    I installed it from the AUR (`yay -S caire).

    When I'm trying this command:

    caire -perc=0 -width 3840 -height 1080 -in everyone.png -out test.png
    

    than I'm getting this:

    ⚡ CAIRE is resizing the image... ✔
    Error resizing the image:
    	Reason: scale option can not be used on image enlargement
    

    How can I expand the image to 3840x1080?

    API related bug

    I don't really know what to write here... (I'm not familiar with go)

    Expected behavior

    That I'm getting the same image just bigger.

    Screenshots

    • [Screenshots, logs or errors]

    image

    Bug with the Desktop version (please complete the following information):

    • Sytem information like OS: [e.g. macOS, Ubuntu]
    • You are using the binary file from the uploaded releases or you are doing a manual build?

    As I said in the beginning I installed it from the AUR and run it via CLI.

    Additional context

    investigation 
    opened by TornaxO7 9
  • imported and not used

    imported and not used "fmt"

    I get this error when I use the lib as described in this tutorial: https://getstream.io/blog/go-1-11-rocket-tutorial/

    # github.com/esimov/caire ../go/pkg/mod/github.com/esimov/[email protected]/process.go:18:2: imported and not used: "fmt"

    Is there something related to the way the tutorial imports CAIRE ? If I delete the "fmt" import myself, then it works fine.

    opened by AdrianTudC 9
  • error while loading shared libraries: libEGL.so.1 (installed caire using snap)

    error while loading shared libraries: libEGL.so.1 (installed caire using snap)

    Installed caire using sudo snap install caire.

    Then I gave caire command. This is the error I'm getting

    /snap/caire/128/bin/caire: error while loading shared libraries: libEGL.so.1: cannot open shared object file: No such file or directory

    opened by NarenZen 8
  • caire never completes

    caire never completes

    Hello, I just gave caire a try against one of my images and it seems to be spinning non-stop.

    ./caire -in ./source.JPG -out ./dest.JPG -width 480 -height 720

    And, I then changed to perc 20 and this worked but produced the image that is almost 3 times the original size.

    Not sure what I am missing. Also, -debug doesn't print anything ...

    Let me know what additional info you need ...

    Thanks!

    opened by foobargeez 8
  • Preserve Jpeg color profile

    Preserve Jpeg color profile

    When I try to use the lib on a jpeg input file to a jpeg output format the colour profile is not preserved resulting in a shifting in the output colours

    The solution would possibly be to retain the input colour profile or being able to assign a desired colour profile to the output

    investigation 
    opened by purplefish32 6
  • Homebrew Installation is Out of Date

    Homebrew Installation is Out of Date

    Trying to take the library for a spin, but it seems like the Homebrew installation is out of date. After installing and running caire --help I get the following output:

    Usage of caire:
      -blur int
        	Blur radius (default 1)
      -debug
        	Use debugger
      -height int
        	New height
      -in string
        	Source
      -out string
        	Destination
      -perc
        	Reduce image by percentage
      -sobel int
        	Sobel filter threshold (default 10)
      -square
        	Reduce image to square dimensions
      -width int
        	New width
    

    Some notable flags missing are -scale and -face. I've tried to use both of these as my images never seem to finish processing.

    I tried the alternative installation method, but when I run go get -u -f github.com/esimov/caire/cmd/caire I get the following error (could open a separate issue for this if needed):

    # gocv.io/x/gocv
    imgproc.cpp:27:26: error: expected '(' for function-style cast or type construction
    imgproc.cpp:30:19: error: expected '(' for function-style cast or type construction
    imgproc.cpp:133:28: error: expected '(' for function-style cast or type construction
    

    Anyone else experiencing something similar?

    opened by quicklikerabbit 6
  • Text detection

    Text detection

    Is your feature request related to a problem? Please describe.

    I'm looking for a mechanism to resize images that contain text overlays.

    Describe the solution you'd like

    I would love if the mechanism for preserving faces could apply to OCR detected text.

    Additional context

    cover2 output2

    opened by jadencarver 2
  • Is a tutorial available

    Is a tutorial available

    Is your feature request related to a problem? Please describe.

    Seems like an amazing library but I can't find any tutorials and wondered if there are any.

    Describe the solution you'd like

    Even a brief tutorial with a few examples would be great.

    Describe alternatives you've considered

    I will inquire of more experienced go programmers than myself but would love to use this tech.

    Additional context

    Thanks I've searched web a bit and for as many stars as this repository I'm amazed there aren't more demo's of it's use.

    opened by esobczak1970 1
Releases(v1.4.5)
Owner
Endre Simo
Programmer, crafting things and ideas mostly around graphics, image processing, computer vision and machine learning.
Endre Simo
Asset storage and on-the-fly image resize

air - Asset & Image Resize Asset storage and on-the-fly image resize powered by libvips. Uploading an asset $ http -f POST http://127.0.0.1:1323/uploa

Alexandre Ferland 0 Feb 5, 2022
Image - This repository holds supplementary Go image librariesThis repository holds supplementary Go image libraries

Go Images This repository holds supplementary Go image libraries. Download/Insta

null 0 Jan 5, 2022
Resize images and animated GIFs in Go

Lilliput relies on mature, high-performance C libraries to do most of the work of decompressing, resizing and compressing images. It aims to do as little memory allocation as possible and especially not to create garbage in Go. As a result, it is suitable for very high throughput image resizing services.

Discord 1.8k Nov 19, 2022
Resize upladed images to s3 bucket with given sizes, and uploades new images back to bucket

Features Resize upladed images to s3 bucket with given sizes, and uploades new images back to bucket Environment Variables IMAGE_SIZES - formax 200x20

null 1 Feb 2, 2022
A collection of (ANSI-sequence aware) text reflow operations & algorithms

reflow A collection of ANSI-aware methods and io.Writers helping you to transform blocks of text. This means you can still style your terminal output

Christian Muehlhaeuser 401 Nov 21, 2022
darkroom - An image proxy with changeable storage backends and image processing engines with focus on speed and resiliency.

Darkroom - Yet Another Image Proxy Introduction Darkroom combines the storage backend and the image processor and acts as an Image Proxy on your image

Gojek 202 Nov 27, 2022
Easily customizable Social image (or Open graph image) generator

fancycard Easily customizable Social image (or Open graph image) generator Built with Go, Gin, GoQuery and Chromedp Build & Run Simply, Clone this rep

Youngbin Han 4 Jan 14, 2022
An API which allows you to upload an image and responds with the same image, stripped of EXIF data

strip-metadata This is an API which allows you to upload an image and responds with the same image, stripped of EXIF data. How to run You need to have

Cristina Simionescu 0 Nov 25, 2021
Imgpreview - Tiny image previews for HTML while the original image is loading

imgpreview This is a Go program that generates tiny blurry previews for images t

Dmitry Chestnykh 8 May 22, 2022
Go package for fast high-level image processing powered by libvips C library

bimg Small Go package for fast high-level image processing using libvips via C bindings, providing a simple programmatic API. bimg was designed to be

Tom 2.1k Nov 23, 2022
Image processing library and rendering toolkit for Go.

blend Image processing library and rendering toolkit for Go. (WIP) Installation: This library is compatible with Go1. go get github.com/phrozen/blend

Guillermo Estrada 61 Nov 11, 2022
A lightning fast image processing and resizing library for Go

govips A lightning fast image processing and resizing library for Go This package wraps the core functionality of libvips image processing library by

David Byttow 795 Nov 25, 2022
A fast, correct image dithering library in Go.

dither is a library for dithering images in Go. It has many dithering algorithms built-in, and allows you to specify your own. Correctness is a

makeworld 262 Nov 19, 2022
ColorX is a library to determine the most prominent color in an image written in golang

ColorX is a library to determine the most prominent color in an image. ColorX doesn't use any sort of complex algorithms to calculate the prominent color, it simply loops over the image pixels and returns the color that occurs the most.

Hesham Abourgheba 1 Nov 11, 2021
A library for basic image processing in Go.

Imaging Package imaging provides basic image processing functions (resize, rotate, crop, brightness/contrast adjustments, etc.). All the image process

null 0 Nov 26, 2021
A library for basic image processing in Go.

Imaging Package imaging provides basic image processing functions (resize, rotate, crop, brightness/contrast adjustments, etc.). All the image process

null 0 Nov 26, 2021
A Go-language library for the automatic generation of image collages.

CollageCreator is a Go-language library for the automatic generation of image collages.

August Schwerdfeger 0 Jan 29, 2022
Image processing algorithms in pure Go

bild A collection of parallel image processing algorithms in pure Go. The aim of this project is simplicity in use and development over absolute high

Anthony N. Simon 3.7k Nov 22, 2022
Decode embedded EXIF meta data from image files.

goexif Provides decoding of basic exif and tiff encoded data. Still in alpha - no guarantees. Suggestions and pull requests are welcome. Functionality

Robert Carlsen 540 Nov 19, 2022