Skip to content
esimovPublic

Repository files navigation

Caire Logo

build Go Reference license release homebrew caire

Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.

How does it work

  • An energy map (edge detection) is generated from the provided image.
  • The algorithm tries to find the least important parts of the image taking into account the lowest energy values.
  • Using a dynamic programming approach the algorithm will generate individual seams across the image from top to down, or from left to right (depending on the horizontal or vertical resizing) and will allocate for each seam a custom value, the least important pixels having the lowest energy cost and the most important ones having the highest cost.
  • We traverse the image from the second row to the last row and compute the cumulative minimum energy for all possible connected seams for each entry.
  • The minimum energy level is calculated by summing up the current pixel value with the lowest value of the neighboring pixels obtained from the previous row.
  • We traverse the image from top to bottom and compute the minimum energy level. For each pixel in a row we compute the energy of the current pixel plus the energy of one of the three possible pixels above it.
  • Find the lowest cost seam from the energy matrix starting from the last row and remove it.
  • Repeat the process.

The process illustrated:

Original image Energy map Seams applied
original sobel debug

Features

Key features which differentiates this library from the other existing open source solutions:

Install

First, install Go, set your GOPATH, and make sure $GOPATH/bin is on your PATH.

$ go install github.com/esimov/caire/cmd/caire@latest

MacOS (Brew) install

The library can also be installed via Homebrew.

$ brew install caire

Supported commands:

$ caire --help

The following flags are supported:

Flag Type Description Default Value
in - Input file
out - Output file
width n/a Desired image width
height n/a Desired image height
preview bool Show GUI window true
debug bool Enable/disable debugging in preview mode false
perc bool Reduce image by percentage false
square bool Reduce image to square dimensions false
blur int Edge detector blur radius 4
sobel int Sobel filter threshold 2
faceDetect bool Activate face detection false
textDetect bool Activate text detection false
textDetInterval int The number of cycles after the text detector should be re-activated 1
cannyHigh float Canny edge detector high threshold 300
cannyLow float Canny edge detector low threshold 50
gaussian float Gaussian sigma for text detection 1.5
angle float Face rotation angle 0
mask string File path to the mask image used to retain specific image areas
rmask string File path to the mask image used to force removing specific image areas.
color string Seam color #ff0000
shape string Shape type used for debugging: circle,line circle

GUI progress indicator

GUI preview

A GUI interface is also incorporated into the library to visualize the process in real time. The GUI is based on the Gio library. Prior running it please make sure that you have installed all the required dependencies noted in the installation section (https://gioui.org/#installation) .

The preview window is activated by default but you can deactivate it any time by setting the -preview flag to false. When the images are processed concurrently from a directory the preview mode is deactivated by default.

Face detection

The library is capable to detect human faces prior resizing the images by using the lightweight Pigo (https://github.com/esimov/pigo) face detection library.

The image below illustrates the application capabilities for human face detection prior resizing. It's clearly visible that with face detection activated the algorithm will avoid cropping pixels inside the detected faces, retaining the face zone unaltered.

Original image With face detection Without face detection
Original With Face Detection Without Face Detection

Sample image source

Enable face detection to avoid face deformation

In order to detect faces prior rescaling, use the -faceDetect flag. There is no need to provide a face classification file, since it's already embedded into the generated binary file. The sample code below will resize the provided image with 20%, but checks for human faces in order tot avoid face deformations.

For face detection related settings please check the Pigo documentation.

$ caire -in input.jpg -out output.jpg -faceDetect=1 -perc=1 -width=20

Text detection

Starting with v1.6.0 the library support text detection capabilities also. This is hidden behind a feature flag named -textDetect.

Once activated the text detector will scan the image to detect text areas with identical characteristics, and mark them as grouped components belonging to the same zone. Then the color intensity is increased for each detected text zone, which tricks the Sobel detector to treat them as important areas which shouldn't be altered by the seam carver.

Note this can generate false positive results also, so adapt the flags for each image specifically.

As a rule of thumb, try to avoid using the -textDetect flag when the image does not contains texts. Otherwise play with -gaussian, -cannyHigh and -cannyLow parameters. These will tell the Canny detector how to threat the gradient changes in edge detection.

Also keep in mind that text detection will drop the performance. You can use the -textDetInterval flag to limit how often the detector should be activated during the seam removal operation.

Example

With text detection Without text detection
text_detection_disabled text_detection_enabled

Demo

Below is a screen recording of the text detector in action.

Resize-with-activated-text-detection.mp4

Usage

$ caire -in input.jpg -out output.jpg

Support for stdin and stdout pipe commands

You can also use stdin and stdout with -:

$ cat input/source.jpg | caire -in - -out - >out.jpg

in and out default to - so you can also use:

$ cat input/source.jpg | caire >out.jpg
$ caire -out out.jpg < input/source.jpg

You can provide also an image URL for the -in flag or even use curl or wget as a pipe command in which case there is no need to use the -in flag.

$ caire -in <image_url> -out <output-folder>
$ curl -s <image_url> | caire > out.jpg

Process multiple images from a directory concurrently

The library can also process multiple images from a directory concurrently. You have to provide only the source and the destination folder and the new width or height in this case.

$ caire -in <input_folder> -out <output-folder>

Support for multiple output image type

There is no need to define the output file type, just use the correct extension and the library will encode the image to that specific type.

Other options

In case you wish to scale down the image by a specific percentage, it can be used the -perc boolean flag. In this case the values provided for the width and height are expressed in percentage and not pixel values. For example to reduce the image dimension by 20% both horizontally and vertically you can use the following command:

$ caire -in input/source.jpg -out ./out.jpg -perc=1 -width=20 -height=20 -debug=false

Also the library supports the -square option. When this option is used the image will be resized to a square, based on the shortest edge.

When an image is resized on both the X and Y axis, the algorithm will first try to rescale it prior resizing, but also will preserve the image aspect ratio. The seam carving algorithm is applied only to the remaining points. Ex. : given an image of dimensions 2048x1536 if we want to resize to the 1024x500, the tool first rescale the image to 1024x768 and then will remove only the remaining 268px.

Masks support:

  • -mask: The path to the protective mask. The mask should be in binary format and have the same size as the input image. White areas represent regions where no seams should be carved.
  • -rmask: The path to the removal mask. The mask should be in binary format and have the same size as the input image. White areas represent regions to be removed.
Mask protection Mask removal
mask.mp4
rmask.mp4

Caire integrations

snapcraft caire

Results

Shrunk images

Original Shrunk
broadway_tower_edit broadway_tower_edit
waterfall waterfall
dubai dubai
boat boat

Enlarged images

Original Extended
gasadalur gasadalur
dubai dubai

Useful resources

Author

License

Copyright © 2018 Endre Simo

This project is under the MIT License. See the LICENSE file for the full license text.

Releases

Sponsor this project

Packages

Used by

Contributors

Languages

⌂ 换网址