Documentation
¶
Overview ¶
Package visual provides perceptual hashing algorithms for image comparison.
The package implements two complementary perceptual hashing algorithms:
dHash (Difference Hash) ¶
dHash is fast and effective for detecting identical or near-identical images. Algorithm:
- Resize image to 9x8 grayscale
- Compare adjacent pixels horizontally (left < right = 1, else 0)
- Produce 64-bit hash (8 rows x 8 comparisons per row)
pHash (Perceptual Hash) ¶
pHash is more robust to minor variations using the Discrete Cosine Transform. Algorithm:
- Resize image to 32x32 grayscale
- Apply 2D DCT (Discrete Cosine Transform)
- Extract 8x8 low-frequency components
- Compute hash from median comparison
Tiles ¶
Both hashes see the page as a whole. TileDiff cuts a page into a grid of 64x90 tiles (90x64 for a landscape page) and returns the largest difference between the mean grey levels of two pages' matching tiles. A tile averages thousands of pixels, so JPEG noise washes out of it, while a stamp or a changed paragraph, which the hashes miss, moves a tile by tens of grey levels. A difference above TileTolerance is a change. TileSimilarity scores it 1 - d/TileScale, which is 0.95 at the tolerance.
Similarity Calculation ¶
Image similarity is computed using Hamming distance between hashes:
h1 := visual.DHash(img1) h2 := visual.DHash(img2) dist := visual.HammingDistance(h1, h2) similarity := visual.Similarity(dist) // 0.0 to 1.0
Combined Similarity ¶
For best results, use both hash types:
similarity := visual.CombinedSimilarity(dHash1, dHash2, pHash1, pHash2)
The combined score is the plain average of the dHash similarity and the pHash similarity: 1 - (d + p)/128 for Hamming distances d and p.
Index ¶
- Constants
- func ChangedPixels(img1, img2 image.Image) (int, bool)
- func CombinedDistance(dHash1, dHash2, pHash1, pHash2 uint64) int
- func CombinedSimilarity(dHash1, dHash2, pHash1, pHash2 uint64) float64
- func ComputeBothHashes(img image.Image) (dHash, pHash uint64)
- func ComputeDHashPair(img1, img2 image.Image) (uint64, uint64, int)
- func ComputePHashPair(img1, img2 image.Image) (uint64, uint64, int)
- func DHash(img image.Image) uint64
- func DHashFromGray(img *image.Gray) uint64
- func HammingDistance(h1, h2 uint64) int
- func HistogramSimilarity(h1, h2 HistogramResult) float64
- func InkScore(n int) float64
- func IsDifferent(h1, h2 uint64) bool
- func IsIdentical(h1, h2 uint64) bool
- func IsNearDuplicate(h1, h2 uint64) bool
- func IsSimilar(h1, h2 uint64) bool
- func PHash(img image.Image) uint64
- func PHashFast(img image.Image) uint64
- func Similarity(distance int) float64
- func SimilarityFromHashes(h1, h2 uint64) float64
- func TileMeans(img image.Image, cols, rows int) []float64
- func TileScore(diff float64) float64
- func ToGrayscale(img image.Image) *image.Gray
- type HistogramResult
- type Tile
Constants ¶
const ( // HashBits is the number of bits in our perceptual hashes HashBits = 64 // IdenticalThreshold - hashes are bit-for-bit identical IdenticalThreshold = 0 // NearDuplicateThreshold - very similar (minor compression differences) NearDuplicateThreshold = 5 // SimilarThreshold - visually similar SimilarThreshold = 10 // DifferentThreshold - clearly different DifferentThreshold = 15 )
const ( // InkContrast is how far, in levels of a colour channel, a pixel must // stand from a flat area of the other page to count as changed: new // ink, or ink gone. InkContrast = 64 // InkTolerance is the most changed pixels a block of 2x2 tiles may hold // and still count as the same. A pixel changed when it stands out from // the flat area around the same place on the other page (see InkDiff). // Two renders of the same content written by different tools differ by // sub-pixel moves, the weight of hairlines and the grey of form-field // borders, all at edges, where the other page is not flat; Ghostscript // and MuPDF rewrites of eight real documents (forms, papers, a 230-page // manual) changed at most 13 pixels of a block at 150 DPI. A 6pt pen // tick on paper changed 16 to 36, two swapped digits 18 to 103. InkTolerance = 15 // InkScale is the changed-pixel count that scores a block 0: // InkTolerance scores 0.95, as TileTolerance does for TileScore. InkScale = 20 * InkTolerance // PhotoTolerance is the largest difference, in grey levels, between two // tiles' mean intensities that InkDiff's verdict stands against. Inside // a photo nothing is flat, so InkDiff sees no edit there; a tile whose // mean moved by more than this changed whatever InkDiff says. Rewrites // and rasterised copies of eight real documents moved no tile by more // than 23 at 150 DPI. PhotoTolerance = 30.0 )
const ( // TileCols and TileRows are the tile grid of a portrait page. A // landscape page is cut the other way round, so a tile is about 1/64 of // the long side wide. TileCols = 64 TileRows = 90 // TileTolerance is the largest difference, in grey levels, between two // tiles' mean intensities that still counts as the same page. Measured // through pdfium on scans of eight seeds, re-encoding between JPEG // quality 92 and 50 moved no tile by more than 2.1 grey levels at 72, 150 // and 300 DPI, while a stamp of 60x30 px or more on an 840x1188 px scan // moved one tile by at least 16 (TestTileSeparation checks the same split // on decoded JPEGs). A second scan of the same paper, shifted by up to // 6 px, leaning by up to a degree, exposed 8% differently, blurred and // noisier, moves no tile of the aligned check (AlignedTileDiff) by more // than 3.4 on soft pages or pages of text (TestAlignedTileSeparation), // while a 6 px dot, a 30 px long 2 px line or a 20x20 px tick moves one by // more than 6; marks as large as 40x40 px move one by more than 20. TileTolerance = 6.0 // TileScale is the difference that scores a tile 0. A tile scores // 1 - diff/TileScale, so a difference of TileTolerance scores 0.95. TileScale = 20 * TileTolerance )
const ( // TileMargin is the number of tiles along each edge of the page that // AlignedTileDiff does not compare. A rescan sits a few pixels off and // leans, so content enters and leaves at the edge and nothing there has a // counterpart. TileMargin = 1 )
Variables ¶
This section is empty.
Functions ¶
func ChangedPixels ¶ added in v0.4.0
ChangedPixels counts the pixels that differ by InkContrast or more on the darkest or the brightest colour channel between two renders. It reports false for pages of different sizes.
func CombinedDistance ¶
CombinedDistance calculates average Hamming distance from multiple hash pairs. Useful for comparing dHash and pHash together.
func CombinedSimilarity ¶
CombinedSimilarity calculates average similarity from multiple hash pairs.
func ComputeBothHashes ¶
ComputeBothHashes computes both dHash and pHash for an image.
func ComputeDHashPair ¶
ComputeDHashPair computes dHashes for two images and their hamming distance.
func ComputePHashPair ¶
ComputePHashPair computes pHashes for two images and their hamming distance.
func DHash ¶
DHash computes a difference hash for an image. The algorithm: 1. Resize image to 9x8 grayscale 2. Compare adjacent pixels horizontally (left < right = 1, else 0) 3. Produce 64-bit hash (8 rows x 8 comparisons per row)
dHash is fast and good at detecting identical or near-identical images. It's less robust to rotation/scaling than pHash but faster to compute.
func DHashFromGray ¶
DHashFromGray computes dHash from an already-grayscale image. This is faster when you know the image is already grayscale.
func HammingDistance ¶
HammingDistance calculates the number of differing bits between two hashes. Uses hardware POPCNT instruction via bits.OnesCount64 for maximum speed.
func HistogramSimilarity ¶
func HistogramSimilarity(h1, h2 HistogramResult) float64
HistogramSimilarity compares two histograms and returns a similarity score [0, 1].
func InkScore ¶ added in v0.4.0
InkScore converts a changed-pixel count to a score from 0 to 1: 1 - n/InkScale, so InkTolerance scores 0.95.
func IsDifferent ¶
IsDifferent returns true if hashes exceed the different threshold.
func IsIdentical ¶
IsIdentical returns true if hashes match exactly.
func IsNearDuplicate ¶
IsNearDuplicate returns true if hashes are within near-duplicate threshold.
func PHash ¶
PHash computes a perceptual hash for an image using DCT. The algorithm: 1. Resize image to 32x32 grayscale 2. Compute 2D DCT (Discrete Cosine Transform) 3. Extract top-left 8x8 block (low frequencies) 4. Compute mean of DCT values (excluding DC component) 5. Set bits based on whether each value exceeds the mean
pHash is more robust to minor image changes (compression, scaling) than dHash, but slower to compute due to DCT. PHash computes a perceptual hash for an image using DCT. The algorithm: 1. Resize image to 32x32 grayscale 2. Compute 2D DCT (Discrete Cosine Transform) 3. Extract top-left 8x8 block (low frequencies) 4. Compute mean of DCT values (excluding DC component) 5. Set bits based on whether each value exceeds the mean
pHash is more robust to minor image changes (compression, scaling) than dHash, but slower to compute due to DCT.
func PHashFast ¶
PHashFast computes an average hash (aHash), not a perceptual hash: it resizes the image straight to 8x8 and sets a bit for each pixel brighter than the mean. There is no DCT, so it is faster than PHash and less tolerant of noise and gradients.
func Similarity ¶
Similarity converts a Hamming distance to a similarity score (0.0 to 1.0).
func SimilarityFromHashes ¶
SimilarityFromHashes calculates similarity directly from two hashes.
func TileMeans ¶
TileMeans cuts a page into a cols x rows grid of tiles and returns the mean grey level of each, row by row. Tiles are fractions of the page, so the result does not depend on the render DPI. It returns nil when the page has fewer pixels than tiles.
A tile averages thousands of pixels, so JPEG noise and ringing wash out of it while a local edit moves it. The perceptual hashes cannot see such an edit: they reduce the whole page to 9x8 or 32x32 samples.
Types ¶
type HistogramResult ¶
type HistogramResult struct {
AverageR float64
AverageG float64
AverageB float64
Luminance float64
}
HistogramResult contains basic color distribution info
func ComputeHistogram ¶
func ComputeHistogram(img image.Image) HistogramResult
ComputeHistogram calculates basic color metrics for an image. This is used as a fast heuristic to skip more expensive dHash/pHash if the images are obviously different in color distribution.
type Tile ¶
Tile is one cell of a page's tile grid.
func AlignedTileDiff ¶ added in v0.4.0
AlignedTileDiff is TileDiff for two scans of the same paper, which are not the same picture: the second sits a few pixels off, leans by a fraction of a degree and is exposed a little differently. So the second page's exposure is fitted to the first's (see fitExposure), the shift between the pages is found (see estimateShift), and a tile is compared with the closest tile within a pixel or so of where it should be, in both directions, so that a mark on either page is found. The outer TileMargin tiles are not compared. The pages must be the same size; otherwise the result is TileDiff's.
Do not use it on rendered text: it forgives a page whose content moved by a few pixels, which is a real change there.
func InkDiff ¶ added in v0.4.0
InkDiff finds where ink appeared or disappeared between two renders of a born-digital page and returns how many pixels changed in the worst block of 2x2 tiles, and its top-left tile. A pixel changed when the pixels around the same place on the other page, one pixel each way, are flat and it stands at least InkContrast from them, in either direction, on the darkest or the brightest colour channel: dark ink on paper, a yellow highlight, light text on a dark slide. Edges that moved by a pixel, or lines drawn a little heavier, sit next to edges on the other page and do not count. It reports false, and no tile, for pages of different sizes or too small to cut into tiles.
Use it on rendered pages, not scans: a rescan moves by more than a pixel (see AlignedTileDiff).
func TileDiff ¶
TileDiff finds the tile that differs most between the pages and returns the difference between its mean grey levels. Both pages are cut on the grid of the first, so a portrait page against a landscape one differs everywhere. It returns no tile when a page is too small to cut into tiles.