Two images of the same photo. One was uploaded as a JPEG, the other was screenshotted, scaled 5%, and re-uploaded as a PNG. They look identical to a human. Their SHA-256 hashes share zero bits in common. Cryptographic hashing detects exact duplicates. Perceptual hashing detects near-duplicates.

Why Cryptographic Hashes Don’t Work#

A single pixel difference changes SHA-256 completely. Re-encoding, resizing, or adding a watermark produces a completely different cryptographic hash. Detecting “this is the same photo someone already uploaded” requires a similarity measure, not an equality check.

Content fingerprinting covers SimHash for near-duplicate text detection. For images, the equivalent is perceptual hashing: dHash, pHash, or aHash.

How dHash Works#

Difference hash (dHash):

  1. Resize the image to 9x8 pixels (discards fine detail, keeps structure)
  2. Convert to grayscale
  3. For each row, compare adjacent pixels: left pixel brighter than right? Output 1. Otherwise 0.
  4. This produces 64 bits (8 rows x 8 comparisons each)

The resulting 64-bit integer is the perceptual hash. Two nearly identical images produce hashes that differ in very few bits. The Hamming distance between their hashes (number of differing bits) is the similarity measure.

Image A hash: 1001101001110110...  (64 bits)
Image B hash: 1001101001100110...  (64 bits)
Hamming distance: 2 bits different
→ Images are likely the same photo with minor variation

Threshold: Hamming distance < 10 = near-duplicate
graph TD A[Uploaded image] --> B[Resize to 9x8 grayscale] B --> C[Compare adjacent pixels row by row] C --> D[64-bit perceptual hash] D --> E[Compare against existing hashes in database] E --> F{Hamming distance < threshold?} F -->|Yes| G[Near-duplicate detected: same image] F -->|No| H[New unique image: store hash] style A fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style B fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style C fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style D fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style E fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style F fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff style G fill:#000000,stroke:#ff0000,stroke-width:2px,color:#fff style H fill:#000000,stroke:#00ff00,stroke-width:2px,color:#fff

Scaling the Comparison#

Comparing a new image against 10 billion stored hashes naively is O(N). That’s too slow.

The trick: split the 64-bit hash into segments (say, 4 segments of 16 bits each). Index each segment separately. Two images with Hamming distance < 10 must share at least one segment that matches exactly (by pigeonhole: 10 differences spread across 4 segments means at most 3 segments can differ in each, so at least 1 must be identical).

Query: find all images where any one of the 4 hash segments matches. That’s 4 exact-match index lookups, returning a small candidate set. Then compute Hamming distance against each candidate. Near-duplicate detection in O(log N) average case.

At Salesforce#

We had users uploading report templates that were essentially identical to existing ones, leading to clutter. We used a content hash of the SQL query plus display configuration to detect exact duplicates. We didn’t implement perceptual hashing because the content was structured, not images. But the problem was the same: find “similar” things without doing a full pairwise comparison. The segmented index approach applies equally to any fixed-width binary fingerprint.

What I’m Learning#

Perceptual hashing trades exact comparison for approximate comparison, enabling similarity search in near-O(log N) time. The segmented index technique is general: it works for any compact binary fingerprint where you want nearest-neighbor search under Hamming distance.

Have you needed to detect similarity rather than equality in your systems, and what approach did you use?