TLDR

Proteus is a next-generation content provenance scheme for AI-generated content on the web.

Designed to overcome the limitations of traditional image watermarking (i.e. digital, latent space, & deep-learning-based), Proteus can track and identify content even after extreme transformations such as crops, screenshots, and image filters.

Proteus employs a unique approach of perceptual hashing and proof of the transformation to resiliently track content origins across the web.

Background

Why don't current schemes work?

High level, there are three approaches used in tracking content on the web:

(1) Embedding data in the real-value domain of an image/video (i.e watermark goes into pixels) (2) Embedding data into the frequency domain of an image/video (i.e. DCT/DWT) (3) Training a neural network to embed data into images (i.e. encoder/decoder)

• Data can be encoded after an image is created, or
• During image creation (i.e. diffusion models can watermark in the latent space)

The fundamental limitation in these standard approaches is: (1) capacity (i.e. how much data can we hide/encode in a piece of content) (2) imperceptibility(i.e. how clear is it that we hid data in this piece of content), and (3) robustness (i.e. how easy is it to corrupt the data that we embedded in this piece of content)

An inherent issue with attempting to embed data into visual content is that images and videos are inherently lossy. The human visual system is not able to detect granular changes in content’s underlying data, which makes modern compression and editing techniques possible. At the same time, this makes designing high-capacity, imperceptible, and robust steganographic/watermarking schemes extremely difficult.

The Proteus System

High-level overview

For a visual explanation, please visit our [system diagram] (https://www.Proteus.pics/) hosted here.

There are three major components to the Proteus system - all running on behalf of different entities.

• Image creation runs on behalf of a content-producing organization (i.e. Midjourney, DALLE, iPhone, Canon)
• Image processing runs on behalf of a content-consuming service (i.e. Twitter)
• Image verification runs on behalf of an end user (i.e. Twitter user)

(1) Image creation: First, an individual produces an image with the help of a content-producing organization (i.e. Midjourney).

Proteus computes the perceptual hashes of this image, as well as the image after a number of applied transformations (i.e. simulated screenshots, crops, rotations, etc), and commits the SHA's of each phash to a public datastore. (2) Background processing: When a new image is uploaded to a content-consuming service (i.e. Twitter), Proteus computes:

• The most similar images available in the public datastore through perceptual hash similarity.
• Whether or not it is possible to go from the uploaded image's perceptual hash to a perceptual hash from the datastore.
• If it is possible, the exact sequence of transformations it took to go from the uploaded image to the original image.

All of these computations are cached & persisted in a verified image datastore on-prem. (3) Image verification: Once a user encounters an image on a content-consuming platform (i.e. Twitter), a query to the verification datastore reveals whether this image was created by Midjourney, any other content-producing organization, or some other means (i.e. iPhone camera).