memorization
When a model reproduces specific training data instead of generalizing from it — the footprint a particular text leaves in the weights, detectable when the model emits it again.
A child who can recite a poem word-for-word has memorized it; a child who can only paraphrase has generalized. The distinction matters because memorization is the signature that a text was seen, which is what a copyright canary looks for. But the castle found the footprint is not one shape: exact repetition leaves a clear print, fuzzy duplicates leave a mosaic print the model assembles from many tiles, and scale dilutes any single print until the ocean of training data washes it faint.
The castle's rooms that lean on it: the-scaling-canary finds that scale dilutes the footprint and deduplication removes the exact repeat, paraphrased-canary tests whether a cluster of paraphrased variants strengthens the footprint, and near-duplicate-canary asks whether minimal edits memorize better than full paraphrases.
Links
mosaic-memory
A language model can remember something without ever seeing it repeated exactly…
WORD · brickdeduplication
Removing near-identical copies from a training set so a model does not see the s…
WORD · brickcanary trap
A canary trap is a mark planted in a work before it leaves your hands — a fictit…
ROOM · wallAs models grow and training data is deduplicated, does an ordinary author's planted copyright trap become more detectable or less — and has anyone shown a trap a frontier-scale model still betrays?
The canary was bred to sing only in one room; as the house grows, does its voice carry further, or does the larger choir drown it out?
ROOM · wallCould an author plant a cluster of paraphrased variants rather than one repeated passage — seeding the mosaic the model assembles — and would that be detectable at frontier scale without training-time access?
The canary cannot sing loud enough alone, so the thought is to seed a choir that hums the same tune in different words — but can a lone hand raise that choir, and would the larger model even hear it?
ROOM · wallCould near-duplicates (minimal edits) rather than full paraphrases stay within the fuzzy-duplicate band the mosaic mechanism rewards without crossing into the brittleness band — and would the cluster be detectable where full paraphrases are not?
The canary's neighbors hum the same note with one word changed — close enough to be the same song, far enough to dodge the filter that silences echoes.