What is bit rot, and does it really happen to photo archives?

Silent data corruption is a stored block that comes back different from what was written, with no error reported by the drive. The largest field study of it, covering 1.53 million drives over 41 months, recorded more than 400,000 checksum mismatches. It is rare per drive per year and it is not zero, and the reason it matters for photographers is that nobody reads an archive back, so nothing finds it.

Will copying my archive to a new drive detect corrupted files?

No. Copy tools decide what to transfer by comparing size and modification time, which a corrupted file usually still matches. rsync calls this its quick check and only compares file contents when you pass the checksum option. A plain copy moves the damage to the new drive and reports success.

How do I actually verify a migration?

Record a checksum for every file on the old drive, copy, then compute the checksums again on the new drive and compare the two lists. Anything that does not match is either corrupt on the old drive or was damaged in transit, and you want to know which before the old drive is wiped. Keep the old copy until the comparison passes.

How often should a photo archive move to new drives?

There is no measured answer for a drive living in a cupboard, and anyone quoting one precisely is guessing. What is measurable is that the published failure and corruption rates are annual, so the risk accumulates with every year you leave it. Pick a cadence you will genuinely carry out, write it in the calendar, and treat the verified copy rather than the new drive as the thing you are buying.

Does a RAID or a NAS solve this on its own?

Only if something reads the data back and checks it. Redundancy repairs a block once a mismatch is found, so the detection step is doing the work. File systems built around this, such as ZFS, expose it as a scrub that reads every block and verifies its checksum on a schedule. A mirror with nothing scrubbing it can hold two copies of a corrupted file.

Moving an Archive to New Drives Copies the Damage Too

Migrating a photo archive onto bigger drives every few years is the right instinct. The risk is in how the copy is made. A file that has silently corrupted keeps its name, its size and its modification date, and those three things are exactly what a copy tool looks at when it decides whether a file needs transferring. So the damaged file is carried to the new drive, reported as copied, and the old drive is then wiped or sold. The archive now contains the corruption and no longer contains the evidence.

This is a different failure from the one photographers plan for. A drive that dies announces itself. This one does not, which is why the fix is not another copy but a verification step, and why the moment to run it is the migration rather than the emergency.

What silent corruption actually is

The storage research literature calls it silent data corruption: a block that reads back differently from what was written, with no error surfaced by the drive. The 2008 study below puts it plainly, describing corruption "where the data is silently corrupted with no indication from the drive that an error has occurred" and noting that, unlike a bad sector, it "cannot be detected or repaired by the disk drive itself".

That last part is the part worth sitting with. A drive can and does report unreadable sectors. It cannot report a sector it believes is fine. Nothing in your operating system, your catalogue software or your backup tool is checking either, unless you have asked it to.

What the only large scale study found

The reference dataset is An Analysis of Data Corruption in the Storage Stack, published at USENIX FAST in 2008 by researchers at the University of Wisconsin-Madison, NetApp and the University of Toronto. It covers 1.53 million disk drives in production storage systems over 41 months, and it remains the largest field study of corruption rather than outright failure. It recorded more than 400,000 checksum mismatches across that period.

Two numbers from it are worth carrying around. Over the first 17 months in the field, 0.66% of nearline drives developed at least one checksum mismatch, against 0.06% of enterprise class drives: an order of magnitude apart, and the paper says so in those words. Averaged per year, the study puts mismatches at 0.466% of nearline drives and 0.042% of enterprise drives.

The rate also is not the interesting part for a photographer. The interesting part is how corruption gets discovered. In that study it is found by scrubbing, which means deliberately reading every block and comparing it to a stored checksum. A wedding archive is the one dataset nobody ever reads back. Delivered, downloaded, and then silent for years. Whatever is wrong in there is wrong undetected by construction.

Why the migration is the dangerous moment

The same study found that 8% of corruptions were detected during RAID reconstruction, which it notes creates the possibility of data loss. That is the worst possible moment to learn about a problem: the system is already down to its last good copy and is reading everything to rebuild.

A photographer has the same shape of moment, and it is the migration. It is the one time the whole archive is read end to end, and it is immediately followed by the irreversible step of clearing the old drive. Everything that makes that dangerous is in the default behaviour of the tools involved.

The rsync manual is explicit about it. Rsync finds files that need transferring "using a quick check algorithm (by default) that looks for files that have changed in size or in last-modified time". Passing the checksum option "changes this to compare a checksum for each file that has a matching size", and the manual warns that doing so means "both sides will expend a lot of disk I/O reading all the data in the files in the transfer". Finder and File Explorer do not offer the option at all.

So the default answer to "did everything copy?" is a comparison of names, sizes and dates. Silent corruption changes none of those.

What a verified migration looks like

The principle is simple enough to state in one line: the new drive has to prove it holds the same bytes, and the old drive is not touched until it has.

Before you copy anything, write a checksum list for the old drive, one line per file. This is the only record of what the archive was, and it is worth keeping afterwards alongside the drive.
Copy with a tool that can verify, or copy first and verify second. rsync with the checksum option does both in one pass at the cost of reading everything twice.
Recompute the checksums on the new drive and compare the two lists, file by file. A clean comparison is the actual deliverable of the migration, not the fact that the copy finished.
Investigate every mismatch before you conclude anything. A file that differs may have been corrupt on the old drive already, which means the good copy is somewhere else, or nowhere.
Only then wipe or retire the old drive, and only if a second copy of the archive exists elsewhere. A migration is not a backup, and for the duration of it you are usually down one copy.

If the archive lives on a file system that checksums data itself, the verification is continuous rather than occasional. OpenZFS documents a scrub as an operation that "examines all data in the specified pools and verifies each block's checksum", and where there is redundancy, ZFS "automatically repairs any damage discovered during the scrub". The manual draws the same distinction this post is about, describing scrubbing as what "examines all data to discover silent errors due to hardware faults or disk failure". It is the reading back that finds the problem, not the storing.

How often to move, and what that has to do with keeping

There is no honest measured answer to how many years a drive in a cupboard gets. What is measurable is that the failure and corruption rates published for managed hardware are annual rates, so the exposure is cumulative and the direction of travel is only one way. The practical cadence is whichever one you will actually perform, and a migration you schedule is worth more than a shorter one you skip.

It is also worth deciding what the archive is for before deciding what to spend on it. How long you are actually required to keep a wedding has a firmer answer than most photographers expect, and it is usually driven by the limitation period on the contract rather than by sentiment. Material you must be able to produce is worth verifying. Material you keep because deleting feels wrong is a different budget.

The two ends of the same chain are covered elsewhere in this series: where the risk actually sits on the wedding day itself, which is the window between the card leaving the camera and a second copy existing, and what cloud durability figures actually guarantee once the archive is somewhere else entirely. This post is about the middle, which is the part that only gets attention every few years and gets it under time pressure.

What to do with an afternoon

Generate a checksum list for your oldest archive drive. The run tells you two useful things: how long a full read takes, and whether the drive can still complete one.
Pick one wedding from more than three years ago and open every file in it. Not a spot check of the highlights, the whole folder.
Write the next migration date in the calendar now, with the checksum list filed next to the drive it describes.
If your archive already lives on a checksumming file system, confirm a scrub is scheduled and that you would hear about it if one failed.

Common questions

What is bit rot, and does it really happen to photo archives?: Silent data corruption is a stored block that comes back different from what was written, with no error reported by the drive. The largest field study of it, covering 1.53 million drives over 41 months, recorded more than 400,000 checksum mismatches. It is rare per drive per year and it is not zero, and the reason it matters for photographers is that nobody reads an archive back, so nothing finds it.
Will copying my archive to a new drive detect corrupted files?: No. Copy tools decide what to transfer by comparing size and modification time, which a corrupted file usually still matches. rsync calls this its quick check and only compares file contents when you pass the checksum option. A plain copy moves the damage to the new drive and reports success.
How do I actually verify a migration?: Record a checksum for every file on the old drive, copy, then compute the checksums again on the new drive and compare the two lists. Anything that does not match is either corrupt on the old drive or was damaged in transit, and you want to know which before the old drive is wiped. Keep the old copy until the comparison passes.
How often should a photo archive move to new drives?: There is no measured answer for a drive living in a cupboard, and anyone quoting one precisely is guessing. What is measurable is that the published failure and corruption rates are annual, so the risk accumulates with every year you leave it. Pick a cadence you will genuinely carry out, write it in the calendar, and treat the verified copy rather than the new drive as the thing you are buying.
Does a RAID or a NAS solve this on its own?: Only if something reads the data back and checks it. Redundancy repairs a block once a mismatch is found, so the detection step is doing the work. File systems built around this, such as ZFS, expose it as a scrub that reads every block and verifies its checksum on a schedule. A mirror with nothing scrubbing it can hold two copies of a corrupted file.