PAR2 Recovery Data Explained: Repairing Damaged Backups in Place

Most backup failures are loud. A drive will not spin up, a profile reports an error, a folder has gone. You know about them straight away and you deal with them.

The failure that does the most harm is the quiet one. A backup file that was perfect when it was written picks up a damaged sector, or a few flipped bits, while it sits on a drive for a year. Its name, size and date are unchanged, so nothing looks wrong. You only find out on the day you need it, when the photo opens half grey or the archive refuses to extract.

PAR2 is a long-established, open answer to that problem. It stores a small amount of carefully calculated extra data alongside a file, and that extra data can later be used to find damage and rebuild the damaged parts, without needing the original. This article explains where PAR2 came from, how it works, what it is good at and where it falls short, and how to use it in SyncBackPro.

The problem: files that go bad quietly

Storage is very reliable, but not perfectly so, and a backup is exactly the kind of data that sits untouched long enough for the small risks to add up. Hard drives and SSDs use error correction internally, and almost all of the time they either return the right data or report that a sector cannot be read. Occasionally neither happens. Damage gets in through a failing controller, a firmware fault, a power cut in the middle of a write, faulty memory during the copy, or flash storage that has been left unpowered for too long. The usual name for all of this is bit rot.

Ordinary backup software is poorly placed to notice. A backup program decides what to copy by comparing files, and a file that has decayed at the destination still has the size and date it always had. Unless something reads every byte and checks it against a record made at the time, the damage goes unseen. Worse, it can be carried forward, because a damaged backup file that is never re-copied stays damaged in every later run.

Two things are needed to deal with this properly: a way to detect damage, and a way to repair it. Hashes give you the first. SyncBackPro has long been able to record a hash of every backup file and check it later, which we describe in Ensuring Data Integrity in Backups and What is Hashing?. A hash will tell you a file has changed. It cannot tell you what the file used to say. For that you need either another copy, or recovery data like PAR2.

A short history of PAR2

PAR2 came out of Usenet. In the late 1990s and early 2000s, large files were posted to newsgroups split into dozens or hundreds of parts, and some parts always went missing on their way between servers. Asking for a missing part to be posted again was slow and often did not work. What people wanted was a way to post a little extra data up front that could stand in for whichever parts happened to go astray.

The first answer was the Parchive project. In July 2001 Tobias Rieper and Stefan Wehlus proposed the Parity Volume Set specification, and version 1.0, now called PAR1, was published in October 2001. PAR1 worked on whole files: each recovery file could replace one complete missing file from the set. That suited a set of equal-sized parts, but it had real limits. A set could hold at most 255 files, every recovery file had to be as large as the largest file in the set, and a single bad byte cost you an entire file's worth of recovery data.

PAR2 fixed that by working on small blocks, called slices, rather than whole files. Building on earlier proposals, Michael Nahas published the version 2.0 specification in September 2002, and the final revision is dated 11th May 2003. PAR2 also moved the arithmetic from 8-bit to 16-bit, which allows far more slices and recovery slices than PAR1 could manage. Peter Clements wrote the first two implementations, QuickPar for Windows and the cross-platform par2cmdline.

More than twenty years on, PAR2 is still the version in everyday use. par2cmdline is maintained by the Parchive project on GitHub, and Yutaka Sawada wrote MultiPar as the successor to QuickPar on Windows, and still develops it. A successor, PAR3, has been in development since 2019 and an alpha specification was published in January 2022, but PAR2 remains the format that tools everywhere can read. Its usefulness has also spread well beyond Usenet, to anyone who wants long-term files to be able to survive a little damage: archivists, photographers, and people keeping data on optical discs and drives that live in a cupboard.

How PAR2 works

The idea in miniature

Suppose a file consisted of just two numbers, 7 and 5. Alongside it you store one extra number, their sum, 12. If the 5 is later lost, you can work it out: 12 minus 7. If the 7 is lost instead, 12 minus 5 gives it back. One extra number can replace either of the originals, without knowing in advance which will be lost. It cannot replace both at once.

That is parity, the same principle RAID 5 uses across disks. PAR2 uses a more powerful family of the same idea, based on Reed-Solomon codes. Instead of one sum, it calculates as many extra values as you ask for, each worked out from all of the original data in a different way. With N of them you can solve for N missing values, wherever in the file they happen to be.

Slices, checksums and recovery slices

To protect a file, PAR2 divides it into equal pieces called slices. It records two checksums for every slice, an MD5 hash and a CRC32, so that later it can tell exactly which slices are intact and which are not. Then it calculates a number of extra recovery slices, each the same size as a data slice.

The redundancy percentage decides how many recovery slices there are. At 10%, a file divided into 300 slices gets 30 recovery slices. Those 30 can repair any 30 damaged slices, in any combination: thirty scattered across the file, or thirty in a row.

A recovery slice is not a copy of any part of the file. Each one is computed from the whole file at once, and that is what makes the scheme so flexible. You do not need to guess where damage will happen and protect that area. You only need to decide how much damage you want to be able to survive.

The index file and the recovery volumes

PAR2 keeps detection and repair in separate files. A protected file called data.bin typically gets:

• data.bin.par2, the index. It holds the file's name, its size and the checksum of every slice, and no recovery data at all. That is enough to say whether the file is damaged and exactly which slices are affected, but not enough to mend it.
• Several recovery volumes, named like data.bin.vol00+01.par2, which hold the recovery slices. They are written in increasing sizes so that a repair only has to read as much recovery data as the damage calls for.

Each recovery volume also carries its own copy of the descriptive information from the index, so the set still works if the index file itself is lost or damaged.

What PAR2 is good at

• It finds damage rather than assuming there is none. Verification reads every slice and checks it against the recorded checksums, so silent corruption is found instead of being carried forward.
• Repair needs nothing but the damaged file and its recovery data. There is no return to the source and no second copy to fetch. That matters most when the source has since changed or gone.
• The file is mended where it stands. It keeps its name, its dates and its place in the backup, and nothing else has to be rewritten.
• It works on any kind of file. Documents, photos, video, virtual disks, archives: PAR2 works on bytes and needs no understanding of the format.
• The space it needs is predictable. You choose the redundancy, and the recovery data costs roughly that percentage of the protected data, plus a little overhead.
• It is an open standard. Your ability to repair a file does not depend on any one program, or on any one company still being around.
• Checking is practical. Verifying a file reads it once and runs at about the same speed as creating its recovery data, so a regular scheduled check is realistic even for a large backup.

What PAR2 cannot do

• It cannot bring back a file that has gone. Deleting a file, or losing the drive it was on, loses every slice at once, and rebuilding a whole file from nothing would take 100% redundancy. The same goes for a file overwritten from end to end, for instance by ransomware: nearly every slice changes at once, far beyond what any sensible redundancy will mend. For that you need versioning and an offline copy, as described in our ransomware recovery guide.
• The budget is fixed. Ten percent redundancy repairs damage to 10% of a file's slices and not one slice more. A file with more damage than that cannot be repaired at all.
• Damage anywhere in a slice costs the whole slice. A single flipped bit and a completely destroyed slice cost the same to repair. Damage scattered thinly through a file uses up the budget much faster than the same number of bad bytes in one place.
• It protects the file as it was when the recovery data was made. If a file was already wrong when it was protected, PAR2 will faithfully protect the wrong content. It proves a file has not changed since; it cannot prove the file was right to begin with.
• The recovery files can rot too. They are ordinary files and are not themselves protected. PAR2 detects damage to them and ignores the affected parts, so a damaged recovery volume lowers how much can be repaired rather than causing a bad repair.
• Small files are expensive to protect. Every protected file gets its own set of recovery files, each repeating the same descriptive information, so the overhead is proportionally much larger for a small file than for a large one.
• It costs disk space, and time when the recovery data is created. The figures for SyncBackPro are given below.
• The arithmetic is not quite perfect. The matrix the PAR2 specification uses is not guaranteed to be solvable for every combination of damaged slices and recovery slices, and a reported case shows a repair that ought to succeed failing because of it. Such combinations are rare, but they are one more reason not to set the redundancy at exactly the amount of damage you expect.

PAR2 compared with other kinds of protection

PAR2 is one tool among several, and each of them answers a different question. The table below is a simplification, but it shows where each one helps.

PAR2's particular strength is that it travels with the files and needs no special hardware or file system. A USB drive formatted with NTFS or exFAT, an ageing NAS, or a network share you do not control can all hold PAR2-protected files, and the protection goes with them if they are copied elsewhere.

Using PAR2 in SyncBackPro

SyncBackPro V12.1.34 and newer can create and maintain PAR2 recovery data for the files a profile writes to its destination, check those files later, and repair any it finds damaged. The recovery data is written as standard PAR2 files, so other PAR2 tools can read and use it. The settings are on the PAR2 page of the profile settings. It is available only in SyncBackPro, not in SyncBackSE or SyncBackFree (see upgrading from SyncBackSE to SyncBackPro).

Watch the video on using PAR2 in SyncBackPro to protect backups from bit rot and repair damaged files.

An Insider Access feature

PAR2 recovery data is an Insider Access feature. Insider Access is part of Upgrade Assurance, and lets subscribers use features destined for the next major release before it ships (see Why Upgrade to V12). PAR2 is planned as a standard feature of SyncBackPro V13; in V12 it is available through Insider Access, so you can use it while you have an active Upgrade Assurance subscription and have turned Insider Access on. Insider Access is turned on and off from the Serial Number menu in the main window.

If Insider Access is turned off later, nothing is lost. A profile with recovery data switched on carries on backing up as normal, but no recovery data is created or checked, and each run says so with a warning. The recovery data already created is kept, and when Insider Access is turned back on the next run brings it up to date. When you turn Insider Access off, SyncBackPro lists the profiles that use it and asks you to confirm.

Getting started

1. Turn on Insider Access from the Serial Number menu.
2. Open the settings of a backup profile that writes to a local drive or network share, go to the PAR2 page and tick Create PAR2 recovery data for destination files. The default 10% redundancy and 1 MB minimum file size suit most backups.
3. Run the profile. The first run protects everything already in the backup, so it takes longer than usual; later runs only protect what they copy.
4. Schedule a regular Integrity Check for the profile, so that damage is found and repaired while it is still small.

Which profiles can use it

Recovery data is most useful where checking it is cheap and going back to the source is expensive, so it is restricted to backups written to a plain local drive or network share. The page is hidden unless the profile qualifies (in Expert mode it is shown greyed out instead). A profile does not qualify if:

• it is a group profile
• its destination is not a local drive or network share, which rules out FTP, cloud services, SyncBack Touch, MTP devices, location scripts and backup of email
• the destination is compressed, into a single zip file or one zip file per file
• it uses Fast Backup, which never scans the destination and so has nothing to maintain the recovery data against
• it deletes everything on the destination before it runs
• it uses delta copy versioning on the destination

Only the destination matters, because the destination is the backup and the backup is what recovery data protects. Copying from a cloud service, an FTP server or SyncBack Touch to a local drive is fully supported.

Why not cloud storage? Checking recovery data means reading every byte of every protected file. On a cloud destination that is a full download, with the time and data charges that brings, just to learn that the copy is still good. The major cloud storage services already guard against silent damage with checksums and redundancy of their own, and the things that really destroy cloud backups, such as a closed account, a lapsed payment, a mistaken deletion or ransomware reaching the destination, are things PAR2 cannot help with. Recovery data pays its way where the storage does no checking of its own and reading it back is cheap: a local disk, a USB drive or a NAS.

Delta copy is left out for a different reason. It keeps versions as patches against a base file and is aimed at large files that change often, the opposite of the long-lived files recovery data is for. Protecting the current file without the chain of versions it depends on would only be half a promise.

The settings

• Create PAR2 recovery data for destination files: turns the feature on. It is off by default. A file is protected when it is first copied, and its recovery data is rebuilt whenever the file is copied again. Switching it on for an existing backup is fine: the next run protects the files already there, without copying them again.
• Redundancy percentage: how much recovery data to create, as a percentage of the size of each protected file. The default is 10 and the range is 1 to 100.
• Minimum file size (KB): files smaller than this are not protected. The default is 1024 KB (1 MB) and the smallest allowed is 64 KB.
• Also create PAR2 recovery data for version files: off by default. When ticked, the older versions kept by versioning are protected as well. Versions are often the copies you reach for after discovering a problem, so this is worth considering, but the disk space used grows with the number of versions you keep.
• Store recovery data in: leave empty to keep the recovery data with the backup, or choose a folder to keep it elsewhere. The folder can use variables, such as %LOCALAPPDATA%, but not a date or time variable, which would put the recovery data in a different folder on every run.
• File Types: include and exclude masks for choosing which file types get recovery data. The profile's own filters are applied first, so a file the profile excludes is never protected; these masks then narrow down what is left.
• Remove Recovery Data: deletes the profile's recovery data, after showing how many files it protects and how much space it uses and asking you to confirm. Your backup files are not touched. Unticking the main setting deliberately leaves existing recovery data alone, so this button is how you reclaim the space.

SyncBackPro chooses the slice size for you. A file is divided into about 100 slices, but no slice is made larger than 512 KB, so files over about 50 MB get more slices, up to 2,000 for a file of about 1 GB. Beyond that the count stays at 2,000 and the slices grow instead (a 2 GB file gets 1 MB slices), which keeps the time taken to create the recovery data in proportion to the size of the file.

Where the recovery data is kept

Recovery data is never written next to the file it protects, so it does not clutter the backup. It goes in a folder named $SBP$ that mirrors the structure of the backup. With the store location left empty, that folder sits at the root of the destination, so a backup at D:\Backup gives:

D:\Backup\sub\data.bin
D:\Backup\$SBP$\sub\data.bin.par2
D:\Backup\$SBP$\sub\data.bin.vol00+01.par2
...

For most files there are five recovery files: one index and four recovery volumes. Very large files get a few more, because each volume is roughly twice the size of the one before it; at 10% redundancy a file of 256 to 512 MB gets seven or eight. If you choose a store location, the recovery data goes to <store location>\<profile name>\$SBP$ instead, so two profiles sharing a store cannot overwrite each other. Moving the store keeps the backup folder tidy or lets you put recovery data on cheaper storage, but it does not protect you against losing the backup drive: a file that has gone cannot be repaired, wherever its recovery data is.

The $SBP$ folder is never synchronized, in the same way SyncBack leaves the $SBV$ versioning folder alone.

Keeping the recovery data in step

You do not have to manage the recovery data yourself. When a file is copied again its recovery data is rebuilt, when a file is renamed or moved its recovery data moves with it, and when a file is deleted from the backup its recovery data goes too. Anything left over, such as recovery data for a file that no longer qualifies, is cleaned up by the next run, and a run that is cancelled part way through does not leave half-written recovery data behind.

A problem with recovery data never fails a backup. If the recovery data cannot be written, for example because a separate store location is unreachable, the files are still copied and the run reports a warning, so you know the backup is not protected until the problem is fixed.

Checking and repairing

Creating recovery data does not check anything. To check a backup, run an Integrity Check on the profile, from the profile's right-click menu. Every protected file is read and verified against its recovery data, and any damaged file is repaired in place if there is enough recovery data to do it. A repaired file is listed with the files that passed, marked as damaged and repaired. A file too badly damaged to repair is reported as a failed integrity check and counted as an error, naming the file. The log also gains a PAR2 Recovery Data section with the number of files protected, verified and repaired.

A file too badly damaged to repair is left exactly as it is, so you can still salvage what you can from it or restore it from another copy.

An Integrity Check verifies and repairs, but it never creates recovery data. A file that has none, for example because it was copied before the setting was switched on, is skipped. Recovery data is created by a normal run of the profile, so run the profile itself to protect anything left unprotected.

Restores are protected too. Before a restore copies anything back, every protected file about to be restored is verified and, if necessary, repaired, so what you get back is the original content rather than damaged bytes.

If the profile also records hash-based file integrity data, the two work together. Integrity checking detects problems on every kind of destination; recovery data is what lets SyncBackPro fix them without going back to the source. Because verification is fast, scheduling a regular Integrity Check is worthwhile.

What it costs

Disk space. At the default 10% redundancy, expect the recovery data to take about 12% of the size of the protected data for files around 1 MB, falling towards 10% for files of 64 MB and above. Watch the number of files as well as the space, because each protected file usually gets five recovery files. In one measured backup of 300 files and 525 MB, the 100 files above the minimum size produced 500 recovery files totalling 55 MB.

The overhead rises steeply for small files. For a requested 10%, the recovery data actually costs:

That is why the minimum file size defaults to 1 MB. If small files matter to you, a second copy of them is usually a better tool than recovery data, so raising the minimum is more often sensible than lowering it.

Time on the run that creates it. In our testing, the 64-bit version creates recovery data at about 110 MB/s for files of a few megabytes, rising to 200 to 240 MB/s for files of 100 MB and above, so protecting 10 GB of changed data adds something like one to two minutes to a run. The 32-bit version is about as fast for files up to a few hundred megabytes but much slower for very large ones, at about 75 MB/s for 1 GB files. The 64-bit version also uses the processor's AVX2 instructions for the arithmetic where they are available, which helps most with large files. The first run after you switch the setting on takes much longer than usual, because everything already at the destination is protected then. That happens once.

Time afterwards. Almost nothing. A run in which no files changed only has to confirm that each file already has its recovery data. In the measured backup above, a run with recovery data switched on and nothing to do took 1.7 seconds.

Time to check. Verification runs at about 170 to 230 MB/s in both the 32-bit and 64-bit versions, faster for larger files, and repairing a damaged 64 MB file took under two seconds.

Choosing a redundancy percentage

The default of 10% suits what recovery data is for. Bit rot and bad sectors damage a small part of a file, not most of it, and 10% covers that with a good margin. Some practical guidance:

• Keep the default for most backups on reasonably healthy drives that you check regularly.
• Go higher, perhaps 15% to 25%, for long-term archives that will be checked rarely, for drives kept unpowered on a shelf, or for older media you trust less. The longer the gap between checks, the more damage can build up before it is caught.
• Do not try to replace a second copy by pushing the figure towards 100%. Past a certain point, a second copy on another drive costs the same space and protects against far more.
• Check regularly. Recovery data repairs damage that has built up; it does not stop it building up. A scheduled Integrity Check catches damage while it is still small enough to repair.

Which files to protect

Not every file suffers equally from a bad bit. In a photo, a song or a video, a little damage is usually a smear, a click or a glitch for a moment, and the file still opens. In a compressed, encrypted or container file, such as a .zip or .7z archive, a .docx or .xlsx document, an Outlook .pst or a .vhdx virtual disk, the same damage can make the whole file unreadable. Those are the files that gain most from recovery data, and they are not always the largest.

By default every file above the minimum size is protected, which is the safe choice. If you want to save space, use File Types to leave out what is easy to replace, such as caches, thumbnails and build output, rather than to pick a few types to include: an include list quietly leaves everything else unprotected.

For scripts and advanced logging

Run-time scripts see each result as it is logged. The RunLogStatus script event receives EFSIntegrityRepaired for a file that was repaired, alongside EFSIntegritySuccess and EFSIntegrityFailed for files that passed or could not be repaired. Advanced Logging templates can use %LOGJSON_INTEGRITYREPAIRED% (or the XML and CSV forms) to list the repaired files, for example to raise an alert in a monitoring system when a backup needed repairing.

Repairing with another PAR2 tool

Because the recovery data is standard PAR2, you are not dependent on SyncBackPro to use it. We have tested it in both directions with par2cmdline: it verifies and repairs the files SyncBackPro writes, and SyncBackPro verifies and repairs the files it writes.

There is one trap worth knowing about. A PAR2 tool expects the protected file to be in the same folder as its .par2 files, and SyncBackPro deliberately keeps them apart. So running this inside the $SBP$ folder:

par2 repair data.bin.par2

reports Repair is not possible. You need 70 more recovery blocks, which is misleading. The tool looked for data.bin next to the recovery files, did not find it, and concluded the whole file was missing rather than damaged. Tell it where the data actually is with the -B (base path) option:

par2 repair -B"D:\Backup\sub" data.bin.par2

Alternatively, copy the .par2 files for that one file next to the data before running the tool. Either way works.

Where PAR2 fits in a backup strategy

The 3-2-1-1-0 strategy asks for three copies of your data, on two kinds of media, one off-site, one offline or immutable, and zero errors when the backups are verified. PAR2 does nothing for the first four numbers. It is aimed squarely at the last one: it lets you verify that a backup is error free, and when it is not, it often lets you make it error free again on the spot rather than discovering the problem on the day you need to restore.

It is most valuable where a backup is kept for a long time and checked only occasionally: archives, external drives that are rotated and stored, and backups on a NAS that has no checksumming file system of its own (see our NAS backup best practices). It is least valuable for small files and for data that changes constantly, where the recovery data is rebuilt on almost every run.

Conclusion

PAR2 has lasted for more than two decades because it solves a narrow problem very well. Given a modest amount of extra space, it can tell you exactly which parts of a file have been damaged and rebuild them, using nothing but the file and its recovery data, in an open format that any PAR2 tool can read.

It is not a substitute for a second copy, and it cannot help with a file that has gone. What it does is close the gap that hashes leave open: a hash tells you a backup has gone bad, and PAR2 recovery data lets you fix it. In SyncBackPro V12 it takes a few clicks to switch on, and a scheduled Integrity Check keeps your backups both verified and repaired.

Further reading: Ensuring Data Integrity in Backups, Upgrade Assurance and Insider Access, and the PAR 2.0 specification. You can find out more about SyncBackPro on its product page.