Compression

<< Click to Display Table of Contents >>

Navigation:  Using SyncBackPro > Expert Mode >

Compression

Previous pageReturn to chapter overviewNext page

 

Expert Mode: Compression Settings

 

SyncBack has the ability to compress files using the industry standard ZIP format as well as the newer BZip2 format. Compression reduces the size of the file and has the potential to save a lot of disk space, especially when files such as text and office documents are being copied. Two methods of compression are supported: all the files can be placed into one single compressed file, or each file can be placed into it's own individual compressed file. The first option uses the least amount of disk space, but has the disadvantage that "all the eggs are placed in one basket", so to speak. Also, versioning cannot be used if all files are placed into a single compressed file.

 

To increase compression performance SyncBackPro can be configured not to try and compress already compressed files, e.g. MP3's, JPG images, etc. Instead of compressing files of these types it will instead store them (without compression) in the Zip file. Note that they are still stored in a Zip file but are not compressed within the Zip file.

 

For maximum security, you can also encrypt and compress the filenames and file details within the Zip file.

 

warning

SyncBackPro is Unicode enabled, so it can store files and folders with names in any language, e.g. Chinese, in a Zip file. However, some compression programs are not Unicode enabled. Because of this if you open a Zip file produced by SyncBackPro using a non-Unicode enabled compression program then it will show the filenames incorrectly (probably with question marks). The problem is with the compression program, not the Zip file produced by SyncBackPro. The solution is to use a compression utility that is Unicode enabled, such as 7Zip, for example. Another possibility is that the compression utility uses UTF-8 Unicode encoding, e.g. WinZip 12. If so, you should configure the profile to use UTF-8 encoding in the Zip file.

 

Also note that if you open a split Zip file created by SyncBackPro in a compression program, e.g. WinZip, then it may give an error saying the file is corrupted. The problem is that it is expecting the split filenames to be named differently. See this section for details.

 

How Compression Is Used

 

When you are using the cloud, FTP, SyncBack Touch, backup to email and Media Transfer Protocol (MTP) then the compression applies to that location. This means that any files copied to that location, e.g. FTP, will be compressed to a temporary file and that compressed file will be uploaded to the location. When a file is copied from such a location then the compressed file is downloaded and uncompressed.

 

In most cases the compressed file stored in the location will have a special filename. This special filename contains details about the file stored in the compressed file. By doing this SyncBackPro can scan the location and get the details of the file within the compressed file without having to download it and extract it, which would be extremely slow. In some special cases, e.g. when using Amazon S3, it doesn't need to do this as those kinds of locations allow SyncBackPro to store meta-data along with the file and this meta-data will contain details of the file within the compressed file.

 

In the special case that you have compressed files on FTP etc. and want to download them uncompressed then you must run the profile as a restore. So you must create a backup profile where you are backing up files to the location using compression and then run that profile as a restore. Keep in mind that if the compressed files are being created by something other than SyncBackPro then they will not have the special filename that contains the details. This means SyncBackPro will not know the size nor the last modification date & time of the file inside the compressed file.

 

If you want to download files from FTP etc. and have them stored locally compressed, then this is not possible with a single profile. Instead you would need to create two profiles: the first profile would download the files to a temporary location and the second profile would then compress those files. Those two profiles would then be put into a group and you would run the group.

 

Compression Settings

 

Compress the files on destination/right into a Zip file: Enable this option to compress files copied to the destination/right into a Zip file. If you are backing up files to an FTP server, please read the Fast Backup section for tips on getting the best results.

 

Put all the files into a single compressed file (by default each file will be placed in its own compressed file): If this option is enabled, then the files will be put into a single compressed Zip file. If this option is not enabled (the default), then each file will be placed into it's own Zip file.

 

warning

When each file goes into its own Zip file, and those Zip files are being stored on an FTP server, we have the problem of knowing what is in a Zip file on a remote FTP server. To know this SyncBackPro would need to download the Zip file and open it to see what file is inside it, what it's uncompressed size is, and what it's last modification date & time is. To avoid this SyncBackPro changes the filename of Zip files stored on an FTP server by embedding this information in the filename itself.

 

However, if the filename does not contain this information, e.g. it was created on the FTP server by some other utility, then SyncBackPro will not know the files uncompressed size or its last modification date & time. This means (depending on your profile configuration) it will always assume the file has been changed since the last profile run.

 

Try and compress files that are being used: By default any file that is being used will not be copied. This is done to avoid corruption, i.e. a file could be changed by another program while a compressed version of it is being created. Enable this option to ignore this safe-guard.

 

Type of compression: There are five types of compression: Deflated (which is the default), Deflated64, Burrows Wheeler, BZip2, and LZMA.

 

Deflated provides the normal type of compression used by the older Zip format (traditional PKZIP 2.04g compression method).

 

Deflated64 (also know as Enhanced Deflate) provides a greater level of compression, but note that it will increase the compression time and is not compatible with older Zip compression programs. Deflate64™ is a trademark of PKWARE Inc.

 

Burrows Wheeler (popularized by the UNIX and Linux BZip2 program) offers significantly better compression than Deflate but takes longer to compress and decompress data. Tests have shown BWT (Burrows Wheeler Transform) to often achieve between 20% to 30% better compression than Deflate on many popular file types such as databases, pictures, text and executable files. BWT is considered to be one of the most efficient compression algorithms for compressing XML data. In comparison to BWT, Deflated64 is slightly faster but does not compress as well.

 

warning

Note that the Burrows Wheeler compression method is not supported by any other compression program. Only SyncBackSE and SyncBackPro can be used to restore Burrows Wheeler compressed files.

 

BZip2 is similar to the Burrows Wheeler compression method except that it is compatible with some compression programs, e.g. WinZip 11. Note that in some cases, e.g. with highly random data, the compression speed can be very slow as compared to the other compression methods. However, it generally provides the best compression level.

 

LZMA (Lempel-Ziv-Markov chain-Algorithm) uses an improved and optimized version of the Lempel-Ziv (LZ77) compression algorithm, backed by a Markov chain range encoder. It uses a variable dictionary size. It is compatible with some compression programs, e.g. WinZip 12. LZMA typically provides much better compression than the Deflate and Deflate64 algorithms at the expense of speed and memory usage when compressing. It also typically provides compression ratios a little better than BZip2/BWT while being a little faster.

 

warning

LZMA maximum level compression uses a huge amount of memory. If you run two or more profiles in parallel, that use LZMA maximum level compression, then you will probably get the following error: Compression error: There is not enough free memory to process the file. It is strongly recommended that you use the 64-bit version of SyncBackSE or SyncBackPro, and have enough memory, if you need to use maximum level LZMA compression.

 

Level of compression: There are ten levels of compression ranging from level 0 (no compression, files are stored in a compressed file but are not actually compressed) to level 9 (highest compression). The more a file is compressed, the slower it takes to compress the file and it will also use more memory. This option allows you to make a trade-off between speed and file size. The Type of compression setting also influences the compression speed and file size. Note that level 9 LZMA compression uses a huge amount of memory and will very likely cause memory failures if two or more profiles are run at the same time using level 9 LZMA compression.

 

 

All Content: 2BrightSparks Pte Ltd © 2003-2017