Cloud, Advanced

<< Click to Display Table of Contents >>

Navigation:  Using SyncBackPro > Expert Mode >

Cloud, Advanced

 

 

Storage Class/Blob Tier: Amazon allows for different storage classes, while Microsoft Azure allows for different blob tiers. They are essentially the same thing. A storage class (or blob tier) defines how a file is stored on the cloud service, e.g. it may have less chance of being corrupted or may be stored at a lower cost on a slower storage device. You are advised to read the Amazon S3 documentation related to storage classes and/or the Microsoft Azure documentation. This option is not available for S3 compatible services.

 

warning

Glacier, and other archiving classes/tiers, are for long term storage (archiving) of files that do not change, not backup. Do not use these classes for backup.

 

Amazon Lightsail storage must use the Standard storage class otherwise you may receive Access Denied errors when trying to upload files.

 

Use Amazon S3 server-side encryption: When this option is enabled SyncBackPro will request that the cloud server automatically encrypt any uploaded files. You do not need to specify a password, and do not require a password to restore any files. All the encryption and decryption is handled automatically and transparently by the server. This is an extra layer of protection in case anyone gets direct access to your files on their servers. If someone has your Amazon S3 login details then this will not stop them accessing your files. To avoid that situation you must use the encryption supplied by SyncBackPro. Note that when you enable or disable this option it will only apply to files uploaded after that point, i.e. any existing files on the server will not be encrypted or decrypted. This option is not available when using an S3 compatible service.

 

Use newer Amazon S3 security: When this option is enabled SyncBackPro will use the newer AWS Signature V4 security. This is required when using buckets located in newer regions, e.g. Frankfurt. Note that when using S3 emulated services it's unlikely to be supported and so setting this option may cause it to fail to connect. SyncBackPro will attempt to automatically use the newer security if it is required by the server. All Amazon S3 regions support the newer security, however using it can reduce performance.

 

Use Amazon S3 transfer acceleration: When this option is enabled SyncBackPro will use the transfer acceleration feature of S3. Please refer to the Amazon S3 documentation on how to enable this feature on a bucket, the restrictions that apply and the increase costs associated with using this feature.

 

Retrieve a list of all the files and folders then filter (faster in most situations): There are two methods for SyncBackPro to request a list of all the files and folders from the cloud storage server. It can either request the lists folder by folder (which is how it does it with local drives, FTP servers, etc.) or it can ask the server for a list of all the files and folders in one go (the default). One method may be faster than the other depending on how many of the files and folders on the cloud service are going to be filtered out by your profiles. If lots of the files are going to be ignored (due to filter and file & folder selections) then the profile may run faster if this option is switched off. If your cloud storage system has  tens of thousands of files on it then you may need to disable this option as SyncBackPro may use a large amount of CPU time retrieving the list. If Fast Backup is being used then this option is always used (if the cloud service supports it).

 

oDo not use the delta API: This option is only available when a cloud service is being used that supports getting a delta changes list, e.g. Dropbox™ is being used. A delta changes list provides the ability to quickly discover what changes have been made since the last profile run. It is recommended that you do not enable this option unless it is causing problems, e.g. changes are not being discovered. Note that if you disable this option then the first run after disabling it may take a long time to run as the delta changes list may be very large. But after that first run with delta API enabled, subsequent profile runs should be much quicker than with this option enabled (delta API disabled). If Fast Backup is being used then the delta API will not be used.

 

Upload the database to the cloud: SyncBackPro keeps a local database for storing the details of the files on the cloud service. This database is used to store details that cannot be stored on the cloud service. For example, some cloud services do not allow the last modification date & time of a file to be changed. To get around such limitations SyncBackPro keeps a record of what those details are. You can optionally also store that database on the cloud. If you are using the cloud service with SyncBackPro from more than one computer then it is strongly recommended you enable this option. This ensures that any changes made to the details will be visible to all SyncBackPro installations. Due to the way SugarSync™, OneDrive for Business (Office 365) and SharePoint™ (Office 365) work this option is not available when using those cloud services. It's also not available when using Amazon S3 or Microsoft Azure as they do not require a database.

 

Move deleted cloud files to trash: Some cloud services, e.g. Google Drive, support an option where files deleted from the cloud service are moved to the trash instead of being permanently deleted. If this option is enabled, which is the default, then SyncBackPro will move files to the trash when they are deleted. Note that some cloud services will move files to the trash regardless and provide no option not to do this, just as some will only permanently delete files and not have a trash feature. This option is only available when a cloud service supports the option of moving files to the trash.

 

Automatically translate invalid filenames: For some cloud services, SyncBackPro will translate filenames so they are compatible with Windows file systems. This option is enabled by default and works the same way as in FTP. This feature is not available (and not functional) with Amazon S3, Backblaze B2, Google Storage, Microsoft Azure, Rackspace, Egnyte and Citrix Share.

 

Mute file change notifications in cloud apps: Some cloud services, e.g. Dropbox, have client applications that notify you when a file is uploaded or changed. By enabling this option you can configure SyncBackPro to tell the cloud service not to notify you when it uploads a file.

 

Number of worker threads to use (too many will degrade performance): File upload and download performance with Amazon S3, Microsoft Azure, Egnyte, Backblaze B2, Rackspace/OpenStack and Google Storage (if using a private key) can be significantly improved by enabling parallel file copying. You can set the number of files to upload in parallel. SyncBackPro will upload smaller files in parallel, and large files serially (as these are already uploaded using multiple threads, see following setting). Note that increasing this value too high can slow down the profile or cause the cloud system to throttle the connection. Also, in some circumstances SyncBackPro will not use worker threads, e.g. if runtime scripts are used. Note that with Egnyte the maximum number is 12 (due to restrictions the service imposes). If you are using multi-zip compression then worker threads will not be used to upload files.

 

Number of upload/download threads to use (too many will degrade performance): Some cloud services, e.g. Amazon S3 and Microsoft Azure, allow for multi-threaded access so upload and download performance can be improved. When SyncBackPro uploads or downloads files, or retrieves the meta-data on the files, it will do so in parallel by using a number of different threads. This can significantly improve performance, but if too many threads are used then it can also significantly reduce performance (by overloading the network, CPU, and increasing memory usage). On cloud services that support this, by default 30 threads are used. If you are limiting the bandwidth, and have more than one thread, then this will be highlighted in the user interface. For cloud services that do not support threaded access then only 1 thread is ever used. The maximum number of threads is 150. Note that with Egnyte the maximum number is 12 (due to restrictions the service imposes).

 

Number of scanning threads to use (too many will degrade performance): Some cloud services, e.g. Amazon S3 and Microsoft Azure, allow for multi-threaded access so scanning for changes performance can be improved. This can significantly improve performance, but if too many threads are used then it can also significantly reduce performance (by overloading the network, CPU, and increasing memory usage). The maximum number of threads is 100. Note that with Egnyte the maximum number is 12 (due to restrictions the service imposes).

 

Send and receive timeout (seconds): Sometimes network problems and disconnections can cause communication with the cloud storage server to freeze, e.g. SyncBackPro may be waiting for a response from the server which may never arrive. You can set a limit for how long SyncBackPro should wait for a response before it tries again. By default the timeout value is 60 seconds.

 

Access Policy applied to uploaded files: This setting only applies to Amazon S3 (Azure sets its security via the container). By default SyncBackPro will use the highest security setting, i.e. only you and nobody else can access the files. However, you may be storing files that you want others to be able to access (e.g. via a web page) and so would want a different access policy. Note that there is also an access policy on the bucket. Please refer to the cloud service documentation on the meanings of the access policies.

 

Limit bandwidth usage to: This limits the bandwidth usage when uploading and downloading files to and from the cloud. Note that the throttling uses an average over time meaning that it will transmit in bursts (at maximum speed) and then pause, i.e. the transfer rate will not be constant. Note that if the bandwidth throttle is set too low then you may experience timeout failures. If you limit the bandwidth then you should also reduce the number of threads (ideally to just 1).

 

Number of days to restore Glacier files for: This setting only applies to Amazon S3. If SyncBackPro needs to retrieve a Glacier file then this is the number of days Amazon will store the temporary copy of that file.

 

Glacier retrieval tier to use: This setting only applies to Amazon S3. If SyncBackPro needs to retrieve a Glacier file then this is the speed at which the file(s) will be retrieved from Glacier. Check with Amazon on the costs associated with different retrieval speeds:

 

oStandard: default, 3-5 hours retrieval time

oExpedited: fastest, most expensive, 1–5 minutes retrieval time

oBulk: slowest, lowest cost, 5–12 hours retrieval time

 

Customer-provided encryption keys (SSE-C): This setting only applies to Backblaze B2 and Amazon S3 and compatible services, e.g. Wasabi and Oracle S3 compatible storage. If you are using server-side encryption with your own encrypted keys (SSE-C) then you can specify the key here. When creating your own SSE-C key, refer to the cloud services documentation for the encryption cypher to be used. For example, at time of writing, Backblaze and Amazon S3 require AES-256-CFB. You can use a secret (the secret type is password) for the key. If you use SSE-C:

 

oAlways use an encrypted connection. This is to protect your key.

oDo not use more than one key with objects in a bucket, i.e. all objects in the bucket must use the same key. If you use more than one key then scanning will fail.

oIf you change keys then you will need to delete the objects and re-upload all of them with the new key. Alternatively create a new empty bucket and use that bucket with the new key.

 

 

Setting the MIME type (Content-Type)

 

You can change the MIME Content-Type for files that are uploaded to a Cloud Storage Service by using a custom MIME.TXT file. Note that not all cloud services allow the Content-Type to be specified.

 

Create a text file called MIME.TXT in the folder where the SyncBackPro executable is. By default, this location is at:

 

32-bit: C:\Program Files (x86)\2BrightSparks\SyncBackPro

 

64-bit: C:\Program Files\2BrightSparks\SyncBackPro

 

The text file must specify the extension and what mime type it should be. Specify one extension/mime per line. For example:

 

txt=text/plain

asf=video/x-ms-asf

webm=video/webm

 

Please note that it is not necessary to put a period before the extension (example - txt instead of .txt), but it wouldn't matter either if you place the period before such extensions.

 

 

Setting the Content-Disposition (Amazon S3 only)

 

You can set the Content-Disposition to attachment for files that are uploaded to a Amazon S3 by using a custom CONTDISP.TXT file.

 

Create a text file called CONTDISP.TXT in the folder where the SyncBackPro executable is. By default, this location is at:

 

32-bit: C:\Program Files (x86)\2BrightSparks\SyncBackPro

 

64-bit: C:\Program Files\2BrightSparks\SyncBackPro

 

The text file must specify the filename extension or the filename (no path). Specify one filename extension or filename per line. For example:

 

.txt

download.exe

 

Please note that it is necessary to put a period before an extension otherwise it is treated as a filename. Any file uploaded that matches a filename or extension, or filename, in this list will have its Content-Disposition set to attachment when uploaded.

 

If you want all files to have the Content-Disposition set to attachment then put a single asterisk in the file:

 

*

 

 

 

All Content: 2BrightSparks Pte Ltd © 2003-2024