Backblaze B2 sends 1 file at a time (how to handle lots of files?)

For technical support visit http://support.2brightsparks.com/
John59224
Enthusiastic
Enthusiastic
Posts: 13
Joined: Tue Aug 15, 2017 5:13 am

Backblaze B2 sends 1 file at a time (how to handle lots of files?)

Postby John59224 » Tue Aug 15, 2017 6:01 am

Im looking for input or experience for those using B2 (back blaze cloud) or even other experiences using cloud backup services would be helpful.

Im trying to setup Cloud backup for a biz client, where they have a few windows servers. On the first server im testing, the issue is the inital backup will take nearly 6 months (after it has been running for about 49h, so its a real estimate).

The issue here is not the internet (fiber, 100/100, with great speeds to B2), nor the server (SSD raid array).

We do need to use Client side encryption (as should be done with any cloud backup- so SBP setting of Low Compression ZIP + AES and dont combine into a single zip, ofcourse), but it seems that SBP will only upload 1 file at a time (so this is the delay, especially when there are lots of small files, as is standard with Business windows file servers). I do have SBP Cloud->Advanced set to use 6 simultaneous connections (support confirms this only applies to the splitting of large files over 100mb though) The total data amount isnt that large ( 800gb to 1tb). Im fine with the inital backup taking a while, but at this rate, even subsequent smaller updates to B2 (ie daily or weekly change only updates) will take way too long as well.

I have confirmed with support that multiple, simultaneous connections are only used on files above 100mb (at which pint they are essentially split and half uses 1 connection and half is sent via a 2nd, simultaneous connections).

What are others experiencing with B2 + SBP? (or even SBP + other cloud destiaions- are files uploaded one a time also?)

(we love SBP and it has been EXCELLENT for all our local backup destinations).

(it would be great if SBP could simultaneous upload all files to B2)

Thanks

John59224
Enthusiastic
Enthusiastic
Posts: 13
Joined: Tue Aug 15, 2017 5:13 am

Re: Backblaze B2 sends 1 file at a time (how to handle lots of files?)

Postby John59224 » Tue Aug 15, 2017 6:19 am

To be clear, my issue is not with scanning or anything other than the sending of the actual files to B2.

(just wanted to clear that up as ive seen other posts about SBP scanning speeds prior to upload to cloud, BTW- ive actually *never* had any slow scanning issues with SBP, even with *lots* of files + filters, and various local destinations (ie local disks, ftps, SMB shares)

keithl
Newbie
Newbie
Posts: 8
Joined: Sun Jan 17, 2016 9:08 pm

Re: Backblaze B2 sends 1 file at a time (how to handle lots of files?)

Postby keithl » Mon Nov 13, 2017 6:50 pm

Did you ever get this resolved?

John59224
Enthusiastic
Enthusiastic
Posts: 13
Joined: Tue Aug 15, 2017 5:13 am

Re: Backblaze B2 sends 1 file at a time (how to handle lots of files?)

Postby John59224 » Mon Nov 13, 2017 8:57 pm

keithl wrote:Did you ever get this resolved?


(same reply from other, related thread):

hi, AFAIK this has not been resolved (back when i posted this, the reply from SBP support was we will add this in the "future").

now keep in mind, the 1 file at a time ONLY applies to when you are using Client side Encryption (which IMO is a REQUIREMENT for any type of cloud backup). if you DONT need client-side encryption, then SBP will use multi connections (ie fast) to B2.

in our use-cases we MUST use client side encryption (ie files must be encrypted with our own passphrase, BEFORE they are sent to B2/cloud, incase B2 gets breached, our files are still safe + B2 employees cant access our files). We had 1 client w about 2TB of data another w 1.5tb, after trying ALL the client options that support B2 on Win (including our SBP lic.) we ended up going with CloudBerry.

Now a HUGE DISCLAIMER about cloud berry: (imo they are deceptive about this limit + not clear on site). if you need to use their client to backup more than 1TB of data to the cloud, you MUST get their CB Ultimate app (i think $299). the $99 CB app will work, but when you hit 1TB of data, it will stop and say u must buy CB ultimate. This is an app side / client side 1tb limit, it applies reguardless of which cloud providers to backup to (and its 1TB hard, no matter how u split up that 1TB)

anyway, CB Ultimate has worked great, offers all features (ie versioning, compression, delete after X days of local delete.. ect) + has AES Client side Encryption, and uses multi-threads when uploading to B2 (ie will max ur line, or max that B2 will allow you the take in, in our experience very fast, ie we often see 100mbit sustained, on our fiber customers).

hope this saves you alot of time that we spent (ie wasted) on getting B2 initially setup on win servers. (its great once u have it setup tho)

John59224
Enthusiastic
Enthusiastic
Posts: 13
Joined: Tue Aug 15, 2017 5:13 am

Re: Backblaze B2 sends 1 file at a time (how to handle lots of files?)

Postby John59224 » Mon Nov 13, 2017 9:08 pm

(dang SBP, we still can EDIT our own posts on this fourm?? turn that on pls, even if w 15m post-post timer)

(see my reply above)- one other important requirement we had, the method in which the client side B2 app STORES our credentials, must be encrypted. (ie alot of B2 supported, lower-end Win apps STORE YOUR B2 App-key / Encryption passphrase IN PLAIN TEXT locally !!!, ie if a hacker, or more importantly ransomware, can see that data in plaintext, they then have the ability to delete your CLOUD backup)

*both* SBP and CB handle this data properly- ie they *can* store your credentials, locally, encrypted. (but many other B2 supported win apps *DONT* , ive contacted their devs and tested/verified this myself).

so that is an important consideration too. AFAIK current randsomware doesn't take "user backup deletion" this far YET, but im sure in the future they will (ie ransomware seeks out local, plaintext, cloud backup credentials, then uses them to delete to your cloud backup data,also).

Also note, that to use this feature in SBP, you must enable it in Options (first read the SBP manual on how to properly enable this, you will cause more issues than its worth if you dont know what your doing). in SBP its under Options-> encryption (BUT READ UP ON HOW TO USE properly!!)


Return to “SyncBackPro (commercial)”



Who is online

Users browsing this forum: Bing [Bot] and 7 guests