Before posting, and to avoid disappointment, please read the following:

  • This forum is not for 2BrightSparks to provide technical support. It's primarily for users to help other users. Do not expect 2BrightSparks to answer any question posted to this forum.
  • If you find a bug in any of our software, please submit a support ticket. It does not matter if you are using our freeware, a beta version or you haven't yet purchased the software. We want to know about any and all bugs so we can fix them as soon as possible. We usually need more information and details from you to reproduce bugs and that is better done via a support ticket and not this forum.

Google Drive backups are slowed by retrieving "delta items" ???

For technical support visit https://support.2brightsparks.com/
Post Reply
Tyree
Knowledgeable
Knowledgeable
Posts: 29
Joined: Tue Aug 12, 2008 3:34 pm

Google Drive backups are slowed by retrieving "delta items" ???

Post by Tyree »

I just ran a backup of my Syncback Profiles to my Google Drive. The backup is a mirror consisting of a total of 309 files and 165MB. Most of those files were already on Google Drive, so only 55 had to be uploaded.

However, the first thing Syncback did before even starting to upload anything was retrieve "delta items." Over 12,000 of them. After retrieving them, it then had to retrieve them again, only this time with a counter "X of 12027 retrieved."

That step took 13 minutes! Once it determined the differences, only 22MB needed to be uploaded. 1 minute later, it was done. 14 minutes for 1 minute of actual uploading??? Why?

My concern is I have MUCH larger backup tasks than this left to setup. If it has to do this "delta items" mess with those operations, the backups will be intolerably slow. To the point of being unusable.

So, what are delta items and are they always like this?
deltaitems.PNG
deltaitems.PNG (9.66 KiB) Viewed 8865 times
Swapna
Expert
Expert
Posts: 1026
Joined: Mon Apr 13, 2015 6:22 am

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by Swapna »

Hi,

Delta changes is a Google Drive’s feature which allows SyncBackPro to quickly retrieve all file changes on the cloud server from a specified checkpoint. Please read this article for more details:

https://www.2brightsparks.com/resources ... ckups.html

However, if you notice the backups are slow when retrieving delta changes, you can try disabling it from Cloud > Advanced settings page. Tick the option ‘Do not use the delta API’ under: Modify > Expert > Cloud > Advanced settings page and retry the profile run to see if it helps?

Thank you.
Tyree
Knowledgeable
Knowledgeable
Posts: 29
Joined: Tue Aug 12, 2008 3:34 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by Tyree »

Interesting. I guess the question would be is how in the world are there 12.000 changes to 305 files and only 55 had actually "changed."

I do notice that each subsequent time I run the job (when I know nothing has changed), the delta items number shrinks considerably.

Thanks!
Tyree
Knowledgeable
Knowledgeable
Posts: 29
Joined: Tue Aug 12, 2008 3:34 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by Tyree »

Definitely going to need to try disabling. This is kinda ridiculous. And I had JUST run this profile and was just running again to verify everything was uploaded. I expected a fairly quick run just to scan and see no changes. Over an hour in and it's still retrieving delta. Gah.
LONG.PNG
LONG.PNG (17.02 KiB) Viewed 8785 times
pluviosilla
Newbie
Newbie
Posts: 6
Joined: Tue Aug 18, 2020 2:12 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by pluviosilla »

I'm having the same trouble, but it gets even worse after all the delta items are retrieved and SyncBackPro V9 just starts "retrieving items".

I have a 7 GB folder on Goggle Drive that I routinely update, and it's reasonably fast. Suddenly, out of the blue, this problem started occurring. I left the machine on 24 hours a day for several days, but it was "retrieving" literally one delta per second. I began to wonder if the delta comparisons were the problem and decided to DELETE THE FOLDER entirely on Google Drive. That sped up the retrieval of delta items alright, but once the delta retrieval phase was over and SyncBackPro started with messages like "1424 items of 252336 retrieved" it slowed to a crawl again.

I just uploaded a half terabyte to OneDrive using SyncBackPro V9 and that took less time than this, so I think maybe GoogleDrive is throttling my uploads or it isn't playing nice with SyncBackPro in some other way.

This seems like such a strange problem. Most of the time SyncBackPro is very efficient.
Tyree
Knowledgeable
Knowledgeable
Posts: 29
Joined: Tue Aug 12, 2008 3:34 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by Tyree »

I've read just a bit about SyncBack 10. It says that cloud storage will be faster. I hope that's true of this feature!!! It's been a while since I have had to upload or download large amounts of data with Google Drive, so I haven't had to deal with the delta items thing.

Good luck getting it straight!
mickyj
2BrightSparks Staff
2BrightSparks Staff
Posts: 479
Joined: Mon Jan 05, 2004 6:51 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by mickyj »

Hi, the Google Drive delta is basically a list of *all* the changes to items in Google Drive since the last time the delta was retrieved. So it's not changes to a specific file or folder, but changes to everything. So if there are lots of changes then the delta list will be large. If there's a long time between delta retrievals then it may also be large. When SyncBackPro asks for the delta list, we of course have to wait for Google Drive to send it to us (all of it). How long that takes, and how much it throttles the transfer, is completely out of our control. We ask for the delta list and we receive it as fast as it is sent to us.

In some cases it may be better not to use the delta. For example, if you have lots of files on Google Drive, but your profile is only using a small subset of those files, then it may be better not to use delta as you don't care about all the files. Also, you may be constantly updating some files on Google Drive. If those aren't part of your profile then don't use delta.

Cloud services like S3 and Google Storage don't throttle because you pay for the bandwidth you use (as well as the storage). With Google Drive, Dropbox, etc. you aren't paying for the bandwidth used, so they will nearly always throttle and always be much slower than S3, Google Storage, etc. When, how often, and how much a cloud service throttles is unknown. There can be many factors, none of which we have any control or influence over.

For V10 you'll be able to use Fast Backup with Google Drive, meaning it won't need to scan Google Drive at all, so scanning time is reduced to zero (except for the first scan after enabling Fast Backup). Fast Backup can only be used if you are not changing those files on Google Drive outside of SyncBack, i.e. if you're only copying the files to Google Drive (and not changing them on Google Drive or using something else to upload/change files on Google Drive) then you can use Fast Backup. V10 also uses less memory when Google Drive is used. If your system is memory constrained (e.g. it only has 2GB) then this may help.

I hope that helps.
Tyree
Knowledgeable
Knowledgeable
Posts: 29
Joined: Tue Aug 12, 2008 3:34 pm

Re: Google Drive backups are slowed by retrieving "delta items" ???

Post by Tyree »

Thanks, @mickyj! Most informative!
Looking forward to using Fast Backup for my off-site Google Drive backups. That's gonna be suh-weet!
Post Reply