UNDER CONSTRUCTION!!!

Tech News

Keeping You Up To Date With The Latest Tech News & Virus Threats
Font size: +

Maxing out your Google cloud storage? How I deleted more than 10TB, thanks to this handy tool

ladder to the cloud
PM Images/Getty Images

There is a hard way. And there is an easy way. Google, despite its willingness to collect a fair amount of bucks from me every month for my enterprise plan, does not offer an easy way. Luckily, there's a tool that does - but first let's recap my never-ending cloud storage saga. 

My Google storage tale of woe

Look, all I want to do is be a good steward of my data. Best practices say you need to use the 3-2-1 approach, which dictates that there should be at least three copies of each file, on at least two devices, and at least one which is offsite.

For my little company, offsite is the cloud. Google's cloud, to be specific.

I produce a lot of data. Between the videos I produce and the VMs I spin up, along with all the other media, documents, 3D models, and instrumentation and telemetry data I produce, my company (consisting of my wife and me) has been using a little over 60TB of offline storage.

Also: Is OneDrive messing with your files? How to get your Windows storage under control

Quick bit of backstory: a few years ago, I moved my Google Workspace account to the enterprise plan because that plan seemed to offer unlimited storage availability. That wasn't strictly true. You were allowed to request more storage, but those requests weren't always granted. After a long series of back-and-forth with Google support, they granted my account a 75TB storage pool.

If you want to read about it, here's the full series of stories, going back to the beginning:

That was the story last year at about this time. I had 57TB of storage in use on Google Drive in a 75TB pool. I knew that I'd eventually need to pare down some of that usage to provide additional room for new backups, because backups keep eating storage space.

And that's where I left it for the year. Other tasks got in the way, as they so often do. Then, a few weeks ago when my storage usage topped 60TB, this warning appeared on my Google Workspace dashboard.

warning-notice
Screenshot by David Gewirtz/ZDNET

Since then, Google has been emailing me warning notices every day. I didn't really feel it was fair for Google to scold me when I was only using 4/5 of their capacity. But since I've been known to gobble up a lot of space, it's probably a good idea for my diminishing capacity (in a storage sense) to be brought to my attention.

After all, I'd prefer to know when I still have 20% capacity left than have to deal with what happened last time Google scolded me. At that time, I was 250% over capacity, and my account faced almost immediate sanctions.

Google doesn't make this easy

Google Drive and the admin console will tell you how much overall storage space you're consuming, but it doesn't let you drill down into individual folders. Google's recommendation is to sync your cloud drive with a local computer so all your data is in both, then use your computer's OS to determine folder capacity.

Also: The best cloud hosting services for 2024: Expert tested

Needless to say, with 60TB in the cloud, downloading it all to determine folder sizes is an exercise in futility. Fortunately, there's a tool that can help, called Filerev. I discussed this last year, when I struggled to identify large caches of data I could delete.

As part of my data survey, I found a 476GB folder that I had apparently duplicated at some point during my YouTube production process. I'd made a second copy of the folder, which contained hundreds of video clips, to try a different editing approach, but I didn't need to keep that extra half a terabyte around on Google Drive.

Deleting it proved to be problematic. I deleted the folder using the Google Drive web interface. Then I went to the Trash, and permanently deleted the files I'd just deleted. That worked well, or so I thought.

empty-trash
Screenshot by David Gewirtz/ZDNET

Notice the two usage numbers. One says I used 60.97TB, the other said I used 61.01 TB, a difference of 40GB. Recall I had deleted 467GB, not 40GB. But it gets weirder.

When I refreshed the page, just to see if the two numbers would sync up, the trash was no longer empty, and this time, my storage utilization was shown as 61.53TB and 61.57TB, still a difference of 40GB, but with different overall utilization values than just a few seconds previous.

What the ever-lovin' heck?

Keep in mind that even if I had managed to get that 467GB to go away, I would still need to clear a lot more space. My goal was to get storage utilization down to 50TB, which would give me a full third of my overall capacity to be able to grow back into. That means I'd need to clear another 10TB once I figured out why my deleted trash kept coming back.

Yes. It just kept coming back. I tried deleting trash a few more times, and each time it showed as deleted. Then, on refresh, it kept showing the files back in place.

Also: The best NAS devices of 2024: Expert tested

So, I used the tech columnist's equivalent of the Who Wants to Be a Millionaire "Phone a Friend" lifeline. I reached out to Brett Batie, the author of Filerev, told him what was happening, and asked him, "What the ever-lovin' heck?"

He explained that Google Drive doesn't do deletes all at once, nor does it delete the trash all at once. When I told Google Drive to delete that folder, it started the process of moving files from my editing bay into the trash.

google-drive-time
David Gewirtz/ZDNET

When I first looked in my trash, it looked full, but in reality, only some of the files had been moved over. So, when I then permanently cleared the trash, only the files that had already made the move were removed.

Since files were still marching from one folder to the other in what seemed like a very slow, single-file procession, when I refreshed and returned to the trash folder, a new tranche of files had made the voyage and were now in the trash. The trash folder was once again no longer empty.

Also: I added 1TB of storage to my sluggish PC with this thumb-sized accessory

This went on for a few hours, and during that time, I had no idea how to tell whether the files I had instructed Google Drive to delete had already been deleted.

That was just for one folder. What would happen when I tried to delete 10TB? Here, Batie told me, Filerev could help once again.

Removing a duplicated server backup

With the help of Filerev, I was able to easily identify the biggest folders on Google Drive.

share-listing
Screenshot by David Gewirtz/ZDNET

Backup Share contains all of my individual computer backups. Studio Share contains most of my YouTube video production work, although some of the older projects are scattered in other shares.

But it was Liberty Tank that was of most interest. That corresponded to one of my NAS boxes, which I have subsequently taken out of service. Of more relevance, everything that was previously on Liberty Tank had been moved to other shares. The Liberty Tank share was out of date and redundant. It was also 11.1TB.

That was a very big piece of low-hanging fruit. Nuking it would clear up my whole Google Drive problem for at least a year or so.

delete-liberty-tank

Screenshot by David Gewirtz/ZDNET 

I checked off that share (shown at 1) and clicked Delete (shown at 2). Now here's where things get really extreme. Are you ready? Maybe you better sit down.

delete-indicator
Screenshot by David Gewirtz/ZDNET

Yep, that's a status window with a percent-done indicator. Filerev actually lets you know what's happening. You'd think, with all the billions of dollars Google has at its disposal to create fake AI podcasts, it could at least provide a percent done indicator for Google Workspace enterprise customers when they're trying to delete terabytes of data.

Also: Google's AI podcast tool transforms your text into stunningly lifelike audio - for free

You'd think. Instead, the solution was provided by a lone developer in Traverse City, MI. At least there is a solution.

It took a few hours for the indicator to indicate that the files had all been moved to Google Drive Trash. Once that was done, I deleted the files permanently and waited overnight for Google Drive to remove them.

The next morning, I got the good news:

50tb-goal
Screenshot by David Gewirtz/ZDNET

And that was that. Big thanks to Bret and his Filerev tool for once again helping solve really big storage challenges.

Now it's time for you to tell us about your storage issues. Do you have to manage terabyte backup libraries? Have you had difficulty with Google Drive? Have you used Filerev? Are you engaged in vast levels of bit production, which then all need to find homes? Let us know in the comments below.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.

×
Stay Informed

When you subscribe to the blog, we will send you an e-mail when there are new updates on the site so you wouldn't miss them.

Why Your PC is Slowing Down and How to Fix It
How To Check if Your Phone's Bootloader is Unlocke...
 

Comments

No comments made yet. Be the first to submit a comment
Already Registered? Login Here
Wednesday, 02 October 2024

Captcha Image

I Got A Virus and I Don't Know What To Do!

I Need Help!