I see folks describe Google Drive errors / limits like this and people describe "a business critical operational system" and I wonder. Is Google Drive even supposed to do this thing?
Not pointing the finger back at them, if Google doesn't make it clear they probably should, but at least as an individual I always see Google Drive as a single user cloud light system that allows for some sharing and organizational functions... but still not an "industrial" type cloud service.
Disclaimer: I'm a Googler but I don't work anywhere near this stuff, and this is just my own personal opinion.
In my mind, Drive isn't a file sharing / storage system, but rather a document sharing / storage system. If you want to store large numbers of files, you should use Cloud Storage, which can handle that. Drive is the cloud equivalent of LAN shared folders - you know those janky NFS and SMB shares that were always a giant headache because of permissions and other nonsense? That's what Drive is trying to be.
Seems like they could do it without the jankiness, though.
But the average user is not writing automated systems to populate google drive with millions of files. Once you are doing something that unusual you should understand that nuance. Sure it is fun to hack a tool designed for one purpose to serve another, but if you are writing code for use in production you have to understand these types of design differences.
Drive is very closely linked with Docs, Sheets, Slides. These aren't "files" in the traditional sense. These, plus things like PDFs, videos, etc are what Drive is designed to help manage.
If you have 100,000 source code files, art assets or something like that, you probably don't want to use Drive to manage them. The UI just isn't built for it and it doesn't scale.
I’ve never had a problem with NFS, and modern filers would laugh at 3 million files.
I really dislike Google Drive. I can’t imagine people would use it at all if not for the fact that it’s bundled, monopoly style, with the office suite and gmail.
Since Google won’t ever fix it, I’m hoping they rapidly make it so crappy some other competitor crops up.
Of course, with our luck, it’ll be some Microsoft offering.
You don't have anything in your HN profile, so I can't be certain, but I'm guessing you haven't been in the Research/Education space? It's scary how much Google Drive is used for storage of stuff.
I work at a university that is fully invested in Google Apps.
We did have unlimited storage, however last week they announced that's soon to change:
> A quota will soon be introduced on the amount of data that staff and students at the University can store in their Google account. This is part of a larger piece of work being carried out by the Google Workspace team in IT Services, to review the overall amount of storage being used by the University.
+1
And, it's not like Google didn't make huge marketing pushes to enter that space and businesses at large. Gsuite, Classroom, ever increasing storage limits. There is even an education package offering "unlimited" storage.
google drive has unlimited storage, they opened that pandora's box themselves. yes it's janky as hell, but I'm looking at tens of thousands per year to backup to cloud services for my lab's data
> at least as an individual I always see Google Drive as a single user cloud light system that allows for some sharing and organizational functions... but still not an "industrial" type cloud service.
That may be how you view the product, but Google Drive absolutely is used as an enterprise cloud service by many large companies. And as a result, there are lots of applications that integrate with its API, etc., to serve those enterprise use cases.
It's funny to read your comment, because to me, Google Drive feels like an enterprise product that incidentally happens to provide a consumer-facing version as well, not the other way around.
The assumption stems from the facts that a/ it started as personal use, like gmail. Only way later workspace and that the gsuite name even came out. Certainly no enterprise offering for years after personal use was broadly well adopted. b/ contrary to most SaaS businesses, Google is rather subtle and gentle when it comes to promoting or upselling, many people don't even know a paid/pro version of this thing exists.
But you are right, google offers enterprise grade solutions so is expected to support over a few million files per account or even per user. Maybe something changed due to hasty cost cutting measures given the sudden interest by many on the issue tracker.
> The assumption stems from the facts that a/ it started as personal use, like gmail. Only way later workspace and that the gsuite name even came out. Certainly no enterprise offering for years after personal use was broadly well adopted.
Yes - I mean, sort of. It depends on what you define as the starting point for Google Drive. Like many Google products, the brand has been reorganized and reused a number of times.
If you look at Google Drive as an extension of Google Docs, which existed prior to 2012, then yes, it started off as personal use. But if you look at it as a thing that launched in 2012 (the introduction of the Google Drive brand), then it arguably belongs as part of the reorganization that involved ending G Suite and turning it into Google Apps for Work[0], which is now known as Google Workspace.
In fact, now that the consumer version of Google Drive has been reorganized into Google One, you could argue that the enterprise version of Google Drive is actually older than the consumer version!
Of course, all of these distinctions are somewhat arbitrary, because it's the same product that you've used all along, and the only thing that changes is the pricing and the marketing. But if we're talking about consumer vs. enterprise versions of a commodity product that essentially operates at enterprise-scale for consumers, that's also the only real difference.
It's actually quite funny that the same product (which has been improved slightly, but is largely identical to what it was over a decade ago) has had so many different brands attached to it, especially when you consider that some of those brands are essentially container formats that repackage the same underlying brand (e.g., Google Workspace contains Google Drive).
[0] There was actually a rebrand in between, from 2012-2015, but because Google's product names are Alphabet soup, I can't be sure of what they called it in the interim, though I'm positive it was a distinct product from before 2012.
At Google’s scale for consumer users, on their backend it has to be an enterprise scale product anyway or the whole thing would have imploded shortly after launch.
Sometimes there are random UX/UI/API limitations, but those usually get fixed relatively quickly when someone hits them, because they’re typically just some hardcoded limit someone put in a front end layer because it was ‘more than enough’ and they needed something. A bit of elbow grease to clean up that path if it seems useful, and it’s usually good again.
Google Drive has the "MS Excel" problem. It's made cloud storage really accessible to people, so they turn to it for everything, even stuff that's more appropriately put somewhere else. In Excel's case, it's database data. In Google Drive, it's files. My company uses it for video assets, because the media folks are really familiar with it, but we're moving it to S3 now because it's not got the chops for our seriously "larger" needs.
I work it in a fly neuroscience lab and we use it to store all our electrophysiology and video data. Each person in the lab is storing on average 5TB of data, and the lab as a whole stores 100TB.
The graphical user interface combined with unlimited storage for Google Workspaces is essentially an unbeatable deal. Researchers can upload their data easily through the interface. Any custom solution based on S3 or equivalent would take some time to teach and more time to maintain. Also, we're paying about $200 / month total to store 100TB of data in the cloud, which is hard to beat with other services.
I tried setting up a single account for the whole lab once, but we ran into the above 5M file limit, so we just have individual accounts per researcher and it's mostly fine for now.
> Also, we're paying about $200 / month total to store 100TB of data in the cloud, which is hard to beat with other services.
You should expect this to go away soon. I support science researchers and our unlimited storage option is going away in the coming months. Options for purchasing space are limited, and not cheap.
This is with a regular Google Workspace plan, independent of the university-wide research plan. The research plan unlimited storage is indeed going away for us as well ( https://itconnect.uw.edu/tools-services-support/software-com... ), which is why we migrated to our own private Google Workspace.
I haven't heard about the google workspace enterprise unlimited storage going away anytime soon, although perhaps you know something I don't?
How expensive are we talking? Looking at the workspace pricing page the enterprise plan gets you a pooled 5TB per user with the option to add more by talking to them, does that mean you have to pay more for the extra storage?
It's "free" cloud hosting. You'd be surprised how often people mix their personal google account with work stuff. It annoys me greatly, but that's how it is.
There is general ignorance of what google does/is/isn't and how online storage works. I agree with the sentiment, but, I've seen many a thing from recipes, shopping lists to FPA/Proposals all hosted on google drive. Even people sell content that is hosted on google drive.
For personal use, Google Drive creates a way for me to have automatic backups of files I wouldn't want to lose in case of drive failure. It's also a way to easily share files between my desktop and my phone.
For professional use, I've usually seen it used as a document repository that can be shared by teams.
In both use cases, 5 million files seems like it'd be a hard limit to hit.
You can't download anything without 3PC. It's the only Google service set up that way, which isn't a coincidence. Drive is essentially a loss leader to protect their cash cow.
What's to do with cookies? I understand the comment to refer to ads being by far their main revenue making business, and shaping users browswer preference to keep cookies helps Google make more money via ads.
I didn't even know 3PC was used by google drive. Not sure what could be non cpincidental otuer than a technical reason for it given the very collaborative nature of this tool.
So that's the reason they're required: to preserve the credentials story between the static resource provider and the non-static layer. Nothing nefarious; just an implementation detail.
That's how all dark patterns are constructed. Present a plausible cover story. Keep the underlings in the dark about the true reason for the system architecture. Don't write anything down.
That's also how all patterns are constructed: present a plausible cover story (because it is true). Keep the underlings in the dark about the true reason for the system architecture (because the delta between the true reason and apparent reason is null). Don't write anything down (because there's nothing else to write down).
Suspicion of dark pattern is not proof of dark pattern.
(ETA: at Google in particular, the dark-pattern-conspiracy you're describing between two unrelated teams, in this case Drive and static storage, is hard to pull off because coordinating those teams requires documenting design, and it turns out a lot of folks working at Google are Hacker News-type personalities who will ask inconvenient questions if they come up. But what they have that we lack is observation of the larger security picture and the ways failure to, for example, 3PC-guard your static files can result in failure modes).
I see folks describe Google Drive errors / limits like this and people describe "a business critical operational system" and I wonder. Is Google Drive even supposed to do this thing?
Not pointing the finger back at them, if Google doesn't make it clear they probably should, but at least as an individual I always see Google Drive as a single user cloud light system that allows for some sharing and organizational functions... but still not an "industrial" type cloud service.