It's worth noting that they are talking about a single Pure disk pack for one of their arrays. Their disk packs include NAND, a bunch of RAM, some NVRAM, and storage device controllers all in a hot-pluggable module. This isn't something you'd buy and install into a PC, it's a part number for a Pure array, presumably FlashArray//C ("C" being "capacity", which is Pure's line of capacity optimized AFAs).
Storage drives aren't 18-inch behemoths anymore. You can stack storage to your heart's content, or, rather, your wallet's limit. Don't tell me about how you plan to offer ten times the storage in three years, tell me about how you plan to offer ten times the storage per dollar.
You basically tell them how much usable space you need (high water mark), and they will make sure you have that amount available. It also comes with an SLA for latency and they'll even upgrade your controllers for free if it goes out of bounds.
Many people who deal with raw hardware don't realize that what the business wants is guarantees that shit is going to work, cost mostly be damned.
And so that's how you get scenarios where you pay through the nose, but you've explicitly called out what you need and you get it, even if the company ends up doing weird things like shipping the top-end hardware (with speed disabled) so that they can make sure they meet the requirements.
In this case, Pure will monitor your arrays for you and determine if the latency issues are coming from the controllers or the nvme subsystem, and they’ll either add more drives or swap out the controllers for faster hardware to make sure the SLA is met.
It’s a pretty good system especially for CFOs who prefer opex model
That’s debatable when you consider the performance you get from Pure.
Our X20 which is on the small end, with only 10 drives can easily do hundreds of thousands of IOPs, supporting over 1000 VMs and a high performance ERP solution.
The whole thing costs us about 50k a year. On an AWS/GCP that would cost a lot, lot more.
Sure, try running a thousand VMs on it though (which is an extremely random workload). A single sequential test at an optimal byte size benchmark for a consumer SSD is not representative of a real workload these arrays would see.
Doing so is a lot more complex and intensive than what a single consumer grade SSD can handle.
My point was just that if you wanted to get dedicated IOPs on AWS to match what you get with modern SANs, it’ll cost you far more.
I get 2 to 2.5 million IOPS for 4k random-reads on my personal server.
Running Linux LVM software RAID over 3 x Samsung NVMe SSDs. That's not a read-write measurement, but it's a satisfying number for a not particularly high end server.
(I use it for a side project's database engine experiments. That level of IOPS supports a very high random query rate.)
100% Read or 100% Write are as far from the real workload as it can get. The only exception would be backups read/restore.
Also 1GB test file is often fits in the SSD's RAM cache. Get iometer, 50R/50W%, blocks from 512 to 16k, at least half the size of the storage. Then you would see the real performance.
NB if you have random read way below the random write means you are measuring anything but the storage performance.
I tried CrystalMark for my desktop SSD with 64GB 4k 50R/50W and got 134K, mostly I think because it has 400K read and 130K write, so writes are bottleneck.
Sorry, lazy to learn how to use iometer, but you probably have SSD too and can report your results.
>> but you probably have SSD too and can report your results.
Uh-uh!
I intended to show you the difference between a single NVMe drive and a SSD SAN (a bit old, but still very performant to handle ~900 VMs).
Sure, I can just ramp up queue depth and see some magical numbers, but for me the real performance is in everyday tasks and running CrystalMark isn't an everyday occurrence.
> Sorry? 900 VMs equals 100% full random access. There is no sequential access there, just as I said in my first comment.
it depends on workload, if they do most of the work in RAM, and most of fs traffic is snapshoting and restoring from snapshots, then you will get 99% io seq traffic. If they do some non-trivial fs operations, then you will get q16t16 io traffic. It is very unlikely you will get q1t1 random.
> Because of your comment[0].
> This comment[1] pretty much summarized what I said in a more eloquent way.
In my view you are jumping from topic (single ssd vs nas) to another topic (your speculations about benchmark not representing real world scenarios) and then back.
I agree that if you ran a synthetic benchmark against a brand new pure x20 with 10 NVME drives you'd see astronomical numbers for iops.
That is absolutely not representative of a real workload of mixed reads and writes, different block sizes, potentially different queue depths, all coming in on hundreds or maybe thousands of different volumes.
A single consumer Samsung SSD would hilariously crumble under a real workload, it will NOT deliver hundreds of thousands of IOPs in that environment.
Your benchmark screenshot is the equivalent of showing your pickup truck can do burnouts in the parking lot, and extrapolating that to think it could keep up with a Ferrari on the Nurburgring.
> That is absolutely not representative of a real workload of mixed reads and writes, different block sizes, potentially different queue depths, all coming in on hundreds or maybe thousands of different volumes.
if you see bottleneck there, it could be that actual SSD speed is maybe irrelevant in your case since upstream software is not optimized.
We switched from EMC to PS for our on-prem stuff about 5 or 6 years ago. I sat in on some of the RFP process. PS was basically the same price for a vastly superior offering. You had to make a leap of faith, though, that PS would survive as a company.
Now there’s no doubt, so they probably have more pricing power. And EMC most likely has an offering that competes with the performance (or at least as close as they can get). Back then it was flash vs spinning disks.
We do most of our stuff in the cloud, BTW, and PS is faster for cheaper, by a wide margin. But we’re not going to bring that compute back (I’d like to).
One thing I remember was them being very open and direct, and they wanted to know a lot of heuristics about our data so they could give an accurate estimate about speed and dedup. Which was really close to the reality. Oh, and we’re using one of their devices for an Itanium OpenVMS cluster, it works like a charm. Try getting a startup to support that kind of setup these days, lol
Given their target 66% profit margin (product price = 3x cost of goods sold), their $/GB compares favorably with any flash based storage that doesn’t use wide erasure coded stripes, even ignoring dedupe and compression.
Not sure how they compare to other enterprise offerings these days, or how hard it is to roll your own RAID 6 open source thing with synchronous replication.
Storage per dollar increases are usually correlated with storage density increases. Seeing news like this makes me confident we aren't hitting the limit of Moore's law for storage for a while.
We are now below $.02 per gigabyte, at $.014-ish at single-unit prices for CMR drives, more or less tracking linearly that last leg of the curve. My guess is that there is a lot of price elasticity in hard disk drives (and that margins aren't as thin as we could hope - which is good, because it allows more R&D spending).
Those aren't that sad looking. You can't go down 4 cents/gb when you're at 4 cents/gb. If you look at the halving period, it's pretty stable at 4-6 years.
Yea spinning rust has been plateauing for anything other than SMR disks. A large part of that is that CMR (the old style) just isn't scaling anymore, even though it has much better performance and characteristics than SMR. Combine that with the fact that flash storage has been considerably increasing in density and speed (spinning rust disks haven't really increased in performance either) there's probably not as much investment happening in improving the spinning rust stuff the same way as it used to be. Flash still isn't quite at the same level of cost per storage but it's definitely starting to approach it and will very likely surpass it in the future.
I'm curious what the performance characteristics will look like. Will they behave more like SMR where they don't work very well in RAID or ZFS due to needing to wipe large portions at a time, or will they be more like CMR where you can reliably do a sector at a time. I really hope it's more like CMR because otherwise the storage just won't really cut it at sizes as small as 50TB by then. With Flash disks being available (though at insane prices) at 100TB right now I'm not convinced there's going to be much of a market with the price/storage and performance stagnation that's been going on.
One of the big problems is the difference between the capacity curve and the access speed curve. The 22TB HDDs are 22x bigger than the 1TB drives of years ago; but they are no where near 22x faster. Speeds have increased to be sure, but they are in the range of 2x to 3x faster than old drives.
This means that the total time needed to read and/or write all the data on a given drive continues to increase. This is very important for things like backup or RAID rebuilds. Reading 50TB from a drive with 300 MB/s speed will take almost 50 hours.
Also, some might think that this problem somehow goes away if you just move to SSDs; but it does not. The speeds are an order of magnitude faster, but the curves are very similar. A 4TB SSD is not 4x faster than a 1TB SSD.
Unless you actually hit the maximum your building can sustain (heat, volume). Building datacenters is incredibly expensive, so reusing existing infrastructure and packing it with more is actually important.
Having existed when 128GB USB flash drives were expensive.
Naturally as the storage amount increases, the price per GB decreases over time.
Consumers have been saying exactly what you have been saying since the dawn of storage. When storage capacities get higher, what you had before only becomes cheaper.
The density is what's most relevant and interesting here. Pure is putting 28 of these drives in 3U - so 8.9P of nand in 3U drawing ~3kw of power. Pure gets ~2.5 data reduction on average (deep compression & dedup) and so the real world density is pretty impressive.
The speed of technology is absurd. I operated the Object store in a small scale public cloud company in the EU in 2014 and we had about that capacity over 12 racks drawing 5-7kW each probably.
I understand these are for enterprise. Consumers seem to be stuck at 5TB for portable SSD storage. Does anyone know if I am missing something or if larger SSD drives are planned soon?
I purchased an 8TB nvme drive over two years ago. Put it into the $30 enclosure (on the same page) and you've got 8TB portable. The biggest issue with these is cooling. As density increases, more heat is dissipated within the same space.
These drives aren't for retail sale. They go into various products. With the newest product they are offering 20 cents per GB max capacity 19.2PB and three years of support. See FlashBlade//E https://blocksandfiles.com/2023/03/01/pure-flashblade-e/
They say it's 3U. It's the size of a large toaster-oven. You can put that in your home, if you really wanted to. Will be probably the most expensive thing you own though, beside the home itself.
I have an older gen m10 here at home. You can in fact run Pure at home, and doing so is costing me around $84/mo just to have it plugged in and consuming electricity. Not something I would recommend for most people but my lab VMs are _screaming_ fast.
My path into programming went through several "Moshe Yanai companies" (as we call them here). So, I see Pure more as a competition rather than something I'd use for my own stuff. I've used similar equipment from eg. Infinidat, but I'd struggle to think about what an individual person would possibly do with such a thing. Being a storage enthusiast in general, I was looking for a reason to convince my wife to spend time on building home NAS, but even in my wildest dreams I couldn't think about a use for this much space. Even though my wife is a researcher with a government org, the datasets she works with can typically fit on a USB stick... Even working on several projects at a time she doesn't have enough data to justify buying a dedicated enclosure for a bunch of disks :(
I guess, maybe if you are in video editing that would pay off? But what else is there that an individual may plausibly be doing that needs so much room?
I've been on calls w/ Moshe :D Infinidat is all about capacity, we tend to see them as B2D targets or for long term warm storage. Pure is more about performance. They have never shipped a device with spinning disks in them, so even their capacity series is all flash. The m series I'm using (now the X series) is designed for ultra low latency, even under heavy load I almost never see read or write times north of 1ms.
That's fine; call their enterprise sales line and be prepared to mortgage your house. The comment I was responding to was implying they were interested in a retail product.
Worked with some of the teams that are bringing this tech out, top-notch people, super fun tech. This not really applicable to commodity drives as pure storage flash controllers aren't limited by SSD dimensions.
I hadn't followed NAND developments since 160 layer was the six-month-out tech.
400-500 layers sounds exciting, that's the biggest news I get out of that.
Flash doesn't scale down with node shrinks, I guess the downside is that layer stacking will be a linear process, but hopefully that process has a longer runway than transistor node shrinks do.
performative IT like this is just saying "I'm willing to waste exceptional amounts of money because it implies I'm important".
conspicuous consumption via Rolls Royce arguably gives you actual, tangible differences, albeit not in the automotive equivalent of bandwidth and latency.
Fragility is the price you pay, and that leads to very demoralizing breakage and repair. I stopped all development since my 400GB 540s main desktop SSD corrupted after only 5 years of 24/7 operation.
Meanwhile my 40GB X25-M runs flawlessly after 10+ years of 24/7 operation.
These newer drives probably have very advanced error correction systems that will hide defects for long enough that you get comfortable. Redundancy saves you from catastrophe, but not from the depression that follows the realization that everything is crap.
Are you saying this based on any data or evidence that relates to this enterprise drive or are you saying it because you personally had a single drive fail on you?
