The problem isn't that our data has become public; it's that businesses accept data as identity. They mostly just mindlessly automated manual paperwork processes that were slow, but also less efficient to defraud. By only looking at costs, and not at risks, business has built our ecommerce infrastructure on sand. The notion of identity as it relates to business transactions needs to be reworked from the ground up.
The solution here is not technical - we already know the technical solution. The solution is to introduce a cost to the risk.
Right now the "risk" has no cost - if you expose people's data there's not that much consequence besides bad PR. We should assign some dollar value to each person's identify leaked. That way, businesses can properly asses risk and reward when they start these types of projects
Overtime, I think businesses would rather avoid the liability of storing people's info in the first place
This is exactly what will happen in Europe next year.
With the General Data Protection Regulation, any leak of private data caused by a major cybersecurity gap in a company will lead to severe financial sanctions such as a fine based on 4% of the company turnover.
As I understand it, Europe does not generally suffer from the disease of banks and courts accepting knowledge of basic demographic data as proof of identity. GDPR is not really comparable to the mess of the US social security number situation.
The risk we need is to introduce is to make banks (not customers) responsible when banks allow themselves to be tricked w.r.t. personal identity. This would motivate them to come up with identity verification schemes reflecting ~anything at all that the security community has figured out in 50+ years.
Exactly, there are best practices to avoid these scenarios. The engineers are just being pushed to produce so quickly that they end up doing things like this.
The problem has been for faaar to long that doing tech right takes much longer than the business side of the company is willing to wait for.
The other cost we can introduce into the equation is holding corporate leaders personally liable for these types of gross negligence.
Once CEOs know that they can go to prison if their customers' data is stolen, I assure you that they will suddenly become very interested in making sure their underlings follow best practices.
It would be a rather different world if people's private data couldn't be used against them in some fashion or other.
So the problem is both that money isn't locked by a more secure scheme than lots of data AND it is a problem that businesses are allowed/able to accumulate massive data on people wind-up being negligent with it.
The biggest single problem is that businesses accept data as identity, but then are allowed place the consequences of that (improperly confirmed) identity on individuals. Currently they get the best of both worlds (all of the benefits of trusting random strangers with almost none of the consequences). This gives them no incentive to properly verify their customers are who they say that they are.
The "rework" to the system would be minimal if the impact of identity theft was a business' responsibility to handle. Businesses would reshape their processes based on the risk that they'd be willing to accept. Instead, we have a system that's broken, where individuals are punished for companies' (deliberate) incompetence.
That's not really true though philosophically speaking. Even if I forget everything about myself, my identity (as far as my creditors are concerned) is the same.
I think that there should be some kind of hardware key or smart card system issued by the government, where if you loose it or get it stolen, you can go to the post office or DMV or something to get the old key revoked and a new one reissued. The post office or DMV would verify your identity by biometrics or N other forms of ID.
Key-based cryptography falls apart once you have a manual component to it. Let's say hypothetically I have an ID card containing my keys -- how do I use it to pay a bill online?
Which is a moot point, as it moves the barrier to getting it to "impersonating another person in front of US authorities", so more like getting a forged passport.
If only most online authentication was that hard, and with so much risk for those attempting to fake it.
You're right that ryacko missed my point. And, you nailed exactly the benefits of an in person government issued asymmetric key process.
Even if showing up in front of US authorities was completely hackable, it'd still be better than the current state of affairs, because you'd have to hack people one at a time in person. (Unless you discovered some 0day in the key generation process or the hardware or something)
That naturally limits identity theft to a targeted attack on one individual, rather than being able to steal half of US adults identities all at once. Also, the hacker (or their accomplice) would only be able to hack one person per DMV (or Post Office) per month. And they'd have to at least vaguely fit your race, height, gender, etc.
A gang would maybe have the organizational skills to farm it out to a bunch of people. But, that's still limited to something on the order of thousands of people.
Let's not let the perfect be the enemy of the good.
True. This is the crux of the issue. In the beginning of commerce, identity was literally your physical presence. As business evolved, it gradually became a collection of information contained in documents, sometimes bearing seals and signatures. It was a lot easier to copy than someone's physical body, but still not trivial. With the advent of the digital age, it's become trivially easy to copy any data used to identify someone. So we moved to secrets, but that requires two people to keep the secret, only one of whom is really incentivized to do so. Then we turned to public key encryption. It's the best thing we've been able to come up with, but still suffers from computational loopholes, often due to implementation flaws or poorly secured private keys.
Does it just come down to the cost of copying? Should all businesses who use identity data indemnify their customers against identity theft (as suggested elsewhere in this thread)?
> For example with certificates on a national card, like spain or estonia.
There is so much wrong with a national identity card. For example, who will pay for it? I don't want to pay for it. I don't want my taxes to pay for it. Even if you find some way to pay for it, I don't want it. What's next? Will you require me to carry a identity card on me at all times? "Random" cavity search for people walking down the road?
> Even if you find some way to pay for it, I don't want it.
Sounds like you are not open to discussing this.
Also, what does a national ID card have to do with "cavity searches"?
Edit: I understand the American insistence on reducing the reach of the federal government. What I don't understand is how many of those same people are fine with allowing Congress to screw us all over on important issues like healthcare and taxation...
> Sounds like you are not open to discussing this.
Should I be open to discussing everything? Why? Personally, as long as this Congress is in session I think we should block every single new legislative idea because I can't trust it to do anything right.
> What I don't understand is how many of those same people are fine with allowing Congress to screw us all over on important issues like healthcare and taxation...
I don't either. Which is why I think they will manage to mismanage this. Remember, getting health care dot gov up and running took heroic efforts of a lot of people. Just something simple as getting a dot gov website (department of education to be exact) took years. The Path station at World Trade Center was 100% over budget at FOUR BILLION DOLLARS. Nobody cares. It isn't my problem, right? We talk about net neutrality and how we gave over $400B to telecoms to deploy fiber across the nation (which to their credit, they did but what about the last mile?). And about health care, I am sorry but Obamacare doesn't even go far enough to call it an achievement. Apparently, even Germany spends less (as a portion of its GDP) on healthcare than we do. This is insane. The main point of the matter is costs have to come down. People get mad when I say while minimum wages should go up, wages and salaries in general should come down. Credits and deductions should go away when it comes to taxes. Yet, the people who want a "simplified" tax code get up in arms when they can't deduct their house or their car from their income... Why can't I deduct my rent? Why can't I deduct my cost of commute? Why should I? There is no rhyme or reason when it comes to taxes. It seems like it is just about who can push their way through...
You seem to be talking emotionally instead of rationally. But I suppose you already have a passport or a drivers licence (what if you don't drive ?). Then you just add a chip to it. This chip generates its own keys and then you can authenticate yourself with it.
What relationship does providing citizens a way to identify themselves, sign documents, to having cavity searches ... can you show how providing a nationwide digital certificate scheme leads direct to random cavity searches??
It's like if someone says "I fancy Chinese takeaway" and you don't want any you say "enjoy your cavity search" as if that's a natural outcome.
Explain.
Moreover, how is identifying yourself to the police bad? Sure if you live in a fascist dictatorship, but in a Western democracy?
If you're in the US, you already have a national ID. The SSN. Because everyone uses it like that. This would just be a much more secure way of having a unique number for your person.
And if keeping citizens secure is not the job of the government and thus paid for with taxes, then what is?
I am from a country with national ID cards and yes people are required to carry it on themselves at all time if they are outside.
It's mostly used to check your identity if the police stops you while driving, which is a little annoying because surely they could just check that electronically, but it's probably a legacy system. You also use it to verify your age when buying alcohol because driving license is not a valid form of ID.
It's not really a problem, I have mine in my wallet and never take it out, it's been sitting there for years. You can only really get in trouble if you don't have it while driving, and in that case I probably won't have my driving license either because it's in my wallet as well.
> people are required to carry it on themselves at all time if they are outside
... while in other contries there is an ongoing discussion about whether it is against the Constitution to (begin to) require to show an ID when voting!
Are you sure of that? There is an almost finished call for algorithms right now from NIST, because RSA and elliptic curve are known to be broken by quantum cryptography.
While you might not personally believe this to be true, it's not a good design decision to enforce upon the entire world a concept of identity that many security professionals will tell you is broken at scale.
This doesn't even address key management issues such rotation, theft, loss, planting, etc.
You will have to have an Equifax like service that will do the mapping.
But then how do you prove to them that you are who you say you are in order to map you to your public key?
Back to, SSN, driving license and what not.
Edit: I don't think having a governmental service dealing with thousands of people every day(lost keys etc..) is something that is going to happen in the US.
In the United States, at least, the U.S. Postal Service is in a unique position to "pivot" to being an identity and "trust" provider. They already provide a physical-to-identity "mapping" service for the vast majority of Americans.
> In the United States, at least, the U.S. Postal Service is in a unique position to "pivot" to being an identity and "trust" provider.
The USPS is nowhere near equipped to handle this. And there's no way I'd trust them to be competent enough to handle the task with the level of security that it would entail.
> They already provide a physical-to-identity "mapping" service for the vast majority of Americans.
They really don't. Even if we ignore the fact that one's identity is completely separate from the question of where they reside, the USPS has no way to verify residence. They don't really even a way to verify mailing addresses, which is at least a more well-defined problem than residence.
> And there's no way I'd trust them to be competent enough to handle the task with the level of security that it would entail.
Why? Obviously it would be a big undertaking, but the post office already issues US Passports. I'm not sure what you mean by "verifying mailing addresses" because the USPS does provide a way to do verify the correctness and deliverability of an address [1].
The point is that the USPS would be in a good position to become the government body that implements a national identity service.
> but the post office already issues US Passports.
The State Department issues US Passports, the Post Office merely accepts your applications on their behalf.
> I'm not sure what you mean by "verifying mailing addresses" because the USPS does provide a way to do verify the correctness and deliverability of an address [1].
It's only vaguely tied to identity. I have my physical address, but because I live in an RV park and move often, I don't actually receive mail there. In about half of the locations I've stayed, you _can't_ receive mail there.
I use a private mailbox along with a mail forwarding service in order to receive postal mail.
Is used to solve the problem of sorting and routing mail, which is really what the Post Office spends most of their effort on. Every postal address in the US can be uniquely identified by an 11 digit code: your ZIP+4 and the last two digits of your house number, but it completely ignores multi-tenancy and has no provisions for linking to identity.
The post office does not issue passports, the US Department of State does, it literally says right on the passport that it's issued by the Dept of State. Additionally, there's a whole host of government buildings that accept your passport application that aren't the post office. I submitted mine at the county clerk's office.
Have you ever filled out a form that uses that system? It will be rejected if you use any "weird" characters. You know, characters like a period after the abbreviation "St." for "Street". It's ludicrous.
>The USPS is nowhere near equipped to handle this. And there's no way I'd trust them to be competent enough to handle the task with the level of security that it would entail
Is that more market ideology ("public services are by necessity so dumb") speaking, or is there something specific with the US postal service?
>"In the United States, at least, the U.S. Postal Service is in a unique position to "pivot" to being an identity and "trust" provider."
The US Post Postal Service is most certainly not in a unique position to "pivot" to being an identity and "trust" provider."
It's actually quite unbelievable to think that a bloated organization with 500K employees[1] that relies on manual and mechanical processes can "pivot" to become a tech organization providing digital identity management.
The USPS "pivoted" to delivering junk mail for direct marketers years ago. Given that there largest customer is direct marketing, that alone makes them rather unfit to be an "identify provider."
The USPS government mandate is delivering mail and despite having a government sponsored monopoly to deliver mail to mailboxes it still loses money hand over fist[2].
Because unlike an SSN, you never hand out your private key. Ever. Instead, you encrypt things with your private key, and whomever you are validating your key to looks up the public key, decrypts your message, and has their proof.
The public keys can be stolen all day long and they're the only part of the equation that needs to be stored anywhere long term. The private keys are just that; truly private, and ideally extremely difficult to steal.
Yes, there will need to be a public service to manage the public keys, and yes this will be able to be compromised in some dangerous ways, but not quite so dangerous as "Whoops, everyone's SSNs are lost, now any attacker can impersonate them because that's all they needed."
If you think they're going to be difficult to steal you're crazy. They will sit unsecured on personal computers in some folder on the Desktop waiting for any malware to scoop them up. They will overnight beclem the highest value target for hackers.
This is even before we talk about how to handle the huge number of people who will lose or delete them.
Tech can't solve this prpblem. Any system that requires a secret won't be.
They will sit unsecured on personal computers in some folder on the Desktop waiting for any malware to scoop them up. They will overnight beclem the highest value target for hackers.
Most modern phones have secure elements that can generate and store a private key that cannot be extracted through software. Also, it is easy to set up things such that a physical confirmation is required to sign something (this is eg. what some U2F keys or touch ID do).
Of course, you still need a procedure to map public keys to identities and people need to secure their phones in order to prevent someone stealing the phone to make signatures.
But using the secure element of a phone for various forms of authentication is orders of magnitude safer than relying on a credit card or social security number.
If you do something like this you actually need to be serious about it and establish a rigorous vetting/auditing process -- not just hand the contract to whoever donated the most to your election campaign.
Maybe set-up a 3-year long NIST competition or something, like they do when choosing new crypto standards, and establish the winner this way.
The other side of the equation, allowing services to interface with these cards securely, is already being solved by the FIDO 2.0 spec.
With smart cards (EMV, PIV, etc.), cryptographic functions executed on secret materials are usually handled on the card. The host sends the data to be encrypted or signed to the card, requests the card to process it, and then reads the encrypted results or signature. Often, there are no ways to get the chip to send the private key off of the device. Once the private key is generated or loaded onto the device, it can only be erased or written over.
Standards like FIDO and others allow for browsers and websites to utilize these cryptographic functions of smart cards in ways to handle website authentication. The technology exists for smartcards to handle authentication, it is simply a matter of us moving to this technology.
That depends on the chosen smart card technology. Some smart cards are USB devices in themselves, such as YubiKeys or associated FIDO-certified devices. Otherwise, you may want or need a smart card reader to read cards in a similar format to what you see on chip-based credit cards.
USB smart card readers are fairly cheap. Many business-focused laptops have been available with smart card reader options for many years.
The same way you can currently do it with smartcards. When you insert your smartcard into the reader, it is being treated as a certificate. Most major browsers support this, I can vouch for Chrome and Firefox personally, as I use a smartcard for auth in them on a fairly regular basis.
Do you have any information on how you've set that up and which cards and readers you use? I've been thinking about playing around with a similar concept.
The government keeps a publicly available list of the public keys of the people in their jurisdiction. Even the government has no need to know the private keys of citizens. In a sense, a person's identity is the public key. You prove who you are to a third party by encrypting a challenge text provided by the third party. The only reason the government needs to be involved at all is to prevent a single person from having multiple identities, but with or without the government keeping track of the public keys, bank fraud is made exponentially more difficult, and has to be done on an one-by-one individual basis, rather than the situation today where a single hack exposes the credentials of millions.
That's false. By centralizing public keys under the purview of the federal government you have created a single point of failure that is susceptible to theft, spoofing, planting, and numerous other issues.
In Slovenia where I'm from you have to go to a government location, same place that gives out IDs and passports and such, show your government issued ID and sign some paperwork. You are then given a digital certificate that you can use for online banking and e-government stuff. Proper RSA stuff. You install it on your computer and your browser uses it to sign requests.
Seems like a pretty good way to do it, if you ask me.
A slightly more efficient, if less secure, way is how Apple does it for their Apple Developer program. You have to prove to Apple in a way they like that you are who you say you are, then you are issued a certificate with which to sign your apps. That could work too.
I think a dedicated device (e.g. smartcard or USB dongle) is a better option. I know they've had their problems, but personal computers get owned all the time, since they're simply too exposed.
The part I liked about the system in Slovenia isn't so much the particulars of where or how the certificate is stored, but about how it's issued. Since the bit I was answering was "But how do you map a key to a person"
You do that in person, at the fed/state agency, using facial tech, fingerprints, etc. Once you prove who you are, you get to submit a public key to the agency, which you can revoke at any point -- you take a unique key from them too, and some combination of the two you use when signing up for new financial accounts.
Actually e-passports have (in 2017) a pretty competent public-key based issuing scheme behind them. You can read them with NFC and cryptographically authenticate the contents.
That's like saying how do you map a bitcoin to a person.
Public Private key cryptography and the blockchain.
Works great for bitcoins, would work great for identity.
A malicious actor can utilize a botnet to skew the crowdsourcing of trust, for example. In other words, in a WOT for identification, one's identification can be invalidated or stolen by someone who purchases cloud time.
Web-of-Trust means there's a link between you and the other entity. The number of people who trust something is not really relevant, so crowdsourcing wouldn't matter. I know there was a service called WOT that relied on that, but in my opinion they were simply misusing the concept.
A pgp wot or the wot that you refer to can be subverted. In the scenario of everyone's identification, the WOT covers everyone in the population not a small WOT inside a relatively much larger population.
So your interactions with the other people in your life (which could be affected by breaches of privacy) are all going to be governed by public key cryptography? Look I like strong crypto but this is getting ridiculous.
You can look at most first world countries outside the USA for inspiration, since the identity theft isn't a universal problem but one limited to USA and a few other places with similar policies. However, it's not a single change but requires a significantly different approach to things like credit.
Key components of such an approach are:
1) strong consumer protection legislation that removes consequences from consumers - mainly, that if a transaction is disputed the lender can't report such transactions to credit rating agencies and the credit agencies can't use such transactions in the credit rating. In essence the dispute resolution is that if you claim that it's not you, then they can either drop the claim (and marking that claim on your credit rating would be libel) or press criminal charges on you for fraud, with the usual due process and level of evidence required.
2) A secure nationwide, centralized physical photo/biometric ID issued to everyone (which is a no-no in USA for political reasons), so that organizations who actually want to verify your identity have a reasonable means to do so in a manner that doesn't rely solely on "something you know". My brother knows all my personal information and would be able to impersonate me in any knowledge-based access; however, but passing an ID check would require either theft or forgery; he can't claim lost/stolen documents and get ID in my name since the face/prints/etc wouldn't match the ones on file.
The immediate effect is that it makes it easier for fraudsters to operate in the current environment that relies solely on knowledge-based authentication (since one can simply take a loan and deny it, and get away with it since the lender now has no way of proving who was it if they followed processes currently popular in USA) so the next-to-immediate effect is that all the organizations "offering trust" either face huge costs or switch their policies to handle identity verification in a reasonable manner. It will mean a temporary slowdown in issuing credit - that's not an inherently bad thing, but it does mean that such a change has to be timed to occur during an economic boom/bubble (where a temporary slowdown in credit is useful) and not during a slump.
"Reworking" identity from the ground up as OP suggests is actually one of the goals that I've been working on with ibGib. No one really cares, but I'm going to describe some of the more interesting (to me) aspects of it, to bounce it off of you and others here.
First, ibGib's structure is like a block chain. I've been developing it for a long time, and I had no idea what a block chain was, and the like. But an ibGib's structure is like this:
* ib - unstructured text, like a name.
* often provides data or metadata for convenience per use case, i.e. data is just in the address, without loading entire record.
* ib + gib (ib^gib) is a "content address", but I think of it as like a memory pointer in an infinite memory space.
* Currently sha256 but that is metadata and can be specified in the data section.
* data - internal data, like a "value" or "content" of the record.
* rel8ns - named "merkle" links to other ib^gib.
* special rel8ns include...
* "past" - provides a linked list of mutations
* "ancestor" - provides linked list of forks
* "dna" - provides event-sourcing-like complete history of how to build the record.
So, it's effectively like a tree-version of a block chain, or a distributed (and scalable) block chain. Or if you're familiar with IPFS (which is where I learned the term "merkle"), it's like a merkle forest. (I've been working on ibGib for 15+ years though - had never heard of IPFS either, but I digress). Basically, you can think of the entire thing as self-similar git repos, but for anything - not just code (currently working on VCS use case for it, which is why I've taken the code off of GitHub. You can see my current "issue" for it at https://www.ibgib.com/as-chat/version%20control%20in%20ibGib...).
So this works with identity in a different way, in that each record is internally associated with multiple identity ibGibs. For the above example, check out the "identity" key in the "rel8ns" section. So, each individual datum is associated with _multiple_ identities for multiple things: users, nodes, sessions, etc. The piece I'm working on right now (in the active process of whiteboarding/coding at this very second) is the public key infrastructure "replacement". Because the data has this entire integrity chain, you can do different things for verifying provenance.
The way that you "prove" who you are is similar to the current SPHINCS algorithm (https://sphincs.cr.yp.to/ or ), which is an ever-expanding many-times hash-based signature scheme. In my algorithm though, you can create "keystones" which act similarly to public/private key pairs. Each stone has a list of hash challenges and the specs of the challenge difficulty. For example, if I have a stone of 100 challenges, the stone may say that a valid challenge requires a minimum of 5 challenges to be answered. The challenges are based on 1-way hashes (recursively called with a depth that is included in the params of the stone). So, when you first communicate between nodes, you provide a public global stone, that is replicated, e.g. to a "public key server" analog or wherever. In the initial contact between any two nodes this global stone is challenged, and if successful any future communications between the two nodes works on a private stone (created also in the handshake). Then, each transaction - in the form of ibGib data structures - is proven in the future using that private keystone. The ibGib internal integrity allows for integrity of the data exchange, as it's basically hashing the entire communication for verification.
And so, identity is established among nodes, and all data is verifiable. It's very tricky to really try to "nail down" the provenance once you get multiple nodes involved, but even if there is a known mistake, that is where another aspect of the data comes into play: non-monotonic (append-only) data.
Again, this is like a version control repository for your data. This leaves a full audit trail, yada yada yada, it's really neat. I've typed enough for people to ignore anyway. If anyone is interested, ask about how this affects identity with users AND IoT devices AND AI! Ah well. At the very least, the website is instructional for navigating around merkle forests.
I get that it's the customer's responsibility to correctly configure their services, but what does it say about the UX of AWS services that they're so easy to misconfigure with disastrous consequences? And despite this happening across a variety of industries, Amazon doesn't seem super concerned about it either, but I could be wrong here.
There's just no excuse for this to keep happening, but the processes meant to prevent this are clearly failing.
The new console for AWS literally has a documentation link containing example bucket policies at the top of the page for S3 buckets. Either someone was given a job they clearly had no experience in or was alarmingly inadequate at performing their job duties. It really shows the level of incompetence so many of these companies operate at with no regard for PII.
It's not that so much but that novice users see two ACLs: "Public" and "API Key I use to deploy." So it's very easy to "fix" an access issue by making a bucket public, with the alternative being to give your application the keys to the kingdom. This combined with S3's most frequent use as a file server means that buckets get opened up. You have to jump through hoops and dig into Amazon's access model for anything more nuanced, and only people experienced with AWS will understand or know how to do that.
Making a bucket public should come with red flags, and there should be a simpler way for client-side code to securely access a bucket. If I save a user's uploaded files to the filesystem, I usually have to go out of my way to expose that. Even novices are less likely to put them inside the web root, so exposing such files involves jumping through hoops. Opening a bucket to the world is too easy in AWS.
Calling this situation "insecure defaults" was imprecise of me, my point was more that Amazon gives you a Big Red Button to "fix" things which has consequences like this.
It does have red-flags. When you try to set a bucket to "public" in S3 it literally tells you: "This bucket will have public access. Everyone will have access to one or all of the following: list objects, write objects, read and write permissions."
I mean, there's a difference between being a novice and being stupid.
They were times when people used to manage their servers and they had to be careful on how to use 'rm -rf' or 'dd' tools.
I find it funny how today there have to be red flags, warning signs and double confirmations everywhere. Are people less competent or are there simply lower requirements when hiring them?
Not if you use aws-cli. But I think the larger issue is that Amazon doesn't provide a low-friction, secure way for applications to access their buckets. A good way would be for each bucket to come with its own access key, or expiring tokens that you could query on the backend and use from the browser. Just spitballing of course.
Windows tells you that using a password is probably a good idea for a user account. Many websites these days force you to use a combination of letters/numbers/special characters.
Why doesn't Amazon say "hey, this is publicly available, you wanna fix it?"
> Why doesn't Amazon say "hey, this is publicly available, you wanna fix it?"
They do. It's clearly warned about in the interface both at the time you make it so and with a big "PUBLIC" sticker label afterwards. What's more, I've received warning emails from AWS notifying of (intentionally) public buckets.
Public buckets for private data are a deliberate and wilful choice by lazy, reckless administrators.
I noticed this when I went into my console the other day. (Ironically, to set up a public bucket for something.) I don't think the UI was bad before but, now, you'd have to be pretty clueless to have a bucket public by accident.
Agreed. So it sounds like Amazon is making efforts to reduce configuration failures, but users don't necessarily take the time to properly learn the tools and configure them correctly because the risk of error doesn't seem to motivate them appropriately. What can be done to change that? Naming and shaming like this?
I'm also not a devops person, so I'm definitely not debating in my wheelhouse here. It's just crazy to keep reading about a seemingly preventable problem that has the potential to do major damage in regards to data exposure.
Not only is private by default, but also when you have a public bucket there's a big freaking "PUBLIC" label in the UI. Doesn't get more clear than that,
Maybe if you house sensitive information you should have a person or multiple people enforce security across your organization? Scapegoating one technology or person is not the correct attitude because it could have been anything else.
Also, amazon is improving their UX for these things already.
>Maybe if you house sensitive information you should have a person or multiple people enforce security across your organization?
They probably didn't care, and should suffer consequences as a result.
This looks like a really small company who contracted this stuff out to a WordPress shop. There's almost no tech employees or positions on job boards for them.
I honestly don't know what the solution is. Making a good user experience (read: convenience) and keeping your stuff secure is always a balancing act. I can't tell you how many times I've told a manager or director not to do something because it would make our client data insecure, only to have them basically reply with, "I'm your boss, do what I say."
This guy may have known exactly what he was doing, but relented to other higher ups who told him to do something. The shitty part is now this is public, all the execs can point to the developer and say it was his fault and fire him without incurring any repercussions themselves.
I suspect this kind of storage was not the original purpose of S3. For instance, it's a common to use S3 buckets to serve up webpage assets rather than using a dedicated CDN or your own servers.
Apple has actually updated their UX recently to let you know if your s3 buckets are public or not. EDIT: The recent update made it explicitly clear that your bucket is public. Previously, you might not notice, especially if you were taking over from a previous sysadmin (or more likely a developer)
It's frustrating that whatever I do to protect my own identity/data, no amount of 2FA or password generators or my own good practice mitigates the loss/theft of data/identify in this way.
I'm as good at it as anyone I know (and unsurprisingly I work in tech) and it's still a complete crapshoot.
Is there a way I can triple check my S3 bucket is secure?
I know I've not enabled public access that I know of - but given the recent focus on this; what are the exact steps that I need to follow so I can sleep at night and show a level of diligence on the issue?
Enable Amazon Macie. It automatically classifies your data in S3 buckets, detects situations where data is more open than it should be, and warns you if access patterns for data change in a way that may indicate that you have been hacked or someone is misusing their level of access to the data.
Neat! Didn't know about that service, hopefully businesses accept that hefty price tag though. It's obvious they don't want to invest too heavily in sec-orgs as it stands it seems.
Pretty soon with the GDPR kicking in it will be more expensive to not protect the data than it is to protect it.
All companies processing the personal data of people residing in the EU regardless of the company’s location who have a breach of data where the organization has been shown to violate basic privacy design concepts can be fined 4% of annual global turnover or €20 million, whichever is greater.
"Under GDPR organizations in breach of GDPR can be fined up to 4% of annual global turnover or €20 Million (whichever is greater). This is the maximum fine that can be imposed for the most serious infringements e.g.not having sufficient customer consent to process data or violating the core of Privacy by Design concepts."
So that 20mil or 4% (whichever is greater) is for companies that have seriously violated the GDPR. It remains to be seen how it is enforced, but my understanding is that this is purposely designed to be very punitive to force companies to have a dollar amount in mind when it comes to designing security and doing the right thing.
How about monitoring via polling? I'm imagining something like this...
- Set up application with your AWS/S3 credentials
- Poll S3 to get list of all your buckets on a regular interval (once a day, every 5 minutes, whatever)
- Get a list of some files in those buckets
- Try to access those files directly w/ no authentication or authorization
- Set up some rules about how to interpret the results (look for any public files, look for specific private buckets, look any buckets that are public & haven't been whitelisted, whatever)
There's probably a ton of ways to do this. For simple use cases, it shouldn't be too tough. That'd be a fun hack for a day project, and I'd be happy to pair with you on it if interested. It's probably spending a little time looking around for an off the shelf solution first.
A private bucket isn’t going to turn itself public on accident. This error wasn’t the bucket suddenly becoming public, it was a developer making it public because that was the easiest way to get the job done.
Your testing system would not help here at all, and if you spent the effort to make the system, you are already the type of person who isn’t going to turn the bucket public to make your life easier.
> Is there a way I can triple check my S3 bucket is secure?
Amazon Trusted Advisor will automatically send you a weekly email if any of your buckets are misconfigured to allow public access. The catch is that this feature is only available if you pay for a premium support contract, which is hundreds or thousands of dollars per month.
If you have Trusted Advisor enabled but aren't paying for premium support, then you will still get the weekly email saying that there is a security vulnerability somewhere in your system, but when you click to see what it is you will just get prompted to hand over your credit card to signup for an annual support contract.
There are several companies that handle this for you, though some more effective than others. Evident.io is a big player in the field, though general complaints are that they're too noisy.
However, this is actually what CloudCoreo does - infrastructure security. More precisely, infrastructure security at deployment and continuous security monitoring (Evident only does the latter).
Finding smart solutions to the issue of corporate data breaches is laudable. Finding solutions that are feasible for use by people who are being paid the lowest wage that the company they work for can get away with will be miraculous, IMHO.
The reason I say this is that knowing not to give a sensitive AWS S3 bucket public access is 101-level AWS expertise, which means either of these scenarios are likely:
- The persons responsible did not know any better, or
- They did know better, but made an error provoked by pressure due to understaffing.
Both of these point to a lack of competent security personnel, either due to understaffing, or hiring insufficiently experienced staff, both of these due to minimizing salary budgets (or clueless hiring managers, perhaps).
It's great to see attention being drawn to responsible use of cloud services, which for most part have safe defaults. I absolutely support, responsible disclosure but it is always uncomfortable to see that security researchers (this particular team) goes beyond vulnerability discovery and handles the data directly. Who is responsible for safe disposal of the data they downloaded and analyzed ?
Back in the paper days. I moved into a new office only to discover a dozen or so boxes crammed full of people's personal files. Mortgage applications and taxes. It took a couple of weeks to track down the parent company buy all sorts of damage could have been done had I been anyone else.
Those AWS “practitioners” who are too stupid or lazy to figure out IAM policies. Thankfully, AWS has added bright yellow labels to identify public buckets. However, labels won’t be enough to motivate some people to learn JSON.
Last year one of mexico's political parties left the nominal list with a lot of citizen's information available in an AWS server, unforgivable; how come this keeps happening.
