Sorry but this is a coverup that Alex is using to defend himself. He had easy access to Wes, as Wes was actually demanding a reply via Facebook's own system in place to communicate with researchers, and not receiving one.

Alex would have been aware via the original RCE bug that Wes was reporting on behalf of himself and not his employer. Also, it is reasonable that Wes would have mentioned that he is reporting the bug on behalf of his employer from the beginning.

I presume that Alex knew these things, but he decided to take a more dramatic approach to get Wes to stop, by contacting his employer. It obviously would be leverage, and Alex knew that he could also leverage his position at Facebook to use a security firm in the industry (who would understandably not want to do anything to jeopardize its relationship with one of the largest internet companies in the world) to ask their employee to stop.

I do not believe that Alex legitimately believed that Synack (Wes' employer) was behind the research, but he knew it would be an effective way to stop Wes from continuing, so he decided to pull those strings.

I'm more questioning the flow of researcher reports vulnerability, company awards bounty, researcher disputes bounty value, CSO of company contacts CEO of researcher's company. Is that normal escalation procedure?

Wait, you just made something up.

Even the researcher doesn't claim that Alex contacted the CEO of Synack because of a dispute over the bounty.

Rather, it's the other way around: the researcher disputed the bounty, and did so by revealing that he'd retained AWS credentials from Instagram long after they'd closed the vulnerability that he used to get them.

Alex contacted the CEO of Synack to ensure the credentials weren't used, because if they were, Alex couldn't be control Facebook's response: they've got a bug bounty participant who has essentially "gone rogue" and is exploiting Facebook servers long after they've told him to stop. They need him to stop.

The "bug" here is that they aren't really keeping track of their AWS buckets and keys at all. Least privilege, access logging, remote IP flagging, etc. These operational failures are ostensibly the responsibility of the CSO.

I'm not saying this researcher was 100% in the right, but this is the CSO ass covering. "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching."

A simple phone call directly to the researcher that cut through the bullshit would have made everything better. But he had to make sure it didn't get out and the only way he could do that was by using the only leverage he had: The researcher's employer.

Alex has in the last few months built one of the best teams in application security at Facebook (Facebook security is now seemingly most of O.G. iSEC Partners). I get it, everyone hates big companies and especially Facebook evil Facebook but, come on. They know what they're doing.

If you understand how security works inside of big companies, this is a really silly theory to run with. CSOs are happy when shit like this gets discovered, because it gives them ammunition to get the rest of the company to adjust policies.

If you were working from the understanding that a CSO comes in and just immediately tells a team of (what is it) NINE THOUSAND developers how to do stuff differently... no. That's not how it works.

The problem is that nobody at Facebook with the possible exception of like 10 people none of whom are Alex can make huge operational changes like "change all the ways we store keys across an entire huge business unit". So, you tell Alex you took AWS credentials he didn't know existed and you're going to start mining them for a story you're bringing to the media, and now Alex is in a position where he's NOT ALLOWED to sit back and try to manage the situation himself.

Delete the keys or I have to tell legal what's happening.

The researcher NEEDED TO HEAR THAT.

>> Delete the keys or I have to tell legal what's happening.

>> The researcher NEEDED TO HEAR THAT.

I'm not in security, but from the outside looking in, how things worked out just doesn't smell right.

If "the researcher NEEDED TO HEAR THAT" is the priority, then why waste time looking up who the guy works for and calling them instead?

The simplest and most obvious way to tell the researcher is to tell him directly in the clearest way possible. It isn't as though there wasn't a pre-existing line of communication with the researcher.

My reading of tptacek's subtext is that Facebook wanted to show the researcher that they were really, ALL-CAPS serious, as in "get you fired and ruin-your-livelihood if you don't stop" serious. These mafia tactics are fine because the Facebook CSO "built a good team and knows what he is doing"

If FB wanted to show the researcher that they were really, ALL-CAPS serious, then they would talk to him directly as in "You've got stolen data and we're going have the FBI arrest you, seize your computers and put you in jail ruin-your-livelihood if you don't stop" serious.

So I still don't see how calling the guy's boss trumps that in terms of scariness. Because if I'm the wronged party (i.e., FB), that's what I'd do if I couldn't resolve it amicably.

If we are disagreeing, I don't quite follow your argument - I never said that this was the worst/scariest thing Facebook could to do (there's no upper limit). What I meant was that the action by Facebook was intended to intimidate (and not that the specific form of intimidation was the worst possible)

We're not disagreeing. I think your interpretation of tptacek's subtext is the same as mine.

In some of his posts, he has been, however, comparing the researcher's dump to criminal activity -- something I am not in disagreement with.

His implication that calling the researcher's boss is a sensible approach to intimidating the researcher for potentially criminal activity -- that in particular seems like a stretch if he's being truly objective.

...which all adds up to a smell of "tptacek knows that team is good, because he's on it".

I don't doubt that he's put together a great application security team. Or that he even knows his shit. And I do understand how it works. CSOs are happy when this kind of shit gets discovered when they can't get other teams onboard to fix it. They're unhappy when it gets discovered when they intentionally ignore it in favor of another initiative (particularly if there's a paper trail showing that someone brought it to their attention). Or when they've already spent a bunch of money and resources fixing it only for everyone to find that they haven't fixed it at all.

There are basic things you can do to mitigate or isolate damage in AWS and they either aren't doing it or have done it badly. Even if he couldn't convince the rest of the company that god-mode keys are bad, he still could have built out some basic infrastructure to track when and where they keys were being used from so red flags could be raised when some random IP address is being used to pull down several buckets.

You're perfectly right, but his employer didn't need to hear it. And that's the whole crux of the matter.

If you read the article, his company does security research and found a vulnerability in Hotmail. Plus he was using his company's email address.

> At this point, it was reasonable to believe that Wes was operating on behalf of Synack. His account on our portal mentions Synack as his affiliation, he has interacted with us using a synack.com email address, and he has written blog posts that are used by Synack for marketing purposes.

That's a big mistake. DO NOT EVER USE YOUR COMPANY EMAIL ADDRESS if you are doing this on your own. The employer has the right to know. Imagine using a company email address on Ashley.com. Yeah, plenty of people were embarrassed after that hack.

Actually his write up makes pretty clear that he didn't use his company email until after Alex went over his head to the CEO.

Second, everything else being equal, Alex going to the CEO without calling or mailing the researcher first was a mistake. Going to someone's boss and saying "please do something, I don't want to get the lawyers involved" IS an implicit legal threat, both to synack and the researcher.

What Alex wrote is a bit interesting given his update (emphasis my own):

>At this point, it was reasonable to believe that Wes was operating on behalf of Synack. His account on our portal mentions Synack as his affiliation, he has interacted with us using a synack.com email address, and he has written blog posts that are used by Synack for marketing purposes.

According to Alex this is the timeline:

1. Researcher not happy with sum

2. Researcher already in contact using Synack email address

3. Alex calls Synack CEO

From the researcher's blog:

>I never contacted Facebook or Alex using my work email account. It was only after Alex contacted my employer via email that I sent a reply from my work account. Alex indirectly contacted me at work, not the other way around.

This means that Alex is lying, is telling exactly the facts needed to come to a specific conclusion and nothing more or the researcher is lying. And he's "written blog posts that are used by Synack"? Come now. Reads a lot like someone looking for a third item so they can make it a comma separated list of reasons. His post smells like bullshit.

I like how we're talking about Stamos warning a guy running around with stolen AWS credentials for all of Instagram in the same fashion as we'd talk about a DMCA threat. "Implicit legal threat"? There's nothing "implicit" or subtle about what was happening here.

You seem levelheaded throughout the thread and make good points more articulately than I ever could but this seems a bit emotionally involved.

It's possible that we're all correct: This guy could be a wildcard researcher that plays fast and loose and the CSO could be covering his own ass. You say he's building a first rate application security team. Is it hard to believe that he could have made the mistake of focusing almost exclusively on that?

I get your point in these threads, but unless I'm misunderstanding, who cares about stolen, potentially undeleted Amazon creds? Revoke the key in the portal and be done with it?

Given who I'm replying to, I'm assuming that I'm missing some key piece of the puzzle.

(And I totally acknowledge it doesn't change the circumstances of what either side has done, I'm just curious)

The point is, having those is a prosecute-able offense, if Facebook chose to prosecute. So it's a big threshold to cross legally, even if not meaningful from a programmer's perspective.

Facebook's terms say they will not prosecute /report whitehats to law-enforcement. Facebook could prosecute, at the price of some goodwill from the security industry (or part of it). I'm sure a competent lawyer to mount a robust defence for the security researcher (beyond reasonable doubt, IMO).

You're missing my point and talking about something entirely different from what I'm talking about. I'm not talking about whether Facebook will prosecute and what the consequences of that will be (whether they'll win or lose whatever).

I'm just pointing out that taking AWS keys is a big deal, because it's legally a big deal.

Facebook's disclosure policy reads:

>If you give us reasonable time to respond to your report before making any information public, and make a good faith effort to avoid privacy violations, destruction of data, and interruption or degradation of our service during your research, we will not bring any lawsuit against you or ask law enforcement to investigate you.

IANAL: but it could be argued (in court) that he had Facebook's permission to getting the AWS keys. In his opinion (and mine) he made good faith efforts to avoid privacy violations.

Facebook's official disclosure policy has legal weight. There is legal concept (whose name is escaping me) that could apply that in laymen's terms say the official disclosure policy gives him Facebook's tacit approval - I first heard about it in the Oracle v Google where Google argued a blog post congratulating Google provided tacit approval.

The part you emphasized is dependent on the first part of that sentence, however. In this hypothetical lawsuit Facebook's lawyers would easily be able argue that they would not have done anything for the initial exploit or even demonstrating that he had recovered valid AWS keys but that attempting to hoover up data from S3, etc. violated the “good faith effort to avoid privacy violations” part.

>but that attempting to hoover up data from S3

That's a mischaracterization given his description. He examined the filenames/metadata specifically to avoid buckets that might contain user data.

1. This assumes his description of his actions is completely accurate

2. This assumes that he was perfectly accurate in his assessment of an unfamiliar project's naming conventions, data structure, etc.

3. This assumes that he was perfectly reliable in making the actual copies and didn't accidentally include potential personal data (e.g. who knows what might be in a log file?)

The problem is that we're talking about someone who already decided to exceed the bounds of what was clearly protected under the bounty program. He'd already reported the initial vulnerability and been paid for it but waited until later to mention that he'd copied a bunch of other data, had access to critical infrastructure, and wanted more money.

It seems fairly likely that this wasn't malicious but rather just poor judgement, but that makes it very hard to assume that outside of that one huge lapse in judgement he did everything correctly. It's really easy to see why Facebook couldn't trust his word at that point since it's already far outside normal ethical behaviour.

To your first point: There's being skeptical and then there's calling someone a liar without actually calling them a liar because you don't have any justification for doing so. This is far from the first time I've seen this on HN and it's really not okay. There's no point in speculating about the veracity of this person's statements until there's a reason to.

To the second and third: They only require that a researcher "...make a good faith effort to avoid privacy violations..." and I'd say he met that. You can argue that the entire endeavor wasn't in good faith but he certainly made a significant and conscious effort to avoid private data.

I think his biggest lapse in judgement was that he brought security operations issues to light in a bug bounty program run by the people that would be most embarrassed by them. Application security bugs are created by the engineering team and the CSO's application security team fixes them (or advises or whatever). Security operations issues are entirely the responsibility of the CSO's department.

Facebook (as an organization) should be thanking him. While he didn't expose application security bugs he exposed significant operational issues and blind spots. Keys with far too much access, lack of log inspection, lack of security around what IP addresses a key can be used from, etc. Operational issues and lapses in operational security are what got Twitter in hot water with the FTC in 2010. It's not as easy to play cowboy with operations as it used to be.

The CSO hasn't been around for long but by all accounts he poured a lot of effort into hiring an application security team. Perhaps that's his specialty but even one experienced technical manager hired for security operations could have caught these basic issues. They probably wouldn't have addressed the lack of least privilege in that time frame but they could have easily spun up logging to catch some rando on an unknown IP address using their keys.

But like I said, he hasn't been there for long so I don't blame him for the failure. What I do blame him for is calling up the employer to threaten them as leverage to shut up the researcher. I blame him for posting a thinly veiled justification for doing so. He could have addressed this openly, talked to the guy directly and went to the other C-level execs with it as a justification for getting everyone on board with fixing it but he tried to keep it contained to his department.

I understand how he must feel being the new guy who's responsible for the outcome but not for creating it. I know he'll get questions that he might not be able to answer since they probably aren't logging bucket access. Questions like, "Who else got a copy of these keys and what did they access?" Saying "I don't know and we may never know" in response to that, even if you weren't in charge more than three months ago, is rough.

Again, you're quibbling about legal details that are not relevant to my point. I'm pointing out that his actions are a big deal because they crossed a legal threshold where a company would have a somewhat decent case to prosecute you. I don't care whether or not they would succeed.

Then why the immediate escalation?

Wouldn't it have made more sense to contact the researcher directly, rather than using his position of power to pressure the researcher's company's CEO?

Why not assume good faith? (Which is what I would think a white hat bug bounty program should assume)

Bottom of his post he replies to Alex his post:

> I never contacted Facebook or Alex using my work email account. It was only after Alex contacted my employer via email that I sent a reply from my work account. Alex indirectly contacted me at work, not the other way around.

If that is true, it is either poor judgement from Alex, or bad intent to call Synack

It says nothing about who initiated contact using his company email address. It could have said, "he contacted us using" or "his facebook account was associated with" but instead it says "he has interacted with us using". Sometimes what's not said tells us as much if not more.

The Facebook reply does not state that he was using his company email address to report issues or to communicate prior to them reaching out. The researcher says that he only used the email after Facebook got his employer involved.

The Facebook post does not, in any way, contest that.

Technically it doesn't contest that but he uses multiple weak points in bullet prior to the one about contacting the employer's CEO. The intent was clearly to establish that his decision to contact the researcher's employer had merit. It was clearly carefully written so it remained factual while implying things that aren't.

I'm not disagreeing with you, only making it clear that yeukhon was played by Alex exactly as intended so he'd be out there defending him on sites like HN.

He didn't use his company address until he was contacted through his company.

In any case, there is one question remains. How do facebook defines a "million dollar" bug if the security team is not aware of the damage it can do. Since this is not the first time this bug was reported, did they actually give a big bounty to the first person who did the initial report(Given that it can lead to this much damage)? Or just another small bounty saying that it's not a very important security flaw.

There are enough laws against "cybercrime". If Alex felt threatened he should have escalated the issue to the FBI. There is no single reason to call the employer. By doing so Alex has threatened Wesley to fuck up his life.

edit: Or -after calling the CEO- he should have contacted Wesley directly and so they could deescalate the problem together.

> The researcher NEEDED TO HEAR THAT.

I don't disagree. But why go through his employer, when they already had a direct line to the researcher himself?

Intimidation.

Relative security teams are almost useless. In 2-3 years FB might have it's shit together, but three months is no where near long enough to fix there problems.

Agreed, after looking at this linked in Profile, it's hard to blame Alex for the problems as he's only been there a short while. However, he can be blamed for creating all this unnecessary drama.

If you understood how big companies work, you'd know it takes more than a few months to build "one of the best teams". This is one thing in Alex's favor though, he's new to the job. Still, if you also understood how big companies work, you'd know that everyone hates the drama queen.

The right move here would have not been to threaten Wes, pay him, and just update the policy.

Lesson learned for Alex and his friends: Do not threaten individual contributors or suffer massive freaking drama. Thank you internet.

> I'm not saying this researcher was 100% in the right, but this is the CSO ass covering. "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching."

The response from FB's CSO is very specific to a very specific blog publication. Not regarding the flaws in how their AWS Buckets are used.

I'm not sure what you're getting at.

Your statement:

> "Don't pay attention to the obvious operational deficits, the problem is the researcher overreaching."

mischaracterizes what the response by FB CSO as one that is attempting to draw criticism away from operation flaws by instead placing focus/blame on the researchers methodology.

I disagree.

A security researcher went public with a story of "I found this massive security hole and Facebook tried to avoid paying what I thought it was worth, and then threatened me with legal action"

The response that Alex thinks he needs to make is "my actions were reasonable because ..."

From external appearances it seems as though he is more concerned about looking like a heavy-handed, lawyer-invoking, CSO than the publicity around FB having an unpatched RCE that allowed access to highly-privileged AWS keys.

What he chooses to write about is reflection of what he saw as the most important news in the original blog post.

I suspect he's actually right. The blog post will probably raise more bad publicity around the way FB handled the research & disclosure than the existence of the bug, and it's the piece that needs to be resolved well.

You're right, that was the purpose of trying to keep him quiet by contacting the CE-freaking-O of his place of employment with an implicit legal threat. The blog post is an attempt to do damage control when he realized the researcher wasn't going to put up with that and went public.

> by revealing that he'd retained AWS credentials from Instagram long after they'd closed the vulnerability that he used to get them.

How would that change anything?

If Facebook did rotate all keys the moment the researcher reported it, they made no difference.

If Facebook did not, then they aren’t taking care of their security properly.

Without defending the researcher here, I thought that was the weakest point in Facebook's response. Was he interacting with Facebook using his synack.com email address during this exchange rather than at some point in the past? Was he signed up on Facebook with his synack.com address? (I haven't used the bug bounty program but it appears to require a user account.) Did he mention his employment with Synack in the course of the exchange? If any of those things were true, I suspect they'd say so, rather than leaving it at "has interacted..."

I don't know, if the guy was just shaking them down then maybe trying to get him fired is indeed a reasonable thing to do, but I don't buy that anyone would have just assumed under the circumstances that he was doing all of this on the clock.

"I never contacted Facebook or Alex using my work email account. It was only after Alex contacted my employer via email that I sent a reply from my work account. Alex indirectly contacted me at work, not the other way around."

I don't think it does. Wes asked for communication via Facebook's own tools for it, didn't get it, and they went around him to his boss. That's crap.

Now, Wes exfiltrating data rather than just looking at it? Not cool. But Facebook's side of the story is just as biased as his.

But it seems obvious that in doing so he wasn't acting in good faith.