This is really cool, you'd expect any VPN provider that cares about security and transparency to act like Mullvad. Some pour thousands of dollars into forcing influencers to say they care about security, while others focus on actually improving security.
> Some pour thousands of dollars into forcing influencers to say they care about security,
Tangential to this, it always irks me how they talk about how they all act as if the majority of the websites their users are going to aren't HTTPS and they act like their main benefits are filling in the gaps that HTTPS actually fills in.
HTTPS isn't a cure all by any means but most of the scare tactics that the big VPN companies that advertise via YouTube act like anyone will rip you credit card because you happened to be on Amazon while you were at the coffee shop.
Tom Scott is the only person I've ever seen have a great video about this [0]
> how they all act as if the majority of the websites their users are going to aren't HTTPS and they act like their main benefits are filling in the gaps that HTTPS actually fills in.
I hear most of them saying "Don't want your ISP spying on where you're browsing? Use a VPN." Which HTTPS does not cover.
With HTTPS, ISPs can see _where_ you are browsing, but not _what_ you are browsing. Of course them seeing the top level domain still violates certain aspects of privacy, personally I’d prefer ISPs couldn’t even see that. But it’s not like they are peering into the actual content of what you are browsing.
True, and with the rise of CDNs it's probably even harder to figure out what's happening. But it still gives them more info than is necessary, given they like keeping logs for the powers that be.
I’d agree with you about HTTPS providing most of the benefit that VPN advertising focuses on if I hadn’t seen repeated direct evidence that even most technical users will blithely click through HTTPS errors’ “accept the risk” bypass. It’s as if knowledgeable users think “sure, this could be a man in the middle attack, but it’s most likely just a benign cert problem, because certs are hard.” Sigh.
To be frank that's also because the cause for an HTTPS certificate error ranges from "malicious hijack" to "misconfigured server setup" to "I lapsed the expiry date" to "I am using a self-signed certificate".
The degree of which these should be scares is not equivalent, yet browsers will treat all of these as equivalent even though they can distinguish between them in the error page. It just results in clickthrough fatigue, where technical users just ignore the warning because it's not worthwhile to deal with even when they really should.
Plus a VPN won't protect you from a malicious hijack, it just prevents them from grabbing your IP address.
The reason the browser doesn't differentiate between them is because the end result is the same - the cett doesn't match the browsers trusted store. The battle has beenosr on self signed certs at this point (unless you're an enterprise, at which point bundle them with your image).
The difference between a misconfiguration and a compromise is intention, both should be treated as equally suspicious.
The problems with clicking past those errors are typically not due to network sniffing but with whatever crazy shit is on the page they are going to.
The only two valid usecases of big VPNs like these are
1. Very mild security increase over public wifi
2. Shifting your risk from the ISP spying to mullvad or the VPN provider spying or slightly anonymizing if mullvad rotates IPs.
(2) is a real benefit because ISPs are pretty terrible, but it's still pretty minor in the grand scheme of most people's threat models.
3. You live in a country where your ISP is legally mandated to record all of your browsing history and make it available to the government.
4. You live in a country where certain websites are blocked because the government doesn’t agree with them, or because those websites don’t want to deal with your country.
There are some countries that block VPNs but there’s also many countries that don’t. For example, TPB is blocked in UK by court order but VPNs work just fine.
>if I hadn’t seen repeated direct evidence that even most technical users will blithely click through HTTPS errors’ “accept the risk” bypass
As far as I recall this is not possible on Chrome if you are MITM'd. If the cert presented doesn't match the cert in the HSTS cache, there is no option to bypass. If the server's cert is expired, then you do indeed see the option, but an expired certificate doesn't necessarily mean danger.
Not to provoke predictable responses, but I find it interesting that the tech-talented VPN providers are not using BSD in favor of Linux, especially with requirements like diskless operation, kernel customization, and tighter security.
For me, the pool of people to hire that know Linux inside and out would be much larger. This is worth any perceived security issues.
In terms of diskless, I've run 25k+ iPXE deployments on diskless blade servers using a highly customized Ubuntu, and it was fantastic.
Regardless of OS choice, being diskless is also quite nice... if there was a security issue or you need an upgrade of some sort, you just reboot. Only thing is that it takes a while to reboot 25k servers... even on gigE. It was a bit of work to build the scheduling system to make that happen reliably, but it worked out quite well.
I guess that is some of their focus around why they got their image down to 200MB.
Even better if you had boxes with 10 gigE and the smaller image. Would take your times down from like 6-10 hours to 1.5 hours.
Also, I doubt a full 25k restart all at once you probably had underlying applications that expected rolling, blue/green or even % or nodes that can go offline at once.
Not sure the actual authors of the various overlapping Linux network subsystems even know the comprehensive picture "inside and out" for chronic lack of consistent documentation.
Last time I managed a small «supercomputer», 50x IBM blades running Suse, it wouldn't support PXE/NFS without kernel customization, but that would void support contracts and finicky third-party software. Made a switch to FreeBSD, where everything worked out of the box one hour later. That was over 15 years ago, I have no idea how much the situation changed.
Any of the three major branches are the first choice for lean, bespoke network appliances. For Mullvad in particular OpenBSD or FreeBSD would be the obvious choice.
I wonder about those VPNs that say "we don't log or store anything". That may be the case, but they probably just send a continuous stream of data to the law enforcement / intelligence services or whoever instead of storing it themselves. They can then correctly say "WE don't log".
Even with an honest company, the pressures on them are twofold – security and legal. Their systems can be compromised through security vulnerabilities and social engineering (including coercion – money, ideology, compromise, ego – classic psyops playbook). Or they can get legal government orders - which pretty much every government in the world have laws on books and operational practices to force any actor to hand-over data in the name of fighting money laundering and terrorism (AML/CFT). It is very expensive to put up strong defenses against these. I don't see a viable business model charging $5/month that covers regular operational expenses and covers these types of events.
Edit: Forgot to mention backdoors built into basic technologies they may already be using – like the Cavium HSM thing that came to light earlier this week.
You know at that point why not just hack Level3, Cogent, Telia, Zayo or some other T1 provider?
Frankly VPNs don't protect you from anything other than the most monitoring systems and the occasional public wifi connection. They're really just glorified Netflix region proxies and nothing more to most people
I don't think that the Houston Police Department (or whichever city you're in) is sophisticated enough to set up an NSA-fiber-optic-backbone-data-siphoning operation. Yes, the NYPD is known to buy illegal million dollar cell phone interceptor machines, but that's about it the maximum level that sort of thing ever goes, for the largest municipal law enforcement agency in the United States.
Like, how would that even work? Without a court gag order, gossip would make its way out of the building in weeks. The cell phone shit only was only quasi-secret because only police department employees were involved, something that's impossible for these VPN outfits. They don't get any of the (unjustified) privilege that the CIA or NSA (or even the FBI, sometimes) receive.
Anything I might do that could pique the curiosity of law enforcement is definitely below the level of federal intelligence agency interest. Maybe your life is more exciting though.
This is how lawful intercept systems have worked since the nineties. You’d have to go look at their jurisdiction to see whether there are laws mandating LI that impact vpn providers. Or Mulvad could clear it up, I guess.
Sweden absolutely has LI requirements for all telecom gear but vpns I have no idea.
Lying about it opens you up to potential litigation and being exposed through discovery. It makes less sense to outwardly lie to paying customers rather than simply lie by omission.
The EFF wants a world where this kind of BS, and consequent litigation, is a thing of the past. If there's water running down their face, it's tears, not drool. We deserve a better world.
You get there by either getting congress to pass laws or by setting legal precedent in the court. That’s the goal of the legal arm. Not to fight every bullshit civil liberties case, but to fight the last one that doesn’t get thrown out in court.
If you don’t think they look forward to those cases, I think you’re the one who has read them wrong, not me.
There should probably be case law before anyone actually believes in warrant canaries.
'If it's illegal to advertise that you've received a court order of some kind, it's illegal to intentionally and knowingly take any action that has the effect of advertising the receipt of that order. A judge can't force you to do anything, but every lawyer I've spoken to has indicated that having a "canary" you remove or choose not to update would likely have the same legal consequences as simply posting something that explicitly says you've received something. If any lawyers have a different legal interpretation, I'd love to hear it.' --Moxie Marlinspike
Why would someone have to lie? They can just say "We can't comment on that" without providing an answer. And then customers can go "sounds pretty suspicious, time to switch VPN services".
The point of the canary, is to turn any non-answer into a practical, or at least a tentative, "yes". If you no longer see an explicit "no", it means "yes".
The difference between a canary and a "no comment" is that "no comment" is an extremely common thing to say whether an allegation is true or not so it's not very suspicious, while stopping a canary is very suspicious.
So it's like the scenario you outlined earlier, but more effective.
No, it isn't. It's pretending that "no comment" means something it doesn't. Just because you want it to mean "yes" doesn't mean it means that. It means "we are not going to comment on this, because we have a sane legal department and we're not going to give you any information one way or another".
If you think "no comment" means either yes or no, you're pretending to know something you don't, and you should absolutely stop and go "wait, why am I lying to myself? And why am I believing my own lie?"
Yes, I was the one saying that _people_ consider it suspicious, but I'm also the one saying that as a company your only course of action is to not comment on legal matters unless legally compelled to. Those two things are not mutually exclusive. People (in aggregate) don't act rationally, so even if it's going to lose you some customers, no comment.
That doesn't explain your previous comment. Was the accusation about [[pretending that "no comment" means something it doesn't]] a misunderstanding of what I was saying? Or a confusion between me and powersnail? Or something else entirely?
If it's about what powersnail is saying, I think they're just wording things imprecisely. The canary doesn't actually affect the meaning of "no comment". The canary means that if it disappears, things are very suspicious, and if ask directly about the canary and get a "no comment" then you not only stay very suspicious, you also know they didn't forget. The no comment itself is not a "yes", but from a security point of view you should treat this active lack of canary as if it is a "yes".
Which they referred to as a "practical/tentative yes". Which I think is a reasonable way to describe the situation. It's not "flat out wrong" or "counter-productive".
> so even if it's going to lose you some customers, no comment
You're going too far here.
If you used to comment on something, and you could easily comment on it, and it loses you customers not to comment... you should comment. If you don't, it is suspicious.
I'd hire a lawyer that knows how to deal with them and knows how to say no comment.
I've always felt that the warrant canary is a nerd's gotcha designed to get out of a sketchy legal process (NSLs) and that judges would be very unsympathetic. But IANAL.
At least from my perspective, it is not just a "gotcha" problem or whatever feelings it conjures up in a judge's mind.
When a company, who already has a canary in place, receives this kind of warrants, what _can_ the company practically do to comply with non-disclosure? It seems that lying is now the only option left, if the company must explicitly post a "no, we didn't receive such a warrant".
> This is what "warrant canaries" are for. dont use anyone who doesnt have one
What if they have one like this quarterly canary at privacy-forward "write.as" last updated 9 months ago?
It should be noted with significance if this message
fails to be updated on a quarterly basis.
2023-01-05 21:06:06 UTC
No warrants have ever been served to Write.as, or
Write.as principals or employees.
I formerly worked for a somewhat-older mainstream consumer VPN provider for a few years, to the extent that you can take my word for it, this is not industry-standard practice at least as far as the provider is able to control it.
Commercial VPNs typically run on rental servers -- usually a mix of the major cloud providers and smaller hosting providers -- and in my former company's case, using dedicated hosting (bare metal where available). Steps were taken to restrict access for physical actors, but ultimately, the mantra's always that physical access basically guarantees data access on a long enough timeline if you assume there's a bad actor in the mix.
That said, to the best of my memory, there were no indications of this kind of data siphoning happening without our knowledge, and we absolutely didn't take part in it ourselves knowingly. Occasional requests would come in from various international law enforcement orgs, and every time they'd be replied to with a message about how we don't store user records (which was a truthful reply AFAIK).
The biggest challenge for us was competing with some of the newer actors in the space, taking advantage of deceptive marketing and engaging in (IMO) unethical business practices for the sector:
- Claims of "no logging," even backed up by audits, are only ever point-in-time measurements, and may not reflect reality if the VPN provider approaches the auditors in bad faith (say, with a sanitized code base); a good auditor in my experience will refuse to make this claim in the report
- Claims about having the corporate HQ in one country making it immune from the laws of countries they operate servers in (this is deceptive marketing; failure to comply with laws will get you shut down, and at my old employer we'd make calls about whether to just drop our server presence in a country entirely in response to local laws and political happenings)
- Commercial resale of user data is (allegedly) rampant among many of the newer providers you see constantly plugged on Youtube. This isn't helped by the massive consolidation of the VPN market under just 2 or 3 holding companies.
I won't name names for the companies I mentioned above, but my recommendation is to adjust your threat model from "nation-state level surveillance" to "commercial data resale just like every other web service."
As far as data collection went for my old company: we collected system metrics like resource usage over time, and kept minimal sanitized logs to help diagnose any production issues that'd come up -- basically the absolute minimum amount of data we needed to keep the service operating smoothly. I have every reason to believe this is an industry norm, since otherwise development and troubleshooting would be nearly impossible.
Anyway, there's also the looming "threat" (lol) of HTTPS and encrypted DNS proliferation and improvement making the core use case for commercial VPNs obsolete. I think anyone who's spent a bit of time in that industry realizes that the business model isn't long for this earth as a result, so I suspect many are trying to milk the industry for all it's worth. Personally, I'm all for HTTPS and encrypted DNS proliferation, and I'm also hoping more and more commercial public networks start using virtual private subnets and other device isolation features to make it even harder to abuse coffee shop Wi-Fi.
> Anyway, there's also the looming "threat" (lol) of HTTPS and encrypted DNS proliferation and improvement making the core use case for commercial VPNs obsolete
For a lot of people the core use case is accessing Netflix in a different country!
I'm constantly amazed VPNs get away with advertising that, more specifically the ones that advertise lower prices for subscriptions/products. I guess Netflix themselves won't really care if you switch regions for different shows and might write off discounted subscriptions as alternative to piracy, but the companies that license content by region don't care?
I'm not sure how this relates to the parent comment, but there are free encrypted DNS services out there, though the same can't be said for encrypted and anonymous ones (which is, frankly, a hard problem to solve, realistically speaking).
With encrypted DNS you're just shifting the burden of data privacy away from the local network to the DNS operator. How you determine which operators to trust will probably vary from person to person.
Anyway, the major difference here would be that a VPN will encrypt all traffic in a tunnel, from your DNS requests to your actual followup web requests. On the flipside, you may use encrypted DNS to look up records for a domain that serves content over an unencrypted connection.
You can use dns over https over tor(dohot)[1]. Safer than a vpn if you dont mind your isp knowing you go to tor.
1.https://github.com/alecmuffett/dohot
The other argument is to frustrate network correlation analysis. Many VPN providers have an internal high-bandwidth network (virtual or otherwise); you can send a packet to $VPN_SERVER_X, it sends it to $VPN_SERVER_Y possibly via other intermediate servers, and $VPN_SERVER_Y then forwards it on to your destination.
If you live in a country with detailed data retention laws, this massively changes the shape of the graph: rather than your computer connecting via HTTPS to lots of other IP addresses, it only connects to one, which a large number of other customers do too. The argument then goes that there's enough inherent jitter and generic "chaff" on the internal network to make it very hard to deterministically work out if one of your packets going in to a popular service is the same as that coming out at any moment in time; the greater the traffic of the network and the provider the better the statistical protection becomes as the packets become indistinguishable.
This, and the fact that it represents a giant "no thanks" to dragnet surveillance, is arguably a good reason to just put a VPN on your router (as many people do).
Audits are IMO worthwhile, but end users should be aware of the scope of an audit. In the context of commercial VPN providers, it's usually just a code security audit -- are there any memory leaks? Is sensitive data being passed around a little bit too loosely? Is there some way for unprivileged users to gain privilege escalation by crafting a malicious request against one of your services?
In this sense, they're valuable. As someone working in software, I can figure out if the bugs were subtle or blatant, which is often a good proxy metric for the competence of the team behind the product. Are the same bugs cropping up year after year, even if they've already been previously fixed in other parts of the code? Again, a good red flag to use there.
Audits do not and often cannot cover things like "is the company reselling connection/user metadata to other companies," though, and in most cases consumers will care that there is an audit rather than caring what's in the audit.
I tried to contact the auditor, Altius IT, in order to confirm whether exfiltrating connection data to a third party would result in the audit failure. They replied, but in a very vague way (refused to answer any questions regarding Altius IT's audit of PureVPN's environment). Well, at least they confirmed indirectly that the audit did exist.
I tried to contact KPMG to verify the authenticity of that report, and also asked the same question - "whether deliberate real-time exfiltration of origin IP addresses, assigned VPN IP addresses, connection timestamps, or connected user activities to a third party by PureVPN, without PureVPN (as opposed to that hypothetical third party) storing anything locally in any form of logs, would have constituted a failure of the privacy assessment." Result: no reply from KPMG at all, so I cannot be sure even that the report indeed comes from KPMG and is not a fake.
There are bad auditors, of course. Having had the displeasure of working with KPMG (not in a code-security-audit setting, mercifully), I genuinely don't understand how their staff can be allowed within a ten mile radius of source code.
The ideal way to authenticate audits IMO would be for the audited entity to link back to a PDF or other report hosted on the auditor's site.
Everyone said this about global surveillance before Snowden and yet. I don't see the value in questioning if this is true or not, rather I think we should assume it's true and act as such, isn't that what modern security is about? (see: HTTPS/DNSSEC/DoH etc)
I frankly wouldn't be surprised if it's actually happening.
> but they probably just send a continuous stream of data to the law enforcement / intelligence services or whoever instead of storing it themselves. They can then correctly say "WE don't log"
This has never made any sense to me. I'm surprised this isn't a massive red flag from anyone on HN.
Running a production-grade service with zero metrics and logs? If there's an outage, or even something as mundane as a VM failing to provision, you're telling me that Mullvad developers just shrug and say "well, we can't do anything, because there's no logs!"
I don't use a third party VPN, but if I wanted to, "we deliberately eschew all observability" is not a positive selling point.
Who said they don't have metrics? The VPN servers don't have any storage, but they can still send metrics to an off-machine API, or a different server that does have storage can send requests to the VPN servers and do metrics that way.
Ditto for logging. They claim to not log activity over the VPN itself, but I don't see any claims about not logging more mundane infra stuff like "a VM failed to provision". I think you're arguing here against claims they aren't making.
> but they can still send metrics to an off-machine API
And that would be the next most interesting post, imo. A post about how they metric and log in a RAM-only environment while obscuring or obfuscating the details that could lead to “compromise”. Even if the answer is something so simple like “we regex and discard this out”, I would feel more trusting of their services.
Yes, because it’s a simple service (VPN) that hasn’t added a billion nonesense features over time. You can log VM provisioning and health logs but as long as you don’t log any wireguard logs or user provisioning logs you’re good.
Sending them over the network to where? "We don't store logs" means they certainly aren't being ingested into any persistent storage. I'm highly interested in how one can run time-series queries over /dev/null.
Metrics are mostly by nature anonymous. Things measured are CPU/Mem usage, network rate. Metrics at IP/user level aren't of much value. Companies add country/device type attr. but they can be done without.
I am not a lawyer, but my understanding is that this generally falls under Section 230, as you can make the same argument about Comcast, AT&T, et.al. who lets the bytes go over their infrastructure.
All this would do would be to lead the investigation to get a warrant/subpoena to have the VPN service provide user details about the account and anything else relevant like logs. This is where the "we don't log shit" bullet points comes into play as well as running only from RAM. If the warrant allows for removal of hardware, all data is lost once power is removed. LEOs would have to bring lots of batteries.
They're going to freeze the whole data center? It's rack after rack of machines that the traffic could have passed through, right? And if they're not logging IPs to RAM then they only have a fraction of a second to get the right one before the register is overwritten with the next user's info.
You do need to know where to send the user's return traffic, so you'll need a table ultimately comprising mappings of network flows to end-user addresses. Of course, once the flows close you don't need to retain this information. In practice, you'll also need information about all currently-open VPN sessions.
If the feds have physical access and considering the high likelihood that these are VMs and not physical, it would be a whole lot easier to get the hypervisor to just snapshot the VM w/ its memory and perform forensics against that file(s).
and then suppose you login to that VPN and are looking up children's sweaters for your kids and keep the session on .. while law enforcement is looking up the ip address associated with the earlier activity which is now assigned to you . Good luck explaining to the the cops about VPNs and IP addresses.
You are not going to be the only person appearing to come from that IP address – many will likely be NATed through it.
The more significant concern is if you are the other side: if you deliberately run some sort of VPN or other proxy that others can use, or less deliberately do so. Many hacked or otherwise suspicious browser add-ons, and other malware, will make HTTP(S) requests & other connections on behalf of their C&C hosts and to your ISP or anyone else those requests will be largely indistinguishable from those that are the result of your activity.
You need a VPN that actually cares about your privacy and goea the extra mile to ensure it. On top of that if the VPN service does not know who you are how can they actually tell the cops. On top of that you don't need to explain it to the cops - if you are ever accused this should be done in a court of law where we understand what ips are (heck, even some cops understand it - it's not exactly rocket science nowadays)
You actually shouldn't even say anything to the cops. If they show up with a warrant for arrest as well as search, you're going to jail no matter what you say. If they show up with just a search warrant, they are going to take whatever they want to take whether its outside the purview of the warrant or not. It will be up to a lawyer to convince a judge it was out of scope at a later date after it has already been taken. You will never convince a team of cops that their warrant is wrong when they show up. The only chance you have is if you're uber criminal and have your attorney present when they arrive.
> You actually shouldn't even say anything to the cops.
Unless you're in the UK, in which case: "You do not have to say anything. But it may harm your defense if you do not mention when questioned something which you later rely on in court. Anything you do say may be given in evidence."
As a Yank, that line always felt odd when watching BritCop dramas. How is the alleged meant to know the specifics of a defence when the full charges haven't even been levied, or how is the alleged meant to read the mind of a lawyer? It just feels like something rigging the system
And the court of public opinion. By the time lawyers and judges are involved, unless you are very lucky, your name and photo is all over the tabloids. Any retractions published when you are later found completely innocent will be the equivalent of a column inch or two on page 17.
The point is that it could happen to anyone, if their choice of connectivity arrangements make it possible, so it is a relevant concern for everyone when planning such connectivity arrangements (whether or not they care about the implications of it potentially happening to you, me, or anyone else).
No, it could not happen to anyone, for a whole host of reasons…
A number that rounds to zero in the 21st century is the number of people who have been randomly cancelled. You can disagree that it is proportionate to their offense, but z e r o people were just sitting around and woke up a) in the public eye and b) were completely and entirely good people who were maligned needlessly in a permanent way.
It is in no way a real concern for 99.9999999% of people.
> but z e r o people were just sitting around and woke up a) in the public eye … who were maligned needlessly
Demonstrably false. There are a significant number of individual cases I could mention off the top of my head. If you want a whole bag of them to look at in one go, read up on the NOTW “name and shame” campaign which resulted in numerous entirely spurious accusations, and actual vigilante attacks as well as legal investigations. The court of public option can be a concern for all.
> b) were completely and entirely good people
On a slightly more facetious note: if anyone claims to be absolutely 100% totally good, someone somewhere will disagree :)
There is no objective definition of good that all will agree on, and even if there are none of us are perfect our entire lives.
Same thing different words. Your point (it doesn't happen to many so no one needs to worry) is an appeal to infinity, much like how Douglas Adam's used an appeal to infinity for humour purposes to “prove” that the population of the universe is zero (and any people you may meet are just figments of a deranged imagination).
You: “It happens to a small number of people, there are many many more people than that, so it averages out to effectively zero, so no one needs to worry”
Me: “Toe stubbing only happened to a small number of people, there are many many more people than that, so it averages out to effectively zero, so no one needs to be careful about their feet”
HHGTTG: “the volume of the universe is infinite there must be an infinite number of worlds. But not all of them are populated; therefore only a finite number are. Any finite number divided by infinity is zero, therefore the average population of the Universe is zero, so the total population must be zero and any people you may meet are just figments of a deranged imagination”
What if they are on cellular and that hasn't been upgraded to IPv6?
Years ago I handled fraud cases for an e-commerce site with local police, at some point they started asking for IP and port numbers for the offenders, rather than just the IP. Turns out that one of the cellular phone providers had basically run out of IPv4 addresses for their 4G network and did some NAT solution. If you didn't have the port number the client had connected from then they could only tell you which cell tower had been used, not who the customer was.
Definitely not. I still am logged into my ex girlfriend wifi so if I wanted to harm her I could easily go stand outside her home at night and download malicious files. That would not make her guilty. They may investigate but that is not proof she did something unlawful.
"They" will just spray the machines with liquid nitrogen, pull them out of the rack, put the DRAM in a thermos w/ LN2 and read the data at their leisure.
With modern encryption protocols, this yields you nothing.
The feature is called Perfect Forward Secrecy, and protects past flows from later key compromise.
Wireguard supports this, which is what Mullvad uses. (For some reason, speculation about which is an exercise left to the reader, WPA in Wi-Fi still does not.)
Not exactly nothing, just not ongoing compromise. TLS session keys can be pretty long-lived; I don’t know how long-lived Wireguard’s equivalent keys are, but even a relatively conservative few minutes can yield valuable traffic and metadata.
(That being said, I think having your RAM frozen to extract ephemeral secrets is firmly in the “fully hosed” threat model, and is not a realistic model for 99.9% of users to plan for.)
AMD processors support encrypted RAM, called SME[1]. The key is internal to the CPU and randomized at boot. Sniffing a live RAM chip or reading a perfectly preserved frozen RAM will give you nothing. It's a big part of why the xbox one was never hacked.
You can enable SME in the BIOS on all AMD-based business laptops and AMD EPYC servers.
The word "just" is doing some heavy lifting here... To "just" do this, the agent would need to more or less completely take over the building infrastructure before Mullvad could react which is a lot easier said than done. Even if it were trivial it's still quite a few cuts above any competing VPN service.
Still doesn’t protect you against hardware based backdoors, or other types of backdoors like memory injection or supply chain, to get data on the fly
> When servers are rebooted or provisioned for the first time, we can be safe in the knowledge that we get a freshly built kernel
Any info what’s the period of time doing so? Do you provision them every day, week? An hour maybe? The more the period the less chance of some attack vectors.
Disks don’t always have a readonly switch these days, though I do still miss the physical notch on floppy disks, and no third-party auditing could exist for proving that switch to be set correctly.
No third-party auditing could exist that proves you only have RAM in the system and don't have a secret disk in there with a magnetic reed switch in-line with the SATA power cable such that without sticking a magnet on the case the disk doesn't show up. Or that you aren't booting off a USB drive that you plug in only after the auditors leave.
Third-party audits are a scam to begin with and don't prove anything.
A third-party audit can prove that the system functions as shown without a hard drive, and a third-party auditor can, using contractually-authorized random unscheduled spot checks, physically inspect the live deployed servers to confirm the absence of any disk media.
Third-party audits prove something. They don't prove everything.
If they claim that they possess no data, and after some years and some nontrivial attempts no one has succeeded in extracting data from them, that is not nothing, even if it is not proof.
> All of our VPN servers continue to use our custom and extensively slimmed down Linux kernel, where we follow the mainline branch of kernel development.
The custom server is a niche security point. While every server is continously researched and patched, we cannot expect the same from a a server like this. If someone were to find a security hole, an attacker would purchase it and no one else would ever know the system was compromised.
But, if anything should be a decentralized anonymous crypto-paid service, it should be a VPN network.
Centralized VPNs are still a single point of failure privacy risk. We have to trust they don't share our identity/account info and activity.
I am surprised dVPNs are not THE first rationale given for crypto. I.e. since separately and together they (ideally) have a clear comparative advantage over other alternatives for strong privacy.
A performant global open-standard dVPN could become an indispensable layer of web access.
I wasn't sure what a decentralized VPN would look like, so I searched and found https://surfshark.com/blog/decentralized-vpn . Obvious bias coming from a VPN provider, but if they are stating the technology correctly, then I think it's important to determine if this is correct:
> A decentralized VPN is a distributed VPN service where volunteers supply your VPN servers instead of a single company – but paid by crypto. Like with regular VPNs, you have to trust that the VPN server isn’t monitoring your data. But instead of there being a single VPN provider company behind it all, you have to trust that none of the thousands of server volunteers are spying on you.
Is this a correct understanding of dVPNs? Is there a rebuttal, especially to that last sentence?
You have a network of VPN point providers. As you communicate, data can be sent through any series of points.
Data is encrypted end-to-end, and the addresses for the point providers are also encrypted so that each point can only decrypt and see the next point to forward data to.
So each point knows where data last came from, and where they are sending it. But they don't know:
1. Which step of a chain of points the data is at.
2. If they are the first in the chain (i.e. the "from" is the source)
3. If they are the last in the chain (i.e. the "to" is the destination)
And (as long as two or more points are traversed, which would be always), no point ever has access to:
4. Both source and destination info.
Finally, since payments to each point are handled through a combination of peer-to-peer point bookkeeping, and a crypto block chain account, no point ever knows:
5. Any identity information about who uses the VPN.
6. Any way to identify activity over time that is related.
Acting as a point, as well as using the network, serves to further cloak activity, as being from you vs. passed through you.
And an alternative to crypto payments, would be earning usage by providing point service.
Any VPN provider that is claiming decentralized VPNs are a greater risk is either misinformed, or willing to misinform users.
I wouldn't trust a VPN provider from either category.
Actual reasons to not use a dVPN might be that it is a work in progress, not supported well, its source code is not open, or not yet vetted by experts, too slow, not many points yet, etc.
Some kind of economics are needed to over come the fact that there are only a few thousand Tor nodes [0], making it relatively easy to compromise the network by any entity willing to pay for a couple of thousand nodes [1], which is a bargain for any intelligence service.
I.e. Tor is pretty safe, but because it’s volunteer, it is also a bit of a honeypot.
Now take all the money people spend on commercial VPN’s, and anonymize accounts while making some privacy first crypto actually useful to the general public.
Millions of nodes, or tens of millions.
The benefits come not just from linear node path anonymity.
By spreading traffic packets in parallel across different paths, and geographically, so it’s near impossible to track anything useful even with a lot of compromised nodes.
Assuming you have a LOT of nodes.
(Geography here meaning Internet topology, verified by minimal latency.
Topological information for millions of nodes will help keep latency low, while increasing the number of nodes in each path, for a better security vs. latency trade off.
So nodes could be incentivized to locate and scale based on topology & usage.)
If there is a way to make Tor anywhere near that secure a lot of people would like to know how.
Economics matter, and this money is being spent already.
I have servers which have literally been running non-stop for multiple years. Unless / until the kernel itself needs to be upgraded for security issues, and so long as backup power is good, they can run indefinitely.
Technically, researchers have proven that you can shutdown a machine, hit the RAM with a cold spray (like liquid nitrogen) and keep the bits "alive" long enough to dump them for analysis.
But, obviously, that's pretty insane. Agree with everything that this is a big leap in the step of better protection for users.
Even if that attacks has close to 100% success rate, I'd imagine it being nigh physically impossible to execute a targeted attack, as you don't know which machine to hit for a specific user. And that seems to be the main threat model we would be concerned about for this.
Mullvad gives you the option to connect to multiple servers. They offer wireguard configs for every endpoint. How does law enforcement know which server the client plans to connect to? There is no metering either, just a flat monthly rate so nothing to track there either.
I find these discussions so tiring. Let me turn it around. In their position, how would you manage this? Might you hook authentication events? Why are you pretending this is hard?
You can connect to mullvad via tor though. If I only ever went to the mullvad site via tor to make an account, paid in monero and only ever accessed the VPN via tor, what is there to hook into?
The likelihood of them showing and doing that is low. However, the likelihood of them showing up with a set of USB drives and just running rsync/cp/dd is higher.
If only the system were open source so you wouldn't have to wonder about that...
But we do still have to trust that they are actually running the code they posted.
Unless that code somehow contains some way to verify itself?
I wonder if there is some way to do that? Have the code include a hash of itself and some way to query the running service that guarantees that the running service must be running the code you are looking at?
At first glance it seems any response could always be faked, but maybe there is some cryptography trick where you submit something, like an encrypted copy of the public code maybe, and it crunches and returns something, and that somehow proves that the running code you can't see must be the same as the code you can see.
Depending on how the protocol for the challenge works, that could still be faked. The challenge has to somehow not be seperable from ordinary traffic so that you can't have one piece of code handle the challenge and another piece of code handle other traffic.
- Multi-party computation. Too much overhead for something like this.
- Remote attestation, as seen in e.g. Intel SGX. Usually provided by the CPU vendor. Not a cryptographic guarantee, more of a "it'd be very hard to defeat this if you're not Intel". Probably not that warrant-resistant.
I think the normal solution to this is all the prove you are the software you say you are calls is proxied to that software and all the normal services calls you want to log and duplicate or otherwise violate the contract are sent to the modified code.
You can talk about hypothetically doing anything. You're not really making a point.
Hypothetically, you can rewrite the firmware for the IPMI with a backdoor and extract data.
Hypothetically, you could kidnap the family of a developer and force them to add a surreptitious side channel for exfiltrating data.
Hypothetically, you could run the universe in a simulator and use the simulator's controls to read data from RAM in the simulated Mullvad servers on the simulated Earth.
And it's all open source btw. https://github.com/system-transparency/stboot