To clarify, outside of the CDN providers or AWS calls and the big 3 (Facebook, Google, Apple), the vast majority of the calls seem to be to marketing providers or developer tools
Branch - these guys provide deep links into phones and tools to analyze who clicked on the links and if they worked.
mParticle, Appsflyer, Braze formerly Appboy, Appboy all provide internal app marketing teams tools like mobile push or analytics from the app on the phone.
While NewRelic, letsencrypt (free SSL certificates), crashalytics etc are all developer tools to monitor usage and issues with your app.
In summary majority are 3 classes of traffic: CDNs which cache data, Marketing tools such as deep linking analytics etc, and finally developer tools.
Seems like a missed opportunity for Apple and Google to allow users to opt in or out but send data back to one place and then push that out to all these guys so the phone isn’t sending the same data over and over to so many partners and wasting battery.
With regard to your last paragraph: that would probably be an excellent application of Ben Thompson's aggregation theory. It would increase Apple and Google's moat by making them the hardware gatekeeper for all mobile app analytics. And battery life is also a strong cover for the business reasons for doing it.
But the public claim, "it saves battery life!" would not make it defensible for most analytics companies, in my opinion. That would mean Google and Apple get duplicated access to just about all mobile analytics data in the world overnight. They already get vast data from the mobile phones through telemetry and their own apps; I think the largest third party analytics providers would revolt. They would all be at the mercy of Apple and Google's benevolence, which is basically backing their business into a corner. You don't want to be reliant on the whims of a giant tech company.
There are probably also some (maybe weak) anti-trust arguments against it, because all analytics other than e.g. Google Analytics become literal second class citizens on the phone. That would basically be telling app developers they're not allowed to send requests to specific hosts within their apps, only Apple and Google can do that (on their respective phones).
So I don't know if this is a missed opportunity, so much as Apple and Google realizing it would burn their walled gardens to the ground.
>think the largest third party analytics providers would revolt
Would anyone care? I don't think a game company is going to refuse to publish on iTunes or Google Play because some tool they use for analytics stops working.
Nothing against analytics companies, but they just aren't a relevant party in Apple's (or Google's) ecosystem.
> Would anyone care? I don't think a game company is going to refuse to publish on iTunes or Google Play because some tool they use for analytics stops working.
I don't see why these companies can't simply push all analytics to their own servers then out to the analytics company, bypassing apple/google.
Most of the biggest mobile games companies have custom analytics engines and likely do this anyway.
That doesn't really solve the battery life / analytics duplication problem.
Furthermore, from experience, duplication within a single app often happens all on its own because, say, different departments use different toolchains with different integrations, thus want different analytics providers and it's easier to just have the app send to both. It's inefficiencies all the way down because the only one to really pay for this is the user, and the user doesn't know they're paying for this (be it in battery life, PII leakage, etc).
I think it would at least create a lot of hostility between Google/Apple, the developer ecosystem and the analytics industry. Third party analytics companies could have their lunch eaten entirely because Google/Apple have greater resources than them and would know how to obviate them using their own infrastructure. I could see very expensive lawsuits being brought against Google/Apple for doing this, or increased pressure for third party app stores or nontraditional app distribution channels.
In the short term you're probably right, nothing would "burn" except a lot of developer good will. But in the long term it'd be a great way to get many different parties thinking hard about how to get off your platform or replace you.
I was not thinking about the anti-trust implications by not allowing folks to send data back but perhaps there is a middle ground.
It could also increase transparency for consumers by allowing the opt in opt out on the device for each app and letting customers know “this app is tracking your clicks on it”.
The one point about them essentially getting all data, don’t you think they are already doing this? Look at the amount of calls the iPhone is doing back to Apple or the amount of data Android is shipping to Google. I believe they literally are already doing all of it, this would just be a way to give developers access to what they want transparently and reduce the number of unnecessary calls, all the calls would still happen but server side.
As a developer knowing the ability of some of these developer tools they have the same ability to log sensitive privacy data as marketing tools. All tools that log to a central server have a high potential of abuse and should have similar oversite to prevent abuse.
Regarding iOS: I stopped using iPhones and (edit typo) quot the ecosystem altogether (apart from an app I still sell in apple app store) because with the lack of an untethered Jailbreak I could no longer install "Firewall IP" and I could not edit the hosts file.
Regarding Android: I switched to Android for the "NoRoot Firewall" and since most Android phones are Root-able I can also edit my hosts file.
The article gives a very good analysis of what I have been telling friends, and my constant complain towards that Cancer called Facebook: why does my e-banking app or Booking.com or practically every air carrier's app, need to alert FB that I am using this or that app?
Anyone with an Android can install that NoRoot Firewall and see in 60seconds what their phones are doing when you are not looking. This in combination with the applications running in the background 24/7 makes privacy a thing of the past.
That's exactly why I uninstalled Robinhood, Spotify, Venmo and half the other apps on my phone. Netguard showed me those apps were notifying Facebook every time I opened them.
Facebook has no business knowing when I'm making trades, listening to music, sending money or anything else.
This is what I did as well, and for the services I still needed, I found the mobile sites nearly equivalent in looks and features. Especially for banking, shopping, and reading sites. I added links to my homescreen in lieu of apps, and that way I know they're not using my phone in the background. The downside is getting hounded to install the app when I visit certain mobile sites.
I tried no root firewall and found that Google groups literally everything under a kitchen sink service that talks to blind IPs with SSL. There is no way to allow Maps without inviting Google to share your bed with you.
What I mean is you wipe the phone, reinstall either AOSP or LineageOS (or other custom ROM), do NOT install any Google Apps (to include Google Play Services, Google Store, Google Maps, etc.). Use F-Droid (https://f-droid.org/) as your App Store.
It's a bit annoying though how much a first-world-only solution this is. OpenStreetMaps was completely useless for my city/country the last five times I tried to use it.
I think you may be in the first world on this one.
I saw a very interesting talk last year about how Google Maps is often useless in third world areas, and OpenStreetMaps provides the only useful coverage.
It's due to armies of volunteers mapping an area during disasters, while Google has no economic interest in mapping the area.
"Google has no economic interest in mapping the area"
Google has an economic interest in mapping anything as long as it's useful to some people. Google is playing a very long term game: provide free services online (Email, Maps, Search...) to entice more and more people to go online. The more Internet users there are in the world, the higher their ad revenues are. It's that simple.
I live in West Africa, and yeah I think I'll take the difference between the two services that I've experienced for myself for what it is. Those "armies of volunteers" aren't exactly interested in the region either.
> Those "armies of volunteers" aren't exactly interested in the region either.
I don't doubt your experience.
The talk is about disaster response, so it only describes maps in regions where a response is taking place.
Elsewhere, the rest of the time, I'm under the impression there is no serious mapping taking place, neither by Google or OSM.
What I found surprising from the talk I linked was instances where there was a hardly usable Google Map to use for the area (just big expanses on the map and the occasional large feature), and the fact that at times up to 1000 volunteers would work together mapping a region at short notice, down to street and building level, by analysing satellite images, and coordinating with responders on the ground.
In the opinion of those disaster responders, those volunteer created maps were much more useful than the Google ones.
We haven't exactly had one of those in decades, so yeah. I'd say it's a bit telling that there's only interest/means for volunteers to map the developing world when there has been a disaster, but I digress.
Google Maps is perfectly usable where I live (apart from its continued if understandable confusion about how...irregular roads and road access can be here), and in my experience has been at least useful in all the cities in my home country I've visited. OpenStreetMaps is...not.
also using OsmAnd but just on android with Tor & NoRoot Firewall. takes a second to load the route. but allows you to download the maps and use offline. always have Play Store disabled unless there's an app i have to download briefly. once done, i uninstall the app and disable the Play Store.
How cheap are you looking for? When my Nexus 5x broke, I simply looked through the lineageOS site to look for phones. I ended up with a Sony Xperia XA2. I found it new for $220, but I found an open box for $150.
Disconnect Pro for iOS is also a great tool. Hasn't failed me yet and makes adding custom trackers super easy via the list of recent connections. I blocked 30.8k trackers and saved 3 GB of data last month. :)
Yes. I use a DNSBL in my router (pfBlockerNG), and I've been considering writing a small script to resolve every IP in the blocklist and then block outgoing connections to those IPs.
Of course this is only useful at home. As soon as my phone leaves the house, everything is open again.
Ooh, good idea. Would probably only have to write ~1 time per day to a separate, parallel list. Or have a small VPS do the work continuously and push changes to a Git repo so others can pull it however often they want.
Serious question--how does one verify the security and privacy of something like NoRoot Firewall?
It wants:
- have full network access
- view network connections
- run at startup
That's a pretty serious list of permissions to consider giving something that while it would seemingly be beneficial, is also a great vector into my phone activity for a malicious actor (not accusing them of being such btw).
Netguard is an equivalent Android app, it is open source and available on fdroid / play store. The fdroid version has to match the source. I'm not sure how play store builds work.
I'm not sure how the situation is with Apple, but it always bothered me that on Android, apps can implement their own logic for TLS certificate validation. Apps can use this to hardcode key-pinning and make it effectively impossible (short of patching the app) to inspect an encrypted connection, even if you're the owner of the device.
I feel the push for DoH will make this even worse - because then you won't even know which servers your apps are connecting to.
Once you install closed source software, you're no longer the sole owner of your computer. If an app wants to hide data, and they can't rely on tls to do it, they'll just add another layer of encryption.
Unfortunately Google both supports and recommends this. Recently they've even made it easy for apps to automatically ignore any custom certificates added to the trust store, so they don't even have to bother to implement pinning.
Yeah, I'm honestly not surprised. Apologies for the cynicism, but sometimes I wonder if the pushes for HTTPS-everywhere, certificate transparency and DoH are really more for the privacy of app developers instead of the privacy of users...
Understandable position for them to take when you have the likes of Facebook / Onavo etc pushing VPNs and root certificates on uninformed users for "research".
Same for Apple, I've had to bypass this multiple times for pen testing engagements. There are ready to use modules to patch it out, no root / jailbreaking required.
There is lots of good security reasons to do this. Further, if they didn't people would just roll a Swift / OBj-C SSL library and do it all themselves, which would be worse I think.
It stops people who have root certificates installed on their phone (e.g. the Facebook research app from a couple weeks ago) from being able to monitor traffic.
But it also allows that very same app to smuggle all kinds of tracking data to facebook without the developers having to worry that anyone would catch it doing so.
Also, we already have several systems to manage app access to things that could potentially be misused. Why not manage user certificates the same way?
E.g., pop up a consent prompt before letting an app install anything - or, if that is too annoying, don't give apps access to the functionality at all and exclusively manage certificates via the system UI.
> It stops people who have root certificates installed on their phone... [emphasis mine]
Indeed. That's my point. I'd consider this a bug, not a feature.
Yes, A tool called Objection can do it. You have to modify and re-sign the app you are interested in bypassing. Requires a developer account and the .ipa of the app you want to work with.
Its a feature not a bug, and also thats the same with Apple. On top of that, with android you at least can easily become root and hook into the TLS library yourself and read the network traffic this way. Which is why this article is pretty underwhelming if you ask me, whats interesting is looking at the content that the apps are sending about you.
Well, yeah, you need to go beyond a traffic sniffer because if a traffic sniffer was enough, where is the security gain?
Apps using custom certificates is a best practice and absolutely essential for communicating securely with devices that can not participate in the web CA (because, duh, they are not websites). Think your local network WiFi camera.
2. Do not install the app if there's a website equivalent you could use (Facebook, Banking Apps).
3a. Force Stop or Disable apps you use frequently despite web equivalents (Google Maps).
3b. Enable permissions required by apps used occasionally only when in use. Disable them again, once usage is complete (Banking Apps).
4. Use websites on mobile on Firefox with uBlockOrigin/uMatrix, PrivacyBadger, CanvasBlocker, WebRTC blocker.
5. Prefer using 'lite' versions of apps, if you must use an app (Uber Lite).
6. Try to use apps that do not require GooglePlayService or slowly force yourself to (OpenStreetMaps).
7. Use privacy-oriented apps as a replacement to apps that you you use very frequently (Signal, ProtonMail, DuckDuckGo) or use a separate user-profile for those apps (WhatsApp) altogether. https://news.ycombinator.com/item?id=18873433
8. Use LawnChair as your default launcher (or some such privacy oriented launcher).
----
Of course there's a big matter of Google services running the show underneath, and you couldn't get rid of that unless you went the microG+LineageOS route. https://news.ycombinator.com/item?id=15617615
> 3b. Enable permissions required by apps used occasionally only when in use. Disable them again, once usage is complete (Banking Apps).
Bouncer - Temporary app permissions seems to be a brilliant tool for this. I installed it the other day together with Glasswire. Both are paid, and I happily pay (reasonable amounts) for good tools.
Together they should hopefully mitigate the risk connected to useful apps with broad permissions.
Haven't tested them too much yet, so if anyone knows problems with those apps, feel free to let me know.
Of course, depending on your threat model some of you might never be safe with a smartphone or any portable phone at all. Personally however I feel this might solve it for me for now.
I will strongly second Bouncer. It's an amazing app for keeping permissions in line on my phone. It's also a shame I rarely have to open the app itself because Sam Ruston is a master of clean UIs.
Apple promotes privacy (which is great), but at the same time they behave like a dictatorship by not providing an opt-out of the iOS walled garden, which they do provide with System Integrity Protection on macOS.
What he is doing will not prevent apps from extracting information and uploading it to their servers. For example, by using an ip address instead of a hostname/domain, an app/service can exclude themselves from the "domain graph" he created with this vpn. Sure you could eventually track down the public ips an app communicates with and block those, but the app will always keep collecting and storing your data, and at some point in time they update their app and change the ip - by the time you notice this your data is already uploaded to the new ip.
The only proper solution is an app firewall for iOS, which is not allowed by Apple.
Apple is crippling our freedom with their walled garden/dictatorship, which makes me sad.
Actually would be interesting to see the content of the HTTP packets that are not encrypted!
I wonder what kind of information is shared by our smartphones without it being properly secured...
A lot. Running my phone traffic through mitmproxy was a rather sobering experience, especially what leaks on boot before firewall and ad blocker are ready.
On Android you can even inspect a lot of encrypted traffic using mitmproxy and the cert in generates, although some apps (like signal) use cert pinning in a way that i haven't managed to get around yet.
In my home network I use a hosts file to block unwanted tracking.
I use a file from this [1] project, which makes it easy to filter out the type of content you don't want.
The nice thing about this is that it blocks requests from any device in my network, especially from those which cannot be configured with a firewall or adblocker.
Under ideal conditions my “smart” tv, internet connected game console, internet connected air filter, etc will never leave my home. At least not u til I get around to documenting them for renters insurance.
Android is designed to leak like a sieve but using an OS by the kingpin of the surveillance industry and expecting privacy and things to be above board is dissonance.
We live in a 'fantastic' world where the same people who have made a billion dollar business model of behavioral targeting and creepily stalking people 24/7 aggressively push things like https claiming to care about user privacy and security.
Where Android can be promoted as 'open' in-spite of abusing all the driving principles of open source. Tech folks cannot be unaware of the massive and ever growing surveillance ecosystem in operation and many are infact actively building it, and pretense of surprise by such articles only serves to affect some kind of fabricated normalcy.
I've enjoyed using an app for iOS that installs a "VPN" which is configured to run a local DNS proxy. It gives you a log of every request on the phone and allows you to block domain wildcards.
It's fascinating to peer into the dark alleys of your iPhone.
I've got it on my list to play with Charles proxy. I'm curious to peer into a few of the requests if possible. But I've read that, especially with mobile apps, they may use cert pinning which defeats something like Charles.
What substantial aggregate profit do you think they can make by analyzing detailed location data?
I'm asking because I'd assume for most of their customers it just doesn't reveal much. Everyone shops at supermarkets, Target vs Walmart isn't going to reveal a whole lot more than a residential address.
If location is correlated with time, they could know the speed at which you travel.
The car insurance could know if you visit dragstrips, which might imply specific driving habits. Or how often you visit the gas station, from which could be estimated your mileage. Or if you already report your mileage, might be used to estimate you fuel consumption, which could imply specific driving habits.
The health insurance might know if you visit the same bad-neighbourhood address as some known-heroin-users do. Or if you just visit the tobacco shop.
Car insurance gets a lot of information from past claims, age and sex/gender, they aren't going to make a bunch of money turning away a few people that go to dragstrips.
In the US, health insurers are specifically prevented from considering such things in setting their premiums, they get to consider age and smoking. Carriers that offer plans to the general public are also subject to a "guaranteed issue" provision, they are not able to refuse coverage to anyone that can pay.
Whenever I read stuff like this, I'm reminded how user-hostile Android and iOS are. Even compared to Windows. Or at least, to Windows XP and 7.
Not that many years ago, I had imagined that microcomputers and cellphones would merge. But I was expecting something like Linux. Or at worst, like Windows.
And it clearly didn't work out that way. We have smartphones that are never really owned by users. They run apps that have more rights than they do. With no practical way to change that.
It's sad, because I can't have a smartphone that I can trust. But so it goes.
Anyone know if a similar process is possible to see what the baseband controller handling data transfer to the tower is doing? The method in this article works as well as the host os can redirect traffic. The baseband chip is often completely separate as I understand it. Likely would require a fake tower implementation, but maybe there are more creative solutions.
That is a great idea. I'm going to try this myself on my Android phone (though ads etc. are already blocked on it). Might be a good way to put a tripwire up to catch if anything suspicious happens.
EDIT:
OpenWRT's adblock package (which I already used) can create a DNS report and each list has a Blacklist/Whitelist button. Superb!
nflxso.net is not a tracker. It is a domain used by Netflix to refer to "small" non-video downloadables (things like the images you see when browsing titles on the web or in the app).
Use Pi-Hole, everything in article I got to know from looking at statistics from it. It also blocks those bad ones but unfortunately I use phone outside my home wi-fi as well.
Branch - these guys provide deep links into phones and tools to analyze who clicked on the links and if they worked.
mParticle, Appsflyer, Braze formerly Appboy, Appboy all provide internal app marketing teams tools like mobile push or analytics from the app on the phone.
While NewRelic, letsencrypt (free SSL certificates), crashalytics etc are all developer tools to monitor usage and issues with your app.
In summary majority are 3 classes of traffic: CDNs which cache data, Marketing tools such as deep linking analytics etc, and finally developer tools.
Seems like a missed opportunity for Apple and Google to allow users to opt in or out but send data back to one place and then push that out to all these guys so the phone isn’t sending the same data over and over to so many partners and wasting battery.