Obviously, this is stupidly oversimplified, I have no idea how Apple has structured their code. But the fact of the matter is, if the scanning routine is already on the phone, and the photos are on the phone, all anyone has to do is change which photos get scanned by the routine...
By the way they already scan photos that aren’t uploaded to iCloud. I’ve never used iCloud and I can go on the photos app and search for food for example
Right, the difference is that now, you can be reported to the authorities for a photo that the CSAM algorithm mistook for child pornography, whereas before the image classifying was purely for your own use.
Right, but neither is the CSAM code being used to detect anything but CSAM.
If a bit can be flipped to make the CSAM detector go evil, surely it can be flipped to make photo search go evil, or spotlight start reporting files that match keywords for that matter.
There is nothing special about this CSAM detector except that it’s narrower and harder to repurpose than most of the rest of the system.
It has already been added. That's the point here. The phone already has the code to report its own owner. People are rightfully pissed about that.
Also, adding a one-liner of code is a lot more than a bit-flip. That's the addition of a feature, which is a major business decision. Code that already exists is a lot closer to "misfiring" than code that simply does not. Flipping a policy config, OTOH, could be "explained" away a lot easier. In fact, Big Tech does it all the time! (Remember Facebook's "Oh. Sorry. It was a bug."?)
----
Please stop this poor attempt at moving the goalposts by saying, "But, similar code could be added at any time in the future! Why're you guys complaining now?!"
It's also not about where the scanning is happening, or whether that's new or old. The old scanning is not a problem, precisely because it does no reporting. The new scanning IS a problem, precisely because it does the reporting.
The former is not a problem on its own. The latter is. That's the point you keep missing.
Right, but the CSAM scanning routine is absurdly narrow and difficult to use for detecting anything but CSAM, whereas a general purpose hash matching algorithm really is just a few lines of code.
This whole ‘now that they have a scanner it’s easier’ reasoning doesn’t make sense. iOS already had numerous better matching algorithms if your goal is to do general spying.
It could be 1 or 2, but the claim is that it is set to only detect people who are actually building a collection of CSAM images because Apple doesn’t want this system to flag people who are unlikely to be actual criminals. I.e. they don’t just want no false positives, they don’t want positives that aren’t highly likely to be criminal. I’m guessing it’s a lot more than 2.
> but the claim is that it is set to only detect people who are actually building a collection of CSAM
From what I read the threshold is for a different purpose, ,the false positives are too many so to reduce the number of reports the threshold workaround is introduced, so is a hack to workaround the false positives and not a threshold to catch people with big collections.
I have seen both in comments from Apple. Certainly spokespeople have talked about wanting any reports to NCMEC to be actionable.
I think a lot hinges on what you call a ‘false positive’. It could mean ‘an image that is not CSAM’ or it could mean ‘an account that has some apparent CSAM, but not enough to be evidence of anything’.
I think it is clear that Apple will have no choice, if they for sure find 1 CSAM image they can't say "it is just 1 img so it is not a big collection..." they have to send it to whoever to handle it.
What would help me with this kind of things would be more transparency, like to know the algorithm, the threshold, the real numbers of false positives, can the hashes be set per country or individual, can independent people look inside iOS and confirm that the code works as described...
What would also help is to know for sure hoe many children were saved by this kind of system,
> I think it is clear that Apple will have no choice, if they for sure find 1 CSAM image they can't say "it is just 1 img so it is not a big collection..." they have to send it to whoever to handle it.
This is a misunderstanding of how it works. The threshold has to be met before a detection occurs. Unless the threshold is met Apple doesn’t get to know about any images.
As for independent auditing and transparent statistics, I fully agree.
Let's say the threshold is 10 , you trigger the threshold so Apple checks all the 10 images , but 9 are false possitive and one appears to be a correct match, Apple will report you for 1 image ... so my point is that is not designed to catch people with big collections of CP, is designed to not trigger to often.
Isn't the scanning routine searching for fingerprints? What happens if someone adds a fingerprint to the database, which matches something other than CSAM?
It’s way more complex than that, and has been engineered to prevent exactly that scenario.
I recommend you actually check out some of Apple’s material on this.
The idea that it’s just a list or fingerprints or hashes that can easily be repurposed is simply wrong, but is the root of almost all of the complaints.
Yes, I did. You do realise that CSAM is just a policy control, and there is literally nothing technical from Apple adding non-CSAM content to the same policy and systems?
I feel like you may not have actually read the paper in depth. The paper clearly shows this is a generalised solution for detecting content that perceptually matches a database. The "what" is CSAM today, but nothing about the technicals require it to be CSAM.
To answer your specific response of safety voucher and visual derivative:
- The 'safety voucher' is an encrypted packet of information containing a part of the user's decryption keys, as well as a low resolution, grayscale version of the image (visual derivative). It is better described as "backdoor voucher".
- The 'visual derivative' is just the image but smaller and grayscale.
None of those two technologies have anything to do with CSAM. Apple's statement that they will only add CSAM to the database is a policy statement. Apple can literally change it overnight to detect copyrighted content, or Snowden documents, or whatever.
It doesn’t matter where the fingerprints are coming from, they’re all just fingerprints. Today they come from NCMEC. Tomorrow they could be from the Chinese Communist Party.
