This is used to show where a program came from the first time you run it. For example, if I download iTerm 2 and then run it, I get the following warning:
The extended attribute records that the file is quarantined, but it doesn't contain the detailed information about where the file came from. It just has an identifier that's used to look up in the database. For example, here's the quarantine info for a file in my Downloads folder: 0002;50e77ce1;Safari.app;BF047C49-2539-4536-A584-E69FDE328A2C. I don't know what the first two fields are, but the third is the app that downloaded it, and the forth is a UDID, which is presumably the identifier for the quarantine event.
Furthermore, even once a file is out of quarantine, the Get Info dialog can still tell you where it was downloaded from, precisely because of this database.
There is a reason why the marking of the file and the database containing the download information are separate. It is for personal privacy! If you copy or move the file or someone else runs whatever it is, the "quarantineness" is still there, but the personal information about where and when you downloaded it will not be accessible. The "taint" associated with the downloaded object will remain with the file, but the personal and potentially private information does not.
Actually I think that warning comes from data stored in the xattr com.apple.metadata:kMDItemWhereFroms on the file itself. So this sqlite database must be for something else.
I'm not a Mac user but wouldn't the test therefore be to clear the DB entries and re-open the file. No prompt implies the link to the sqlite db and not the file attr.
The dialog only shows up when that xattr is set, the sqlite db has nothing to do with this security prompt. In fact all the prompt does is unset the xattr if you deem it ok.
The design decision of storing the application a file came from in file metadata, but the URL it came from in an sqlite database ... makes my head hurt.
Kind of brilliant, actually. Not the fix or the tracking method...just that I knew subconsciously that if OSX could know to warn me that I'm trying to open up a file downloaded from the internet (and which webpage it came from), then it must have some way of tracking it that supersedes the standard browser wipe. But I just never thought to look around for it.
Windows will warn you about files downloaded from the internet, too. It uses "Alternate Data Streams" feature of NTFS to mark the file. I suppose once you delete the file, the alternate data stream is gone as well. From what I read, alternate data streams were made for compatibility with a Mac filesystem!
No, the resource fork is dead for good reasons. (1) The resource fork is fragile and easily corrupted, like the Windows registry in the bad old days. (2) Four-character namespaces suck. (3) 24 MB limits suck. (4) Extended attributes and bundles are better in every conceivable way.
The resource fork was designed for a different world. (A) In the old days, there was a blurry line between OS and app, so there was no reason to make the resource fork "opaque". Without opacity, there is no graceful way to use it in a preemptive multitasking environment. (B) In the old days, pointers were 24 bits. The high 8 bits didn't go anywhere, so people used these high bits to store extra data. Resource forks make use of 24-bit quantities for this reason — which went out of favor when Macs with the 68020 were introduced in 1987. (C) The real feature of resource forks was to reduce the disk space usage and load times when you have lots of small pieces of data, but — data is bigger and we have better archive formats.
There is also a maximum of one officially supported fork per file (i.e. in a single format), and even a single stored byte will occupy a full block. On HFS+ attributes are written as individual records directly to a per-filesystem b-tree, so wastage is no worse than any other metadata. HFS+ also has named forks but they were never a supported feature AFAIK
This is amazing, whoever feels comfortable about it should band together and see what files are in common, or domains. I want to delete this yet I don't. This contains all (most?) of the files I have ever downloaded, those I thought I lost when clearing browsing data from 2008.
That's actually how I felt about the iPhone coordinates database (also in SQLite I believe)...I saved a copy of it before upgrading to the iOS that didn't have that "feature"
Nice article, but unless the database is "VACUUM"ed after the "DELETE" the rows are still readable in the database file.
This kinda defeats the purpose of the whole article b/c the user didn't gain any more privacy than he had before the deletion :)
The idea is that first you run the DELETE statement from TFA, which marks the records as deleted and "hides" them but doesn't completely "forget" them, then you run the VACUUM one, which should basically erase any trace of those records ever existing in the sqlite db.
Hm, they could also have tagged the file as "came from the Internet" in the filesystem metadata associated with the file itself. That way, the "came from the Internet" tag is only around as long as it needs to: for the lifetime of the file. Avoiding the privacy problem, but also more sensible for another reason:
If you'd rename or copy the file, the "came form the Internet" tag will remain or be copied with the file. With the sqlite database approach, either this association breaks, or you need to check the database every time a file is copied, moved or renamed and if it's in there, update the database, if you want to be able to track a file when it's copied or renamed.
I don't know if OSX has extra logic for this, or if they just allow the association to break. But with the metadata tagging approach, you only have to run the tagging logic when the file is downloaded (to set the tag) and when it's about to be executed (to check the tag), not with every other file-operation.
A strange choice, IMO: the sqlite approach makes it harder to achieve the intended goal because you need extra effort/logic required to track a file as it's copied, renamed or moved, while at the same time it makes it easier for an unintended goal: tracking users by keeping the information about the file around even when it's deleted and the "came from the Internet" warning is no longer useful.
As I mentioned in another comment, apple do tag "com.apple.quarantine" as an extended attribute on the program folder/zip itself. The download log has nothing to do with the warning dialog.
That's not so much a "sane defaults dotfile" as a highly-opinionated personal defaults dotfile. I don't consider that a bad thing, but do at least give it the deserved appellation.
Words can have different meanings in a jargon than they do in the dictionary, but that does not make those meanings invalid. For instance, in computing "sane" means "I'm an overly opinionated git and everyone who disagrees with me is wrong."
Files get a "quarantine flag" set on them as metadata when downloaded on OS X. Gatekeeper uses this (along with the developer signature) to check if an app is "safe" to open or not. Not sure where this sqlite database fits in, but it's very likely related to that.
Strange, on my machine this file only contains the URL from when I installed Firefox and a bunch of URLs for Adium updates. Not any of the many other things I've downloaded...
Is it only listing things downloaded through Safari?
I'm experiencing pretty much the exact same thing, with a few minor additions to your items: I also have iTerm2 in my list, some ADC files, and strangely some RSS xml files I have downloaded ages ago.
$ sqlite3 -column ~/Library/Preferences/com.apple.LaunchServices.QuarantineEvents <<END|pbcopy
> .mode tabs
> .headers on
> SELECT strftime('%Y-%m-%d %H:%M:%S', LSQuarantineTimestamp+ 978307200, 'unixepoch', 'localtime') as date,
> LSQuarantineAgentName as App,
> LSQuarantineDataURLString as URL
> FROM LSQuarantineEvent
> --where LSQuarantineDataURLString like '%bankofamerica%'
> order by LSQuarantineTimestamp;
> END
$
Then open your favorite spreadsheet program and paste in the results.
The 978307200 number corrects for the fact that Apple is using 2001-01-01, the year OS 10.0 was released, as its epoch.
I got way fewer rows than expected. Apparently, downloads by Firefox are not logged. When I screened for B of A downloads, I was puzzled to see only downloads in the past year or so. That's because I only recently started using Chrome to access that site.
Also, some apps doesn't even need to integrate, as the OS will override and auto-apply files for some blacklisted apps, see /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/Exceptions.plist for LSFileQuarantineEnabled
But firefox doesn't seem to be in the list, though I am using the Firefox Nightly so that has something to do with it. I have Adium (lot of sparkle update checks), Chrome (just used for testing), iChat and a couple other random one-offs.
Depending on how long you have had your Mac, you will have both com.apple.LaunchServices.QuarantineEvents and com.apple.LaunchServices.QuarantineEventsV2
Only if you are paranoid enough to think that this is a problem. Of course, there are other problems with installing Linux that are ignored, but that's a topic for another discussion.
Funny, but sometimes you do want to clean up your laptop - for example, before crossing the US border.
I would be irked if I had taken steps to remove a downloaded file from my laptop, including secure deletion, only to leave evidence that I'd downloaded it in a sqlite database.
If you are worried about that kind of thing, I'd recommend doing that stuff in a VM with an encrypted file system. Then, when you delete it, you know everything is gone.
What's perhaps more concerning is that Safari enters URLs in this database, even when Private Browsing is enabled. When you enable private browsing, Safari states:
"Safari can keep your browsing history private. When you turn on private browsing, Safari doesn’t remember the pages you visit, your search history, or your AutoFill information."
Well it does remember files you downloaded, even if you subsequently delete them.
It's really hard to tell what back doors proprietary operating system have created and you lose your rights to privacy as soon as you start using proprietary blob.
There's quite a difference right there. I thought it was some kind of "hidden" file (not unlike the GPS location which was saved by default on any iPhone with a GPS -- up to the latest iPhone!?) which had now been discovered.
It's just in your user account so it's no big deal.
Why is it different just because it's in your user account?
Everyone with access to your Mac can access it just as well, i assume. I guess it can also be read by applications, uploaded to the internet? I don't have a Mac so i don't know the restrictions but i would consider it privacy information that i would like to know if and where it's stored. There is a reason you can delete your browser history. Or your recently accessed files list.
No, he wouldn't. "If you don't want Google to tell the government you've been searching for illegal things, you shouldn't tell Google you've been searching for illegal things" is what he'd try to say. But he'd mince the words and make himself sound evil, somehow. So maybe you're right.
What? I doubt it. I was pissed when I heard the full context of the quote you're referring to and saw what he actually meant. Here's what everyone likes to quote:
> "If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place"
Here's the quote in full context:
"I think judgment matters. If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place, but if you really need that kind of privacy, the reality is that search engines including Google do retain this information for some time, and it’s important, for example, that we are all subject in the United States to the Patriot Act. It is possible that, that information could be made available to the authorities."
http://i.imgur.com/IbUWj.png