how could that guy hook up an eSATA drive to his mac? As far as I can see, none of the currently available machines provide eSATA support.
I thus assume that he was using OSX on a custom-built machine and it's very well possible that OSX doesn't have built-in eSATA support (due to the lack of official hardware) and thus doesn't recognize the external drives as external ones.
You see, when I plug a USB or Firewire drive into my Mac, Time Machine doesn't try to back them up.
Presumably using either an ExpressCard (in a MacBook Pro) or PCIe (Mac Pro) eSATA controller. Not exactly what I'd call "custom built". Interesting though, that it recognises USB and Firewire as external, but not eSATA. I wonder if it supports hotplugging/ejecting of eSATA drives at all?
I use an ExpressCard. It didn't even occur to me that a drive connected this way might appear to the OS to be an internal drive, but that actually makes a certain amount of sense.
When the backup drive get full Time Machine is going to delete the oldest backups, that's how it works. Using it for a permanent backup solution isn't too smart.
On my systems I also backup to a BackupPC system for longer term storage.
Exactly. Time machine is for those "oh shit, my disk is toast/I really didn't mean to delete those files/I'm switching to a new mac" moments.
It's not intended to be used to retain a complete history of your machine and as such if used for this purpose you're eventually going to get a nasty surprise.
On some of my servers, I made a couple scripts that does it in several levels. It first deletes incremental backups beyond a give horizon but keeps the full ones. Beyond the next horizon, it deletes every other full backup and does so until it reaches the size constraints I set.
If it cannot delete anything else and the storage is beyond its limits, it e-mails me telling me to please do something about it.
On my Time Machine preferences pane it says 'Notify after old backups are deleted'. I read this to imply if TM decided it needed a monstrous amount of extra storage and decided it needed to delete a lot (or all) of the old backups, there wouldn't be any notification until they were gone. I don't see any option to notify before deleting them; perhaps a confirm option there would have been useful?
I'm becoming less and less comfortable with hard drives as long-term backup media. The other day I plugged in my external to back up some family photos, but the drive wasn't coming up and while I was investigating (this was Linux so it could have been any number of reasons) this faint noise was bugging the crap out of me. "What is that, a metronome?" It took a few minutes before the "oh shit" set in.
Time machine is nice because its nearly impossible to remember to do backups regularly (unless it's your job) and the backups are highly available, but really its more like an extended trash bin, not an archive. Storing your backups on-site is one thing, but keeping them plugged in and mounted is just asking for it.
Whatever happened to tapes? They're like comically expensive now. Another option is to copy your Time Machine out to S3 once a month. (Would probably take that long) but I don't know how you'd preserve the aliases / hard links.
<Begin rant>
Oh jeez. I worked at a company that made tape drives and you do not want to use tapes ever. They are the slowest, most unreliable thing imaginable. Especially now that they are making them more and more dense and the media is exposed to the elements (unlike a disk platter).
Yeah, they use more redundant ECC than disks but I still wouldn't trust them at all. I had definitely drunk the kool-aid for a while and happily used tape for data transfer and not just backup. I can't tell you how many times I got stuck with a bad tape hours away (physically) from the original data, frustrated beyond belief. I firmly believe most people don't even know just how many of their tapes are corrupted because they've never tried to restore except for the initial testing period and the occasional file or two.
Tape sucks. Tape is dead. Long live disks.
<end rant>
> Another option is to copy your Time Machine out to S3 once a month. (Would probably take that long) but I don't know how you'd preserve the aliases / hard links.
One of the interesting things about time machine is how it deals with network backups. It creates an HFS+ disk image on the remote machine and then mounts that and does the backup to it instead of directly to the remote volume. It uses the "sparse bundle" style disk image which partitions the disk image into a bunch of 8mb slices. So you could rsync those slices to S3 daily or monthly or whatever and not worry about the format details at all.
Just have two external hard drives. I have one at home I use daily, and then one I keep in my desk drawer at work I use weekly [1]. The chances of both failing at once are pretty unlikely. You could do the cloud, but I'm not comfortable having all my data in the could. If you have to use an external provider, use Rysnc [2] . They seem trustworthy [3].
Does it have to be "the cloud" as in S3? What about good old space on a traditional host? I have a Dreamhost account that claims to offer "unlimited" space for a flat monthly rate, and they support traditional *nix rsync/scp/etc. Seems like a good fit to me.
Uncomfortable with the cloud in general or with the current cloud-based offerings? I'm uncomfortable with Mozy, Backblaze, Carbonite because there's no way to verify they've encrypted my data, stored it redundantly, etc.
Yeah rsync.net by far looks like the best functionality, but they are almost 10x more expensive than Amazon or Rackspace, $1.20/Gb/Mo vs. $0.15/Gb/Mo. For 20-30 gigs of photos and movies, that starts to add up.
Hah I actually signed up for jungle disk after this happened.
I like it so far but I don't think it solves the alias/hardlink problem, so you'd just get one or two good snapshots before it got to be really expensive. The great feature of time machine (or rsync --link-dest) is that your snapshots share storage of unchanged files, and I don't know how to make S3 do that.
Tapes are still probably viable - but only at the high end.. Slap a SL8500 / TS3500 with Lots of LTO4 drives hundreds of tapes and the cost / GB starts coming down. (esp if you look at things like cooling, power, etc (High End storage like a Symm can power drives down, but their not exactly cheap - or really needed for backups)).
I haven't used it myself, but I've heard good things about Backblaze. $5/month, you download a small piece of software that runs in the background and backs up your data to their servers.
>> Newly mounted external drives should be excluded from backup by default.
Hey! Let's solve a backup trimming problem by potentially never backing data up unless the user digs around in their preferences! This seems like a cure that's worse than the disease.
I wouldn't call irretrievably wiping out all the backups of my primary hard drive in order to make room to back up a media drive that I just mounted a "backup trimming problem."
And by the way, I had that volume on my exclude list, but TM forgot about it. (That's a problem that has been around since TM first appeared.)
>> almost bordering on legally actionable negligence
Good luck with that, and please do call us when every single kernel contributor gets sued because someone accidentally overwrites their Windows partition when installing Ubuntu.
YOU EXPRESSLY ACKNOWLEDGE AND AGREE THAT USE OF THE APPLE SOFTWARE IS AT YOUR SOLE RISK AND THAT THE ENTIRE RISK AS TO SATISFACTORY QUALITY, PERFORMANCE, ACCURACY AND EFFORT IS WITH YOU.
and
TO THE EXTENT NOT PROHIBITED BY LAW, IN NO EVENT SHALL APPLE BE LIABLE FOR...LOSS OF DATA.
and for good measure,
THE APPLE SOFTWARE IS NOT INTENDED FOR USE IN THE OPERATION OF NUCLEAR FACILITIES, AIRCRAFT NAVIGATION OR COMMUNICATION SYSTEMS, AIR TRAFFIC CONTROL SYSTEMS, LIFE SUPPORT MACHINES OR OTHER EQUIPMENT IN WHICH THE FAILURE OF THE APPLE SOFTWARE COULD LEAD TO DEATH, PERSONAL INJURY, OR SEVERE PHYSICAL OR ENVIRONMENTAL DAMAGE.
Bottom line is that these licenses don't mean much anyway - they just make it slightly harder to sue someone, in the same way that those personal liability forms you sign before doing anything are just to scare you. In the absence of an explicit contract - as in Apple guarantees "this will not cause any data to be deleted" - they've got a very good case that while it's sad you lost your data, it's not their fault.
Paying for something may create more of an expectation, but does not (in and of itself) guarantee you anything or increase liability.
I knew that these disclaimers were in both licenses. However I think that the fact that Linux is given away for free and that OS X is a product makes a difference in the clause TO THE EXTENT NOT PROHIBITED BY LAW. I mean, you can't buy an Apple computer without OS X on it. Are they really allowed to disclaim all liability for everything? In my legally uneducated mind that seems unlikely.
In most jurisdictions, whenever you accept money for a product there are implied warranties of merchantability and fitness for a particular purpose. Since TM is widely touted by Apple as one of the features of OS X they cannot disclaim the warranty that it does what it is supposed to do, namely, back up your data. What Apple has done is analogous to selling a fire extinguisher that actually sets your house on fire. You can't disclaim liability for something like that.
Apple have always sold TM as a way of doing backups not archiving. The original article seems to want to have used TM as an archive, not a backup, and that's where the problems came in.
That said, maybe I've missed something in the marketing that Apple have done for the product - your comment seems to indicate that. Do you have any specific examples of false claims made in marketing TM by Apple?
TM would have deleted every backup they had if they didn't stop it in time. Wanting every backup you've ever made is one thing; wanting the last backup you made to be available doesn't sound like archiving.
No, it wouldn't. If it doesn't have enough space to keep at least one backup, it won't delete previous backup, it will tell you that you need more space.
Happily, the logs record exactly what happened. The request was for 877.54 GB. The TM volume is 731.95 GB. It deleted 14 old monthly backups (so just over a year's worth) before I stopped it. But given that it had requested more space than was actually available there is no reason to believe it would have stopped before wiping them all, and in particular, before wiping all my Leopard backups, which would have made it impossible for me to revert.
I don't have a copy of the software in question, I was basing my understanding of Time Machine's operation on the blog entry. When the author says things like "There are apparently people out there who have lost all of their backups because...", it seems reasonable to assume all means all.
Your logic is sound, but reasonable people can disagree. No operating system is designed to crash, certainly not advertised as doing so, yet all operating systems crash and these crashes sometimes result in serious monetary loss. But no one has ever won a verdict against an OS manufacturer for such losses (except for maybe like some defense contractor missile guidance OS or something. I'm clearly talking about Windows and Mac OS here.)
I just checked Windows and OS X licenses and there is specific language disclaiming liability for damages. Apple's license in particular disclaims liability for loss of data, in explicit terms. So your state or country would need laws specifically dealing with this kind of thing. Anyone know of any such laws?
It is not clear from the article whether this happened because the author upgraded to Snow Leopard, or because they switched their external hard drive to eSATA. If the former, the behavior might be caused by the fact that Snow Leopard is a new operating system, and many of the system files that were in OS 10.5 have changed.
Have any readers that have upgraded to OS X 10.6 experienced the same issue?
It's both. I upgraded to Snow Leopard, and I remounted external drives. In Leopard, there was an option to warn before deleting old backups. In SL that option has changed to "notify after deleting old backups." That plus deleting the external volumes from the exclude list was the nasty combination.
You also say that you switched from USB to eSATA. There is a very good chance that from the perspective of the OS these are entirely new dives, maybe even seen as internal drives. There may not be an issue in the Time Machine code, but rather this whole thing is unfortunate edge case of intended behavior.
From what I can tell the behaviour in Leopard and Snow Leopard appears to be identical, it's just that Leopard had a falsely-reassuring label for the option which has now been changed to a more accurate one. Either OS would have deleted your old backups in the same way.
The bottom line is that Time Machine is a backup solution with a certain level of archiving as an added feature; it is not a bulletproof archival solution and shouldn't be relied on for that purpose.
This behaviour sounds perfectly reasonable for the common case, for whom it was designed.
Furthermore it sounds like this guy is using Time Machine as an archiving solution, not a backup solution. You shouldn't be all that worried to lose backups a year old if you know newer ones are intact. You should never delete a document there's any chance of ever needing again, then rely on TM or similar to save it for you.
I'm not worried about losing the old backups. But I would be worried about losing all of my backups, which is what would have happened if I had not noticed what was happening and stopped it manually.
I thus assume that he was using OSX on a custom-built machine and it's very well possible that OSX doesn't have built-in eSATA support (due to the lack of official hardware) and thus doesn't recognize the external drives as external ones.
You see, when I plug a USB or Firewire drive into my Mac, Time Machine doesn't try to back them up.