Using DU, I used to clone Snow Leopard on my intel iMac to a partition on my external APM formatted firewire drive. I also cloned Leopard from my PPC iMac to another partition on that drive. In both cases, I could restore from the firewire drive to the parent computer using DU.

I've got a third partition on the firewire waiting to take Lion. With Lion, "ought" the cloning and restoring to behave as before -- that is, given that I'm again using the APM formatted firewire drive? (I've googled but can't get confirmation from any using an APM external drive.)

I understand that with Lion I have to do the cloning and the restoring by going first to the new recovery "partition" in Lion (by holding down command-r). That's to be expected.

By the way, those using a GUID external drive report that while DU clones the recovery "partition", CCC and Super Duper don't.

Okay; so I'm impatient.

I tried to clone Lion on my Intel iMac to my APM external firewire using DU and the Recovery "partition" on my Intel iMac. DU said: No way Jose -- you have to have the GUID formatting on your external to use Restore with Lion.

I had worried about this -- hence my prior post.

So it looks like I need an additional drive to clone both my PPC Leopard and Intel Lion. Good thing drives don't cost much anymore. And if I wanted to use Time Machine for my two iMacs, wouldn't I also need two backup drives?

The Time Machine backups aren't bootable, so it shouldn't be necessary to have a separately-partitioned drive for each.

I've heard, though I haven't verified, that Lion *will* boot from an Apple-partitioned drive, even though DU says you need a GUID-partitioned drive. Third-party cloning programs like Carbon Copy Cloner may be able to clone Lion to an Apple-partitioned drive.

Thanks for that. I went over to the CCC forum. Mike B says there (in general terms -- not with respect to just Lion) that to clone a PPC and an Intel Mac to an external drive, two such drives are needed, one formatted as APM and one as GUID. I know that is wrong up to and including Snow Leopard -- at least it is wrong for DU as the cloning tool.

I posted a question there. So I'll look to see Mike B's answer.

I'm not clear about using Time Machine and one external for both my PPC and Intel Mac. I'd have to have, of course, two partitions on the external. But how should the external be formatted -- APM or GUID?

I have yet to run into an intel that won't boot from an APS partitioned hard drive. There are some limitations, some real, some artificial:
1) you cannot run an intel firmware update while booted off a volume on an APS partitioned drive
2) you cannot select a volume on an APS partitioned drive using a Mac OS X operating system installer (the installer will lie and tell you that you cannot start off that volume)

I don't know why people keep spreading the misinformation that intels can't boot off APS partitions. I do it several times a day here. All my service drives are partitioned APS because I use them for both PPC and intel.

I'm certain at some point Apple will drop APS support from the boot firmware, but afaik they haven't done it yet.

Well, you're right -- as I found out this morning. I used the latest CCC to clone Lion to a partition on my external APM firewire. I could boot that partition.

The Recovery Partition is not copied. But so what if you've got a clone to copy back to the parent machine. And if you "really" want the Recovery Partition on the parent machine, you can reinstall the latest Lion there.

CCC is rather impressive. (I've never used it.) I have only 29 gigs on my Intel machine but CCC copied them in 15 minutes! And if I do the same clone in a week, CCC will copy just the new stuff. I'm going to have to give Mike B some money.

CCC is definitely worth making donations for. I have used it personally and with my clients on countless occasions. I have a policy that any time I use it with a paying client, I always donate some money (again). It's a great piece of software that has saved my bacon several times.

Free candy!

http://vftp.net/virtual1/temp/MFIF/Klone_2011.09.23.A.zip

let us know how this compares with CCC.

Sorry I've been so slow to get to this.

The first thing I learned about Klone is that, unlike CCC, it does not permit cloning your boot volume.

So I booted into my 10.5.8 partition and...

1. I erased the partition in which I store my clone.
2. I cloned my boot volume using Klone.
3. I erased the partition again.
4. I cloned my boot volume using CCC.

Results (I've saved copies of Klone's Terminal window, CCC's log, and assorted disk usage info from Disk Utility in case you're interested in seeing it.):

• Sat Oct 22 23:29:04 EDT 2011 Starting Klone...
• Sat Oct 22 23:41:53 EDT 2011 -15584976 KB remains, 00:0-5:0-45:0-24 at 752 KB/sec current, 00:00:0-41:0-2 at 6329 KB/sec average
  
  Klone finished with return code 1, stopping scoreboard... Terminated
  logout
  
  [Process completed]
• CCC: 10/22 23:59:51 Time elapsed: 00:09:51.

As you can see, Klone and CCC took 13 minutes (plus the time it took to boot into my Leopard volume and back into my boot volume) and 10 minutes (in total), respectively, to complete the same task.

I didn't boot into the Klone cloned volume to confirm this, but judging from volume sizes as reported in Disk Utility it looks like Klone maintained HFS+ Compression as does CCC.

Edit: I forgot to mention that my boot volume is about 5.5GB in size.

ditto will clone the booted volume, but you will need to use -X lest it try to copy /dev/* and /Volumes/* which will turn ugly.

I've had several "experts" tell me that they heard an expert somewhere say cloning/backing up a booted volume is a bad idea, but nobody could tell me why or provide anything more than hearsay to back up their words so I am ignoring them

In theory, backing up a booted Unix volume is a bad idea because there are a large number of open files all the time on any Unix system, and it's long been considered bad form to attempt to copy an open file. On top of that, a Unix boot volume is unstable; it's constantly being read from and written to even when the computer is just sitting there.

In practice, well-behaved systems limit the file reads and writes, especially with temporary files or VM files, to well-known directories that can be excluded from the backup.

There are times when backing up a booted volume can lead you into trouble. For example, if you use Apple's Mail.app, it keeps incoming messages stored as files on the hard drive, and it indexes the mailboxes using a SQLite database that's always open for writing as long as Mail.app is running. Backing up the mailbox directories or the SQLite database at the moment mail comes in will result in a backup that isn't coherent, is missing some of the files, or both.

I make clones of my booted volumes all the time, but I make a point to ensure that no user-level applications are running when I do it.

> I've had several "experts" tell me that they heard an expert somewhere say cloning/backing up a booted volume is a bad idea, [....]

If correct, wouldn't that put the kibosh on Time Machine backups?

> I make clones of my booted volumes all the time, but I make a point to ensure that no user-level applications are running when I do it.

That's the route I've traveled since giving a moment's thought to the implications of cloning a volume while booted into it.

Time Machine backups are a special animal. They don't back up operating-sysem-level files and aren't intended to be bootable, and they're more database entities than file-level entities.

Aaah! Thanks for the clarification.

That's what I used to do, back in the days when I was backing up my MacOS 8.6 and earlier systems using Retrospect. Retrospect had the nice feature that, after copying all the files that had changed, it would go back and compare them with the copies. (They didn't do that right for a while: they'd just read back what they'd just written to see if they wrote it right, but that doesn't detect read errors. I got some very bad backups because of that. They eventually fixed it so that they copied all the files, then made a second pass comparing all the files, making sure they were reading everything from disk again and not just from cache.)

But when I went to OS X, I discovered to my dismay that "no user-level applications" wasn't nearly good enough. I was astonished at how much OS X does in the background, a lot of it with important files that really needed to be backed up. For a long time I made vain attempts to shut down more and more stuff, but it was becoming unwieldy to remember all the things that needed to be turned off, and increasingly problematic to remember to turn it all back on again after the backup. (At the least, you had to disconnect from the internet during the backup, because so many background processes are constantly updating files with information that becomes available online.)

I finally conceded that the only safe way to do it (and in practice the easiest way also) was to boot from a different volume. I never back up a running boot volume anymore.

Except with Time Machine. TM isn't perfect, but it knows it's backing up a running system, and goes to extraordinary lengths to mitigate the risks.

One thing it does is almost childish in its simplicity, but an effective stratagem nevertheless: it keeps all the backups. If one of them is bad, you can restore from another. This still isn't perfect (you need to know the backup is bad, you need to know which files to restore, etc.), but it was enough to justify giving it a chance to prove itself.

Another thing it does is that every time it does a backup, it does two backups in rapid succession, back to back. Each time it backs up, it backs up only the files that have changed since the last good backup, using the FSEventLog mechanism to find those changes very fast. Much much faster than the time it would take to look at every file to see if it had changed. (Other backup programs spend much more time figuring out which files have changed than they do actually copying them.) When it does that second backup, the "last good backup" is the first one, so the only files that have changed are the ones that changed during the copy phase of the first backup. (The first backup has already caught all the files that changed while it was evaluating what files had changed.) It only takes a few seconds to find and copy those, so the second backup is a consistent copy across the whole filesystem to within a few seconds. (This not only guards against single files changing at the time, but related files that need to be updated and backed up in a consistent state.) It's still not perfect, but the window for failure is getting narrower.

Some files it knows cannot be backed up. If it finds a disk image file that's currently opened read/write, it knows there's no way that file can be backed up. Instead, it punts, saying that the previous backup of that file is "good enough". (Stale data is better than bad data.) One way to look at that is, as far as TM is concerned, a disk image file doesn't actually change until it's closed, and it'll get backed up then.

Some it knows how to deal with. It knows what an sqlite database is, and backs it up in a consistent state even if it's being updated at the time. (How? Talk to the database gurus. Database designers have to deal with multiple simultaneous updates all the time, and many a doctoral dissertation has been written on each tiny aspect of that problem. TM just plays by the rules that SQLite sets up.) That means it gets a consistent view of your Mail data even if Mail is retrieving mail at the time.

During a backup, it uses an FSEvent stream to watch what files are being changed. It can detect a file that changes while it's being backed up, and knows the copy is no good. If it happens in the first pass, it puts that file in the list of things to copy in the second pass. If it happen in the second pass, it will either ignore the change (just like it does for disk images mounted read/write), or will sometimes mark the entire backup as bad, putting up an error alert to that effect. (I don't know how it decides which action to take. I've seen it do both.) No matter. The next backup, an hour later, will probably be good. Two hours between backups is still a lot better than any other backup method can strive for.

Is it perfect? Not if you believe in Murphy, but it's not shabby.

The “aren't intended to be bootable” part is true enough (in the immediate sense), and easily overcome by installing an OS on the backup disk (assuming we're not talking about a NAS, which wouldn't be bootable with any backup software). However, the backups Time Machine produces certainly are intended to be bootable —once restored back to some disk.

Therefore, the “don't back up operating-sysem-level files” part may be a bit overstated. [or, i'm not sure exactly what you meant.] In either event, stuff that gets skipped by TM is generally considered to be expendable (though a few exceptions may be argued by some users).

$ defaults read /System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions

Not merely expendable, but in most cases stuff you wouldn't want to restore.

• While a case might be made that log files might be useful for historical review, actually restoring a log file would constitute a loss of information.
• Restoring temporary files would only create orphans; the application that originally created them has forgotten them, and won't use them even if they're restored.
• A similar case can be made for cache files. In most cases the application that cached them has forgotten about them, and won't use them even if they're restored.
• Restoring your virtual memory file would be disastrous, even if possible.
• Your Spotlight database is actually a cache (in the sense that everything in it came from elsewhere and can be recreated if need be), but more importantly it's supposed to reflect what's on the disk volume now. Restoring your Spotlight database would break its contract, because you'd be setting it back to a reflection of what was on the disk volume then.

If it would be a mistake to restore a file, it's futile to back it up. Most of TM's exclusion rules are a consequence of this principle if futility, and not based on mere expendability.

It is true that some of the rules reflect expendability:

• A case could be made for restoring log files as part of a whole-disk recovery, even though you'd never restore individual logs, but not a strong enough case to justify the expense of backing them up over and over again. Who ever looks a log files anyway? (Present company excluded.) They're nearly expendable as is.
• Files in the trash have been explicitly marked as expendable by the user, but additionally they're probably redundant, having been backed up from wherever they were before they were put in the trash.

A few exclusion rules are for the sake of security. Backing up /Users/Guest would violate the contract that when Guest logs out all of its files are gone, gone, gone.

Some files that would be excluded for one of those other reasons could also be excluded for the sake of efficiency. Repeatedly backing up a large constantly changing file (like Outlook data) quickly becomes expensive enough, both in time and in the space it chews up in the backup, to make you feel that, even if the file is not expendable, hourly backups of it are. I think most user-defined exclusion rules are based on efficiency.

Of course, there's a lot of overlap. Your virtual memory file could never be restored, it's expendable, and backing it up would be a security breach as well as horrendously expensive. (It's also probably impossible to back it up even if you were so inclined.) A file that won't ever be restored must be expendable in some sense. The expense of backing up a file needs to be weighed against the cost of rebuilding it (or even of losing it).

But when I review TM's built-in exclusions, the most common basis I see is futility, not expendability. Don't back up what you're not planning to restore.

Well, from the point of view of time, cloning by booting from a different volume is not the easiest way. The easiest way is to clone from the boot volume.

And, in my eight or so years of experience using Mac OS X's DU for cloning, up to Snow Leopard and now with Lion using CCC, cloning from the boot volume works just peachy fine. And I never bother to close down the internet connection either. Unlike Tacit, I don't bother to close down all the apps I've just been using. So what if Text Edit is open and so is Safari? Makes no difference.

After doing a clone, I always test it by booting from it for a test day. Never found a problem.

I'll happily and thankfully bow to Ganbustein if he can produce a good sized sample of competent Mac users who have had trouble cloning from the boot volume. Even better, if that sample was statistically significant. (Most samples aren't.)

But without some negative data on cloning from the boot volume, warnings against it are ... to put it politely ... just unsupported opinions. And they look really foolish to some of us empiricists who have been cloning from the boot drive for years without trouble.

Empirical data is what counts in everything -- wonderful theoretical opinions unsupported by empirical data are fun to contemplate and possibly the ringing bells for future improvements, but nothing to take seriously in their unsupported clothing.

As Ganbustein said elsewhere in these forums, let's get real.

I don't recall seeing an 'RHV' back at the MacFixIt forums, but there were frequent threads there about unbootable clones. Same at macosxhints. Or —if the clones were bootable —the other fun thing that happened is that utilities like DU and DW would refuse to repair the original while booted from the clone. (which was one of the primary reasons for the clone in the first place... not mere backup, but a boot drive from which to repair the HD when things got borked up).

The main reason a cloned OS refused to repair a mounted original was that it somehow believed that it was still booted from the original. Some "link" seemed to exist between the two. [the clone operation had worked "too well" for such purposes.]

Tons of times i had to advise folks to **install** an independent OS for emergency booting. Booting into the same "exact" OS that we want to repair or restore is problematic.

For real.

I always use SuperDuper to clone my main drive and have never had a problem repairing it with either DU or DW when booted from the clone. I have never tried Ditto so I can't comment.

Well... obviously if everyone had this sort problem, then i wouldn't need to mention it to anyone. But quite clearly such has not been the experience of 100% of all users. So if you want to take chances by having your only backup be the same OS which you rely on for emergency booting, be my guest... but that is not my advice in any case. An autonomous boot volume is always going to be more prudent (both for backup integrity and boot OS isolation).


If you ever used Carbon Copy Cloner version 1 (back during Puma and/or jaguar), then you've tried ditto. It's actually a fantastic little command line utility, for backing up user folders and such. (but dealing with system files and making stuff bootable requires additional tools).

With those two programs in particular, you're unlikely to have problems. A TextEdit file that's open isn't held open for writing on the disk; the only way you'll have an issue is if you save that file at the exact moment it's being cloned, which is unlikely. Having Safari open while you clone may result in the Safari cache files on the clone being a bit scrambled, but that's no big deal.

Other programs are more problematic. Some programs, including Mail and iPhoto, use SQLite databases. Carbon Copy Cloner doesn't know how to deal with these in a coherent way; it just treats them as files. If you're cloning your Mail database while new email is arriving, or cloning your iPhoto database while you are importing, tagging, rearranging, rating, scanning for faces, or sorting photos, you may end up with a clone that has a scrambled database. This isn't too big a problem in Mail--there are ways to force it to rebuild mailbox indexes--but in iPhoto, your library may be unreadable and have to be rebuilt. I've seen this happen. Rebuilding an iPhoto library is not easy, and you lose all your album names, ratings, tags, and faces.



I haven't seen that problem, and I'm a bit mystified how it could happen. The only way I can see for such a thing to happen is if the clone were still referencing files on the original, via aliases or symlinks or some other mechanism, and as a result the original drive still had open files. That's a problem that's not necessarily related to cloning per se.

Certainly I've never experienced that problem. I've been using tools like CCC to clone active boot volumes for many years, and have probably at this point done it literally thousands of times (not only on my own systems but on clients' systems as well) without problem.

Like Jon, I have always used SD to clone my main boot volume while that was running. Never a problem. That said, (and with the occasional exception of a safe boot) I follow the manual's recommendations.
While stating that this is not strictly necessary, the SuperDuper! manual recommends in Step 1: Prepare for your backup to quit all running applications, and perhaps even do a safe boot into the volume to be cloned. Final note: I am NOT running Lion, but Leopard (PPC Mac).

For the record: I never had any clone problems either. (those few i bothered to test anyway)

But i did help [and/or read about] dozens who did have problems. If y'all want to wander the MFI archive and dig up some old threads, maybe you can explain what actually went wrong. [tacit is basically right... and like i said: there was some "linkage" in the clone which made the utility believe it was booted from the same disk it was supposed to repair. No one has ever figured out how exactly that happened (e.g., it's this symlink in /usr/bin or wherever), so no one ever "fixed" whatever it was, nor can anyone predict when it might happen to them.]

Meanwhile, i don't see any need to take a chance, when avoiding the potential issue requires only a basic partition/install.

The SuperDuper philosophy is this: backup... and when the HD fails, you can boot from that backup and "get back to work immediately" (meet deadlines, etc). And that's fine... unless this bug bites (or perhaps if your backup has a "backup" of the same problem).

To dispel any doubt, my previous post was not intended to argue against your position, but just a statement of practice. FWIW, I sure do recall those florfed clones, and I have this dark-green suspicion (no hard supporting figures here either) that many—if not most—of those problems had to do with cloning while continuing to work and/or with hardware glitches.

In conclusion, I have no argument whatsoever with your backup approach, but I haven't had occasion (yet) to follow it either.

Nice Try Hal.

You are way, way off the topic. I was surprised by that. You are no dummy.

But you were addressing the question whether or not clones are reliable. I was not -- though I said, as a personal comment, that my clones were reliable after testing for a day even when they were done from the boot volume.

But I was addressing in my post a different issue. And surely that issue was apparent.

I was addressing the issue as to whether clones done from the boot volume are less reliable than those done from another volume. Ganbustein thinks that clones done from the boot volume are less reliable. I challenged that on the basis of no properly acquired data provided by Ganbustein.

And you did not deal with that issue in your reply to me -- but dealt with the issue as to whether clones in general are reliable. On that one, I suspect that proper data will show that they are.

And yes; I've been on Macfixit since about 2000 and on OSX Hints forums since about 2001 -- if that matters to the issue dealt with in my post. I bought my first Mac in 1984 -- a Mac that had all the guys (some gals?) sign the interior backside panel of the computer. I think that I paid $5,000 for the computer, an external drive, and the printer.

How would you know if my answer addressed your question or not? Perhaps clones done that way are more susceptible to the phenomena which i discussed. No one ever found out for sure what the exact mechanism was... that was my point.

Anyway... it should be self-evident that cloning an OS which is actively running (with open files, etc.) would indubitably be more susceptible to any sort of quirk, than cloning a totally inactive OS from some other volume. That's just basic logic, and certainly nothing over which to get all worked up or demand detailed analysis about.

Perhaps CCC and SD have managed to solve the matter over the years, and found the exact file(s) to exclude/tweak/whatever... idunno. But some special step was probably needed to overcome the potential for such problems... which don't present themselves when cloning an inactive OS volume. If you're that interested, try asking in the forums over at bombich.com or shirt-pocket.com maybe.

It's funny: I was thinking about a reply to your post when I noticed Hal's response (I read both on my iPod, but don't like to use that for more than a brief reply). My take on Hal's posts was that his reservations were about (making) clones of a running boot volume, not so much about clones in general. After rereading those post I still think that way, and the funny thing of course is that neither you nor Hal don't quite seem to. Shows you how easy it can be to misunderstand each other.

And that's another reason for my post: although I understand how one might view this otherwise, I took Hal's comment about not having seen you at MFIF at face value, and not as the slight you may have taken it for.

As to your original Mac, I bought one too in early '84, but I recall paying about 3K (US) for it including Apple printer and (later) external floppy drive. Your 5K must have been Canadian, right?

In 1989, even with an educational discount, the price for a Mac IIci with Portrait Grayscale monitor ran up to 5K (at least in Boston it did). They sit in storage now, as i don't have the heart to toss them in a scrap heap.

[and yes, the MFI forums reference was strictly about one having seen the numerous clone issue threads.]

I wasn't talking about the easiest way to do it (which, by the way, is to not do it al all), but rather the easiest way to do it reliably.

The first rule of optimization is: any optimization must be correct. No one will be impressed if you get the wrong answer faster.


There was a formative moment in my earlier career. I was working as a Tech Rep for Burroughs Corporation, and one of my fellow Tech Reps told a customer, a heavy user of Burroughs' reader/sorter machines (used to sort checks based on their MICR coding) "Don't worry about that. There's only a one-in-a-million chance that that would ever happen." The customer replied "At Bank of America, one in a million happens fifty times a day."

That made a big impression on me. It doesn't matter how unlikely something is. If it can happen, it will. Testing once or twice or even a million times isn't good enough. You have to design your algorithms and procedures to make errors impossible, not merely unlikely.

If you tell me you've been making clones of running systems for 8 years (and also tell me you test each one by booting from it for a day), I figure you've only tested it about 400 times (once a week at most), and "running with it for a day" hardly comes close to a thorough test. I am not impressed.

Backing up any digital file while it's being updated is a well-known danger. Look up the "readers and writers problem" in any text on databases, but don't think for a minute that the problem only affects "database applications". Or look up "readers and writers problem" in any text on concurrent programming. Even something as simple as incrementing a number in memory can be problematic on a machine with multiple cores.

This is not a hypothetical risk. It's common, especially among neophytes who think "I tested it and it works" means that it will always work. Experienced programmers know better. If you want to take a digital snapshot of something, you have to freeze it for the duration of the snapshot. One of the reasons we prefer digital copies over analog copies is that successive analog copies get blurrier and blurrier, but digits don't blur. Trouble is, when you make a digital copy of a changing file, you eventually discover that digits don't blur, they break.

Edsgar Dijkstra, one of the pioneers of Computer Science, famously said "Testing can only reveal the presence of bugs, never their absence." The tests you've made fall under that heading. The fact that you haven't found a bug yet doesn't mean you won't find one tomorrow, or even that there weren't undiscovered bugs in the past.

My data is more valuable to me than that. I want to know that my backups are good. Hoping isn't good enough.

A day? How thorough a test can you do in a day? I've got 1.6 million files on my boot volume. How could I ever test all of them in a single day?

And you think booting off the clone (and not getting any real work done during that day, because you'd be updating the wrong disk) is easier than what I do, which only costs me about an hour of down time?


Hal has already pointed out where you can find some, and most users who have problems won't say "I cloned a running system and this is what went wrong." Usually, if they say anything at all, they'll say "I'm having a problem with X", and if we're lucky we'll eventually pull out of them the oh-by-the-way comment about how they restored from such a backup two years ago, and it turns out the problem has been lurking in their system all that time. But usually, even if the backup is bad, they don't know, because most users never test their backups, and the few that do make only a few spot-checks, usually limited to "It boots, so it must be good."

I also talked at length about all the files that Retrospect found changing during each backup. Many of those were files I really wanted reliable backups of. And there's also the problem that many applications cache data in RAM, and don't bother keeping their disk files consistent until the program quits. Many of those applications are actually "system processes" that run in the background, so you don't even know you're running them, yet cloning their disk files will give you inconsistent data.

I have at times had to recover from backup. Most of those times, the problem was because the disk catalog got corrupted. For that reason, I don't trust anything that copies the catalog itself. In particular, I don't trust a sector-level copy, like with DU. Even if I'm not booted from the volume. If I had used a sector level copy in those situations, my "backup" would be just as bad as the original.

You're looking at it wrong. It's not a question of "what are the odds?" It's a question of "what's reliable?" For bugs, even a single occurrence is "statistically significant", especially if that occurrence happens to me.

To make a long story short, my understanding is that both you and Hal are advocating cloning of a "resting" volume. Does this mean that you need to have three drives: the active volume that has the cloning software running, the inactive source (also bootable) and the target drive? I kind of doubt that this is a standard setup unless one has a desktop with two drives and an external. Are you guys doing it that way?

P.S. I started a thread at the SuperDuper home about this issue. Will report when David or someone else answers.

Why would you need three drives? It's entirely possible to have three volumes on a single drive, and to be booted from volume B while cloning volume A to volume C.

Obviously, since cloning from one volume to another on the same drive does you no good if the drive itself fails, in practical terms you do need a second drive—typically, an external—but a second bootable volume on either the internal or external drive is sufficient to avoid cloning a booted volume.

I was wondering the same thing as Alex. (Everyone in class is always so happy when some other kid is willing to raise his arm first) So, to be clear, this is what I now understand.

I would repartition my backup drive into two volumes. In one partition I install a system and Super Duper. The other partition would be reserved for my Clone.

When I make my clone I reboot from the drive that has the system and Super Duper, and I instruct SD to make a clone of my main drive onto the partition reserved for it.

N'est-ce pas? (My Time Machine is on a separate drive)

> When I make my clone I reboot from the drive that has the system and Super Duper, and I instruct SD to make a clone of my main drive onto the partition reserved for it.

You slipped... You "reboot from into the drive that has the system and Super Duper...."

....which demonstrates why I'm always at the back of the class.

Please be sure not to sit in my seat.

Actually, I think ryck had it right. One boots from a particular volume or device, and boots into a particular OS or mode. Check these Google searches to see what I mean:

• "boot from" - Google Search
• "boot into" - Google Search

Admittedly this is non-"standard" practice, but every disk/drive i own (or have ever owned since the early 90's) is partitioned. The externals have at least one bootable system, and my internal HDs have at least two. I suppose another alternative is a bootable flash drive. [one could also use Disk Utility to do a "restore" (clone) while booted from a system dvd perhaps, but i reserve dvd booting for full-blown emergencies when no other option is available.]

This part of the discussion is mostly about *full* clone operations. Once that has much been achieved successfully, all the "incremental" updates (which both SD and CCC offer) are safe enough in my opinion.

That all goes for Time Machine too i suppose. I was just wondering to myself how they design those programs to run while users are busy renaming, re-saving, and/or moving stuff around in realtime... while the backup is still underway. I guess they have error handlers that silently avoid potential conflicts and deal with them somehow.

That's why a full clone is more susceptible... because it takes so long to complete. If it all happened in one second, not much will have changed from location to location. But the fact that it takes a few minutes increases the chance that "something" might get out of sync (e.g., when one item backed up early on depends on the state of something backed up later in the game). In most cases it might not matter, but there's always a chance that one day . . . it might.

I agree with your reasoning, it seems very logical. So, why didn't Apple make a couple of partitions on their computers to facilitate the cloning procedure? We heard a lot of things here about TM "clones" not being bootable… Do they think that only backing up user's data is necessary and the rest can be restored from original disks if push comes to shove?

A brief bit of history: back in 10.1 we had two Apple programs... Disk Utility and Disk Copy. Only the latter was capable of copying disks. Then later (10.2 maybe?) that functionality was merged into DU, and DC was dumped. My recollection is fuzzy, but i think it was during that interim in which CCC came on the scene with its ability to make bootable clones. [can't recall if DC did that or not in OSX, and i'm fairly certain that Super Duper didn't exist yet.]

But anyway, "cloning" to create a bootable backup was never anything that Apple offered initially.. as far as i recall. In fact Mike Bombich went to work for Apple right around the time that CCC and NetRestore were becoming huge hits... so perhaps it was around that same time that DU started to get beefed up. I'm not sure though.

There is a similar phenomenon with Software Update. Remember back in the "modal" days before OSX, when doing a software update meant not being able to do anything else? Then OSX came along and didn't enforce any such restriction. Subsequently, some users would continue browsing or playing WoW whilst a software update was underway... only to show up at MacFixIt's fora later on talkin' 'bout how the update hosed their system.



You know Apple... everything needs to be (or at least seem) as simple as possible.

"Subsequently, some users would continue browsing or playing WoW whilst a software update was underway... only to show up at MacFixIt's fora later on talkin' 'bout how the update hosed their system".

Like!

This is similar to what I do. On every computer that I've ever had that had two internal drives, they were both bootable. When I started using computers with only one internal drive, I started partitioning it into two bootable partitions. Let's call them "Normal" and "Alternate". I partition a same-size external drive the same way, into "Normal2" and "Alternate2". Normally, that external disk is unmounted and powered off, and I'm booted off of "Normal". Time Machine is backing up to a separate external drive.

When I want to back up, the procedure is:

1. Power up the external.
2. Use a copy of SuperDuper on Normal to clone Alternate to Alternate2.
3. Boot off of Alternate (which, incidentally, has Time Machine turned off).
4. Use a copy of SuperDuper on Alternate to clone Normal to Normal2.
5. Unmount and turn off the drive containing Normal2/Alternate2.
6. Run Software Update to bring Alternate up to date.
7. Update anything else on Alternate that needs updating. (This is quick. Alternate is a nearly-virgin standard OS X install. The only extras are SuperDuper and Xcode.)
8. Boot off of Normal, logging into the admin account
9. (Optionally, turn off Time Machine here, if the following steps are expected to be lengthy.)
10. Run Software Update to bring Normal up to date.
11. Update anything else that has updates
12. (Turn Time Machine back on if I turned it off above.)
13. Log out of the admin account, and back into my normal user account.

Things to notice:

• I never use SuperDuper to clone the volume I'm booted from.
• The external backup drive is unmounted and turned off during all software updates, to guard against updaters grabbing the wrong disk.
• I get a full current backup of everything before it gets updated. If an update botches things, I can back out by restoring from the backup.
• The last TM snapshot before I start, and the first after I'm done, also give me full before/after versions to restore from. I don't let TM back up a partially updated system.
• The internal drive (where I work), SuperDuper's external, and TM's external, are three separate drives. Two backups onto the same drive are really only one backup. (There are other externals, for a backup of my TM backup, and offsite backups, but that's another topic.)

Why would you think that? Any backup of a running system is risky. The "increment" would be mostly my user files, which are the files I care most about.



I wrote a very lengthy post on this topic. TM knows it's backing up a running system, knows its risky, and goes to great lengths to mitigate the downside. I won't repeat the whole thing, but the highlights are:

• It keeps multiple backups. If one is bad, there are others.
• It uses the FSEventLog mechanism to locate changed files very quickly, reducing typical backup times from several tens of minutes to several tens of seconds. The list of changed files includes even the files that change while it's building the list.
• It always does two backups, back to back. The second backup catches all the files that changed during the copy phase of the first backup. The second backup typically takes only a second or two, so it's an almost-simultaneous snapshot of all files everywhere.
• It has intelligence about which files can be safely backed up while they're being changed. (It knows disk images cannot be backed up while they're mounted read/write; it knows how to get a valid copy of an SQLite database even if it's being updated.)
• It accepts advice from applications about what can't be backed up. For example, iPhoto tells TM not to back up it's photo library while iPhoto is running.
• Lion has a feature called File Coordination that lets any application safely copy a document being edited by another application. The editing application needs to support File Coordination, of course, but Finder and Mail both invoke it. I can only imaging that TM on Lion will invoke it too, but I haven't verified this yet.
• It uses the FSEventStream mechanism to detect files that are changing while being copied. It knows such copies are not to be trusted, and won't keep them.
• For files that cannot be backed up this time around, it keeps the version from the previous backup, preferring stale data to corrupted data.
• When, despite precautions, it doesn't trust the backup it just made, it discards it (with an error alert). There'll be another attempt in an hour, or the user can do a Backup Now. Either way, such errors are transient, and the next backup will probably succeed.

You're correct as relates to ryck's words, "reboot from the drive that has the system and Super Duper," but he hasn't got a drive that's got OS X and SD, rather he's got a partition on a drive that's got them, so his intention is to "reboot into the partition...," and that's what I reacted to.

Edit: In other words, I should have said "You 'reboot from into the drive partition that has the system and Super Duper....'"

"The external backup drive is unmounted and turned off during all software updates, to guard against updaters grabbing the wrong disk."

Never happened to me. In fact, Apple software updater only updates the main disk, which is used to boot the computer (it shows with a green downward arrow); my external (bootable and with a clone of the main one) always shows up in orange color with an exclamation mark on it.

The OS X 10.6.8 Combo offers to update any eligible volume it can find, but I'm not certain which, if any, other updaters work similarly.

Edit: It just tried, and it offered to update my main volume, my bootable clone on the same disk, and a bootable clone on an external HD.

The App Store application also tries to update all the copies it can see. It's not always successful. (For example, it keeps complaining that Xcode on my "Alternate" volume is not up to date, even though it actually is, probably because I updated it using the "Install Xcode" application that it downloaded to "Normal". Xcode can only be installed on the boot volume. Irritatingly, the App Store doesn't tell you which copy of an app it thinks is not up to date.) It did update the copy of "Install Mac OS X Lion.app" that I had squirreled away on a different volume where I thought it would be safe.

But I've been doing this for a long time, not because I mistrusted Apple's installers (until now), but because third-party installers are a very mixed bag from vendors with very different levels of competence. I've even seen installers try to update copies of their application on the Time Machine backup. (Fortunately, TM is very good at protecting its backups.)

Another problem, less common than it used to be, is that an alias my resolve to a target on a volume different from the one you expect, especially after you've restored from a backup. (For example, a restored alias may resolve to a file on the backup.) If you even run an application, it may think its documents are on the backup volume, and you'll silently update the wrong document. Then all your changes get wiped out on the next backup. As I say, that's less common now that Apple has changed the algorithm for resolving aliases so that it gives priority to the embedded symlink over the embedded volref and inode numbers.)

And, there was briefly a problem with Apple's installer, in that it would damage a drive using a very specific Firewire interface chip if it was powered on (it didn't need to be mounted) during an install. Apple fixed the problem, and it would never have affected me since I didn't have any drives using that chip, but I still power off my backup drives before running the installer.

Besides, you're going to power off your backups eventually, right? They're too valuable to leave mounted as a tempting target for bugs and/or operator error. Why tempt fate?

Bugs, like clothing styles, come and go, and sometimes come back again. Just because a problem has been fixed doesn't mean it'll stay fixed. Better to let someone else report the resurgence of an old problem than to experience it firsthand.

You are right. However, the installer offers to first update the booted disk (green arrow) by default. One can only manually choose the other disk(s). Just updated Microsoft autoupdate and it did not even ask me where to go: updated the booted disk as well. Therefore, my impression that the installers do not install something "by mistake" and try the booted disk first still stands.

> Therefore, my impression that the installers do not install something "by mistake" and try the booted disk first still stands.

I agree that installers try the booted disk by default, but I can't argue with ganbustein about the "by mistake" part; he's got too much history to my too little.

Apropos of that, did you see his "The App Store application [...] update(d) the copy of "Install Mac OS X Lion.app" that I had squirreled away on a different volume where I thought it would be safe?"

I think it happens because it must be the only copy the updater can find. Again, does not seem to be a mistake.

This is what the creator of SuperDuper replied about cloning:

It's always "most reliable" to copy a totally inactive system from a 3rd drive. But it's sufficient, in most cases, to run from the drive you're copying from.
__________________
--Dave Nanian

Well if you're gonna be that way about it (), ganbustein said "The external backup drive is unmounted and turned off during all software updates, to guard against updaters grabbing the wrong disk," which is precisely what the App Store did to him.

Why did the App store grab the wrong disk? Was the updater also on the main disk? If not, seems like it did it right. Anyway, the update process is not always fully automatic; the disk for an update is usually chosen manually, unless it is a direct download from software update (and maybe from the App store).

I've shunned the App Store so far, but my understanding is that it doesn't allow you to maintain an archive; it searches every volume it can find and updates anything it finds that is subject to an update.

Remember, ganbustein said "It did update the copy of "Install Mac OS X Lion.app" that I had squirreled away on a different volume where I thought it would be safe."

Edit: The App Store performed as directed, if not necessarily expected; it "grabbed the wrong disk" only in the sense that it affected a disk that ganbustein did not want touched.

I use the App Store extensively and your understanding has little or no basis in facts.

• It only installs apps into the application folder of the boot drive and these are essentially drag and drop installs.
• It only updates apps in the /Applications folder of the boot drive. Here again the update is t all intents and purposes a drag and drop install that overwrites the previous version, unless the previous version has been moved to a different location.
• There is no automatic update per. se., the user has to open the App Store application, click on updates, if there are updates either select All or the specific app(s) to be updated, and enter the Apple account password. From that point on the update is automatic.

When you say it does not allow you to maintain an archive it does not do so automatically, rather it requires some responsibility on the part of the user. There are "archives" of previous versions in the Time Machine backup and there is nothing to prevent the user from "archiving" a copy of the app before installing an update.

I'm saying: Some installers sometimes do the wrong thing.
You're saying: No, no. Sometimes the installer does the right thing.

I can't know, until I run one, what any particular installer is going to do. The safe course of action is to assume the worst, and take defensive measures.

Which, come to think about it, is what this thread is about. Sometimes, backing up a running system works. Sometimes it doesn't. I don't want my backups to work only sometimes.

I guess the first part of your post is about one thing, the second one is about the other. I tried to be conservative but since you seem to misunderstand my position, I will make it more strict. I don't know of any instances when the installer did the wrong thing. If you do, please tell us so that we would be more cautious in dealing with updates.

I've given examples where installers do the wrong thing.

I think the heart of your problem is revealed when you refer to "the installer". There are many installers, for many products, from many vendors. Some vendors are more conscientious than others, some are less so.

Apple is among the more conscientious vendors, but as I've pointed out even the App Store will sometimes be overly aggressive about trying to update files that are not on the boot volume. Since I want my backup volume to reflect what was on disk before the update, that's unsatisfactory. Never mind why it sometimes does that; the point is, it sometimes does do that.

Even if you focus on just Apple's Installer.app, you have to keep in mind that even that app doesn't always behave the same. Any vendor can use the PackageMaker.app (typically installed as part of Xcode, in <xcode_directory>/Applications/Utilities) to create an installer package. An installer package can be thought of as a "document" for Installer.app. During its creation, the vendor can customize the installation process in many ways. (Examples: there are checkboxes for whether the payload can be installed on only the boot volume, whether the user has to agree to a EULA before installation, whether an admin password is required, whether a prior version has to be present, etc., and options for minimimum OS, hardware requirements, which components to install by default, where they are installed by default and whether the user can customize the installation location, and many others.)

In case the pre-defined options are insufficient, the package can also include scripts to be run pre-flight, pre-install, and post-install. The most common use for install scripts is to do things that cannot be described as "install these files/folders", but in fact those scripts can do pretty much anything. For example, they can change permissions on existing items (Remember when all of a sudden "group:everyone deny delete" ACLs suddenly appeared on major folders?), they can change visibility (Like, making /Users/*/Library invisible.), they can add users and groups (the Xcode installer creates the _developer group), edit package receipts in an attempt to make Repair Permissions spout fewer spurious messages (I sure wish they'd eventually get that right), verify package signatures, delete incompatible software, edit your shell startup scripts, and on and on.

The sheer flexibility built into PackageMaker.app means that, even for packages installed by Installer.app, there is no single behavior that describes what "the installer" must always do. It does whatever that particular package tells it to do.

And Installer.app is far from the only installer out there. The VISE installer is rarer than it used to be, but still crops up from time to time, and vendors are free to use others or roll their own. As I recall, many of the "software bundles" use their own installer. Adobe Flash Player uses its own custom installer. (Other Adobe products may, too, but that's the only one I have installed.)


You want me to "tell [us] so that we would be more cautious in dealing with updates", but I've already given you something very simple that is completely safe for updates and installs: just unmount your backup volume after you finish backing up to it.

You should be doing that anyway. Your backup volume only needs to be mounted while you're backing up to it or restoring from it. At all other times, prudence dictates that it should be unmounted, for its own protection. It's not just installers and updaters that may grab it at inappropriate times: "open with..." will suggest applications on the backup, and some applications try to update files on their volume. Once you run an application from the backup volume, LaunchServices thinks that's the preferred copy. Spotlight will find items on all mounted volumes, so documents you find via Spotlight may save changes in a place where they'll be lost the next time you back up. Merely touching an alias on the backup may cause the alias to auto-update.

In short, you don't want to leave your backup mounted anyway. Why postpone the unmount? (I also turn it off, so it won't remount automatically on the next restart.)

Thanks...much appreciated!

Thanks! A good advice. Will follow immediately.

I use ditto to clone boot media to other drives. you can also just install to something it's "happy" with, and then ccc it over to say, APS.

I have tried to dig into the installer and find and override the stupid architecture and partition table types, without success.