An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Topic Options
#33015 - 02/09/15 04:09 AM time machine excluding flash drive
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
I have a flash drive here I'd like to include in my time machine backup, but it has automatically added it to the "do not back up" list and it seems I can't turn it on. I thought external storage was off by default but could always be turned back on?


Edited by cyn (03/16/15 03:13 PM)
Edit Reason: Topic moved from "Mac OS X Applications" to the "Mac OS X System" forum.
_________________________
I work for the Department of Redundancy Department

Top
#33016 - 02/09/15 04:42 AM Re: time machine excluding flash drive [Re: Virtual1]
jchuzi Online


Registered: 08/04/09
Loc: New York State
I'm sure that this is a stupid question, but is the drive formatted correctly?
_________________________
Jon

OS 10.14.6, iMac Retina 5K 27-inch, late 2014, 3.5 GHz Intel Core i5, 1 TB fusion drive, 16 GB RAM, Epson SureColor P600, Photoshop CC, Lightroom CC, MS Office 365

Top
#33017 - 02/09/15 05:24 AM Re: time machine excluding flash drive [Re: jchuzi]
grelber Offline


Registered: 08/05/09
Loc: North of 49th ||
Originally Posted By: jchuzi
I'm sure that this is a stupid question, but is the drive formatted correctly?

If it's a USB thumb drive, I'd bet it's formatted as MS-DOS (FAT32) – so that it can access PC and Mac systems equally.

Top
#33019 - 02/09/15 07:26 AM Re: time machine excluding flash drive [Re: Virtual1]
joemikeb Online

Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
I was curious and had a 128GB Lexar Flash Drive sitting around from a failed experiment and found
  • if the flash drive were formatted Mac OS Extended (HFS+) with a GUID partition table I could remove it from the Time Machine exclusion list.
  • the same flash drive formatted ExFAT with a Master Boot Record partition table could NOT be removed from the Time Machine exclusion list.
Since MBR and ExFAT are the de-facto standards for high capacity thumb drives and SD cards and my results verified what you are seeing, I didn't attempt to test all the possible permutations of partition maps and format options, but I would venture the key is whether or not the drive is formatted Mac OS Extended. Given Time Machine's reliance, or should I say dependence, on the HFS+ volume structures that would not be surprising.

In the name of full disclosure, I should have said I am running Yosemite (OS X 10.10.2) on a Late 2012 Mac mini (Macmini6,2).


Edited by joemikeb (02/09/15 07:34 AM)
Edit Reason: add configuration data
_________________________
joemikeb • moderator

Top
#33021 - 02/09/15 08:17 AM Re: time machine excluding flash drive [Re: jchuzi]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
Originally Posted By: jchuzi
I'm sure that this is a stupid question, but is the drive formatted correctly?


Yeah that's almost certainly the issue then. It's the Secretary drive for a club I belong to, to which I am this year's new secretary, so it has a number of important files on it that, to my knowledge, are not backed up anywhere. So rather than drag-n-drop backups, I was just going to let TM take care of it. I'll probably just convert it over for the time I'm using it, and convert it back when the time comes.

thx
_________________________
I work for the Department of Redundancy Department

Top
#33030 - 02/09/15 01:05 PM Re: time machine excluding flash drive [Re: joemikeb]
alternaut Offline

Moderator

Registered: 08/04/09
Originally Posted By: joemikeb
I ... had a 128GB Lexar Flash Drive ... and found the ... drive formatted ExFAT with a Master Boot Record partition table could NOT be removed from the Time Machine exclusion list.

I’ve got a couple of inexpensive 4T USB-3 Seagate hard drives. They come NTSF-formatted (MBR partition table) with an NTFS driver for Mac OS X, and are listed as ‘fully compatible’ with Time Machine. I wasn’t planning to use them with that format & driver for TM, but now I am wondering how these would play with the TM exclusion list. Any idea?
_________________________
alternaut moderator

Top
#33034 - 02/09/15 01:51 PM Re: time machine excluding flash drive [Re: alternaut]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
Originally Posted By: alternaut

I’ve got a couple of inexpensive 4T USB-3 Seagate hard drives. They come NTSF-formatted (MBR partition table) with an NTFS driver for Mac OS X, and are listed as ‘fully compatible’ with Time Machine. I wasn’t planning to use them with that format & driver for TM, but now I am wondering how these would play with the TM exclusion list. Any idea?


From what I can see now, the "exclusion list" is a combination of things you can't include, and the things you don't want to include. The former entries are put in there by the OS itself, and only show up while those devices are attached. (when I unplugged that FAT flash drive, it disappeared from the exclusion list)

So it should be compatible. I think those companies saying "time-machine compatible" is a gimmick. It's also USB compatible, XLS compatible, oxygen/nitrogen-atmosphere compatible, and a bunch of other "really, you don't say?" compatbilities.

After backing up and repartitioning the drive, I looked in time machine and it was again in the "exclusions" list, probably because it's a newly-discovered external drive. I was able to remove it from the exclusions list though, so it's being backed up now.
_________________________
I work for the Department of Redundancy Department

Top
#33074 - 02/12/15 09:53 AM Re: time machine excluding flash drive [Re: Virtual1]
ganbustein Offline


Registered: 08/04/09
Originally Posted By: Virtual1
I have a flash drive here I'd like to include in my time machine backup

Not a good idea.

You don't want to have TM back up a drive that will sometimes be unmounted. If TM runs while the drive is unmounted, it will record in its snapshot that the volume was not present. The next time TM runs after you mount the drive, it'll notice that the volume is new, and will back up the whole thing. Unmount it, let TM run, mount it again, and let TM run again, and TM will again decide that the entire volume is new and copy all files all over again. All this copying not only consumes time but also chews up precious space on your backup volume.

You're probably better off letting SuperDuper clone the flash drive to a sparse bundle disk image, and letting Time Machine back that up. A sparse bundle splits the disk image into 8MB chunks. When SD updates the image, chances are only some of those chunks will have changed, and TM will copy only those.

Be mindful that TM will not back up any disk image, sparse bundle or otherwise, while it's mounted. It treats a mounted disk image as "nothing has changed here". If you want TM to back it up, you need to occasionally (usually?) leave it unmounted.

Top
#33075 - 02/12/15 11:54 AM Re: time machine excluding flash drive [Re: ganbustein]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
Originally Posted By: ganbustein
Originally Posted By: Virtual1
I have a flash drive here I'd like to include in my time machine backup
You don't want to have TM back up a drive that will sometimes be unmounted. If TM runs while the drive is unmounted, it will record in its snapshot that the volume was not present. The next time TM runs after you mount the drive, it'll notice that the volume is new, and will back up the whole thing. Unmount it, let TM run, mount it again, and let TM run again, and TM will again decide that the entire volume is new and copy all files all over again. All this copying not only consumes time but also chews up precious space on your backup volume.



I would expect time machine to identify the volume based on its UUID, and be able to tell that it's the same volume, but I will take a look at it and see if it can tell the difference.

If it does indeed see it as a unique structure, it should show up numerous times in my /Volumes folder when I browse time machine, once for each copy it has made. If it does, I'd call it a whopper of a TM bug. But I'll take a look and report back.
_________________________
I work for the Department of Redundancy Department

Top
#33096 - 02/13/15 04:08 PM Re: time machine excluding flash drive [Re: Virtual1]
ganbustein Offline


Registered: 08/04/09
Originally Posted By: Virtual1
I would expect time machine to identify the volume based on its UUID, and be able to tell that it's the same volume, but I will take a look at it and see if it can tell the difference.

If it does indeed see it as a unique structure, it should show up numerous times in my /Volumes folder when I browse time machine, once for each copy it has made. If it does, I'd call it a whopper of a TM bug. But I'll take a look and report back.

It's not a question of Time Machine mis-identifying the drive. Let's give an example. Suppose you have told TM to back up your boot disk which for some bizarre reason you've left with the default name of "Macintosh HD". You also have a flash drive containing a single HFS+ volume that you've cleverly named "Flash Drive". TM is backing up your computer to a folder which might be named /Volumes/"Time Machine"/Backups.backupdb/"My Computer", but we'll just call it "Backups" for short.

At time Time_A, TM takes a snapshot and sees that both "Macintosh HD" and "Flash Drive" are present. It backs them up to:
  • Backups
    • Time_A
      • Flash Drive
      • Macintosh HD
So far so good.

Then you detach the flash drive, and at time Time_B TM takes another snapshot. This time it sees only "Macintosh HD", so it updates the backup to:
  • Backups
    • Time_A
      • Flash Drive
      • Macintosh HD
    • Time_B
      • Macintosh HD
Notice that TM has dutifully record the absence at Time_B of the flash drive.

Now you re-mount the flash drive, and at time Time_C TM backs up again. You now have:
  • Backups
    • Time_A
      • Flash Drive
      • Macintosh HD
    • Time_B
      • Macintosh HD
    • Time_C
      • Flash Drive
      • Macintosh HD

Whether TM recognizes that the "Flash Drive" that was present at time Time_A is the same drive as the one that was present at time Time_C is irrelevant. What it does recognize is that every single one of its files that was present at time Time_C was absent at time Time_B, and must therefore be treated as a brand new file. It's copying rule is simple: it copies everything that has changed since the last good backup, and the last good backup was at time Time_B.

It would be nice if Apple amended the rule to say: copy everything that has changed since the last good backup of this source volume, but they haven't done that. (Yet.) This is not mere carelessness. There are technical problems to be overcome, having to do with the extended attributes that TM attaches to the file. It doesn't expect gaps in the timeline for any particular file.


As for looking in /Volumes while in the Time Machine view, that won't show you anything useful, because:
  1. When in the TM view, TM is showing you what was present at a particular time. Even if it mounted "Flash Drive" as different mount points, it would be mounting them only one of them at a time, and at whatever time you looked the most you could hope to see would be the single mount that was in effect at that time.
  2. /Volumes is invisible
  3. Even if you make it visible, it's empty in the backup. TM backs up the folder itself, but nothing inside.
  4. When in TM view, you can see a list of mounted volumes in the sidebar or wherever you normally see mounted volumes, but TM isn't getting that list from ./Volumes. At Time_A, for example, it sees that Backups/Time_A contains two items: "Flash Drive" and "Macintosh HD". Those are the currently mounted disks as of that snapshot.

Top
#33153 - 02/19/15 05:10 AM Re: time machine excluding flash drive [Re: ganbustein]
Ira L Online


Registered: 08/13/09
Loc: California
Good explanation. Nicely done! smirk
_________________________
On a Mac since 1984.
Currently: 27" iMacs, Macbook Air, macOS 10.14.x,; iPhones, iPods and iPads galore!

Top
#33161 - 02/19/15 03:10 PM Re: time machine excluding flash drive [Re: ganbustein]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
a folder disappearing is different than a volume disappearing. Time machine tracks volumes by their UUID. (GUID?) So it knows quite well that you've plugged in a drive that has been gone awhile and is now back, and will NOT recopy the entire flash drive.

you can see this by unplugging a drive that's not excluded, and let TM run a few backups. If you browse to that folder on the TM drive, you'll see it there, in the most recent backups, as though it were still plugged in. As far as TM is concerned, it's still there, and nothing's changed on it until it determines otherwise. (when you next plug it back in)

To browse a disconnected drive in TM: open TM in finder, and opt-click or otherwise navigate all the way back to your computer, where you can see your hard drive listed, along with any other attached storage. (whether or not it's excluded)

Now click back ONCE. All attached volumes that are excluded should disappear AND you should see any volumes that weren't excluded, but are not currently present, pop into view. I see my club flash drive there. And it's back behind me sitting on a shelf, definitely not plugged in wink
_________________________
I work for the Department of Redundancy Department

Top

Moderator:  alternaut, dkmarsh, joemikeb