An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Topic Options
#42027 - 10/02/16 01:21 PM AppleFS, fragmentation, and SSDs
joemikeb Offline

Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
Originally Posted By: jchuzi
As I understand it, directory fragmentation is not nearly as important as corruption so don't assume that a score of 10 means that you don't have a problem.

Indeed with Apple's forthcoming new file system, files and directories will — by design — be extremely fragmented. Directory and file fragmentation are significant factors effecting the time required to write or read a file on rotating media (a.k.a. Spinning rust, or hard drives.) Neither is much of an issue with Solid State or Flash Media drives and any existing issue is primarily a function of file management systems and data structures carefully optimized for rotating media. Corruption, on the other hand, means the directory or file is physically damaged. Directories (volume structures) can be recovered. If file damage is a result of directory damage it may or may not be recoverable (remember when cross-linked files were a frequent occurrence?). If the file content is damaged, hopefully you have a good reliable and current backup.


Edit: Originally an off-topic reply to Re: -43 Error Code, this post and the followups were moved to a separate thread.


Edited by cyn (10/06/16 10:23 AM)
Edit Reason: Changed subject line of relocated branch.
_________________________
joemikeb • moderator

Top
#42032 - 10/03/16 03:49 AM Re: AppleFS, fragmentation, and SSDs [Re: joemikeb]
Pendragon Offline


Registered: 08/04/09
Loc: Georgetown, Texas, USA
Originally Posted By: joemikeb
Originally Posted By: jchuzi
As I understand it, directory fragmentation is not nearly as important as corruption so don't assume that a score of 10 means that you don't have a problem.

Indeed with Apple's forthcoming new file system, files and directories will — by design — be extremely fragmented. Directory and file fragmentation are significant factors effecting the time required to write or read a file on rotating media (a.k.a. Spinning rust, or hard drives.) Neither is much of an issue with Solid State or Flash Media drives and any existing issue is primarily a function of file management systems and data structures carefully optimized for rotating media. Corruption, on the other hand, means the directory or file is physically damaged. Directories (volume structures) can be recovered. If file damage is a result of directory damage it may or may not be recoverable (remember when cross-linked files were a frequent occurrence?). If the file content is damaged, hopefully you have a good reliable and current backup.


I suppose a new file system is inevitable and will, for many, have significant benefits. Though (even) if pressed, I could not make a compelling argument for such. Perhaps if I knew what I was talking about…

While the pitfalls you note indeed got my attention, I’m sure (i.e., hope) Apple is diligently working to mitigate any/all dire consequences. I can deal, reluctantly, with getting new hardware/SSDs, but not loss of data.

For sure I’ll be watching the progress(?) of this matter and begin doing my homework.
_________________________
Harv
27" i7 iMac (10.13.6), iPhone Xs Max (12.1)

Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42035 - 10/03/16 06:53 AM Re: AppleFS, fragmentation, and SSDs [Re: Pendragon]
joemikeb Offline

Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
Originally Posted By: Pendragon
I suppose a new file system is inevitable and will, for many, have significant benefits. Though (even) if pressed, I could not make a compelling argument for such. Perhaps if I knew what I was talking about…

While the pitfalls you note indeed got my attention, I’m sure (i.e., hope) Apple is diligently working to mitigate any/all dire consequences. I can deal, reluctantly, with getting new hardware/SSDs, but not loss of data.

For sure I’ll be watching the progress(?) of this matter and begin doing my homework.
The existing HFS+ is not only relatively antiquated among modern file systems it is predicated minimizing the time lag created when the drive heads on rotating media drives have to mechanically move to another locations on the drive and wait for the data to revolve under the head. There is no mechanical movement in SSD/Flash drives so volume or file optimization produces no detectable benefit. On the other had by randomly scattering file segments throughout the media Apple's new file system will be far more secure. There is no increased risk of lost or damaged data, but a dramatically decreased risk of data theft.
_________________________
joemikeb • moderator

Top
#42045 - 10/03/16 06:09 PM Re: AppleFS, fragmentation, and SSDs [Re: Pendragon]
tacit Offline


Registered: 08/03/09
Loc: Portland, Oregon, USA
Originally Posted By: Pendragon
I suppose a new file system is inevitable and will, for many, have significant benefits. Though (even) if pressed, I could not make a compelling argument for such. Perhaps if I knew what I was talking about…


AppleFS has a whole bunch of benefits that are compelling to folks with modern computers:

1. It's optimized for Flash and other solid state storage devices.

2. It allows you to take a "snapshot" of your hard drive, and go back to that snapshot if something goes wrong.

3. It allows you to change partition sizes of partitioned drives on the fly without running software like Disk Utility.

4. It allows you to "clone" a file, which is kind of like a way to copy a file instantly. (The clone and the original share the same disk space, so the clone updates automatically when the original does.)

5. AppleFS allows filesystem-level file and disk encryption.

6. It is far more reliable and robust. If your computer is turned off in the middle of saving a file or making updates to the directory, the directory won't be corrupted.

7. It allows far larger hard drives, and stores files more efficiently on large drives. For example, if you have a 6 TB hard drive, the minimum block size on HFS+ will be about 150KB. This wastes space; a file that contains only one byte still takes up 150KB on disk. If you have many small files, you lose a lot of disk space.
_________________________
Photo gallery, all about me, and more: www.xeromag.com/franklin.html

Top
#42046 - 10/03/16 06:18 PM Re: AppleFS, fragmentation, and SSDs [Re: tacit]
artie505 Online


Registered: 08/04/09
Originally Posted By: tacit
2. It allows you to take a "snapshot" of your hard drive, and go back to that snapshot if something goes wrong.

How does that differ from Time Machine?
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42047 - 10/03/16 10:41 PM Re: AppleFS, fragmentation, and SSDs [Re: artie505]
tacit Offline


Registered: 08/03/09
Loc: Portland, Oregon, USA
Time Machine saves a backup copy of your files on a separate drive. Snapshot takes snapshots of your hard drive on that drive; you don't need a second drive.
_________________________
Photo gallery, all about me, and more: www.xeromag.com/franklin.html

Top
#42048 - 10/04/16 12:31 AM Re: AppleFS, fragmentation, and SSDs [Re: tacit]
artie505 Online


Registered: 08/04/09
Thanks.

That sounds like an extremely useful feature...one that will likely induce me to upgrade when it becomes available.

Edit: "3. It allows you to change partition sizes of partitioned drives on the fly without running software like Disk Utility."

That's potentially useful, although it's hardly a must have.


Edited by artie505 (10/04/16 02:40 AM)
Edit Reason: More
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42049 - 10/04/16 04:56 AM Re: AppleFS, fragmentation, and SSDs [Re: tacit]
grelber Offline


Registered: 08/05/09
Loc: North of 49th ||
Originally Posted By: tacit
Time Machine saves a backup copy of your files on a separate drive. Snapshot takes snapshots of your hard drive on that drive; you don't need a second drive.

Hah?! confused
And if that drive fails, then what? I thought the whole point of a backup on a second drive is for just such occurrences. A "snapshot" on a failed drive cannot be accessible.
Please explain.

Top
#42051 - 10/04/16 09:42 AM Re: AppleFS, fragmentation, and SSDs [Re: grelber]
artie505 Online


Registered: 08/04/09
Originally Posted By: grelber
Originally Posted By: tacit
Time Machine saves a backup copy of your files on a separate drive. Snapshot takes snapshots of your hard drive on that drive; you don't need a second drive.

Hah?! confused
And if that drive fails, then what? I thought the whole point of a backup on a second drive is for just such occurrences. A "snapshot" on a failed drive cannot be accessible.
Please explain.

As I see it, the "snapshots" aren't meant to be backups other than ephemerally, rather they will give those of us who don't need or even want Time Machine the immense benefit of having an on-board means to revert from an unhappy update/grade without it.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42065 - 10/05/16 08:25 PM Re: AppleFS, fragmentation, and SSDs [Re: grelber]
tacit Offline


Registered: 08/03/09
Loc: Portland, Oregon, USA
Originally Posted By: grelber

Hah?! confused
And if that drive fails, then what? I thought the whole point of a backup on a second drive is for just such occurrences. A "snapshot" on a failed drive cannot be accessible.
Please explain.


A snapshot isn't intended to be a backup. It's intended to be a safe rollback mechanism, in case you do something like install a software or system update that goes haywire, or an errant program crashes and corrupts valuable files (*ahem* Microsoft Word *ahem*), or for some other reason you find yourself facing an "oh my God no!" moment and need to roll back.
_________________________
Photo gallery, all about me, and more: www.xeromag.com/franklin.html

Top
#42066 - 10/06/16 02:42 AM Re: AppleFS, fragmentation, and SSDs [Re: tacit]
grelber Offline


Registered: 08/05/09
Loc: North of 49th ||
Originally Posted By: tacit
Originally Posted By: grelber

Hah?! confused
And if that drive fails, then what? I thought the whole point of a backup on a second drive is for just such occurrences. A "snapshot" on a failed drive cannot be accessible.
Please explain.


A snapshot isn't intended to be a backup. It's intended to be a safe rollback mechanism, in case you do something like install a software or system update that goes haywire, or an errant program crashes and corrupts valuable files (*ahem* Microsoft Word *ahem*), or for some other reason you find yourself facing an "oh my God no!" moment and need to roll back.

How is that "rollback mechanism" not the equivalent of a backup?! Why wouldn't Time Machine capability be just as good? And how might it work following drive failure?

Top
#42067 - 10/06/16 02:53 AM Re: AppleFS, fragmentation, and SSDs [Re: grelber]
ryck Offline


Registered: 08/04/09
Loc: Okanagan Valley
Originally Posted By: grelber

How is that "rollback mechanism" not the equivalent of a backup?! Why wouldn't Time Machine capability be just as good? And how might it work following drive failure?

I've been following this and wonder if the confusion is in the earlier statement.

Originally Posted By: tacit
Snapshot takes snapshots of your hard drive on that drive; you don't need a second drive.

I didn't take this to mean you don't need a second drive to do a backup. I thought it meant you don't need a second drive to take the snapshots….while a drive is required to do the equivalent thing with Time Machine.


Edited by ryck (10/06/16 02:55 AM)
_________________________
ryck

iMac (Retina 5K, 27", 2017), 3.4 GHz Intel Core i5, 8GB RAM, 2400 MHz DDR4
OS High Sierra 10.13.6
Canon MX710 Printer
Epson Perfection V500 Photo Scanner
Time Machine on 320GB OWC Mercury OTG Pro
Carbon Copy Clone on 500GB OWC Mercury OTG Pro

Top
#42068 - 10/06/16 03:13 AM Re: AppleFS, fragmentation, and SSDs [Re: grelber]
artie505 Online


Registered: 08/04/09
Like ryck said. And the snapshot feature is a backup, but it's useless if your drive fails; it's only a one-time, local, precautionary feature as per tacit.

Time Machine is specifically a backup mechanism, but as well as benefitting users who don't use TM(*), the snapshot feature may also benefit TM users by enabling them to avoid unscheduled backups.

(*) I don't use TM because I'd either have to keep a second drive permanently cabled to my MacBook Pro...a deal breaker, or spring for a Time Capsule that would not be worth the expenditure.


Edited by artie505 (10/06/16 10:06 AM)
Edit Reason: Expand & clarify
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42070 - 10/06/16 05:41 AM Re: AppleFS, fragmentation, and SSDs [Re: artie505]
grelber Offline


Registered: 08/05/09
Loc: North of 49th ||
Thanks, artie. That clears it up.

Top
#42260 - 10/24/16 08:07 AM Re: AppleFS, fragmentation, and SSDs [Re: grelber]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
Originally Posted By: grelber
Originally Posted By: tacit
Time Machine saves a backup copy of your files on a separate drive. Snapshot takes snapshots of your hard drive on that drive; you don't need a second drive.

Hah?! confused
And if that drive fails, then what? I thought the whole point of a backup on a second drive is for just such occurrences. A "snapshot" on a failed drive cannot be accessible.
Please explain.

Snapshots are a tool for rapid recovery and minimizing downtime, and have nothing to do with backups.

"OOPS I just deleted the wrong folder. That was a big folder too. I can...."
- spend 30 minutes restoring it from backup
- roll back one minute and it's all there right where it was before, in less than ten seconds.

All modern databases have some variation on START TRANSACTION, COMMIT, and ROLLBACK, because it's frighteningly easy to clobber a huge database in the blink of an eye with a typo. Modern filesystems have always had a passing similarity to databases, and the distinction is growing shorter all the time. This ugprade is long overdue.
_________________________
I work for the Department of Redundancy Department

Top
#42278 - 10/24/16 01:54 PM Re: AppleFS, fragmentation, and SSDs [Re: Virtual1]
artie505 Online


Registered: 08/04/09
Originally Posted By: Virtual1
Snapshots are a tool for rapid recovery and minimizing downtime, and have nothing to do with backups.

"OOPS I just deleted the wrong folder. That was a big folder too. I can...."
- spend 30 minutes restoring it from backup
- roll back one minute and it's all there right where it was before, in less than ten seconds.

That makes it sound like MacOS takes snapshots on its own, and regularly (if not constantly), while I ASSumed from tacit's post that they were user initiated when appropriate.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#42282 - 10/24/16 08:46 PM Re: AppleFS, fragmentation, and SSDs [Re: artie505]
tacit Offline


Registered: 08/03/09
Loc: Portland, Oregon, USA
It's not clear from Apple's documentation, but it appears that snapshots are created by the user. If that's the case, common troubleshooting wisdom in the future will probably include "always remember to take a snapshot before doing a system update or upgrade."
_________________________
Photo gallery, all about me, and more: www.xeromag.com/franklin.html

Top
#42284 - 10/24/16 10:03 PM Re: AppleFS, fragmentation, and SSDs [Re: tacit]
artie505 Online


Registered: 08/04/09
Thanks for the clarification; it sounds reasonable.

It would be nice if, rather than relying on users remembering to take snapshots prior to, at the least, system updates/grades, Apple built a "Take Snapshot" command into each installer.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top

Moderator:  alternaut, dkmarsh, joemikeb