An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Previous Thread
Next Thread
Print Thread
Deleting and Overwriting
#46820 11/10/17 05:13 PM
Joined: Aug 2009
MG2009 Offline OP
OP Offline

Joined: Aug 2009
Just wondering . . .


When I name a file (or folder or photo) the same as an existing file, I receive the message : Already exists. Would you like to replace?

When I select "replace", does this mean that the "old" file is immediately overwritten with the new, or is the "old" file simply moved/flagged to be overwritten when more space on the HD is required at a later time (i.e. similar to emptying the trash)?



Many thanks,
MG2009
Re: Deleting and Overwriting
MG2009 #46821 11/10/17 05:17 PM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
In my experience, the old file has invariably been overwritten when I've clicked on "Replace".

Sometimes I'm given the option to keep both, but I think it's only with apps.

Last edited by artie505; 11/10/17 05:54 PM. Reason: Clarify and expand

The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Deleting and Overwriting
MG2009 #46823 11/10/17 06:26 PM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
Technically what happens in Apple's HFS+ file system is:
  • The file directory is changed to point to the replacement file
  • That has the effect of deleting any references to the old file
  • The disk map is changed to mark the data segments used by the old file as "available" for reuse
There is no algorithm or rule that causes those newly available segments to be immediately overwritten so the data is still technically available to be recovered and there are a number utilities specifically designed to do just that. Systems intended to make data recovery more reliable often include backup copies of the directory structure so there is a "memory" of where the original file was located on the drive. However data recovery from replaced files is notoriously difficult, especially if the drive is in continuous active use because the data is frequently overwritten by chance alone. If you want the old data badly enough there are labs that can drill down through even overwritten magnetic impressions to recover data that has been overwritten, even several times overwritten. But you will need very deep pockets for that level of recovery.

Apparently, there are features included in Apple's new APFS that when fully implemented will allow recovery of replaced file content/versions down almost to the character level, but as far as I know those functionalities are still TBI (To Be Implemented).


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: Deleting and Overwriting
joemikeb #46830 11/11/17 01:23 PM
Joined: Aug 2009
MG2009 Offline OP
OP Offline

Joined: Aug 2009
Thanks, Joe:

Fortunately, I haven't lost anything I need to recover. I was just curious if the "space" of the old file is used up immediately with the new file overwriting it . . .

OR, as you explained, the two files still exist - except that the old one gets buried (and may or may not be written over in time).


Many thanks,
MG2009
Re: Deleting and Overwriting
MG2009 #46833 11/11/17 05:26 PM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
If you have 50 or so minutes to spare you might find this video presentation on HFS+ and APFS by OWC's engineering manager interesting. Among other things it covers your specific question and the differences in what happens between HFS+ and APFS.


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: Deleting and Overwriting
joemikeb #46835 11/11/17 10:57 PM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
Originally Posted By: joemikeb
Technically what happens in Apple's HFS+ file system is:
• The file directory is changed to point to the replacement file....

shocked blush

Had MG2009 asked about trashing an item, I'd have had replied more or less - mostly less - as you did, but without ever having given it any particular thought, I've always taken Apple's "overwrite its current contents" literally.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Deleting and Overwriting
artie505 #46837 11/12/17 11:36 PM
Joined: Aug 2009
MG2009 Offline OP
OP Offline

Joined: Aug 2009
So, said another way, when a file is "replaced" with a new one of the same name, the old one, in essence, is automatically "trashed" (i.e. moved to be overwritten as some later date) . . . bypassing the step of having to manually delete the old file by dragging (or right-cicking) to move it to the Trash Can.


grin

Re: Deleting and Overwriting
MG2009 #46838 11/13/17 12:03 AM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
So it seems, and if so, Apple's
Quote:
A file or folder with the same name already exists [at that location]. Replacing it will overwrite its current contents.
is technically incorrect.

Based on joemike's input, the pop-up should say "Replacing it will result in its immediate deletion."...the difference between the two results being that a file that's simply been deleted is (considerably?) more easily recoverable than one that's been overwritten.

And if that's not confusing already, it changes depending on whether you're dealing with a HDD or a SSD. grin


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Deleting and Overwriting
artie505 #46839 11/13/17 02:49 AM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
Originally Posted By: artie505
More: I just did some searching, and the references I found to recovering a replaced/overwritten file on a Mac all point to Time Machine, suggesting that joemike's response may be incorrect. confused

Given that recovering a previous version of any given file is a prime design goal of Time Machine, those recommendations are within the technical competence of virtually any user and are entirely dependent on tools provided free within MacOS, but they all assume the availability of a Time Machine backup. I was talking about the situation that too often arises where a file is replaced/changed/deleted and there there is NO Time Machine backup. Recovery from Time Machine is just another version of "overwriting" the file again and works the same way as the original replacement. A new file is created and the directory modified to point to the new (third) location of the target file and may or may not overwrite the physical location of either of the two (or more) previous versions.

I could go on at length about what happens when you modify a file, replace a file, the differences in what happens in HFS+ and APFS, and magnetic media (HDD) vs electronic (SSD) media but if you will take the time (some 50 minutes) to watch this video I referenced in my previous post all should become clear (or at least clearer grin )


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: Deleting and Overwriting
joemikeb #46840 11/13/17 02:53 AM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
We crossed, and you responded to a quote that I deleted while you were typing.

I deleted it, because further searching indicated that Time Machine is merely the best option, but data recovery software can also be used as well.

I guess "overwrite" in the sense we're discussing doesn't mean the same thing as "overwrite" in the secure delete sense. (It would be helpful if a distinction were made between the two.)

50 minutes will be awfully taxing, but I"ll give your link a try.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Deleting and Overwriting
joemikeb #46842 11/13/17 12:06 PM
Joined: Aug 2017
Offline

Joined: Aug 2017
Originally Posted By: joemikeb
If you have 50 or so minutes to spare you might find this video presentation on HFS+ and APFS by OWC's engineering manager interesting. Among other things it covers your specific question and the differences in what happens between HFS+ and APFS.

Very interesting lecture/session. Besides the shiny new features and promises, APFS comes with some serious warnings. Not just the part on Low level data recovery (0:38:45), but also the (lack of) speed (0:30:38) and the recommendations at the end (0:41:04). I had thought about converting all my volumes to APFS, but not anymore. So, thanks for that insight.

Last edited by Urquhart; 11/13/17 12:09 PM.
Re: Deleting and Overwriting
artie505 #46843 11/13/17 03:38 PM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
Originally Posted By: artie505
I guess "overwrite" in the sense we're discussing doesn't mean the same thing as "overwrite" in the secure delete sense. (It would be helpful if a distinction were made between the two.)

The problem is not a lack of accurate descriptive terms, it is so many of us use a term sloppily that it becomes common usage, and invariably creates the opportunity for misunderstanding.


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: Deleting and Overwriting
joemikeb #46846 11/14/17 07:00 AM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
Originally Posted By: joemikeb
Originally Posted By: artie505
I guess "overwrite" in the sense we're discussing doesn't mean the same thing as "overwrite" in the secure delete sense. (It would be helpful if a distinction were made between the two.)

The problem is not a lack of accurate descriptive terms, it is so many of us use a term sloppily that it becomes common usage, and invariably creates the opportunity for misunderstanding.

I'll take issue with your placing the blame on us, because we're really dealing with a nomenclature inconsistency on Apple's part.

In Snow Leopard's Disk Utility > Erase Free Space Options pane "erase" and "overwrite" are used interchangeably, and interchangeability is implied in the pop-up referenced in the OP.

Thinking about it, rather than erasing data (Nothing is ever actually erased from either a HDD or SSD in an OS context.) the OS actually loses it, so maybe "erase" should go by the boards in favor of "lose", the "Trash" should be renamed the "Lost & Found", and "overwrite" should be restricted to use when data is actually overwritten.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Deleting and Overwriting
artie505 #46847 11/14/17 01:27 PM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
When I say "us" it is intended as all inclusive and extends to the entire computer using community from those that do well to read their email to those who spend their careers in creating the hardware and software. The documentation for the last major project I worked on began with a 500 page dictionary clarifying exactly what was meant by words like "overwrite" In the context of this specific software project. (If 500 pages sounds excessive, one complete printed set of the documentation filled an 18 wheeler semi-trailer truck.)


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: Deleting and Overwriting
MG2009 #46853 11/14/17 08:44 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
I think I would say the most accurate concise description for that is it replaces the directory entry for the file.

First, the original file is RM'd. this may not delete the file, if there are additional hard links to it. RM lowers the link counter on the file, and if the counter hits zero, it
then marks the blocks of the file as available for reuse.

Then the replacement file's directory entry is moved from its current folder to the folder of the object it is replacing.

Terminal won't let you replace a folder with a file, it will move the file INTO the folder. Trying to replace a file with a folder will get you an error.

Quote:
When I select "replace", does this mean that the "old" file is immediately overwritten with the new, or is the "old" file simply moved/flagged to be overwritten when more space on the HD is required at a later time (i.e. similar to emptying the trash)?


CAT and DD could do an in-place-overwrite as you are considering, but that's pretty unusual. Normally when replacing, a file's directory entry is overwritten ("totally destroyed, immediately") and the blocks of the overwritten file are marked as available for reuse later when the file system finds them convenient to use. (which could be in a minute, a week, or four years) Blocks aren't necessarily reclaimed in order, and may get overwritten in segments as time goes on because files aren't always stored using one continuous strip of blocks. This makes deleted-file-recovery an uncertain and unpredictable affair. (see my other recent post regarding aliases for an explanation of how multiple hard links to a file can complicate deleting a file)

If you do a "secure delete" however, THAT will wipe the blocks to random values before freeing them. HOWEVER, if you've copied or edited the file, there's a very good chance there are many copies of older versions of the file scattered around the hard drive, NONE of which the secure delete will do anything to. Some may have already been overwritten completely, some may have had parts of them overwritten, and some may be fully intact, years later. If you have a file that you want to be able to edit AND secure-delete later, your only good alternative is to store the file on a (non-growable) disk image, and secure delete the dmg when you need to make the file(s) go away for certain.


I work for the Department of Redundancy Department
Re: Deleting and Overwriting
joemikeb #46866 11/17/17 06:47 AM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
Originally Posted By: joemikeb
When I say "us" it is intended as all inclusive and extends to the entire computer using community from those that do well to read their email to those who spend their careers in creating the hardware and software. The documentation for the last major project I worked on began with a 500 page dictionary clarifying exactly what was meant by words like "overwrite" In the context of this specific software project. (If 500 pages sounds excessive, one complete printed set of the documentation filled an 18 wheeler semi-trailer truck.)

That must have been one heck of an espresso machine! (On the positive side, though, the quick-start guide was probably only a 5x7 index card. tongue )

Seriously, I can't even conceive of a project that big, and I wonder how many people you'd have to pack into a room to have complete understanding in one place.

"...what was meant by words like "overwrite" In the context of this specific software project" is critical to our discussion, because Apple's use of the word "overwrite" indicates that they've lost sight of the concept of differing contexts.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Moderated by  alternaut, dkmarsh, joemikeb 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 7.4.33 Page Time: 0.027s Queries: 46 (0.018s) Memory: 0.6610 MB (Peak: 0.7826 MB) Data Comp: Zlib Server Time: 2024-03-28 08:32:04 UTC
Valid HTML 5 and Valid CSS