An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Previous Thread
Next Thread
Print Thread
SRM is gone. Long live SRM!
#45600 07/08/17 01:46 AM
Joined: Aug 2009
OP Offline

Joined: Aug 2009
Well the comments at the top of the script basically describe the issue but I'll repeat it here anyway.

I was asked to securely remove some images off an SD card following a shoot, and I was quite shocked to find that my recent upgrade to Sierra had removed the option to securely empty the trash. (lots of "where did it go? I could have sworn it was here, and here, and here...?) It turns out they didn't just get rid of the GUI access, they got rid of the whole command line tool too: srf - aka Secure RM.

Digging a bit I found that Apple had decided to remove the feature since they thought everyone ought to be using filevault and SRM was no longer necessary. (translation: SSDs broke it and we don't feel like fixing it, so we're fixing the feature by removing the feature")

This answer is totally irrelevant where external storage of any kind is concerned.

Now I realize that overwriting a file on a wear-leveled storage isn't going to be AS effective, but it certainly raises the bar for recovery. (and things like 7x overwrite start to raise concerns about cell write count durability), but I need it for this SD card, and I'm sure others need it for their external storage in general, so here we go...


Code:
#!/bin/bash

vers=2017.07.07.A

# Apple foresaw a problem with SRF (Secure Remove) - since everyone's boot drive is
# now an SSD, and since SRF wasn't written with SSDs in mind and won't work on them,
# APPLE REMOVED THE FEATURE.  That's right, they fixed it by removing it.

# Apple's official response to this outcry is that everyone is using SSDs and
# everyone should also be using filevault2, so just turn that on, since it doesn't
#  matter if files aren't securely erased since if you don't have the key you can't
# get at the data anyway.

# That's a fair (if not rather PUSHY) response, but doesn't address what happens
# when you need to secure-remove a file on say, an external spinning drive.  Or in
# my case, SD card.

# Now most flash memory (including SSDs and SD cards) will implement "wear leveling",
# and that's what's causing the issue here.  A. if you do say a 7x overwrite on cells
# before freeing them you do wear them out a bit faster, and B. the wear leveling
# software will probably just move the block on you when you try this so the data
# won't actually end up getting overwritten.  The data will still be there, but will
#  be a lot harder to get at.

# My response to A and B is simple.  Do a 1-pass erase on the file, then remove it.
# If it's a spinner, at least it's either zero'd or randomized once, and that will
# secure it from all but the best forensics software and hardware forensics.  If
# it's an SSD or other flash-based media, the wear leveler will likely move the
# cell, and when it does that, it will mark the True Cell as zero'd regardless of
# what its state is, so if you try to read the True Cell (again with anything except
#  expensive forensics software or  hardware) you will get all zeros anyway.

# The end result is the same either way.  This won't protect your privacy from
# people with a grudge and deep pockets, but otherwise will continue to work
# acceptably well, instead of just becoming 100% broken.

# This program is less complex than the SRF it replaces.  It doesn't support the "-v"
# switch and does not confirm.


# switch for live or dry-run, in case you're either testing or don't trust it to behave
live=1

# temp file used to catalog folders.  it too will get secure erased when we're done
tempfile="$TMPDIR/srm.tmp"
 
# sub to wipe a single file - overwrites from Random once
wipefile () {
size=$(stat -f "%z" "$1")
((blocks=(size+511)/512))
# this isn't terribly efficient for large files, but stops us from getting too abusive with the writes and
# speeds things up a bit rather than doing it a byte at a time.
# 512 is the minimum block size.  most SSDs and SD cards use 32k block size, which is a multiple of 512 anyway
if [ "$live" == "1" ] ; then
  dd bs=512 count=$blocksif=/dev/random of="$1"
  rm "$1"
  echo "$1 has been randomized and removed"
else
  echo "CMD: dd bs=512 count=$blocks if=/dev/random of=\"$1\""
  echo "CMD: rm \"$1\""
  echo "$1 would have been randomized and removed"
fi
}

# sub to wipe a folder, recursively.  wipes contents, then removes the folder itself.
# DOES NOT change the name of the folder prior to removing it.  technically most of the folder NAME is recoverable.
wipefolder () {
local r f n i  # to allow recursion
r=$1
ls "$r" > "$tempfile"
wipetemp=1
IFS=$'\n'
IFS2=$IFS
n=0
while read x ; do
  f[n]="$r/$x"
  ((n++))
done < "$tempfile"
IFS=$FS2
for ((i=0;i<n;i++)) ; do
  if [ -f "${f[i]}" ] ; then
    wipefile "${f[i]}"
  else
    wipefolder "${fp[i]}"
  fi
done
if [ "$live" == "1" ] ; then
  rm -d "$r"
  echo "directory \"$r\" has been removed"
else
  echo "CMD: rm -d \"$r\""
  echo "directory \"$r\" would have been removed"
fi
}


# main program loop.  remove all specified files/folders.  does not like tilde (~)
while [ -n "$1" ] ; do
 if [ "${1:0:1}" == "-" ] ; then
    echo "ignoring switch \"$1\""
    continue
  fi
  if [ -f "$1" ] ; then
    wipefile "$1"
  elif [ -d "$1" ] ; then
    wipeolder "$1"
  else
    "OBJECT NOT FOUND TO WIPE: \"$1\""
  fi
  shift
done


if [ $wipetemp ] ; then
  wiptefile "$tempfile"
fi

echo "all specified files and folders processed"


I work for the Department of Redundancy Department
Re: SRM is gone. Long live SRM!
Virtual1 #45601 07/08/17 02:03 AM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
Thanks for posting your script, but I think it's incorrect in one aspect of its "necessity".

I can't put my finger on the thread, but we had one recently in which I believe it was reported that Disk Utility still offers secure delete with n overwrite options for connected external spinning rust drives. (Haven't got one to try.)

More: I think that relates to BOTH INternal and EXternal spinning rust.

Last edited by artie505; 07/08/17 02:25 AM. Reason: More

The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: SRM is gone. Long live SRM!
artie505 #45605 07/08/17 11:50 AM
Joined: Aug 2009
Likes: 16
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 16
Artie is correct, at least as far as Disk Utility in MacOS 10.13 (High Sierra) is concerned.


If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein
Re: SRM is gone. Long live SRM!
Virtual1 #45640 07/10/17 04:59 PM
Joined: Aug 2009
OP Offline

Joined: Aug 2009
so the "supported method" to secure delete a file is to wipe the entire hard drive?

pardon me but I don't like that option very much wink


I work for the Department of Redundancy Department
Re: SRM is gone. Long live SRM!
Virtual1 #45650 07/10/17 07:55 PM
Joined: Aug 2009
Likes: 15
Online

Joined: Aug 2009
Likes: 15
Originally Posted By: Virtual1
so the "supported method" to secure delete a file is to wipe the entire hard drive?

I've got no idea if that's the case.

Have you checked?


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: SRM is gone. Long live SRM!
artie505 #45660 07/11/17 06:53 PM
Joined: Aug 2009
OP Offline

Joined: Aug 2009


I work for the Department of Redundancy Department
Re: SRM is gone. Long live SRM!
Virtual1 #49981 09/17/18 07:09 PM
Joined: Sep 2018
Offline

Joined: Sep 2018
I believe I have found two bugs:

Line 55:

Code:
  dd bs=512 count=$blocksif=/dev/random of="$1"


should be:

Code:
  dd bs=512 count=$blocks if=/dev/random of="$1"


Line 106:

Code:
      wipeolder "$1"


should be:

Code:
      wipefolder "$1"


Am I correct?

Re: SRM is gone. Long live SRM!
macrevival #49982 09/17/18 08:43 PM
Joined: Aug 2009
Likes: 5
Moderator
Offline
Moderator

Joined: Aug 2009
Likes: 5
[The_Duke_Speak] Well, that's pretty good proof-reading, Pilgrim.... [/The_Duke_Speak] grin


Freedom is never free....thank a Service member today.
Re: SRM is gone. Long live SRM!
macrevival #49992 09/18/18 07:54 PM
Joined: Aug 2009
OP Offline

Joined: Aug 2009
hmmm paste apparently somehow failed me?


I work for the Department of Redundancy Department
Re: SRM is gone. Long live SRM!
Virtual1 #50255 10/17/18 10:01 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
Originally Posted By: Virtual1
so the "supported method" to secure delete a file is to wipe the entire hard drive?

I'm not sure even that will work!

The reason secure delete doesn't work on flash drives is that it doesn't work. Period. There's no way for Apple to fix something that's unfixable.

Even overwriting the file won't delete the information. On a spinning disk each sector has a fixed physical location on the platter. Writing new information to that sector writes to the same physical location, replacing whatever was there. It's like editing a manuscript by erasing a sentence and writing a new sentence in its place.

On a flash drive, a sector does not have a fixed physical location. Writing new data to the "same" sector writes the new data to new physical location, and then an internal translation table on the chip gets updated to show the new physical location for the sector. The old physical location is not touched, and retains its original data. It's like editing a manuscript by writing a new sentence on a new blank page at the end, leaving the old sentence still there but updating a table of contents that tells the reader which order the sentences should be read in.

By and by, the chip (NOT the operating system) notices that the old physical location is not being used, and can be erased in preparation for a later rewrite. No off-chip software has any knowledge or control of when that erasure happens, nor even what physical location corresponds to what sector number. The erasure is postponed because it's very slow (much slower than either reading or writing), and can't erase a single sector anyway. Pushing the manuscript analogy further, it's like you can only erase an entire page at a time. To "erase" a single sentence, you have first copy all the still-in-use sentences on the page to somewhere else and then erase the entire page.

Re: SRM is gone. Long live SRM!
ganbustein #50266 10/18/18 10:59 PM
Joined: Aug 2009
Likes: 14
Offline

Joined: Aug 2009
Likes: 14
Hey....nice to see you again.


ryck

"What Were Once Vices Are Now Habits" The Doobie Brothers

iMac (Retina 5K, 27", 2020), 3.8 GHz 8 Core Intel Core i7, 8GB RAM, 2667 MHz DDR4
OS Ventura 13.6.3
Canon Pixma TR 8520 Printer
Epson Perfection V500 Photo Scanner c/w VueScan software
TM on 1TB LaCie USB-C

Moderated by  alternaut, dkmarsh, joemikeb 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 7.4.33 Page Time: 0.038s Queries: 36 (0.032s) Memory: 0.6300 MB (Peak: 0.7330 MB) Data Comp: Zlib Server Time: 2024-03-28 11:37:20 UTC
Valid HTML 5 and Valid CSS