An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Topic Options
#45600 - 07/07/17 06:46 PM SRM is gone. Long live SRM!
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
Well the comments at the top of the script basically describe the issue but I'll repeat it here anyway.

I was asked to securely remove some images off an SD card following a shoot, and I was quite shocked to find that my recent upgrade to Sierra had removed the option to securely empty the trash. (lots of "where did it go? I could have sworn it was here, and here, and here...?) It turns out they didn't just get rid of the GUI access, they got rid of the whole command line tool too: srf - aka Secure RM.

Digging a bit I found that Apple had decided to remove the feature since they thought everyone ought to be using filevault and SRM was no longer necessary. (translation: SSDs broke it and we don't feel like fixing it, so we're fixing the feature by removing the feature")

This answer is totally irrelevant where external storage of any kind is concerned.

Now I realize that overwriting a file on a wear-leveled storage isn't going to be AS effective, but it certainly raises the bar for recovery. (and things like 7x overwrite start to raise concerns about cell write count durability), but I need it for this SD card, and I'm sure others need it for their external storage in general, so here we go...


Code:
#!/bin/bash

vers=2017.07.07.A

# Apple foresaw a problem with SRF (Secure Remove) - since everyone's boot drive is
# now an SSD, and since SRF wasn't written with SSDs in mind and won't work on them,
# APPLE REMOVED THE FEATURE.  That's right, they fixed it by removing it.

# Apple's official response to this outcry is that everyone is using SSDs and
# everyone should also be using filevault2, so just turn that on, since it doesn't
#  matter if files aren't securely erased since if you don't have the key you can't
# get at the data anyway.

# That's a fair (if not rather PUSHY) response, but doesn't address what happens
# when you need to secure-remove a file on say, an external spinning drive.  Or in
# my case, SD card.

# Now most flash memory (including SSDs and SD cards) will implement "wear leveling",
# and that's what's causing the issue here.  A. if you do say a 7x overwrite on cells
# before freeing them you do wear them out a bit faster, and B. the wear leveling
# software will probably just move the block on you when you try this so the data
# won't actually end up getting overwritten.  The data will still be there, but will
#  be a lot harder to get at.

# My response to A and B is simple.  Do a 1-pass erase on the file, then remove it.
# If it's a spinner, at least it's either zero'd or randomized once, and that will
# secure it from all but the best forensics software and hardware forensics.  If
# it's an SSD or other flash-based media, the wear leveler will likely move the
# cell, and when it does that, it will mark the True Cell as zero'd regardless of
# what its state is, so if you try to read the True Cell (again with anything except
#  expensive forensics software or  hardware) you will get all zeros anyway.

# The end result is the same either way.  This won't protect your privacy from
# people with a grudge and deep pockets, but otherwise will continue to work
# acceptably well, instead of just becoming 100% broken.

# This program is less complex than the SRF it replaces.  It doesn't support the "-v"
# switch and does not confirm.


# switch for live or dry-run, in case you're either testing or don't trust it to behave
live=1

# temp file used to catalog folders.  it too will get secure erased when we're done
tempfile="$TMPDIR/srm.tmp"
 
# sub to wipe a single file - overwrites from Random once
wipefile () {
size=$(stat -f "%z" "$1")
((blocks=(size+511)/512))
# this isn't terribly efficient for large files, but stops us from getting too abusive with the writes and
# speeds things up a bit rather than doing it a byte at a time.
# 512 is the minimum block size.  most SSDs and SD cards use 32k block size, which is a multiple of 512 anyway
if [ "$live" == "1" ] ; then
  dd bs=512 count=$blocksif=/dev/random of="$1"
  rm "$1"
  echo "$1 has been randomized and removed"
else
  echo "CMD: dd bs=512 count=$blocks if=/dev/random of=\"$1\""
  echo "CMD: rm \"$1\""
  echo "$1 would have been randomized and removed"
fi
}

# sub to wipe a folder, recursively.  wipes contents, then removes the folder itself.
# DOES NOT change the name of the folder prior to removing it.  technically most of the folder NAME is recoverable.
wipefolder () {
local r f n i  # to allow recursion
r=$1
ls "$r" > "$tempfile"
wipetemp=1
IFS=$'\n'
IFS2=$IFS
n=0
while read x ; do
  f[n]="$r/$x"
  ((n++))
done < "$tempfile"
IFS=$FS2
for ((i=0;i<n;i++)) ; do
  if [ -f "${f[i]}" ] ; then
    wipefile "${f[i]}"
  else
    wipefolder "${fp[i]}"
  fi
done
if [ "$live" == "1" ] ; then
  rm -d "$r"
  echo "directory \"$r\" has been removed"
else
  echo "CMD: rm -d \"$r\""
  echo "directory \"$r\" would have been removed"
fi
}


# main program loop.  remove all specified files/folders.  does not like tilde (~)
while [ -n "$1" ] ; do
 if [ "${1:0:1}" == "-" ] ; then
    echo "ignoring switch \"$1\""
    continue
  fi
  if [ -f "$1" ] ; then
    wipefile "$1"
  elif [ -d "$1" ] ; then
    wipeolder "$1"
  else
    "OBJECT NOT FOUND TO WIPE: \"$1\""
  fi
  shift
done


if [ $wipetemp ] ; then
  wiptefile "$tempfile"
fi

echo "all specified files and folders processed"
_________________________
I work for the Department of Redundancy Department

Top
#45601 - 07/07/17 07:03 PM Re: SRM is gone. Long live SRM! [Re: Virtual1]
artie505 Online


Registered: 08/04/09
Thanks for posting your script, but I think it's incorrect in one aspect of its "necessity".

I can't put my finger on the thread, but we had one recently in which I believe it was reported that Disk Utility still offers secure delete with n overwrite options for connected external spinning rust drives. (Haven't got one to try.)

More: I think that relates to BOTH INternal and EXternal spinning rust.


Edited by artie505 (07/07/17 07:25 PM)
Edit Reason: More
_________________________
The new Great Equalizer is the SEND button.

Top
#45605 - 07/08/17 04:50 AM Re: SRM is gone. Long live SRM! [Re: artie505]
joemikeb Offline

Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
Artie is correct, at least as far as Disk Utility in MacOS 10.13 (High Sierra) is concerned.
_________________________
joemikeb • moderator

Top
#45640 - 07/10/17 09:59 AM Re: SRM is gone. Long live SRM! [Re: Virtual1]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
so the "supported method" to secure delete a file is to wipe the entire hard drive?

pardon me but I don't like that option very much wink
_________________________
I work for the Department of Redundancy Department

Top
#45650 - 07/10/17 12:55 PM Re: SRM is gone. Long live SRM! [Re: Virtual1]
artie505 Online


Registered: 08/04/09
Originally Posted By: Virtual1
so the "supported method" to secure delete a file is to wipe the entire hard drive?

I've got no idea if that's the case.

Have you checked?
_________________________
The new Great Equalizer is the SEND button.

Top
#45660 - 07/11/17 11:53 AM Re: SRM is gone. Long live SRM! [Re: artie505]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
_________________________
I work for the Department of Redundancy Department

Top
#49981 - 09/17/18 12:09 PM Re: SRM is gone. Long live SRM! [Re: Virtual1]
macrevival Offline


Registered: 09/17/18
I believe I have found two bugs:

Line 55:

Code:
  dd bs=512 count=$blocksif=/dev/random of="$1"


should be:

Code:
  dd bs=512 count=$blocks if=/dev/random of="$1"


Line 106:

Code:
      wipeolder "$1"


should be:

Code:
      wipefolder "$1"


Am I correct?

Top
#49982 - 09/17/18 01:43 PM Re: SRM is gone. Long live SRM! [Re: macrevival]
MacManiac Online
Moderator

Registered: 08/04/09
Loc: Paradise....on the central Ore...
[The_Duke_Speak] Well, that's pretty good proof-reading, Pilgrim.... [/The_Duke_Speak] grin
_________________________
Freedom is never free....thank a Service member today.

Top
#49992 - 09/18/18 12:54 PM Re: SRM is gone. Long live SRM! [Re: macrevival]
Virtual1 Offline


Registered: 08/04/09
Loc: Iowa
hmmm paste apparently somehow failed me?
_________________________
I work for the Department of Redundancy Department

Top
#50255 - 10/17/18 03:01 PM Re: SRM is gone. Long live SRM! [Re: Virtual1]
ganbustein Offline


Registered: 08/04/09
Originally Posted By: Virtual1
so the "supported method" to secure delete a file is to wipe the entire hard drive?

I'm not sure even that will work!

The reason secure delete doesn't work on flash drives is that it doesn't work. Period. There's no way for Apple to fix something that's unfixable.

Even overwriting the file won't delete the information. On a spinning disk each sector has a fixed physical location on the platter. Writing new information to that sector writes to the same physical location, replacing whatever was there. It's like editing a manuscript by erasing a sentence and writing a new sentence in its place.

On a flash drive, a sector does not have a fixed physical location. Writing new data to the "same" sector writes the new data to new physical location, and then an internal translation table on the chip gets updated to show the new physical location for the sector. The old physical location is not touched, and retains its original data. It's like editing a manuscript by writing a new sentence on a new blank page at the end, leaving the old sentence still there but updating a table of contents that tells the reader which order the sentences should be read in.

By and by, the chip (NOT the operating system) notices that the old physical location is not being used, and can be erased in preparation for a later rewrite. No off-chip software has any knowledge or control of when that erasure happens, nor even what physical location corresponds to what sector number. The erasure is postponed because it's very slow (much slower than either reading or writing), and can't erase a single sector anyway. Pushing the manuscript analogy further, it's like you can only erase an entire page at a time. To "erase" a single sentence, you have first copy all the still-in-use sentences on the page to somewhere else and then erase the entire page.

Top
#50266 - 10/18/18 03:59 PM Re: SRM is gone. Long live SRM! [Re: ganbustein]
ryck Online


Registered: 08/04/09
Loc: Okanagan Valley
Hey....nice to see you again.
_________________________
ryck

iMac (Retina 5K, 27", 2017), 3.4 GHz Intel Core i5, 8GB RAM, 2400 MHz DDR4
OS High Sierra 10.13.6
Canon MX712 Printer
Epson Perfection V500 Photo Scanner
Time Machine on 320GB OWC Mercury OTG Pro
Super Duper on 500GB OWC Mercury OTG Pro

Top

Moderator:  alternaut, dkmarsh, joemikeb