An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Previous Thread
Next Thread
Print Thread
Bad blocks question
#31750 11/13/14 08:29 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
My daughter took her MacBook to the Genius Bar, Apple's diagnostic tool immediately spit out an I/O error, and the Genius sold her a Seagate external (best for cross-platform use, he said) and assisted her in saving her (un-backed-up shocked ) data. (It didn't occur to him to advise her to back up ~/Library. tongue )

Unfortunately, she was unable to save all her data, because numerous good files lived in folders that also housed bad files, and once the Genius's software choked on the first bad file in a folder she was reduced to dealing with the rest of its contents with drag & drop...one file at a time to avoid choking on bad ones that weren't absolutely necessary, and she just plain ran out of time, because the store was closing.

So, my question is this: Since Apple hasn't got one, I assume that it's impossible to develop a cloning program that can detect a bad block, leapfrog past the file in which it resides, and keep chugging away until it detects the next bad block..., and I'm wondering why not?

Thanks.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31755 11/13/14 05:22 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
I've actually written several wink

I'd also invite you look at ddrescue if you're so inclined. (not bundled, but is free)

quick n dirty is to just use ditto. it skips bad files, unlike say, Finder. (bundled with os x)


I work for the Department of Redundancy Department
Re: Bad blocks question
artie505 #31756 11/13/14 05:25 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
klone.command

Code:
#!/bin/bash

# need to auto save klone log to destination drive

##  need to add catch for console:
##   device/channel is not attached
##   media is not present"

vers="2012.02.02.A"
rv="306"  # rsync version
rf="-aHAXx --delete --ignore-errors"
debug=0

# change to folder script lives in
cd "${0%/*}"


####################################################################################################
###  SUBS
####################################################################################################


# two digit numbers
twodigits () {
if [ $1 -lt 10 ] ; then
  echo -n "0$1"
else
  echo -n $1
fi
}


# timestamp format
digitaltime () {
totalseconds=$1
totalminutes=$((totalseconds/60))
totalhours=$((totalminutes/60))
totaldays=$((totalhours/24))
numseconds=$((totalseconds-(totalminutes*60)))
numminutes=$((totalminutes-(totalhours*60)))
numhours=$((totalhours-(totaldays*24)))
tfs="$(twodigits $totaldays):$(twodigits $numhours):$(twodigits $numminutes):$(twodigits $numseconds)"
#tfs="$(twodigits $numhours):$(twodigits $numminutes):$(twodigits $numseconds)"
echo $tfs
}


# verify/install support files
verifyulb () {
local d1 d2 f f1 f2
f=$1
d1="/usr/local/bin"
d2="/Volumes/Service Data/Tools/usrlocalbin"
f1="$d1/$f"
f2="$d2/$f"
if ! [ -d "$d1" ] ; then
  mkdir -p /usr/local/bin/ ; rc=$?
  if [ $rc != 0 ] ; then
    echo "Return code $rc when trying to create directory /usr/local/bin/"
    exit
  fi
fi
if [ -f "$f1" ] ; then
  return
fi
if ! [ -f "$f2" ] ; then
  echo "Cannot find support file to copy: $f2"
  exit
fi
cp "$f2" "$f1" ; rc=$?
if [ $rc != 0 ] ; then
  echo "Return code $rc when trying to copy \"$f2\" to \"$f1\""
  exit
fi
}

####################################################################################################
###  THREADS
####################################################################################################

# klone will run a copy of itself as a background process to print progress indication during the copy,
# this block is the progress instance.  When klone is done this instance will exit automatically
started=
if [ "$1" == "track" ] ; then
  # "$0" "track" "$trgp" $smu $tmt $tmu &
  # we are the scoreboard instance
  trap exit sigterm  # we will quit if told to do so with a "kill -s sigterm ourprocessid", otherwise we quit when our PPID goes away
  trgp=$2  # destination disk path ("/Volumes/Macintosh HD 1")
  smu=$3   # source MB total
  tmt=$4   # target MB total (not used)
  tmu=$5   # goal MB used, assuming 100% of source is copied
  sku=$((smu*1024))
  tkt=$((tmt*1024))
  tku=$((tmu*1024))
  pmu=$tmu # start previous mb used on target to current mb used on target
  pku=$((pmu*1024))
  line=$(df "$trgp" | grep " $trgp$" | tr -s " ")
  bmu=$(($(echo $line | cut -d " " -f 3)/2/1024))  # beginning mb used on target needed to calculate gmr
  bku=$((bmu*1024))
  et=0
  i=10  # interval between samples in seconds
  while [ -n "$(ps -x | grep "$PPID " | grep -v "grep")" ] ; do  # exit loop when main klone quits (finishes)
    if [ -n "$(tail -n 10 /var/log/system.log | grep 'is not attached')" ] ; then  #  source or target disappeared
      # copy has hung and is probably porking the system.log file with errors and making ASL hammer the system
      # make it stop
      echo "DRIVE DISAPPEARED - killing rsync/ditto" | wall
      killall rsync_${rv}_intel &> /dev/null
      killall rsync_${rv}_ppc &> /dev/null
      killall ditto &> /dev/null
    fi
    et=$((et+i))
    line=$(df "$trgp" | grep " $trgp$" | tr -s " ")
    cmu=$(($(echo $line | cut -d " " -f 3)/2/1024))  # current mb used
    cku=$(($(echo $line | cut -d " " -f 3)/2))  # current kb used
    cmr=$(((cmu-pmu)/$i))  # current mb/sec copy rate based on last two samples
    ckr=$(((cku-pku)/$i))  # current kb/sec copy rate based on last two samples
    gmr=$(((cmu-bmu)/$et))  # going mb/sec copy rate since copy start
    gkr=$(((cku-bku)/$et))  # going kb/sec copy rate since copy start
    cml=$((smu+tmu-cmu))   # mb left assuming no overwrite
    ckl=$((sku+tku-cku))   # kb left assuming no overwrite
    if [ $ckr -lt 1 ] ; then  # copy stalled
      if [ $started ] ; then  # only bonk if we've started
        ctogo="(--:--:--:--)"
        ctogo="("$'\e[1;93;101m'" STALLED "$'\e[0m'")"
        echo -n $'\a'  # i/o error beep
      else
        ctogo="(STARTING)"
      fi
      #echo
      #echo "line=$line"
      #echo "cmu=$cmu, pmu=$pmu"
    else
      started=1
#     if [ $cmr == 0 ] ; then  # ckr > 0 but cmr = 0, avoid div by zero
#       csl=0
#     else
#       csl=$((cml/cmr))  # seconds left at current rate
#     fi
      csl=$((ckl/ckr))  # seconds left at current rate
      #ctogo=$(date -j -f "%s" $((csl+21600)) "+(%H:%M:%S)")  # convert to hrs:min:sec
      ctogo=$(digitaltime $csl)  # -j is not supported before 10.5
    fi
    if [ $gmr -lt 1 ] ; then  # copy hasn't started, or rate has dropped below 1mb/sec
      gtogo="(-:--:--:--)"
    else
#     gsl=$((cml/gmr))  # seconds left at average rate
      gsl=$((ckl/gkr))  # seconds left at average rate
      #gtogo=$(date -j -f "%s" $((gsl+21600)) "+(%H:%M:%S)")  # convert to hrs:min:sec
      gtogo=$(digitaltime $gsl)  # -j is not supported before 10.5
    fi
#   echo -n $'\r'"$(date)      $cml MB remains,  $ctogo at $cmr MB/sec current,  $gtogo at $gmr MB/sec average       "
    echo -n $'\r'"$(date)      $ckl KB remains,  $ctogo at $ckr KB/sec current,  $gtogo at $gkr KB/sec average       "
    pmu=$cmu
    pku=$cku
    sleep $i
  done
  # parent quit, klone is done
  exit
fi


####################################################################################################
###  MAIN
####################################################################################################

# banner
clear
echo
echo "*********************************"
echo "***  KLONE vers $vers  ***"
echo "*********************************"
echo
echo "Duplicate entire disk volumes quickly and easily"
echo
echo

# must run as root
if [ $UID != 0 ] ; then
  echo
  if [ -z "$(echo " $(dscl . -read /Groups/admin | grep "GroupMembership: " | cut -d ':' -f 2) " | grep " $USER ")" ] ; then
    echo "You will need to login as an admin to run this program."
    echo
    exit
  fi
  echo -n "Please enter your admin " # Password:
  sudo "$0"
  exit
fi

# determine rsync app
echo "Using rsync vers \"$rv\""
echo
echo -n "Checking platform support... "
if [ -n "$(system_profiler SPHardwareDataType | grep -i "intel")" ] ; then  #  intel
  echo "Intel processor found"
  ra="rsync_${rv}_intel"
else
  echo "PPC processor found"
  ra="rsync_${rv}_ppc"
fi
echo


# verify support files are present
echo -n "Verifying support files are present... "
verifyulb sethidden
verifyulb fixlinks
verifyulb $ra
echo "all required files found."
echo

# constants
tmpfile="/tmp/klonetmp0"
volfile="/tmp/klonetmp1"
runfile="/tmp/klonetmp2"
errfile="/tmp/klonetmp3"

# select to ignore service (and boot) volumes or not
ignoreservice=
while [ -z "$ignoreservice" ] ; do
  read -n1 -p "Ignore service/boot volumes? [y/n] : " x
  if [[ ("$x" == "y") || ("$x" == "Y") ]] ; then
    ignoreservice=1
  elif [[ ("$x" == "n") || ("$x" == "N") ]] ; then
    ignoreservice=0
  else
    echo $'\a\a\a'
  fi
done
echo
echo

echo "Building list of eligible volumes..."

# calculate offset of volume name in du's output, varies by version of os x
dx=$(df -ht hfs | grep "Mounted on$")
dx=${dx%Mounted on}
dx=${#dx}

# determine boot volume's volume path
bi=$(ls -la /Volumes | grep " \.$")
bvp="/Volumes/$(ls -la /Volumes | grep " -> /$" | cut -c ${#bi}-999 | sed 's/ -> \/$//g')"

# loop through volumes, building volume list
vc=0
df -ht hfs | grep "%" > "$tmpfile"
while read y ; do
  # determine volume name and root folder
  n=${y:$dx}  # "" or "/Volumes/Macintosh HD" etc
  if [ "$debug" == 1 ] ; then echo "checking \"$n\"" ; fi
  if [ "$n" == "/" ] ; then
    n=$bvp
    if [ "$debug" == 1 ] ; then echo "boot volume path was updated to \"$n\"" ; fi
    if [ "$ignoreservice" == 1 ] ; then  # ignore boot drive
      if [ "$debug" == 1 ] ; then echo "ignoring boot drive $n" ; fi
      continue
    fi
  fi
  if [[ ("$ignoreservice" == 1) && (-f "$n/.Service") ]] ; then  # ignore service drive
    if [ "$debug" == 1 ] ; then echo "ignoring service drive $n" ; fi
    continue
  fi
  vol[vc]=$n  #  load volume paths into vols[] array
  ((vc++))
done < "$tmpfile"
rm "$tmpfile"
echo "done, $vc volumes loaded."
echo


echo "Enabling owners on eligible volumes..."
for ((i=0;i<vc;i++)) ; do
  v=${vol[i]}
  echo -n "testing $v : "
#    vsdbutil -c "/Volumes/$x" 2>&1 | grep "are enabled" > /dev/null
# vsdbutil -c is broken in 10.6.7?  6/9/2011
  diskutil info "$v" | grep "Owners" | grep "Enabled" > /dev/null
  if [ $? == 0 ] ; then
    echo "owners are enabled"
  else
    echo -n "owners are disabled, "
    if [ -f "$v/.dneo" ] ; then
      echo "not enabling owners for $v"
#      if [ "$debug" == 1 ] ; then echo "$v was removed from vol[] array" ; fi
#      ((vc--))
#      vol[i]=${vol[vc]}
    else
      echo -n "enabling... "
      vsdbutil -a "$v"
      diskutil info "$v" | grep "Owners" | grep "Enabled" > /dev/null
      if [ $? == 0 ] ; then
        echo "successfully enabled"
      else
        echo "ERROR ENABLING OWNERS ON \"$x\""$'\a\a\a'
#        if [ "$debug" == 1 ] ; then echo "volume was removed from vol[] array" ; fi
#        ((vc--))
#        vol[i]=${vol[vc]}
      fi
    fi
  fi
done
echo "done."
echo

# verify at least two
if [ $vc -lt 2 ] ; then
  echo "Cancelled, Klone requires two eligible HFS formatted volumes."
  echo
  exit
fi

# list sources
echo "Available Sources:"
echo "------------------------------------------------------"
for ((i=0;i<vc;i++)) ; do
  v=${vol[i]}
  vf="$v/System/Library/CoreServices/SystemVersion.plist"
  if [ -f "$vf" ] ; then
    vers[i]="$(cat "${vf}" | grep -A 1 "ProductUserVisibleVersion" | grep "string" | cut -d ">" -f 2 | cut -d "<" -f 1)"
  else
    vers[i]="not found"
  fi
  line=$(df "$v" | grep "/dev/" | tr -s " ")
  mt=$(($(echo "$line" | cut -d " " -f 2)*512/1024/1024))
  mu=$(($(echo "$line" | cut -d " " -f 3)*512/1024/1024))
  mf=$(($(echo "$line" | cut -d " " -f 4)*512/1024/1024))
  pc=$((100*mu/mt))
  xmu=$((mu/1024)) ; xmu=$(echo -n "    $xmu" | cut -c ${#xmu}-99)
  xmt=$((mt/1024)) ; xmt=$(echo -n "    $xmt" | cut -c ${#xmt}-99)
  xpc=$((pc))      ; xpc=$(echo -n  "  $xpc" | cut -c ${#xpc}-99)
  echo "$((i+1))) ${xmu} of${xmt} GB (${xpc}%) used on $v (Mac OS X ${vers[i]})"
done
echo

si=
while [ -z "$si" ] ; do
  read -p "Select source by number: " si
  si=$((si))
  if [ $? == 1 ] ; then
    si=0
  fi
  ((si--))
  if [[ ($si -lt 0) || ($si -ge $vc ) ]] ; then
    echo $'\a'
    si=
  fi
done

# prepare source statistics
srcp=${vol[si]}
line=$(df "$srcp" | grep "/dev/" | tr -s " ")
smt=$(($(echo $line | cut -d " " -f 2)*512/1024/1024))  # mb on source total
smu=$(($(echo $line | cut -d " " -f 3)*512/1024/1024))  # mb on source used
smf=$(($(echo $line | cut -d " " -f 4)*512/1024/1024))  # mb on source free
spc=$((100*smu/smt))
echo
echo "Selected source is \"$srcp\""
echo

# list targets
echo "Available Targets:"
echo "------------------------------------------------------"
for ((i=0;i<vc;i++)) ; do
  v=${vol[i]}
  vf="$v/System/Library/CoreServices/SystemVersion.plist"
  if [ -f "$vf" ] ; then
    vers[i]="$(cat "${vf}" | grep -A 1 "ProductUserVisibleVersion" | grep "string" | cut -d ">" -f 2 | cut -d "<" -f 1)"
  else
    vers[i]="not found"
  fi
  line=$(df "$v" | grep "/dev/" | tr -s " ")
  mt=$(($(echo "$line" | cut -d " " -f 2)*512/1024/1024))
  mu=$(($(echo "$line" | cut -d " " -f 3)*512/1024/1024))
  mf=$(($(echo "$line" | cut -d " " -f 4)*512/1024/1024))
  pc=$((100*mu/mt))
  xmu=$((mu/1024)) ; xmu=$(echo -n "    $xmu" | cut -c ${#xmu}-99)
  xmt=$((mt/1024)) ; xmt=$(echo -n "    $xmt" | cut -c ${#xmt}-99)
  xpc=$((pc))      ; xpc=$(echo -n  "  $xpc" | cut -c ${#xpc}-99)
  if [ $i == $si ] ; then  #  asterisk source
    echo -n "*"
  else
    echo -n "$((i+1))"
  fi
  echo ") ${xmu} of${xmt} GB (${xpc}%) used on $v (Mac OS X ${vers[i]})"
done
echo

if [ $vc == 1 ] ; then
  x=0
  echo "only one available target, automatically selected"
else
  ti=
  while [ -z "$ti" ] ; do
    read -p "Select target by number: " ti
    ti=$((ti))
    if [ $? == 1 ] ; then
      ti=0
    fi
    ((ti--))
    if [[ ($ti -lt 0) || ($ti -ge $vc ) || ($ti == $si) ]] ; then
      echo $'\a'
      ti=
    fi
  done
fi

# prepare target statistics
trgp=${vol[ti]}
line=$(df "$trgp" | grep " $trgp$" | tr -s " ")
tmt=$(($(echo $line | cut -d " " -f 2)*512/1024/1024))  # mb on target total
tmu=$(($(echo $line | cut -d " " -f 3)*512/1024/1024))  # mb on target used
tmf=$(($(echo $line | cut -d " " -f 4)*512/1024/1024))  # mb on target free
tpc=$((100*tmu/tmt))
echo
echo "Selected target is \"$trgp\""
echo

# check for sufficient free space on target
rmu=$((smu+tmu))  # mb on target after copy used
rmf=$((tmf-smu))  # mb on target after copy free
rpc=$((100*rmu/tmt))
echo "Target \"$trgp\" will go from $((tmf/1024)) GB free to $((rmf/1024)) GB free"
echo
if [ $rmf -gt 100 ] ; then
  echo "Target has enough free space."
else
  x=
  while [ -z "$x" ] ; do
    read -p $'\a\a\a'"Not enough space for full copy, attempt anyway? (y/n) : " x
    x=$(echo "$x" | tr a-z A-Z)
    if [ "$x" == "N" ] ; then
      echo "(cancelled)"
      echo
      exit
    elif [ "$x" != "Y" ] ; then
      echo $'\a'
      x=
    fi
  done
fi

# check for existing data on target
echo
limit=1025
if [ $tmu -lt $limit ] ; then
  echo "(Target has less than $limit MB of space used (it's essentially empty)"
else
  x=
  while [ -z "$x" ] ; do
    read -p $'\a\a\a'"Target has more than $limit MB used already, existing data may be overwritten, continue? (y/n) : " x
    x=$(echo "$x" | tr a-z A-Z)
    if [ "$x" == "N" ] ; then
      echo "(cancelled)"
      echo
      exit
    elif [ "$x" != "Y" ] ; then
      echo $'\a'
      x=
    fi
  done
fi

# build exclude list for peaceful kloning of boot volume
excludelist="${excludelist} --exclude=DATA_XFER"
excludelist="${excludelist} --exclude=/Network"
excludelist="${excludelist} --exclude=/automount"
excludelist="${excludelist} --exclude=/private/tmp"
excludelist="${excludelist} --exclude=/private/var/vm"
excludelist="${excludelist} --exclude=/private/var/spool/postfix/private"
excludelist="${excludelist} --exclude=/.Spotlight-V100"
excludelist="${excludelist} --exclude=/home"  # new for leopard
excludelist="${excludelist} --exclude=/.fseventsd"  # new for leopard
excludelist="${excludelist} --exclude=/.journal*"  # recent security update?
excludelist="$(echo "${excludelist}" | cut -c 2-999)"

# select klone mode
echo
echo "Select Klone mode"
echo "---------------------"
echo "1) ditto (overwrites, tolerates minor errors)"
echo "2) rsync entire volume (resumes, deletes)"
echo "3) rsync /Users folder only (resumes, deletes)"
echo "4) rsync all but /Users folder (resumes, deletes)"
echo "5) ditto /Users folder only (overwrites, tolerates minor errors)"
echo "6) rsync all but /Applications folder (resumes, deletes)"
echo
mode=
while [ -z "$mode" ] ; do
  read -p "Select mode by number: " mode
  mode=$((mode))
  if [[ ( $mode -lt 1 ) || ( $mode -gt 6 ) ]] ; then
    echo $'\a'
    mode=
  fi
done
echo
echo "Selected Klone mode = $mode"
echo
if [ $mode == 1 ] ; then
  echo "Klone command:  ditto -rsrc -X \"$srcp\" \"$trgp\""

# rsync vers 2.6.0
#elif [ $mode == 2 ] ; then
#  echo "Klone command:  /usr/local/bin/$ra -ax --eahfs --delete ${excludelist} \"$srcp/\" \"$trgp/\""
#elif [ $mode == 3 ] ; then
#  echo "Klone command:  /usr/local/bin/$ra -ax --eahfs --delete \"$srcp/Users/\" \"$trgp/Users/\""
#elif [ $mode == 4 ] ; then
#  echo "Klone command:  /usr/local/bin/$ra -ax --eahfs --delete --exclude=/Users/* \"$srcp/\" \"$trgp/\""

# rsync vers 3.0.6
elif [ $mode == 2 ] ; then
  echo "Klone command:  /usr/local/bin/$ra $rf ${excludelist} \"$srcp/\" \"$trgp/\""
elif [ $mode == 3 ] ; then
  echo "Klone command:  /usr/local/bin/$ra $rf \"$srcp/Users/\" \"$trgp/Users/\""
elif [ $mode == 4 ] ; then
  echo "Klone command:  /usr/local/bin/$ra $rf --exclude=/Users/* \"$srcp/\" \"$trgp/\""
elif [ $mode == 5 ] ; then
  echo "Klone command:  ditto -rsrc -X \"$srcp/Users\" \"$trgp/Users\""
elif [ $mode == 6 ] ; then
  echo "Klone command:  /usr/local/bin/$ra $rf --exclude=/Applications/* \"$srcp/\" \"$trgp/\""

fi

# remove journal files if ditto
if [ $mode == 1 ] ; then
  jf="${srcp}/.journal"
  if [ -e "$jf" ] ; then
    echo
    echo -n "Removing ditto-unfriendly file on source: \"$jf\" ..."
    rm -Rf "$jf"
    if [ -e "$jf" ] ; then
      echo "could not remove deadly file \"$jf\", cannot use DITTO"$'\a\a\a'
      echo
      exit
    else
      echo "done."
    fi
  fi
  jf="${srcp}/.journal_info_block"
  if [ -e "$jf" ] ; then
    echo
    echo -n "Removing ditto-unfriendly file on source: \"$jf\" ..."
    rm -Rf "$jf"
    if [ -e "$jf" ] ; then
      echo "could not remove deadly file \"$jf\", cannot use DITTO"$'\a\a\a'
      echo
      exit
    else
      echo "done."
    fi
  fi
fi

# final confirmation
echo
echo "(make sure terminal window width is set wide (140) for scoreboard  -----------------------------------------------------------------------)"
#     00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000011111111111111111111111111111111111111111
#     00000000011111111112222222222333333333344444444445555555555666666666677777777778888888888999999999900000000001111111111222222222233333333334
#     12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

echo
x=
while [ -z "$x" ] ; do
  read -p "Begin Klone now? (y/n) : " x
  x=$(echo "$x" | tr a-z A-Z)
  if [ "$x" == "N" ] ; then
    echo "(cancelled)"
    echo
    exit
  elif [ "$x" != "Y" ] ; then
    echo $'\a'
    x=
  fi
done

# start scoreboard
if [ $mode == 1 ] ; then  # ditto does not resume
  "$0" "track" "$trgp" $smu $tmt $tmu &  # assumes data on source is how much that must be copied
elif [ $mode == 2 ] ; then
  "$0" "track" "$trgp" $smu $tmt $(($tmu-$tmt)) &  # assumes any data on dest has already been copied
elif [ $mode == 3 ] ; then
  "$0" "track" "$trgp" $smu $tmt $(($tmu-$tmt)) &  # assumes any data on dest has already been copied
elif [ $mode == 4 ] ; then
  "$0" "track" "$trgp" $smu $tmt $(($tmu-$tmt)) &  # assumes any data on dest has already been copied
elif [ $mode == 5 ] ; then  # ditto does not resume
  "$0" "track" "$trgp" $smu $tmt $tmu &  # assumes data on source is how much that must be copied
elif [ $mode == 6 ] ; then
  "$0" "track" "$trgp" $smu $tmt $(($tmu-$tmt)) &  # assumes any data on dest has already been copied
fi
scoreboard=$!
# thread will wake every 10 sec to update the scoreboard
# it will quit if it sees its parent process disappear,
# or when we SIGTERM it
#echo "Scoreboard is $scoreboard"
#echo
#while ! [ -f /users/nfisher/stop ] ; do sleep 3 ; done

# start klone
echo
echo "$(date)  Starting Klone...  (you will hear beeps if there are I/O errors or slow I/O)"
kstart=$seconds
echo
if [ $mode == 1 ] ; then
  ditto -rsrc -X "$srcp" "$trgp"

# rsync 2.6.0
#elif [ $mode == 2 ] ; then
#  /usr/local/bin/$ra -ax --eahfs --delete ${excludelist} "$srcp/" "$trgp/" 2>&1 | grep -v "^mknod " > "$runfile"
#elif [ $mode == 3 ] ; then
#  /usr/local/bin/$ra -ax --eahfs --delete "$srcp/Users/" "$trgp/Users/" 2>&1 | grep -v "^mknod " > "$runfile"
#elif [ $mode == 4 ] ; then
#  /usr/local/bin/$ra -ax --eahfs --delete --exclude=/Users/* "$srcp/" "$trgp/" 2>&1 | grep -v "^mknod " > "$runfile"

# rsync 3.0.6
elif [ $mode == 2 ] ; then
  /usr/local/bin/$ra $rf ${excludelist} "$srcp/" "$trgp/" 2>&1 | grep -v "^mknod " > "$runfile"
elif [ $mode == 3 ] ; then
  /usr/local/bin/$ra $rf "$srcp/Users/" "$trgp/Users/" 2>&1 | grep -v "^mknod " > "$runfile"
elif [ $mode == 4 ] ; then
  /usr/local/bin/$ra $rf --exclude=/Users/* "$srcp/" "$trgp/" 2>&1 | grep -v "^mknod " > "$runfile"
elif [ $mode == 5 ] ; then
  ditto -rsrc -X "$srcp/Users" "$trgp/Users"
elif [ $mode == 6 ] ; then
  /usr/local/bin/$ra $rf --exclude=/Applications/* "$srcp/" "$trgp/" 2>&1 | grep -v "^mknod " > "$runfile"

fi
rc=$?

# quit scoreboard (it will also quit automatically if it sees us disappear)
echo
echo
echo -n "Klone finished with return code $rc, stopping scoreboard... "
kill -s sigterm $PPID  # this is going to generate a Terminated message and I don't know how to stop it
sleep 1
echo "done."

# hide system files and folders on target
echo
echo
echo "Hiding system files on target..."
echo
/usr/local/bin/sethidden "$trgp" /usr/local/bin/dothidden

# bless OS 9 system folder on target
echo
if [ -f "${trgp}/System Folder/" ] ; then
  echo -n "Blessing OS 9 on ${trgp}... "
  bless -folder9 "${trgp}/System Folder/"
  echo "done."
else
  echo "(no OS 9 system to bless on ${trgp})"
fi

# bless OS X system folder on target
echo
if [ -d "${trgp}/System/Library/CoreServices/" ] ; then
  if [ -f "${trgp}/System/Library/CoreServices/bootx.efi" ] ; then
    echo -n "Blessing OS X CoreServices and EFI on ${trgp}... "
    bless -folder "${trgp}/System/Library/CoreServices" -file "${trgp}/System/Library/CoreServices/bootx.efi"
    echo "done."
  else
    echo -n "Blessing OS X CoreServices on ${trgp}... "
    bless -folder "${trgp}/System/Library/CoreServices"
    echo "done."
  fi
else
  echo "(no OS X system to bless on ${trgp})"
fi

# fixlinks if rsync
if [[ ( "$mode" == "2" ) || ( "$mode" == "3" ) ]] ; then
  cat "$runfile" | grep " failed: " | grep -v "^mknod " > "$errfile"
  count=$(($(cat "$errfile" | wc -l)))
  echo
  if [ "$count" == 0 ] ; then
    echo "(no links to fix)"
  else
    #read -p "Press [return] to begin fixing $count links: " x
    #echo
    echo -n "Fixing $count links... "
    if [ "$mode" == "2" ] ; then
      /usr/local/bin/fixlinks 4 "$srcp/" "$trgp/" "$errfile"
    elif [ "$mode" == "3" ] ; then
      /usr/local/bin/fixlinks 4 "$srcp/Users/" "$trgp/Users/" "$errfile"
    fi  
  fi
  rm "$errfile"
  rm "$runfile"
else
  echo "(mode $mode does not fixlinks)"
fi

# log
log="klone log $(date '+%Y-%m-%d %H-%M').txt"
# hmm I can't save terminal output like I need to

# done
kend=$SECONDS
kduration=$((kend-kstart))
kdur=$(digitaltime $kduration)
echo
echo
echo "$(date)  Klone complete in ${kdur}."$'\a\a\a\a\a'
echo
echo
exit


I work for the Department of Redundancy Department
Re: Bad blocks question
Virtual1 #31802 11/18/14 09:23 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
Many thanks, V1; your response is very much appreciated.

My first thought is, of course, if ditto is so easy, why did the Genius have my daughter copying files from one drive to another, one file at a time, with drag & drop?

Oh, well.

Waaay back, probably in FTM, you posted these instructions, which I successfully followed before I began using CCC:

Quote:
f you want to clone a drive to another using terminal, here's a quickie on how: (you can clone from the volume you're booted on if you like, that's what the "-X' flag is for, handy for making a backup)

1. erase the hd you are copying to. it helps to have a fresh start, though this can be optional if needed.
2. make sure ignore privs is not checked on either drive. Be sure to do this after step 1, which tends to recheck the box.
3. in terminal, type sudo ditto -rsrc -X and add a single space after "X".
4. drag the source drive into terminal, it will type the path and a space. If this is the boot volume, all you'll get is a slash, that's ok.
5. drag the destination (empty) drive into terminal, it will type the path (and will again add a space)
6. hit return, and enter your admin password. If copying from the boot volume, you'll see about 5 warnings over the next 10 minutes, something about device type not supported, you can ignore these errors, it's skipping dynamically created folders on the boot mount point that aren't really folders.
7. go grab a snack. this will take awhile.

When you get a prompt back, the data is copied but the system is NOT blessed. (it will not boot) Run the Startup Disk system pref, select the system on your destination drive. Close system prefs. There will be a brief pause (1-2 sec) while the detination is blessed. Re-open startup disk and reselect your normal boot system.

What I propose to do now is mount my daughter's bad HDD on my desktop using an eSATA to USB connector cable, and use ditto to copy across the viable files remaining on it that have not yet been copied across elsewhere, so my question is "Can I substitute folders for volumes in your command (as seems likely)?" (I haven't got the drive in-hand, so I can't experiment.)

Thanks, again.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
Virtual1 #31803 11/18/14 09:30 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
Wow! Thanks for posting your script. (No ©?)

Unfortunately, I'm at a loss about what to do with it, although I suspect that "klone.command" may be a clue.

My initial thought was to make the script executable with chmod a+x (path...), and then stash it ~/bin, but that apparently is not the way to go.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31808 11/18/14 02:37 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
Originally Posted By: artie505
Wow! Thanks for posting your script. (No ©?)

Unfortunately, I'm at a loss about what to do with it, although I suspect that "klone.command" may be a clue.

My initial thought was to make the script executable with chmod a+x (path...), and then stash it ~/bin, but that apparently is not the way to go.

I've been off the ftm forum for a few months and only just getting back to it now. I changed jobs, no longer working at an AASP but now I'm the "mac guy" at a community college. smile

Anyway, I don't take my laptop to work anymore, probably better to keep me focused on things here since I don't really need it like I did before. So long story short, I remoted home and grabbed the script (which I haven't used here yet actually...) but it may not run without some support files. (fixlinks, setfile) But yes you should be able to chmod +x the script and then put it anywhere in PATH. A lot of software sets up the /usr/local/bin/ folder and adds it to PATH. If you have that folder, just stick it there. Probably simplest is to just stick it on your desktop and double click it. Finder knows to pass .command files off to Terminal to execute, as long as you've changed them to +x first.

And yeah it's been a long time since I sold any of my code. Though I did actually have someone contact me earlier this year and get a copy of something I'd written ages ago (that was also given free) and because it was so useful he sent me a little coin. I think I prefer to keep my coding "hobby grade" rather than trying to make a living on it. I've got enough areas of expertise I don't have to make money on everything I do. Some of it needs to stay a hobby. So many times I've gotten online needing something and finally found an answer in someone's code they posted freely, I feel this is a really good thing to give back to. (oh boy am I running into that A LOT here recently!) If anything, I don't post up enough of my stuff as it is. surfacescan, watchdrives, and klone are probably the most popular of my works, and are all uncommonly large and complex scripts.


I work for the Department of Redundancy Department
Re: Bad blocks question
Virtual1 #31817 11/19/14 07:49 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
First, congrats and lotsa luck on your new job! Quoting from Blazing Saddles: "Woohee, civil service!" smile (It's nice to know that schools now have real "Mac guys"; in the early '90s, my daughter was the "Mac guy" at U Toronto, but it was by virtue of the fact that she was pretty much the only person on campus who knew how to turn one off other than by pulling the plug. tongue )

And second, welcome back to FTM; you've been missed! (Your returning the very day I posted a question with you in mind is wonderfully fortuitous.)

I dunno... I can't make your script executable; I copy it to a plain text file, chmod +x it, and... I've still got a plain text file.

However, if I change .txt to either .command or .exec and open my file in Terminal, (omitting a lot of empty space) I get

Quote:
Last login: Wed Nov 19 03:08:34 on ttys000
Artie's-MacBook-Pro:~ artie$ /Users/artie/Desktop/klone.exec (or klone.command) ; exit;


*********************************
*** KLONE vers 2012.02.02.A ***
*********************************

Duplicate entire disk volumes quickly and easily



Please enter your admin


*********************************
*** KLONE vers 2012.02.02.A ***
*********************************

Duplicate entire disk volumes quickly and easily


Using rsync vers "306"

Checking platform support... Intel processor found

Verifying support files are present... Cannot find support file to copy: /Volumes/Service Data/Tools/usrlocalbin/sethidden
logout

[Process completed]

Under any circumstances, though, your script is directed at copying entire volumes, and I only want to copy specific files, so what I still hope to do is run

Code:
sudo ditto -rsrc -X ... (posted by you)

but using a folder, rather than a volume, as my source, and a folder as my destination. (I plan to mount the HDD with the bad blocks on my desktop with an eSATA to USB cable.)

Will that work?

Thanks.


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31821 11/19/14 12:10 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
yep I figured it would have support file issues. ok on to your question since I am short on time

type "ditto -rsrc -X " (note trailing space, -x should be default as of 10.5 I think and can be omitted, X is case sensitive. do not hit return

drag and drop from finder into terminal, the folder you want to copy from. drag and drop the folder you want to copy to. ITEMS IN THE DESTINATION FOLDER MAY BE OVERWRITTEN if they have the same name as items in the source. hit return.

there is no progress indicator. watch disk space. (which is what klone does) it will return a prompt when it's done. I suggest trying it on a small test folder you have backed up before trying it live. to copy a folder to a destination that does not contain the folder yet, create a destination folder (does not have to be the same name) and drag that in as the destination. (2nd drop) you can also use the same method to copy a single file. If the file has io errors, it will unfortunately not copy parts of the file it can. (use ddrescue for that) files with io errors will be skipped but copying will continue.

very little stops ditto. some rare directory problems, and hitting an uncompleted .journal are about it. other problems print errors but continue copying. great for folders that have a problem Finder refuses to copy and aborts on.

media type does not matter, it'll read anything the file system can read or write. source does not have to be moutned with write enabled. results will be mixed when the source or destination are other than hfs. (from NTFS for example, may not copy some files or folders due to filename issues, flags, path problems, etc)


I work for the Department of Redundancy Department
Re: Bad blocks question
Virtual1 #31822 11/19/14 04:12 PM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
Thanks! smile

I'll let you know how I make out.

By the way, I forgot to say that your making your software freely available is quite admirable; thanks for being a gentleman! cool


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31898 11/27/14 01:36 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
As promised...

1. ditto worked a treat. (Judging from the time it took to do its job, it appears to work hard to resolve "difficult" stuff.)

2. Not that it runs, but I finally figured out how to make your script executable...a good lesson for down the road.

Many thanks for your help. smile


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31899 11/27/14 02:23 AM
Joined: Aug 2009
Offline

Joined: Aug 2009
taking a little time now at home to look at that, it requires sethidden, a little script I wrote that uses setfile to hide a file.
Code:
#!/bin/bash
vol=$1
hfile=$2
if [[(-z "$vol") || (-z "$hfile") ]] ; then
  echo "usage: ${0##*/} targetpath hiddenfile" 1>&2
  exit 1
fi
if ! [ -d "$vol" ] ; then
  echo "target path \"$vol\" not found" 1>&2
  exit 2
fi
if [ "${vol:$((${#vol}-1))}" != "/" ] ; then
  vol="${vol}/"
fi
if ! [ -f "$hfile" ] ; then
  echo "hiddenfile \"$hfile\" not accessible" 1>&2
  exit 2
fi
n=0
while read f[n] ; do
  ((n++))
done < "$hfile"
if [ $n -gt 50 ] ; then
  echo "too many entries ($n) in hiddenfile \"$hfile\""
  exit 3
fi
for ((i=0;i<n;i++)) ; do
  ff="${vol}${f[i]}"
  if [ -f "$ff" ] ; then
#    echo "setfile -a V \"$vol/$ff\""
    setfile -a V "$vol/$ff"
#  else
#    echo "file \"$ff\" not found"
  fi
done
exit 0



setfile is in developer tools and is also included on os installer discs iirc. it also looks for rsync, which you can download compiled in a few places, or install/compile it fron a package manager. technically neither is needed to run the basics of klone - rsync gives it a very useful option in addition to ditto, and sethidden hides files that are hidden on the source following an rsync. (ditto retains hidden flag)


I work for the Department of Redundancy Department
Re: Bad blocks question
Virtual1 #31900 11/27/14 08:10 AM
Joined: Aug 2009
Likes: 15
OP Online

Joined: Aug 2009
Likes: 15
Thanks for the update, but since I've long owned CCC, I'm not sure why I'd need klone.

Does it in some way surpass CCC's capabilities?


The new Great Equalizer is the SEND button.

In Memory of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire
Re: Bad blocks question
artie505 #31901 11/27/14 01:08 PM
Joined: Aug 2009
Offline

Joined: Aug 2009
CCC is a subset of klone. Klone could be though of as a combination of CCC (ditto) and RsyncX's features.

Rsync allows klone to update a drive rather than just copy the files. In a few minutes, it can turn a week-old CCC/klone snapshot drive into an up-to-the-minute updated copy.

It can also exclusively target the /Users folder, when accessing a drive causes the drive to shut down, or when time is critical. I've dealt with a few dozen drives that only allowed me ~4-5 minutes of access time before overheating. (or LESS!) Back into the freezer for 10 min. back out, to run klone again to resume saving just the /Users folder. A few of them required all afternoon for complete recovery. Try that with CCC wink

Surprisingly, klone makes CCC look like a one-trick-pony.

(will CCC even do non-HFS drives? klone will)

I also don't think you can CCC off a diskwarrior preview? Most disk imagers require unmounting the volume during copy, and that causes DW to disconnect the preview. THAT was my bread-and-butter use for Klone. Also, the ONE thing that will reliably kill ditto is duplicating from a drive with an open and dirty journal. Unless they've found a way around it, CCC will hang on such a drive. Dozens of times I ran into a drive that required a DW preview to get it on the desktop, and it had a dirty journal. Can't delete it because the preview is read-only. Thus ditto (for the entire drive) is not an option. Either ditto /Users only to avoid encounting the journal, or rsync then entire drive, because klone tells rsync to skip that killer file. (ditto cannot be told to skip a file) CCC offers neither option. (klone can also copy everything EXCEPT /Users from a failing drive, for cases where you need proprietary software fully recovered because reinstallation isn't a good option)

Trust me, if CCC did what I needed it to do, I wouldn't have spent all that time developing klone.

On a sidenote, I have another smaller script that specifically uses rsync for syncing volumes (boot or otherwise) either locally or over the LAN or even over the internet. Rsync is an excellent tool to have in your kit for automatic offsite backup.


I work for the Department of Redundancy Department

Moderated by  alternaut, dianne, dkmarsh 

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Release build 20200307)
Responsive Width:

PHP: 7.4.33 Page Time: 0.030s Queries: 40 (0.018s) Memory: 0.6723 MB (Peak: 0.8080 MB) Data Comp: Zlib Server Time: 2024-03-29 01:54:35 UTC
Valid HTML 5 and Valid CSS