An open community 
of Macintosh users,
for Macintosh users.

FineTunedMac Dashboard widget now available! Download Here

Topic Options
#51721 - 05/16/19 01:27 AM Terminal can't run dd
artie505 Online


Registered: 08/04/09
This really isn't the correct forum for this post, but I"m not sure where it really ought to be.

Many years ago, someone (V1, I think) posted (this Terminal command)

Code:
sudo dd if=/dev/rdisk0 of=/dev/null conv=noerror bs=1024000

as an alternative way to run a surface scan in the absence of one of the expensive utilities, and I've run it successfully on both HDDs and SSDs any number of times (including on my SSD/macOS 10.14.5 just yesterday).

The need has now arisen (in another thread) to run the command on a fusion drive, but it refuses to run.

Using the original command as a template and this as an "update guide," I adjusted the command to

Code:
sudo dd if=/dev/rdisk1 of=/dev/null conv=noerror bs=1024000

but Terminal returns

Code:
Last login: Wed May 15 11:49:50 on ttys000
iMac:~ myname$ sudo dd if=/dev/rdisk1 of=/dev/null conv=noerror bs=1024000
Password:
dd: /dev/rdisk1: Operation not permitted
iMac:~ myname$

It returns the same error if "disk1" is substituted for "rdisk1".

Has anybody got an idea of why the command won't run on a FD and how it might be adjusted to get it to run?
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#51723 - 05/16/19 12:18 PM Re: Terminal can't run dd [Re: artie505]
joemikeb Online
Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
As I don't have a Fusion Drive to experiment on, I cannot test or otherwise verify this, but Fusion Drives are virtual entities somewhat along the lines of APFS volumes. I believe that in order to run a Surface Scan you would have to address only the HD portion of the drive.

NOTE: Beginning with High Sierra (MacOS 10.13) Apple does NOT permit a surface scan of the boot drive.
_________________________
joemikeb • moderator

Top
#51730 - 05/16/19 04:01 PM Re: Terminal can't run dd [Re: joemikeb]
ryck Offline


Registered: 08/04/09
Loc: Okanagan Valley
Originally Posted By: joemikeb
NOTE: Beginning with High Sierra (MacOS 10.13) Apple does NOT permit a surface scan of the boot drive.

Well, that's a pain....I'm 10.13.6. I suppose that's the reason I get "dd: /dev/rdisk1: Operation not permitted". I wonder what the logic is behind a surface scan prohibition.

On the downside I suppose I just have to wait and see if the world comes to an end but, on the upside, I save a hundred bucks by not having to buy TechTool 11.


Edited by ryck (05/16/19 04:03 PM)
_________________________
ryck

iMac (Retina 5K, 27", 2017), 3.4 GHz Intel Core i5, 8GB RAM, 2400 MHz DDR4
OS High Sierra 10.13.6
Canon MX710 Printer
Epson Perfection V500 Photo Scanner
Time Machine on 320GB OWC Mercury OTG Pro
Carbon Copy Clone on 500GB OWC Mercury OTG Pro

Top
#51732 - 05/16/19 04:36 PM Re: Terminal can't run dd [Re: ryck]
artie505 Online


Registered: 08/04/09
Originally Posted By: ryck
Originally Posted By: joemikeb
NOTE: Beginning with High Sierra (MacOS 10.13) Apple does NOT permit a surface scan of the boot drive.

Well, that's a pain....I'm 10.13.6. I suppose that's the reason I get "dd: /dev/rdisk1: Operation not permitted". I wonder what the logic is behind a surface scan prohibition.

On the downside I suppose I just have to wait and see if the world comes to an end but, on the upside, I save a hundred bucks by not having to buy TechTool 11.

I've already posted that the command runs on my boot SSD from my Mojave 10.14.5 boot volume, so joemike's statement is inaccurate on at least some level.

I think - just for the heck of it - you ought to try booting from your external and seeing if the command runs.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#51734 - 05/16/19 11:30 PM Re: Terminal can't run dd [Re: joemikeb]
artie505 Online


Registered: 08/04/09
Originally Posted By: joemikeb
As I don't have a Fusion Drive to experiment on, I cannot test or otherwise verify this, but Fusion Drives are virtual entities somewhat along the lines of APFS volumes. I believe that in order to run a Surface Scan you would have to address only the HD portion of the drive.

That thought crossed my mind immediately upon learning that the command didn't run on ryck's fusion drive, but "diskutil list" shows it as one seamless unit, and with "disk1" already having returned an error, I don't see another number in the readout that looks even remotely like what we may be looking for.

And then again, the command's running on a freestanding SSD but not the SSD portion of a fusion drive doesn't seem logical.

We may need some input from V1 to get to the bottom of this.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#51735 - 05/17/19 12:02 AM Re: Terminal won't run dd [Re: joemikeb]
artie505 Online


Registered: 08/04/09
Originally Posted By: joemikeb
NOTE: Beginning with High Sierra (MacOS 10.13) Apple does NOT permit a surface scan of the boot drive.

Thinking about it, I'm now wondering whether Apple's prohibition is on only whatever variety of surface scan TTP, etc. run, but not on the "dd" command we're discussing.

ryck's error "dd: /dev/rdisk1: Operation not permitted" suggests that "dd" is permissible...that it's "/dev/rdisk1" that's a no-go.

Maybe the command must be run separately on the HDD and SSD portions of a FD, but if that's the case, it seems like "diskutil list" should provide the necessary info.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#52097 - 07/21/19 03:28 AM Re: Terminal won't run dd [Re: artie505]
artie505 Online


Registered: 08/04/09
VIR-TU-AL 1, WHERE AAARE YOU?

(Hope you're OK!)
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#52101 - 07/21/19 12:52 PM Re: Terminal won't run dd [Re: artie505]
joemikeb Online
Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
It is worth noting that Apple, Micromat (TechTool Pro), and Prosoftengineering (Drive Genius) have all removed surface scans from their applications (granted Micromat and Prosoftengineering followed Apple's not too subtle lead) because in the opinion of their engineering staffs surface scans of SSDs could result in reducing the usable lifespan of those drives.

In the case of a Fusion drive the idea of scanning only the rotating rust segment is intuitively appealing, but the fusion takes place at a very low level within the logical structure of the volume and might require breaking that fusion to access only the rotating rust portion of the drive. In my experience it is possible to separate the two drive segments, but the process is destructive so all the data on the fused drive is lost. The process would look something like…
  1. clone the fusion drive to another disk
  2. boot from the cloned drive
  3. break the fusion
  4. perform the surface scan of the now blank HD portion of the former fusion drive using whatever tool you desire
  5. re-fuse the SSD and HD portions
  6. Clone the data back to the new fusion drive
  7. over the next several weeks or months MacOS will re-optimize what is stored on the SSD and what is stored on the HD.
_________________________
joemikeb • moderator

Top
#52110 - 07/22/19 10:35 AM Re: Terminal won't run dd [Re: joemikeb]
ryck Offline


Registered: 08/04/09
Loc: Okanagan Valley
Originally Posted By: joemikeb
The process would look something like…
  1. clone the fusion drive to another disk
  2. boot from the cloned drive
  3. break the fusion
  4. perform the surface scan of the now blank HD portion of the former fusion drive using whatever tool you desire
  5. re-fuse the SSD and HD portions
  6. Clone the data back to the new fusion drive
  7. over the next several weeks or months MacOS will re-optimize what is stored on the SSD and what is stored on the HD.

I looked at the links and I figure that with the break-the-fusion, then restore-the-fusion, there would be too much con-fusion for me. Add in the fact that I have a partitioned drive, which is probably an additional wrinkle, and we'd probably be into straight-jacket territory. The two partitions now each have two CCC backups, so I'll take my chances that it may not be a bad sector issue.

Thanks for the education, though.


Edited by ryck (07/22/19 10:37 AM)
_________________________
ryck

iMac (Retina 5K, 27", 2017), 3.4 GHz Intel Core i5, 8GB RAM, 2400 MHz DDR4
OS High Sierra 10.13.6
Canon MX710 Printer
Epson Perfection V500 Photo Scanner
Time Machine on 320GB OWC Mercury OTG Pro
Carbon Copy Clone on 500GB OWC Mercury OTG Pro

Top
#52115 - 07/22/19 05:54 PM Re: Terminal won't run dd [Re: joemikeb]
artie505 Online


Registered: 08/04/09
Originally Posted By: joemikeb
It is worth noting that Apple, Micromat (TechTool Pro), and Prosoftengineering (Drive Genius) have all removed surface scans from their applications (granted Micromat and Prosoftengineering followed Apple's not too subtle lead) because in the opinion of their engineering staffs surface scans of SSDs could result in reducing the usable lifespan of those drives.

In the case of a Fusion drive the idea of scanning only the rotating rust segment is intuitively appealing, but the fusion takes place at a very low level within the logical structure of the volume and might require breaking that fusion to access only the rotating rust portion of the drive. In my experience it is possible to separate the two drive segments, but the process is destructive so all the data on the fused drive is lost. The process would look something like… (Omitted for brevity)

That's educational; thanks.

Considering the vulnerability of HDDs and the major role they play in fusion drives, you'd think that Apple would provide a means to do a surface scan.

Which is why I'd love for V1 to get involved in this conversation; he's apparently quite adept at thinking his way through problems that leave others tearing their hair out. (He hasn't posted in more than 3 months...very unusual, and as I said, I hope he's OK.)

Food for thought, though... This post by MMT3 (When's the last time I typed that?), albeit 6 years old makes what I think is a valid point.

Originally Posted By: MMT3

The name of the Surface Scan test is not particularly apt for SSD drives, but the test is still valid. It is simply reading blocks, one after another, to make sure that they can be read. We have had customers run the Surface Scan to check SD cards used in cameras, and a few have been found to be defective.

Save the results of the Surface Scan, so it a later lest finds errors (unremapped bad blocks), you will have a good basis for a warranty claim.

It seems as if running a surface scan on an SSD when it's brand new and then again periodically but infrequently is a good idea.

And as for its shortening the drive's life, I think it was tacit who did some math a few years back and demonstrated that unless you're doing an immense amount of graphics editing, your computer will die long before your SSD does...a few surface scans notwithstanding, I assume.
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#52121 - 07/23/19 07:43 AM Re: Terminal won't run dd [Re: artie505]
joemikeb Online
Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
Originally Posted By: artie505
The name of the Surface Scan test is not particularly apt for SSD drives, but the test is still valid. It is simply reading blocks, one after another, to make sure that they can be read. We have had customers run the Surface Scan to check SD cards used in cameras, and a few have been found to be defective.

Save the results of the Surface Scan, so it a later lest finds errors (unremapped bad blocks), you will have a good basis for a warranty claim.

It seems as if running a surface scan on an SSD when it's brand new and then again periodically but infrequently is a good idea.[/quote]
S.M.A.R.T. was intended to be a reliable guide to drive health, but in practice it turned out to her unreliable because drive manufacturers set the tolerance levels so high S.M.A.R.T. became relatively meaningless. That was particularly true because all of the utilities, except TechTool Pro, reported only a single pass/fail value and provided no opportunity for users to observe trends. The ability to see all of the S.M.A.R.T. values and their acceptable range as provided in TechTool Pro provides a good tool for analyzing hard drive health.

Then came SSD with a different set of S.M.A.R.T. values that could be used to infer drive health if drive health was indicated by the same factors that had been used for hard drives. But that has evolved in later SSDs into NVMe data which includes among many other things information that can be used to analyze drive health such as:
  • unsafe shutdowns
  • % available spares
  • % available spare threshold
  • % percentage used
  • % percentage remaining
  • data units read
  • data units written
  • available space below threshold (Y/N)
  • over temperature threshold (Y/N)
  • NVM subsystem reliability degraded (Y/N)
  • Media in read only mode (Y/N)
  • volitile memory backup device failure (Y/N)
Once again Macromedia has the only products that give the full report, TechTool Pro and Drive Scope. Drive Scope is a tool that is specifically for reporting this data from SSD drives and that is all it can do other than report SMART logs.

IMHO the full S.M.A.R.T. data report provides essentially the same information as a surface scan and a LOT more.

Disk Utility still reports only the SMART verified/fail. 😠

FULL DISCLOSURE: I receive no remuneration whatsoever from Micromat other than being a satisfied customer.

CONFESSION: For many years I have touted the value of surface scans based on the results of a study conducted by google LABS. It is only recently I have become fully convinced of the value of SMART when all of the SMART values are available for analysis.
_________________________
joemikeb • moderator

Top
#52133 - 07/24/19 10:12 PM Re: Terminal won't run dd [Re: joemikeb]
artie505 Online


Registered: 08/04/09
I couldn't find a study of failure rates of different manufacturers' SSDs, but I did find an interesting and presumably knowledgeable read from Backblaze:

Are Solid State Drives / SSDs More Reliable Than HDDs?
_________________________
The new Great Equalizer is the SEND button.

In Memory Of Harv: Those who can make you believe absurdities can make you commit atrocities. ~Voltaire

Top
#52140 - 07/25/19 10:10 AM Re: Terminal won't run dd [Re: artie505]
joemikeb Online
Moderator

Registered: 08/04/09
Loc: Fort Worth, Texas
Originally Posted By: artie505
I couldn't find a study of failure rates of different manufacturers' SSDs,...

That is not surprising as almost all reliable drive failure rate studies are based on data from large server farms and those server farms are not likely to be using SSDs any time soon. HDs are still fast enough to meet requirements, have dramatically higher storage capacity, and cost up to 25 times LESS per unit of data storage capacity. That may change in the foreseeable future, but I wouldn't hold my breath waiting for the changeover.
_________________________
joemikeb • moderator

Top

Moderator:  alternaut, cyn