Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

How to erase data?

  1. Feb 28, 2018 #1
    I have read in several places, I don't remember exactly where, that to effectively and securely erase stored date from most modern digital storage devices, it is necessary (or at least this is one often-used technique) to repeatedly, in several, maybe dozens, of passes, write randomly selected strings of 0's and 1's over the data. My Bitdefender antivirus app now has what is called a "file shredder" to do just that. (I imagined from the name that it used this technique, and a call to Bitdefender indicated that it probably did.)

    My question is: Why is it necessary to use such a time-consuming technique, when, it seems, just writing all 0's (or all 1's) would be quicker and even more effective? It is true that the ultimate limit set by quantum mechanics on the accuracy with which the state of every atom, free electron, etc. in the storage device after writing the 0's could be known would allow, at least in some situations and for some storage locations, the determination of whether a 0 or a 1 had been stored there before writing the 0's, but this could not be done with present technology, or any likely future technology. I could be wrong about this. Am I wrong, or were instead the descriptions I read of erasing techniques, and the Bitdefender phone tech person, both wrong, or, more unlikely, are the erasing techniques used generally, including by Bitdefender, obviously needlessly complicated and time-consuming?
     
  2. jcsd
  3. Feb 28, 2018 #2

    DrClaude

    User Avatar

    Staff: Mentor

    For magnetic storage devices, such as hard disks, there is residual magnetism that can be measured. So even if all bits are set to 0 or 1, with the proper tools one can still read the information that was there before. Cycling many times between 0's and 1's reduces this residual magnetism, making it harder to find the previous state of each bit.

    I do not know if the same is true of other types of memory, like SSD, but I assume it could be. You have to remember that each bit is not held by a single atom/molecule, but by a huge bunch of them, such that writing a new bit doesn't change completely the state of the underlying medium (most atoms/molecules change their state, not all of them).

    As for Bitdefender's approach of using random strings, I don't know if this is more efficient than just cycling 0's and 1's.
     
  4. Feb 28, 2018 #3

    jedishrfu

    Staff: Mentor

    When data is written to a hard disk, it is streamed to a magnetic head while the disk spins underneath it. An actuator moves the head from track to track as it writes your data. Each track has a predefined width that the head travels in. Because of vibration the head doesn't travel in a simple circle but randomly wiggles back and forth a bit.as the track is written.

    https://en.wikipedia.org/wiki/Hard_disk_drive

    Whether you writing zeros or ones there will still be some residual data because the head never passes over the same spot on the track to completely overwrite/erase the prior data written there meaning it could be read back so multiple writes of random data obscure it more completely.

    The other part is that when you delete files on your hard disk they aren't actually deleted. The file is unlinked from the filename to disk sector mapping but the data will still be there for awhile (hence recovery programs can often bring back deleted files in the near term). In the longer term, the OS will begin to reuse the deleted tracks once its cycled thru its free tracks.

    Imagine having a column of names on a whiteboard new names are added to the bottom of the list and deleted names are simply erased. Once you hit the bottom of the whiteboard writing new names then you go back to the top and fill in the blanked/erased rows with new names. That's kind of how a hard disk works. THis strategy also benefits the hard disk in not reusing the some areas more than others which could lead to faster hard disk failures as the head may scrap the disk surface slightly with each pass and and disk shock.

    http://www.explainthatstuff.com/harddrive.html

    SSDs and USB sticks are electronic memory based with different operating characteristics from hard drives. I don't think they have the same issue when overwrites are done. However, they do practice the notion of minimal writes to a given location to preserve the SSD.

    https://en.wikipedia.org/wiki/Solid-state_drive
     
  5. Mar 6, 2018 #4
    DrClaude and jedishrfu,

    Thank you for your answers. Since I read them I've been looking online for more articles about erasing data, and found, as is often the case online, considerable differences of opinion about what techniques are best in different situations. Without going into tedious details, the consensus, to the extent it exists, seems to be that to erase moderately sensitive data on magnetic drives that are going to be resold or junked, several, but not dozens, of passes writing randomly selected 0's and 1's is desirable and also sufficient. For highly confidential information, destruction of the drives is advisable. For personal computers being resold or junked, maybe just writing all 0's or all 1's is good enough.

    Actually what I had in mind by my "most modern digital storage devices" were SSD's and USB flash drives. For them, although it's not entirely clear, writing all 0's or all 1's may always be sufficient in those cases where it can be done, but often the construction and programming of such drives makes the overwriting of certain data on them about impossible, so different techniques, such as encryption of the data before storage or destruction of the drives, should be used. You probably already know more about this than I do.

    Wiggling of the write head on a hard disk is something I hadn't thought about.
     
  6. Mar 6, 2018 #5

    jedishrfu

    Staff: Mentor

    I think you summarized the state of data erasure well.

    If the computer has been used to conduct online banking, shopping, home finance, tax preparation... then I'd be inclined to remove the hard drive.

    There are folks who buy junk computers just to see what they can find on the hard drive.

    I wouldn't want to risk that scenario so I remove the drive and destroy it instead.
     
  7. Mar 6, 2018 #6
    There was some experiments where overwritten data could be restored based on residual magnetism, but that was back in the MFM era. Around 30 (or more) years ago.
    These days, it's true that with complicated instruments remnants of previous data can be identified, but that's all. It can be identified, and things stops at this point. Like a skeleton can be identified - it is a skeleton, see?

    Overwriting is completely safe for sensitive data too. What makes the difference is, that for sensitive data the staff who does the overwriting is not to be trusted, so it is a requirement to identify the wreck of the drive before disposing it.

    The very point is, that for magnetic storage it has to be physical overwriting. Just logical 'delete' won't do.
     
  8. Mar 6, 2018 #7

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2017 Award

    I think you need to decide who you want to keep from reading your stuff. A random tech-savvy criminal? Or a major world government? That will determine the steps you need to take in protecting it. For me, a secure erase followed by a few hammer blows is plenty.

    The more interesting question is what to do with a failed drive that used to have sensitive data on it.
     
  9. Mar 6, 2018 #8
    For my failed drives, I crack the drive open using a hammer, and for drives with platters, I grind the surfaces down with a belt sander. I physically break up the rest of the drive and put the pieces into a crock pot full of salty brine overnight. I let them cool in the brine after that until it's trash day, then throw the parts out. Any working drive I am disposing of, I tend to follow the above steps for as well. I work in IT. The last thing I want is someone one-upping me on drive security.

    I second Vanadium's response - in general you are probably good with even a basic secure erase that overwrites the drive with random data 1-3 times. If you're a bit paranoid, or a perfectionist, overwrite ten times.

    I would add one more thing - if you are truly worried about your data being taken, encrypt your drive - both because if the drive is stolen it cannot be read, and because if you later wipe the drive, the data you are obscuring is encrypted - if someone did get the data, it wouldn't be much good to them.
     
  10. Mar 13, 2018 #9

    Svein

    User Avatar
    Science Advisor

    CCleaner has a Drive Wiper tool.
    So does the Paragon Hard Disk Suite.
    I seem to remember several freeware tools that do the same thing.
     
  11. Mar 14, 2018 #10

    OCR

    User Avatar

    I don't believe CCleaner can erase your Local Disk, though...?
    It won't on my computer, anyway...

    upload_2018-3-14_0-17-24.png

    .
     
  12. Mar 14, 2018 #11
    Then get a Linux stick and use the BADBLOCKS program in destructive mode.

    Better to have some cooling for the drive, because it'll get some heat...
     
  13. Mar 14, 2018 #12

    Vanadium 50

    User Avatar
    Staff Emeritus
    Science Advisor
    Education Advisor
    2017 Award

    No Windows program can erase the boot drive since the code it needs to run is on that drive.
     
  14. Mar 14, 2018 #13

    OCR

    User Avatar

    :check:... Those were my thoughts too.
     
    Last edited: Mar 14, 2018
  15. Apr 12, 2018 #14
    I don't know why Svein recommends the CCleaner tool , only those who have used it know that it is a waste of time. I've used a tool that's helpful but it's not freeware.
     
  16. Apr 12, 2018 #15

    Filip Larsen

    User Avatar
    Gold Member

    Another approach to allow for secure erasing data on a drive is to employ encryption on the full drive or selected files from the start and then securely erase the key when the disc or files are to be erased.

    For devices such as SSD that may not overwrite the same physical memory block when you overwrite the logical blocks belonging to a file, encryption (and physical destruction) is a much more secure choice for erasing data than overwriting. Also, some USB drives come with built-in encryption and secure reformatting.
     
  17. Apr 12, 2018 #16

    FactChecker

    User Avatar
    Science Advisor
    Gold Member
    2017 Award

    Physically destroying the drive is an order of magnitude faster than over-writing the drive even once. Over-writing several times would take a very long time. Unless you are worried that a professional spy will be trying to get your data, you can do a reasonable job of "destroying" it in 10 minutes with a sledge hammer or a drill.

    A year ago, I had to discard 15 to 20 old drives. If I over-wrote them, I would still be working on it. (an exageration)
     
    Last edited: Apr 12, 2018
  18. Apr 12, 2018 #17

    Svein

    User Avatar
    Science Advisor

    Well, I only said that CCleaner had the option to wipe a drive. I also said that Paragon Hard Disk Suite has that option, which I am currently using - it takes 4.5 hours to wipe a 74Gbyte drive!
     
  19. Apr 12, 2018 #18

    phyzguy

    User Avatar
    Science Advisor

    I don't think anyone has answered the question with respect to solid state drives and USB "thumb" drives. On these devices, the data is stored in floating gate non-volatile memory cells. Unlike magnetic drives, I don't think there is any way to recover the data from these devices once it is erased. So a one-pass simple erasure (with all 1's, all 0's, or any random pattern) should be sufficient. Does anyone know differently?
     
  20. Apr 12, 2018 #19

    Filip Larsen

    User Avatar
    Gold Member

    I would image that a general flash drive tool would have difficulty ensuring that any bad blocks on the drive has been securely erased. A bad block would not normally be readable, but if bad block management (BBM) is only implemented in the host computer driver and not the on on-chip controller (I am not aware if all flash drives have on-chip BBM, though I think most have) then it seems plausible that special hardware readers may be able to read sensitive data that has "gotten stuck" in those bad blocks. For drives with on-chip BBM special data forensic equipment may still be able to recover content of some of the bad blocks on the drive.
     
  21. Apr 22, 2018 #20
  22. Apr 22, 2018 #21

    FactChecker

    User Avatar
    Science Advisor
    Gold Member
    2017 Award

    The only completely sure way:
     
  23. Apr 22, 2018 #22
    @ Fact Checker, it's true that the most reliable way to destroy data is to destroy the integrity of the physical medium that holds it, e.g. by extreme heat, perhaps by lowering it into a molten metal crucible as the video clip you posted illustrates, or maybe by tossing it into a magma pool of an active volcano ...

    Orodruin12.jpg
     
  24. Apr 27, 2018 #23
    I saw this on Google yesterday.maybe you can get what you want ....
     
  25. Apr 27, 2018 #24
    It looks to me like you're panning well-established freeware and plugging a trial of something that is less well-established. The freeware version of CCleaner is able to implement the same erasure options as are listed at the site at the location a link to which you posted.

    From a page at the CCleaner site:
    CCleaner has four methods of secure deletion: a Simple Overwrite (1 pass), DOD 5220.22-M (3 passes), NSA (7 passes), and Gutmann (35 passes). A 'pass' refers to how many times CCleaner writes over the spot on the hard drive. The more times CCleaner writes to that spot, the harder the file will be to recover by any means. The drawback is that it will take CCleaner longer to complete the job.​
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Have something to add?
Draft saved Draft deleted