As daunting as securing your Linux system might seem, one thing to remember is that every extra step makes a difference. It’s almost always better to make a modest stride than let uncertainty keep you from starting.
Fortunately, there are a few basic techniques that greatly benefit users at all levels, and one of them is knowing how to securely wipe your hard drive in Linux. Because I adopted Linux primarily with security in mind, this is one of the first things I learned. Once you have absorbed this lesson, you will be able to part with your hard drives safely.
As you might have deduced, the usual way of deleting doesn’t always cut it. The most often-used processes for deleting files — clicking “delete” in the operating system or using the “rm” command — are not secure.
When you use one of these methods, all your hard drive does is mark the area where the deleted file used to be as available for new data to be written there. In other words, the original state of the bits (1s and 0s) of the deleted file are left intact, and forensic tools can recover the files.
This might seem like a bad idea, but it makes sense. Hard drives are designed to optimize hardware integrity, not security. Your hard drive would wear out very quickly if it reset the bits of a deleted file to all 0s every time you deleted a file.
Another process devised with hard drive lifespan in mind is “wear leveling,” a firmware routine that saves each new file in a random location on the drive. This prevents your drive from wearing out data cells, as those near the beginning of the drive would suffer the most wear if it saved data sequentially. However, this means it is unlikely that you ever would naturally overwrite a file just through long-term use of the drive.
So, what does it mean to “securely wipe” a hard drive?
Moving Raw Bits
Secure deletion involves using a program to overwrite the hard drive manually with all 0s (or random data). This useless data overwrites the entire drive, including every bit of every saved and deleted file. It even overwrites the operating system, leaving nothing for a malicious actor to exploit.
Since the command line is usually the simplest way of going about manual operations like this, I will go over this method. The best utility for this is the “dd” command.
The “dd” commamd can be used for many things besides secure deleting, like making exact backups or installing Linux distributions to USB flash drives, but what makes it so versatile is that whereas commands like “mv” and “cp” move around files as file objects, “dd” moves data around as a stream of raw bits. Essentially, while “mv” and “cp” see files, “dd” only sees bits.
What “dd” does is very simple: It takes an input and sends it to an output. Your Linux system has a stream of 0s it can read located at /dev/zero. This is not a normal file — it’s an endless stream of 0s represented as a file.
This will be our input for a wipe operation for the purpose of this tutorial. The output will be the device to be overwritten. We will not be overwriting an actual running system, as 1) you probably wouldn’t want to, and 2) it actually wouldn’t work because your system would overwrite the part of the system responsible for performing the overwrite before the overwrite was complete.
Securely erasing external storage devices, like USB flash drives and external hard drives is pretty straightforward, but for wiping your computer’s onboard hard drive, there are some extra steps involved.
The Live-Boot Option
If you can’t use a running system to wipe an onboard drive, how do you perform the operation? The answer is live-booting. Many Linux distributions, including those not explicitly specialized for the purpose, can be loaded and run on a computer from a connected USB drive instead of its onboard drive. When booted this way, the computer’s onboard drive is not accessed at all since the system’s data is read entirely from the USB drive.
Since you likely installed your system from a bootable USB drive, it is best to use that. To live-boot, we have to change the place where the computer checks to find an operating system to run by entering the BIOS menu.
The BIOS is the firmware code that is loaded before any part of any OS is run, and by hitting the right key at boot time, we can access its menu. This key is different on different computers. It’s usually one of the “F” keys, but it might be something else, so it might take a few tries to figure it out, but the first screen that displays should indicate where to look.
Once you find it, insert the live-boot USB, reboot the computer directly into the BIOS menu, and select the option to change the boot order. You should then see a list of storage devices, including the inserted USB. Select this and the live system should come up.
Locating the Right Address
Before we do any deleting, we have to figure out which address our system assigns to the drive to be deleted (i.e., the target drive). To do that, we will use the “lsblk” command for “list block devices.” It returns information about attached block devices, which are essentially hard drive-type devices.
Before running the command, take note of the target drive’s storage size, and detach all devices connected to your computer EXCEPT the drive storing the system you are live-booting from. Then, run “lsblk” with no arguments or options.
$ lsblk
The only device that should appear is your onboard hard drive and the live-booted USB. You will notice that “lsblk” returns a name (under “NAME”) beginning with “sd” and then a letter, with branching lines to the same name appended with a number. The name the branches originate from is the name of the “file” serving as the address of the drive in the /dev directory, a special directory that represents devices as files so the system can interact with them.
You should see an entry with the size of the USB drive hosting the live-boot system and a path under “MOUNTPOINT”, and (only) one other entry with the size of your target drive with no mount point listed. This second entry gives you the address for the output of “dd”. For instance, if your target drive corresponds to the name “sdb”, then that means /dev/sdb is the address.
However, to identify the address of an external drive you want to delete, run “lsblk” once with no device attached, check the (single) entry against your onboard drive’s size and make a note of its address, connect your target drive, run “lsblk” again, and check that its size corresponds to that of one of the entries in the output.
The output of the second “lsblk” command should now return two entries instead of one, and one of them should match target’s size. If your system is configured to automatically access inserted drives, you should see a path including “/media” under “MOUNTPOINT”, but otherwise the target drive should list nothing in that column.
As these addresses correspond to hard drives, it is important to be EXTREMELY careful to give the right one, because otherwise you will delete the wrong drive. As I noted earlier, if you accidentally give the address of your running system as the output, the command will immediately start writing zeros until you stop it (by hitting “Ctrl-c”) or your system crashes, resulting in irrecoverable data loss either way.
For example, since the letters are assigned alphabetically, starting (usually) with the running system, if a single connected external drive is the target, it probably will be addressed as /dev/sdb. But, again, check this carefully because it may be different for you.
Foiling Identity Thieves
Now we’re ready to delete. All we do is invoke “dd,” give /dev/zero as the input, and give our target (for this example, /dev/sdb) as the output. “dd” is an old command from the time before Linux, so it has a somewhat odd syntax. Instead of options prepended with dashes (“-“), it uses “if=” for “input file” and “of=” for “output file.” Our command, then, looks like this.
$ dd if=/dev/zero of=/dev/sdb
Depending on how big the target drive is and how fast your processor is, this could take a while. With a powerful processor wiping a 16-GB flash drive, this could take as little as 10 minutes. For an average processor writing over a 1-TB drive, though, it could take a whole day. You can do other things with your computer (though not with that terminal), but they probably will be slower, as this is a comparatively processor-intensive task.
Though this is probably not something you’ll do often, knowing how definitely will serve you well in the rare instances when need to. Identity theft from forensically analyzing discarded drives happen all the time, and this simple procedure will go a long way toward defending against it.
Loading ...
I beg to differ with the nay-sayers. Overwriting every addressable byte on your disk with ‘0’ will definitely add a significant barrier to anyone trying to recover that data.
Can some magnetic residue of the original data be remaining on the disk? Maybe. But the cost of the equipment and expertise to recover it is going to steep. Maybe if your of interest to a well funded spy or criminal organization will you have a real concern.
Flash is a little different story. Flash cells should erase cleaner. But there is the reserve sectors and the question of whether they are rotated through with the usual wear leveling algorithms. But this can be addressed with multiple passes. I would probably use 0 (/dev/zero) on the first pass and then 0xff on the second and repeat the cycle again. If the spare sectors are part of the wear leveling cycle 4x the storage capacity will be more than enough to hit them. I leave off with the 0xff because that leaves the flash in the native "erased" state, ready for writing new data.
I don’t know of a readily available source of 0xff bytes but the code to generate them with a compiled language is stupid simple. I wrote such a tool a long time back, which I still use. I would use a compiled language because we are talking about generating a *LOT* of bytes and interpreted languages will take much longer.
Some things to consider:
‘dd’ writes to the kernel’s cache, like most programs. So you need to make sure the cache gets flushed to the device. Using the "sync" command as root can do that or shutdown/reboot will force the flush.
You can pipe any source of data into dd if you want to use a different erase pattern. Just use the standard pipe (|) arrangement. You could also use /dev/random or /dev/urandom to write random data. The first is probably completely useless as it would take a very long time for your system to generate the random data. The latter is faster because the random data is of lesser quality. It will still take a long time.
I would contend with the authors remark that this procedure is CPU bound. Although his formula is not as efficient as it could be its mostly an I/O bound operation. On a decent SATA or SCSI system I have routinely kept a drive saturated with I/O, this way, while not observing a noticeable impact on the rest of the system. The main thing you have to do is tell dd to use a larger buffer with the "bs", block size, option. I use 1M and it seems to provide real good throughput… and yes it will write a partial block if needed.
So something like:
dd if=/dev/zero of=/dev/sda bs=1M
And, yes, a tool like "shred" will do a more thorough job at trying to prevent someone scrounging magnetic residue.
The title is misleading. This is not a way to "secure your Linux system", rather to "securely delete your files". And it fails at that since the usage of dd for security wipe is not considered 100% safe, other tools are available (shred for example).
This is bad advice, I think.
A while back I did some research on wiping data. While I don’t have time now to refer to my notes, (someone correct me if I’m wrong) the impression I got was that:
1.) The DD command is an unreliable way to wipe data on today’s more technologically complex hard drives.
2.) Thumbdrives are entirely different from disk drives and cannot be securely deleted using conventional means, if at all. As far as I know, a readily available solution does not exist for this.
3. The safest way to wipe a hard disk drive is to download a data wiping program from the drive manufacture’s website designed for that specific drive. They know how their drive is built. (This is what I did before donating an old personal computer to a business.)
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4…)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I’ve heard encryption recommended as a sure way to wipe drives… just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It’s been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn’t toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive’s firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive’s firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I’ve already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers… and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday’s encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it’s not, the drive should be incinerated and you should save the receipt (smiles).
We’re taking disk drives here, not thumbdrives… or email servers or ISP’s or Google, which is a whole different subject.
Regards,
(continued)
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4…)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
(continued)
IS DRIVE ENCRYPTION A SOLUTION?
I’ve heard encryption recommended as a sure way to wipe drives… just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
Other points worth consideration:
DOES IT MATTER WHAT FILE SYSTEM YOU USE?
Before manually wiping a drive using DD or formatting, have you ever changed the file system (NTFS, Ext4…)? There may be isolated sections of the older filesystem that are untouchable by the new filesystem, even with formatting.
A filesystem does not perfectly fill all the space on a drive, and you might even had set up multiple partitions.
IS DRIVE ENCRYPTION A SOLUTION?
I’ve heard encryption recommended as a sure way to wipe drives… just encrypt the entire drive using an impossibly long password, then forget the password. However, I think even this may be a misrepresentation.
One of the concerns of safeguarding data is that a recovery expert might disassemble the drive, place it in a expensive scanner, and recover residual information, (such as between the tracks on a disk drive) or otherwise piece together enough leftover data to recover some partial files.
This is why, when wiping data, a drive is formatted several times over and/or is overwritten with random data instead of zeros.
If you are concerned that data might be recovered in this manner, (by accessing underlying/residual data or data fragments) then any encrypted data would be bypassed. You might just as well have written random data over your drive.
IS RESIDUAL DATA RECOVERY REALLY POSSIBLE?
It’s been suggested that on modern hard disk drives, with their densely compacted data tracks, such recovery techniques are not practical or at least unrealistically expensive to attempt.
But I wouldn’t toss such concerns entirely out the window. And there may be other, simpler techniques such as scanning the sectors that have been marked as bad or unreliable by the drive’s firmware.
You may have old data on sections of the drive that are now entirely unaccessible to you and your formatting attempts, but which could be easily read if the drive’s firmware chip was replaced. This may apply to thumdrives as well as disk drives.
WILL IT REALLY TAKE 1,000 YEARS TO CRACK A GOOD ENCRYPTION?
I’ve already mentioned how relying on encryption to wipe data may not be a perfect solution. Besides this, there is another factor which *nobody* seems to consider.
While the premise of "1,000 years to crack" might be correct if you used the same computer for that 1,000 years, does such analysis account for the ever-increasing speed of computers… and can it?
What speeds might the next breakthrough in processor technology achieve? Who can say what the next discovery in mathematically decryption will be? Yesterday’s encryption could be rendered obsolete tomorrow.
Encryption is good for information in transit at that time. But if the same information might be of value in the future, a sinister character may simply hold on to it until such time it can be readily decrypted.
This applies to stolen data drives or intercepted internet transactions. And such risk factors *must* be assessed for any important data protection strategy.
THE CONCLUSION?
To keep from going bonkers, it helps to remember that data protection is really about risk assessment.
For your own personal or small business computer that you want to give away, a DD or full format on the existing filesystem followed by a data wipe using manufacturer software should be sufficient. And, really, if it’s not, the drive should be incinerated and you should save the receipt (smiles).
We’re taking disk drives here, not thumb drives… or email servers or ISP’s or Google, which is a whole different subject.
Regards,
Hi,Thank you for sharing this blog now we can know more thing about linux, these all point are very effective and important.