Intro To 'shred' Command In Linux

2023-06-03 - By Robert Elder

I use the 'shred' command to overwrite files with the intention of destroying their data:

shred data.txt

Overwriting Sensitive Data

Here, I have some extremely private information in a file called 'my-diary.txt' that I don't want the public to see:

March 15, 2023

Dear Diary, Today was a really tough day.  I posted my opinion on Twitter, and
some people disagreed with me!  How could they be so self-centered and
inconsiderate?  Don't they realize that my ideas and opinions are highly
advanced and also superior?  Some people are stuck living inside their own head!

March 16, 2023

Wow!  I'm literally shaking with excitement!  Today is going VERY well.  I
just WON the comment section on a reddit thread today!  It was a post on proper
etiquette to use when creating a pull request on github.  I talked about some
academic studies that researched the effect of emjoi use in collaborative
coding work.  I mean, I just made it all up, but people upvoted it anyway (lol)
but now everybody can see how smart and honest I am!

It was a close call though.  At first, I made a comment where I confused 'git'
with 'github' because I said that they're the same thing, but then someone
made a detailed reply and politely explained that I'm wrong, but it's OK,
because I deleted my comment as soon as they replied to me.

March 18, 2023

Today was a rough day again...   Some people in discord chat said mean things
about me, so I demanded that the admin make me a mod so I could ban those
people, but the admin said no!   If only people would acknowledge my high IQ...

I can destroy the data in this file using the 'shred' command like this:

shred my-diary.txt

and now the data in the file has been overwritten with random bits:

xxd my-diary.txt | head -n 10

00000000: 5376 21d5 5129 df87 fb75 f3ef 6569 0c95  Sv!.Q)...u..ei..
00000010: ed92 8d04 417e cc14 1866 893d 8f5e 4361  ....A~...f.=.^Ca
00000020: 4819 6f40 8494 10f8 ed43 b6ee cd25 e3eb  H.o@.....C...%..
00000030: f10b d042 704c 00fb 78b5 93bf 2b4d 26e6  ...BpL..x...+M&.
00000040: 0a70 e1cb ddb2 0f73 3bfa 8e10 a455 1752  .p.....s;....U.R
00000050: 3cb1 60c8 6b38 db1d 9c65 0f0e 49ba 85df  <.`.k8...e..I...
00000060: a311 a759 54aa f177 f9f0 307e 1d80 f8db  ...YT..w..0~....
00000070: de2a 0417 08b2 136f 190f 758f d76c 6bb9  .*.....o..u..lk.
00000080: 8767 8214 9d96 1dfa 66df 09aa f3a7 4c80  .g......f.....L.
00000090: 2cc2 03c6 5bb0 f5d9 1487 e84e a2e8 ddd8  ,...[......N....

The 'shred' command supports various flags that change its behavior, such as the number of bytes in the file to be shredded, the source of randomization, or what to do after the file has been overwritten.

Not Very Useful

If security is an important consideration, the 'shred' command is unlikely to be useful when erasing files within the context of a file system. There are many implementation details like caching and wear leveling they can create duplicate copies of the data that the 'shred' command won't know about.

Some of these caveats are documented in the man page of the 'shred' command:

       CAUTION: Note that shred relies on a very important assumption: that the file system overwrites data in  place.
       This  is  the traditional way to do things, but many modern file system designs do not satisfy this assumption.
       The following are examples of file systems on which shred is not effective, or is not guaranteed to  be  effec‐
       tive in all file system modes:

       *  log-structured  or  journaled  file systems, such as those supplied with AIX and Solaris (and JFS, ReiserFS,
       XFS, Ext3, etc.)

       * file systems that write redundant data and carry on even if some writes fail, such as RAID-based file systems

       * file systems that make snapshots, such as Network Appliance's NFS server

       * file systems that cache in temporary locations, such as NFS version 3 clients

       * compressed file systems

       In the case of ext3 file systems, the above disclaimer applies (and shred is  thus  of  limited  effectiveness)
       only  in  data=journal  mode,  which journals file data in addition to just metadata.  In both the data=ordered
       (default) and data=writeback modes, shred works as usual.  Ext3 journaling modes can be changed by  adding  the
       data=something  option  to the mount options for a particular file system in the /etc/fstab file, as documented
       in the mount man page (man mount).

When Is The 'shred' Command Actually Useful?

The 'shred' command may be useful when destroying on an entire block device like this:

#  Destroy all data on /dev/sdx
shred /dev/sdx

In this case, the 'shred' command offers a quick way to write random data to the entire disk surface. Since we're destroying all of the data, including filesystems and partitions, there is a lower chance of sensitive data remaining on the disk without our knowledge. Although there is a lower chance of missing sensitive data, the chance is still not zero due to things like internal drive buffers, caches, of security sectors, over-provisioned sectors etc. Having said all this, the same 'shredding' effect can be accomplished using this 'dd' command:

#  Destroy all data on /dev/sdx
dd if=/dev/urandom of=/dev/sdx

so the 'shred' command doesn't really add much.

And that's why the 'shred' command is my favourite Linux command.

Intro To 'stty' Command In Linux Published 2023-10-04	$1.00 CAD Terminal Block Mining Simulation Game	Intro To 'nproc' Command In Linux Published 2023-07-15	Intro To 'comm' Command In Linux Published 2023-09-06
How To Force The 'true' Command To Return 'false' Published 2023-07-09	A Surprisingly Common Mistake Involving Wildcards & The Find Command Published 2020-01-21	A Guide to Recording 660FPS Video On A $6 Raspberry Pi Camera Published 2019-08-01	Intro To 'chroot' Command In Linux Published 2023-06-23

Why Bother Subscribing?

Free Software/Engineering Content. I publish all of my educational content publicly for free so everybody can make use of it. Why bother signing up for a paid 'course', when you can just sign up for this email list?
Read about cool new products that I'm building. How do I make money? Glad you asked! You'll get some emails with examples of things that I sell. You might even get some business ideas of your own :)
People actually like this email list. I know that sounds crazy, because who actually subscribes to email lists these days, right? Well, some do, and if you end up not liking it, I give you permission to unsubscribe and mark it as spam.