Defragmenting Linux EXT3 Filesystems Using Shake And Defrag

I know everybody says EXT3 and EXT4 filesystems don't need defragmentation and in most cases, that's true! But sometimes they actually do need defragmentation. There is an entire discussion rather or not EXT3 needs defragmentation but I'll not get into that. It just depends on the way a filesystem is used, when it was last formated and how full it is. So don't take my word for it, and go take a look at the fragmentation level (will tell you how to do that in just a moment) of your hard drive.

I've recently scanned my drives and found 1 partition (which hasn't been formatted in a very very long time) having a fragmentation level of over 70%. Yes, I was amazed myself, but it seems this is possible! That must be because I use that partition for BitTorrent downloads so I constantly download new files, delete others, etc. BitTorrent clients constantly read and write files which is quite bad from the fragmentation point of view. For instance, my home partition has a fragmentation level of under 1%, but I never download torrents there.

How to check the fragmentation level in Linux

Firstly, the partition you want to check, must be unmounted. For that you can either use the icon next to a drive in Gnome (Nautilus), Gparted (Ubuntu: sudo apt-get install gparted) or palimpsest (in Ubuntu Karmic Koala - comes by default). Or do it from command line.

Then run fsck in a terminal:

fsck -nvf /dev/hda1

replace "hda1" with your partition.

You will then see something like this:

fsck from util-linux-ng 2.16
e2fsck 1.41.9 (22-Aug-2009)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

1955 inodes used (0.08%)
    1554 non-contiguous files (79.5%)
0 non-contiguous directories (0.0%)
# of inodes with ind/dind/tind blocks: 1660/1224/0
3706004 blocks used (72.40%)
0 bad blocks
1 large file

1825 regular files
121 directories
0 character device files
0 block device files
0 fifos
0 links
0 symbolic links (0 fast symbolic links)
0 sockets
--------
1946 files

Notice the "non-contiguous" part - that's the fragmentation level for that partition (smaller is better).

Defragmenting the Linux partitions

There are a few tools for this, and I'm going to cover Shake and Defrag.

1. Shake is a defragmenter that runs in userspace, without the need of patching the kernel and while the system is used (for now, on GNU/Linux only).

It works by rewriting fragmented files. But it has some heuristics that could make it more efficient than other tools, including defrag and, maybe, xfs_fsr.

As an example, it allows you to write find -iname '*.mp3' | sort | shake to defrag all mp3 in a directory, putting together on the disk those close in lexical order.

Installing Shake

In Ubuntu, you can add Shake PPA.

Alternatively, you can simply download Shake .deb files:

• shake-fs_0.99_amd64.deb (23.3 KiB)
• shake-fs_0.99_i386.deb (22.6 KiB)

The .deb and repository are for Ubuntu Intrepid, but I just ran it in Ubuntu Karmic Koala and it works just fine.

For other Linux distributions: There is an installer that you can run as a shell script.

Preparing for Defragmentation

After installation, to use Shake, you need to edit your fstab file:

gksu gedit /etc/fstab

And add the user_xattr option for the partition you want to defragment (even if you only want to defragment files).

Here is a preview on how the line of the partition in question will look like after adding the user_xattr option:

/dev/sdb6 /media/JOCURI ext3 noatime,nodiratime,user_xattr,errors=remount-ro,data=writeback  0 1

(don't worry if for your partition, there are other options in the fstab file).

Now, if the partition wasn't unmounted, run this command:

mount -o remount MY_PARTITION

Replacing MY_PARTITION with the partition you want to run Shake on.

If the partition was unmounted, simply run this in a terminal:

mount -a

You can also use Shake to only see the fragmentation level, using this command:

shake --pretend --verbose --verbose my_dir

Where "my_dir" can be the whole partition or just a folder, but not the partition as in /dev/sda1, but the folder where it is mounted, as in /media/disk1.

Running the actual defragmentation using Shake

Simply run:

shake my_folder

Again: "my_dir" can be the whole partition or just a folder, but not the partition as in /dev/sda1, but the folder where it is mounted, as in /media/disk1.

Now go do something useful until this Shake finishes it's job :)

Then, check again the fragmentation level.

For more info on Shake, visit it's HomePage.

Defragmenting using Defrag

2. Defrag is a shell script. Download the latest version from HERE.

Using Defrag is very easy: copy the script in a folder you want to defragment. This can be just a normal folder or the top level directory of a partition.

To run it, open a terminal, navigate to the folder where defrag was copied (this also being the folder you want to defragment), and type:

sh defrag

P.s.: You need to run it on a mounted partition.


The maximum benefit seems to occur with two runs, and it seems to be effective only if the initial fragmentation is > 2.5%.



Note: I've only used Shake and defrag on my EXT3 filesystem and couldn't find any info if they work on EXT4, so use it on EXT4 at your own risk!