df vs du. Is my disk really full?

Question

Strange problem...

Why i have full / partition used, but it's not really used?

Fast info:

xwing ~ # df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs           16G   15G   75M 100% /
/dev/root        16G   15G   75M 100% /
devtmpfs        5,9G     0  5,9G   0% /dev
tmpfs           5,9G  552K  5,9G   1% /run
rc-svcdir       1,0M   72K  952K   8% /lib64/rc/init.d
cgroup_root      10M     0   10M   0% /sys/fs/cgroup
shm             5,9G     0  5,9G   0% /dev/shm
cachedir        4,0M  4,0K  4,0M   1% /lib64/splash/cache
/dev/sda1       124M   43M   76M  36% /boot
/dev/sda5        63G   25G   36G  42% /home
/dev/sda6       483G  147G  312G  33% /mnt/data
tmpfs           8,0G     0  8,0G   0% /var/tmp/portage

Maybe i-nodes? Noo...

xwing ~ # df -i
Filesystem       Inodes  IUsed    IFree IUse% Mounted on
rootfs          1048576 548459   500117   53% /
/dev/root       1048576 548459   500117   53% /
devtmpfs        1525561    517  1525044    1% /dev
tmpfs           1525918    374  1525544    1% /run
rc-svcdir       1525918     61  1525857    1% /lib64/rc/init.d
cgroup_root     1525918      5  1525913    1% /sys/fs/cgroup
shm             1525918      1  1525917    1% /dev/shm
cachedir        1525918      2  1525916    1% /lib64/splash/cache
/dev/sda1         32768    314    32454    1% /boot
/dev/sda5       4194304 145765  4048539    4% /home
/dev/sda6      32153600  11325 32142275    1% /mnt/data
tmpfs           1525918      1  1525917    1% /var/tmp/portage

But:

xwing ~ # du -sxh /
8,3G    /

Maybe i have something broken in mount? (but anyway, notice i used du with -x which calculates files ONLY from / filesystem...)

/spaces added by me for better readibility/

xwing ~ # mount
rootfs      on /                        type rootfs   (rw)
/dev/root   on /                        type ext4     (rw,noatime,data=ordered)
devtmpfs    on /dev                     type devtmpfs (rw,relatime,size=6102244k,nr_inodes=1525561,mode=755)
proc        on /proc                    type proc     (rw,nosuid,nodev,noexec,relatime)
tmpfs       on /run                     type tmpfs    (rw,nosuid,nodev,relatime,mode=755)
rc-svcdir   on /lib64/rc/init.d         type tmpfs    (rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755)
sysfs       on /sys                     type sysfs    (rw,nosuid,nodev,noexec,relatime)
debugfs     on /sys/kernel/debug        type debugfs  (rw,nosuid,nodev,noexec,relatime)
cgroup_root on /sys/fs/cgroup           type tmpfs    (rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755)
cpuset      on /sys/fs/cgroup/cpuset    type cgroup   (rw,nosuid,nodev,noexec,relatime,cpuset)
cpu         on /sys/fs/cgroup/cpu       type cgroup   (rw,nosuid,nodev,noexec,relatime,cpu)
cpuacct     on /sys/fs/cgroup/cpuacct   type cgroup   (rw,nosuid,nodev,noexec,relatime,cpuacct)
freezer     on /sys/fs/cgroup/freezer   type cgroup   (rw,nosuid,nodev,noexec,relatime,freezer)
devpts      on /dev/pts                 type devpts   (rw,nosuid,noexec,relatime,gid=5,mode=620)
shm         on /dev/shm                 type tmpfs    (rw,nosuid,nodev,noexec,relatime)
cachedir    on /lib64/splash/cache      type tmpfs    (rw,nosuid,nodev,noexec,noatime,size=4096k,mode=755)
/dev/sda1   on /boot                    type ext2     (rw,noatime)
/dev/sda5   on /home                    type ext4     (rw,noatime)
/dev/sda6   on /mnt/data                type ext4     (rw,noatime)
tmpfs       on /var/tmp/portage         type tmpfs    (rw,size=8G)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)

Kernel:

morsik@xwing ~ $ uname -a
Linux xwing 3.6.2-gentooxwing #8 SMP Sun Oct 21 21:36:53 CEST 2012 x86_64 Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz GenuineIntel GNU/Linux

Really... I'm out of possible ideas about it.

What i tried:

• check if there are some fd still open after rm -rf /usr/portage/distfiles/*
• sync
• reboot
• fsck

EDIT AND SOLUTION:

As always i found solution after asking for help (-;

Mounting (as Luke404 said) / partition somewhere else and doing du -sh /mnt/something told there's 15G of data, so the same like in df.

What was the problem?

mount!

I created /home partition later, added to /etc/fstab, I copied files from /home (on / partition) to /home (on /home partition) but i FORGOT to remove files from /home at / partition.

So I still had my old files on /home (on / partition).

But why du -sxh told me 8G? Easy...

-x forces to not check different mountpoints, so... it didn't checked /home while it was mounted. And there was my missing /home.

Thanks to Luke404 and pidpawel who helped me on IRC.

After writing this EDIT section i saw answers here, and Scrivener got point here!

Stupid problems are stupid (-;

Answer 1

This kind of problem is always really fun.

My guess? You have files underneath one of your mountpoints. What does that mean? Well, du -sh will go by hand through all of the files that it can find.

Let's say I have a directory, A. If I dump a bunch of files into A, and then mount, say, /dev/sdb on top of A, df -h will say I have a bunch of files that du won't be able to find!

This happens most often with mountpoints that mount and umount regularly, like USB drives, network drives, etc. It's much rarer to see it when the mounts happen automatically during boot, since the window for it to occur is so much smaller.

Answer 2

Have you checked out for files in (currently) unreachable places? Like stuff under /home on the root filesystem, that your du can't see because you have mounted another filesystem on /home ?

You can take a look at those by bind-mounting the filesystem somewhere else:

# mkdir /foo
# mount -o bind / /foo
# cd /foo
# du...