The Wiert Corner – irregular stream of stuff

In the mean time I’ve made a bit of progress on btrfs free space. It’s complicated. Still.

Let me start with an example system that has details further below.

• total of quotas is slightly more than 1.1 Gibibyte
  • Sometimes this helps making the quota list better:
    btrfs quota rescan /
• the disk partition itself is 10 Gibibyte
• btrfs indicates there is 6.6 Gibibyte used
• df indicates there is 11 Gigabyte total, 6.9 Gigabyte used and 2.6 Gigabyte available.

In short: the used 6.6 Gibibyte (which matches 6.9 Gigabyte) does not match the 11 Gibibyte. A situation very similar to [WayBack] Disk usage is more than double the snapshots exclusive data — Linux BTRFS.

Reminder to self: try bedup that is supposed to deduplicate btrfs data: [WayBack] Re: Disk usage is more than double the snapshots exclusive data — Linux BTRFS

• Repository g2p/bedup: Btrfs deduplication
• Not sure which package is needed (bedup, python3-bedup nor burp are found)

I need to check out on de-duplication (as I know this particular machine has quite a bit of duplicate data).

But first lets get the size down a bit with this series of commands:

sftp-host:~ # btrfs balance start -dusage=0 -musage=0 /
Done, had to relocate 0 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=10 -musage=10 /
Done, had to relocate 1 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=20 -musage=20 /
Done, had to relocate 1 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=30 -musage=30 /
Done, had to relocate 2 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=40 -musage=40 /
Done, had to relocate 1 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=50 -musage=40 /
Done, had to relocate 2 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=60 -musage=40 /
Done, had to relocate 2 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=60 -musage=60 /
sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 8.88GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=7.82GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=263.47MiB
GlobalReserve, single: total=22.67MiB, used=0.00B

Compare this to the initial situation:

sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 10.00GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=8.94GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=264.27MiB
GlobalReserve, single: total=23.48MiB, used=0.00B

Now you see that:

• far less of the partition is actually used by the filesystem (was 10 Gibibyte, now 8.88 Gibibyte)
• far less storage is needed for the data (was 8.94 Gibibyte, now 7.82 Gibibyte to store 6.35 Gibibyte)

If the above succeeds

Continue with steps closer to 99 (which is a percentage) and if that succeeds try this:

btrfs balance start --full-balance /

In my experience it needs at least 60% free dh -f disk space to run to completion. If it fails, it’s no problem: it merges the final almost full blocks. But those blocks will be split soon anyway because of file system write activity.

Nicer overview

You can even get a nicer view by executing btrfs filesystem usage -T / (which I did after continuing up to 99):

Overall:
    Device size:          10.00GiB
    Device allocated:          8.22GiB
    Device unallocated:        1.78GiB
    Device missing:          0.00B
    Used:              7.00GiB
    Free (estimated):          2.72GiB  (min: 1.83GiB)
    Data ratio:               1.00
    Metadata ratio:           2.00
    Global reserve:       24.55MiB  (used: 48.00KiB)

             Data    Metadata  System              
Id Path      single  DUP       DUP      Unallocated
-- --------- ------- --------- -------- -----------
 1 /dev/sda2 7.41GiB 768.00MiB 64.00MiB     1.78GiB
-- --------- ------- --------- -------- -----------
   Total     7.41GiB 384.00MiB 32.00MiB     1.78GiB
   Used      6.47GiB 269.88MiB 16.00KiB

If the above fails

Three things to try now:

1. Try to start with lower values of -dusage and -musage.
2. Split-dusage and -musage in different btrfs balance start commands.
3. Try to remove any snapper snapshots that you do not need.

Log:

sftp-host:~ # df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        176M     0  176M   0% /dev
tmpfs           182M     0  182M   0% /dev/shm
tmpfs           182M  868K  181M   1% /run
tmpfs           182M     0  182M   0% /sys/fs/cgroup
/dev/sda2        11G  6.9G  2.6G  73% /
/dev/sda2        11G  6.9G  2.6G  73% /var/spool
/dev/sda2        11G  6.9G  2.6G  73% /tmp
/dev/sda2        11G  6.9G  2.6G  73% /boot/grub2/i386-pc
/dev/sda2        11G  6.9G  2.6G  73% /boot/grub2/x86_64-efi
/dev/sda2        11G  6.9G  2.6G  73% /var/crash
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/named
/dev/sda2        11G  6.9G  2.6G  73% /var/opt
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/mailman
/dev/sda2        11G  6.9G  2.6G  73% /var/tmp
/dev/sda2        11G  6.9G  2.6G  73% /var/log
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/pgsql
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/machines
/dev/sda2        11G  6.9G  2.6G  73% /srv
/dev/sda2        11G  6.9G  2.6G  73% /usr/local
/dev/sda2        11G  6.9G  2.6G  73% /opt
/dev/sda2        11G  6.9G  2.6G  73% /.snapshots
/dev/sda3       5.5G   36M  5.5G   1% /home
tmpfs            37M     0   37M   0% /run/user/1000
sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 10.00GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=8.94GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=264.27MiB
GlobalReserve, single: total=23.48MiB, used=0.00B
sftp-host:~ # btrfs qgroup show /
qgroupid         rfer         excl 
--------         ----         ---- 
0/5          16.00KiB     16.00KiB 
0/257         1.05MiB      1.05MiB 
0/258         2.55GiB     51.11MiB 
0/259         2.36MiB      2.36MiB 
0/260        16.00KiB     16.00KiB 
0/261        16.00KiB     16.00KiB 
0/262        16.00KiB     16.00KiB 
0/263        36.00KiB     36.00KiB 
0/264        16.00KiB     16.00KiB 
0/265        16.00KiB     16.00KiB 
0/266        16.00KiB     16.00KiB 
0/267        16.00KiB     16.00KiB 
0/268        16.00KiB     16.00KiB 
0/269       533.83MiB    533.83MiB 
0/270        16.00KiB     16.00KiB 
0/271        48.00KiB     48.00KiB 
0/272        16.00KiB     16.00KiB 
0/289        16.00KiB     16.00KiB 
0/401         2.80GiB    396.91MiB 
0/402         2.55GiB      9.57MiB 
0/403         2.55GiB     12.91MiB 
0/404         2.54GiB    676.00KiB 
0/405         2.54GiB    660.00KiB 
0/406         2.81GiB     60.34MiB 
0/407         2.55GiB      8.66MiB 
0/408         2.55GiB      4.57MiB 
0/409         2.56GiB     24.31MiB 
0/410         2.55GiB      7.28MiB 
0/411         2.57GiB     20.55MiB 
255/289      16.00KiB     16.00KiB 
sftp-host:~ # !~
~/Versioned/btrfs-size/btrfs-size.sh 
===============================================================================================
Snapshot / Subvolume                                               ID   Total    Exclusive Data
===============================================================================================
257 gen 505741 top level 5 path .snapshots                         257  1.05MB   1.05MB   
258 gen 505796 top level 257 path .snapshots/1/snapshot            258  2.55GB   51.11MB  
259 gen 505736 top level 5 path boot/grub2/i386-pc                 259  2.36MB   2.36MB   
260 gen 452028 top level 5 path boot/grub2/x86_64-efi              260  16.00KB  16.00KB  
261 gen 452028 top level 5 path opt                                261  16.00KB  16.00KB  
262 gen 505720 top level 5 path srv                                262  16.00KB  16.00KB  
263 gen 505791 top level 5 path tmp                                263  36.00KB  36.00KB  
264 gen 505717 top level 5 path usr/local                          264  16.00KB  16.00KB  
265 gen 452028 top level 5 path var/crash                          265  16.00KB  16.00KB  
266 gen 452028 top level 5 path var/lib/mailman                    266  16.00KB  16.00KB  
267 gen 452028 top level 5 path var/lib/named                      267  16.00KB  16.00KB  
268 gen 452028 top level 5 path var/lib/pgsql                      268  16.00KB  16.00KB  
269 gen 505795 top level 5 path var/log                            269  533.83MB 533.83MB 
270 gen 452028 top level 5 path var/opt                            270  16.00KB  16.00KB  
271 gen 505796 top level 5 path var/spool                          271  48.00KB  48.00KB  
272 gen 505771 top level 5 path var/tmp                            272  16.00KB  16.00KB  
289 gen 452028 top level 5 path var/lib/machines                   289  16.00KB  16.00KB  
401 gen 451786 top level 257 path .snapshots/92/snapshot           401  2.81GB   396.91MB 
402 gen 465358 top level 257 path .snapshots/93/snapshot           402  2.55GB   9.57MB   
403 gen 465363 top level 257 path .snapshots/94/snapshot           403  2.55GB   12.91MB  
404 gen 471598 top level 257 path .snapshots/95/snapshot           404  2.54GB   676.00KB 
405 gen 471603 top level 257 path .snapshots/96/snapshot           405  2.54GB   660.00KB 
406 gen 471658 top level 257 path .snapshots/97/snapshot           406  2.81GB   60.34MB  
407 gen 487231 top level 257 path .snapshots/98/snapshot           407  2.55GB   8.66MB   
408 gen 490073 top level 257 path .snapshots/99/snapshot           408  2.55GB   4.57MB   
409 gen 490081 top level 257 path .snapshots/100/snapshot          409  2.56GB   24.31MB  
410 gen 505715 top level 257 path .snapshots/101/snapshot          410  2.55GB   7.28MB   
411 gen 505739 top level 257 path .snapshots/102/snapshot          411  2.57GB   20.55MB  
===============================================================================================
                                                                Exclusive Total: 1.11GB    
sftp-host:~ # 

–jeroen