The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for 2019

« Previous Entries
Next Entries »

Follow up on “btrfs free space. It’s complicated. Still.”

Posted by jpluimers on 2019/02/14

In the mean time I’ve made a bit of progress on btrfs free space. It’s complicated. Still.

Let me start with an example system that has details further below.

  • total of quotas is slightly more than 1.1 Gibibyte
    • Sometimes this helps making the quota list better:
      btrfs quota rescan /
  • the disk partition itself is 10 Gibibyte
  • btrfs indicates there is 6.6 Gibibyte used
  • df indicates there is 11 Gigabyte total, 6.9 Gigabyte used and 2.6 Gigabyte available.

In short: the used 6.6 Gibibyte (which matches 6.9 Gigabyte) does not match the 11 Gibibyte. A situation very similar to [WayBack] Disk usage is more than double the snapshots exclusive data — Linux BTRFS.

Reminder to self: try bedup that is supposed to deduplicate btrfs data: [WayBack] Re: Disk usage is more than double the snapshots exclusive data — Linux BTRFS

I need to check out on de-duplication (as I know this particular machine has quite a bit of duplicate data).

But first lets get the size down a bit with this series of commands:

sftp-host:~ # btrfs balance start -dusage=0 -musage=0 /
Done, had to relocate 0 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=10 -musage=10 /
Done, had to relocate 1 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=20 -musage=20 /
Done, had to relocate 1 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=30 -musage=30 /
Done, had to relocate 2 out of 18 chunks
sftp-host:~ # btrfs balance start -dusage=40 -musage=40 /
Done, had to relocate 1 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=50 -musage=40 /
Done, had to relocate 2 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=60 -musage=40 /
Done, had to relocate 2 out of 17 chunks
sftp-host:~ # btrfs balance start -dusage=60 -musage=60 /
sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 8.88GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=7.82GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=263.47MiB
GlobalReserve, single: total=22.67MiB, used=0.00B

Compare this to the initial situation:

sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 10.00GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=8.94GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=264.27MiB
GlobalReserve, single: total=23.48MiB, used=0.00B

Now you see that:

  • far less of the partition is actually used by the filesystem (was 10 Gibibyte, now 8.88 Gibibyte)
  • far less storage is needed for the data (was 8.94 Gibibyte, now 7.82 Gibibyte to store 6.35 Gibibyte)

If the above succeeds

Continue with steps closer to 99 (which is a percentage) and if that succeeds try this:

btrfs balance start --full-balance /

In my experience it needs at least 60% free dh -f disk space to run to completion. If it fails, it’s no problem: it merges the final almost full blocks. But those blocks will be split soon anyway because of file system write activity.

Nicer overview

You can even get a nicer view by executing btrfs filesystem usage -T / (which I did after continuing up to 99):

Overall:
    Device size:          10.00GiB
    Device allocated:          8.22GiB
    Device unallocated:        1.78GiB
    Device missing:          0.00B
    Used:              7.00GiB
    Free (estimated):          2.72GiB  (min: 1.83GiB)
    Data ratio:               1.00
    Metadata ratio:           2.00
    Global reserve:       24.55MiB  (used: 48.00KiB)

             Data    Metadata  System              
Id Path      single  DUP       DUP      Unallocated
-- --------- ------- --------- -------- -----------
 1 /dev/sda2 7.41GiB 768.00MiB 64.00MiB     1.78GiB
-- --------- ------- --------- -------- -----------
   Total     7.41GiB 384.00MiB 32.00MiB     1.78GiB
   Used      6.47GiB 269.88MiB 16.00KiB

 

If the above fails

Three things to try now:

  1. Try to start with lower values of -dusage and -musage.
  2. Split-dusage and -musage in different btrfs balance start commands.
  3. Try to remove any snapper snapshots that you do not need. (you can delete ranges using snapper, see [Wayback/Archive] How to delete disk snapshots in SUSE)

Log:

sftp-host:~ # df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        176M     0  176M   0% /dev
tmpfs           182M     0  182M   0% /dev/shm
tmpfs           182M  868K  181M   1% /run
tmpfs           182M     0  182M   0% /sys/fs/cgroup
/dev/sda2        11G  6.9G  2.6G  73% /
/dev/sda2        11G  6.9G  2.6G  73% /var/spool
/dev/sda2        11G  6.9G  2.6G  73% /tmp
/dev/sda2        11G  6.9G  2.6G  73% /boot/grub2/i386-pc
/dev/sda2        11G  6.9G  2.6G  73% /boot/grub2/x86_64-efi
/dev/sda2        11G  6.9G  2.6G  73% /var/crash
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/named
/dev/sda2        11G  6.9G  2.6G  73% /var/opt
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/mailman
/dev/sda2        11G  6.9G  2.6G  73% /var/tmp
/dev/sda2        11G  6.9G  2.6G  73% /var/log
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/pgsql
/dev/sda2        11G  6.9G  2.6G  73% /var/lib/machines
/dev/sda2        11G  6.9G  2.6G  73% /srv
/dev/sda2        11G  6.9G  2.6G  73% /usr/local
/dev/sda2        11G  6.9G  2.6G  73% /opt
/dev/sda2        11G  6.9G  2.6G  73% /.snapshots
/dev/sda3       5.5G   36M  5.5G   1% /home
tmpfs            37M     0   37M   0% /run/user/1000
sftp-host:~ # btrfs filesystem show
Label: none  uuid: 6492a1c6-5fbc-4938-bf11-57d6194e6b8f
    Total devices 1 FS bytes used 6.61GiB
    devid    1 size 10.00GiB used 10.00GiB path /dev/sda2

sftp-host:~ # btrfs filesystem df /
Data, single: total=8.94GiB, used=6.35GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=512.00MiB, used=264.27MiB
GlobalReserve, single: total=23.48MiB, used=0.00B
sftp-host:~ # btrfs qgroup show /
qgroupid         rfer         excl 
--------         ----         ---- 
0/5          16.00KiB     16.00KiB 
0/257         1.05MiB      1.05MiB 
0/258         2.55GiB     51.11MiB 
0/259         2.36MiB      2.36MiB 
0/260        16.00KiB     16.00KiB 
0/261        16.00KiB     16.00KiB 
0/262        16.00KiB     16.00KiB 
0/263        36.00KiB     36.00KiB 
0/264        16.00KiB     16.00KiB 
0/265        16.00KiB     16.00KiB 
0/266        16.00KiB     16.00KiB 
0/267        16.00KiB     16.00KiB 
0/268        16.00KiB     16.00KiB 
0/269       533.83MiB    533.83MiB 
0/270        16.00KiB     16.00KiB 
0/271        48.00KiB     48.00KiB 
0/272        16.00KiB     16.00KiB 
0/289        16.00KiB     16.00KiB 
0/401         2.80GiB    396.91MiB 
0/402         2.55GiB      9.57MiB 
0/403         2.55GiB     12.91MiB 
0/404         2.54GiB    676.00KiB 
0/405         2.54GiB    660.00KiB 
0/406         2.81GiB     60.34MiB 
0/407         2.55GiB      8.66MiB 
0/408         2.55GiB      4.57MiB 
0/409         2.56GiB     24.31MiB 
0/410         2.55GiB      7.28MiB 
0/411         2.57GiB     20.55MiB 
255/289      16.00KiB     16.00KiB 
sftp-host:~ # !~
~/Versioned/btrfs-size/btrfs-size.sh 
===============================================================================================
Snapshot / Subvolume                                               ID   Total    Exclusive Data
===============================================================================================
257 gen 505741 top level 5 path .snapshots                         257  1.05MB   1.05MB   
258 gen 505796 top level 257 path .snapshots/1/snapshot            258  2.55GB   51.11MB  
259 gen 505736 top level 5 path boot/grub2/i386-pc                 259  2.36MB   2.36MB   
260 gen 452028 top level 5 path boot/grub2/x86_64-efi              260  16.00KB  16.00KB  
261 gen 452028 top level 5 path opt                                261  16.00KB  16.00KB  
262 gen 505720 top level 5 path srv                                262  16.00KB  16.00KB  
263 gen 505791 top level 5 path tmp                                263  36.00KB  36.00KB  
264 gen 505717 top level 5 path usr/local                          264  16.00KB  16.00KB  
265 gen 452028 top level 5 path var/crash                          265  16.00KB  16.00KB  
266 gen 452028 top level 5 path var/lib/mailman                    266  16.00KB  16.00KB  
267 gen 452028 top level 5 path var/lib/named                      267  16.00KB  16.00KB  
268 gen 452028 top level 5 path var/lib/pgsql                      268  16.00KB  16.00KB  
269 gen 505795 top level 5 path var/log                            269  533.83MB 533.83MB 
270 gen 452028 top level 5 path var/opt                            270  16.00KB  16.00KB  
271 gen 505796 top level 5 path var/spool                          271  48.00KB  48.00KB  
272 gen 505771 top level 5 path var/tmp                            272  16.00KB  16.00KB  
289 gen 452028 top level 5 path var/lib/machines                   289  16.00KB  16.00KB  
401 gen 451786 top level 257 path .snapshots/92/snapshot           401  2.81GB   396.91MB 
402 gen 465358 top level 257 path .snapshots/93/snapshot           402  2.55GB   9.57MB   
403 gen 465363 top level 257 path .snapshots/94/snapshot           403  2.55GB   12.91MB  
404 gen 471598 top level 257 path .snapshots/95/snapshot           404  2.54GB   676.00KB 
405 gen 471603 top level 257 path .snapshots/96/snapshot           405  2.54GB   660.00KB 
406 gen 471658 top level 257 path .snapshots/97/snapshot           406  2.81GB   60.34MB  
407 gen 487231 top level 257 path .snapshots/98/snapshot           407  2.55GB   8.66MB   
408 gen 490073 top level 257 path .snapshots/99/snapshot           408  2.55GB   4.57MB   
409 gen 490081 top level 257 path .snapshots/100/snapshot          409  2.56GB   24.31MB  
410 gen 505715 top level 257 path .snapshots/101/snapshot          410  2.55GB   7.28MB   
411 gen 505739 top level 257 path .snapshots/102/snapshot          411  2.57GB   20.55MB  
===============================================================================================
                                                                Exclusive Total: 1.11GB    
sftp-host:~ #

–jeroen

Posted in *nix, *nix-tools, btrfs, File-Systems, Power User | Leave a Comment »

Ancient Unix based keyboard shortcuts

Posted by jpluimers on 2019/02/14

Somehow I’ve been doing Unix like work off and on for most of my life. So I know many ancient shell shortcuts (like Ctrl-E for end-of-line and Ctrl-A for start-of-line) by heart.

Many don’t, or even list shortcuts as if they are for the shell, but in fact aren’t.

So here are some links that distinguish between various kinds of keyboard shortcuts and give some insight in the history of these shortcuts (that go back into the 1970s, yes I’m that old).

–jeroen

via:

Posted in *nix, *nix-tools, Development, Power User, Software Development | Leave a Comment »

VCL Root units

Posted by jpluimers on 2019/02/13

Delphi VCL units that are not using any other VCL units:

  • Vcl.Bind.Consts
  • Vcl.Consts
  • Vcl.ComStrs
  • Vcl.HtmlHelpViewer
  • Vcl.IdAntiFreeze
  • Vcl.Imaging.GIFConsts
  • Vcl.Imaging.JConsts
  • Vcl.Imaging.pnglang
  • Vcl.OleConst
  • Vcl.ShellAnimations
  • Vcl.Shell.ShellConsts
  • Vcl.Touch.GestureConsts
  • Vcl.Touch.KeyboardTypes
  • Vcl.VDBConsts
  • Vcl.XPMan

–jeroen

Posted in Delphi, Development, Software Development | Leave a Comment »

Accessing storage (NAS) over the Internet via FTP | FRITZ!Box 7490 | AVM International

Posted by jpluimers on 2019/02/13

Of course you don’t want this. So by the time you read this, the connection has been closed.

For testing some Internet of Shit stuff from a client that cannot do SFTP, I needed a temporary FTP accessible connection.

These links helped:

TL;DR:

  1. Preparing the USB stick:
    1. Ensure the USB disk is FAT/FAT32/NTFS
    2. Create a directory in the root of the USB disk for the FTP user (for now: FtpDirectory)
    3. Insert the USB disk in the Fritz!Box
  2. Logon to the Fritz!Box web UI
    1. Configure a user for FTP:
      1. In the menu, go to System, then FRITZ!Box Users
      2. Click Add user
      3. Name the user (for now: MyFtpUser)
      4. Ensure that user *only* has a checkmark for `Access to NAS contents`
      5. Click the button Add directory
      6. In the popup click Select folder
      7. Choose the FtpDirectory you just made
      8. Click OK
      9. Ensure the read and write checkboxes are enabled
      10. Click OK
    2. Configure the USB stick for FPT access
      1. In the menu, go to Home Network, then USB Devices
      2. Observe if the device is visible and has the correct file system (if not: ask AVM)
      3. In the menu, go to Internet, then Permit Access
      4. Click on the FRITZ!Box Services
      5. Ensure there is a checkmark at Internet access to your storage media via FTP/FTPS enabled
      6. At TCP Port for FTP/FTPS, fill in 21 (many IoT devices cannot use a different port)
      7. Ensure there is *no* checkmark at Allow only secure FTP connections (FTPS)
      8. Make a note of the value after FTP address (something like ftp://example.org:21`)
      9. Click Apply
  3. Test

–jeroen

Posted in Communications Development, Development, Fritz!, Fritz!Box, FTP, Internet protocol suite, IoT Internet of Things, Network-and-equipment, Power User, Software Development, TCP | Leave a Comment »

web based tools to simulate electronics

Posted by jpluimers on 2019/02/13

Via seedstudio, I bumped into EasyEDA – Wikipedia. It seems there are a lot of on-line tools that allow you to draw and often simulate electronics.

Time to put them on my list of stuff to play around with:

Circuitlab also has a nice free interactive book: [WayBack] “Ultimate Electronics” Book – CircuitLab A free interactive electronics textbook, “Ultimate Electronics”

Later – via issue #1 of [WayBack] HackSpace magazine: The new magazine for the modern maker – I also found out about Fritzing which is off-line native software that I want to check out as it is open source:

[Archive.is] Fritzing

It helps a lot knowing there are various Electronic symbol – Wikipedia: Standards for symbols, covering much more than the one below:

To make it easier, power supplies have a host of different symbols for the same things. For instance, DC can have 3 or more different symbols. [WayBack] Electric Current Symbols

–jeroen

Posted in Development, Hardware Development | Leave a Comment »

Berlin 10.1.2 Vcl.RibbonConsts removed, so now Vcl.ScreenTips compilation fails – Pascal Today

Posted by jpluimers on 2019/02/12

The Ribbon controls got removed since Delphi 10.1 Berlin, but the dependency in Vcl.ScreenTips remained, so:

After installing the official Delphi Berlin Update 2 I have faced nasty problem. One of my unit was using the Vcl.ScreenTips unit (for TScreenTipsWindow). And when you compile such a project you ge…

Source: Berlin 10.1.2 Vcl.ScreenTips compilation fail – Pascal Today

In the mean time however, it has been moved to GetIt: [WayBack] Ribbon Controls in RAD Studio 10.1 Berlin.

Note that the XE8 introduced [Archive.is] GetIt package manager is under the Tools menu, which is not covered by [Archive.is] IDE Insight – RAD Studio.

Anyway: here you can get it in Delphi 10.1 Berlin (now also in 10.2 Tokyo, where at first it was not available through GetIt):

—jeroen

Posted in Delphi, Delphi 10.1 Berlin (BigBen), Delphi 10.2 Tokyo (Godzilla), Development, Software Development | Leave a Comment »

Build a Power Bank in $2: 7 Steps (with Pictures)

Posted by jpluimers on 2019/02/12

Cool: [WayBack] Build a Power Bank in $2: 7 Steps (with Pictures)

Via:

TL;DR:

  1. salvage 18650 batteries from laptop battery packs
  2. test to separate good ones from bad ones
  3. assemble together the good ones
    1. put them in a holders
    2. solder plusses to plusses and minuses to minuses
  4. add charger electronics
  5. test
  6. put acrylic plate on front/back

Stuff you need:

–jeroen

 

Posted in 18650, Batteries, Development, Hardware Development, Hardware Interfacing, Li-Ion, Power User | Leave a Comment »

Raspberry Pi and relays – follow up on Having one Raspberry Pi reset another Raspberry Pi through relay or transistor

Posted by jpluimers on 2019/02/12

I did some more research because of Having one Raspberry Pi reset another Raspberry Pi through relay or transistor.

  • [WayBack] How To Add a Reset Switch To Your Raspberry PiRemoving and replacing the USB power cable puts undue wear and tear on your Raspberry Pi, particularly the power port itself. What the system really needs is a reset switch, but sadly none was included.
  • Grove Relay board:
    • has two versions; the V1.2 schematic adds a XC6206P302MR voltage regulator to regulate 3V through the relay coil and an extra 47k Ohm pull-down resistor.
    • has a trigger on high supporting a voltage of 3V, so it works with the Raspberry Pi 3.3V GPIO pins.
    • is “normal open”, so suits the reset scenario (connect on trigger) well.
    • has no “normal closed” header, so if you need that, you’re out of luck
    • does not have optocouplers:
      • Be careful with high voltages on supplies that differ from the one powering your Raspberry Pi
        • It’s fine for resetting another Raspberry Pi powered from the same source
      • The relay is rated 250V ~ but I’d be careful (I’m not sure if this is mains electricity 250V RMS or 250V peak; if the latter, it would be suitable to 175V RMS (approximately 250/1.42 volt).
  • An excellent description (sans optocoupler) on how to connect a relay to power, ground, signal-input and both outputs is at [WayBack/Archive.is] gpio – How to add isolation between raspberry pi and relay board? – Raspberry Pi Stack Exchange (thanks [WayBack] ppumkin).
  • Many 5V optocoupler (or optical-isolator, see video below) based relay boards work fine with the 3.3V GPIO pins from the Raspberry Pi.
    If they don’t, then there are two basic solutions:

    1. Easiest: solder an extra resistor next to the signal input of about the same value (so the voltage drop over it halves), see for instance [WayBack] Controlling a relay board from your RPi · foosel/OctoPrint Wiki
    2. Harder: put an extra transistor in between to pump up the voltage to 5V, see one of the schematics below.

Details of the above can be found from the below links and images from those links.

There is also an Android App with a RaspberryPi distribution that allows you to operate relays:

Finally there are USB relays, shown way down in this post.

Often these are part of some home automation (domotica), IoT, or other, so these are relevant too:

Read the rest of this entry »

Posted in Development, Hardware Development, Raspberry Pi | Leave a Comment »

2018 Hard Drive Reliability Stats by Manufacturer and Model

Posted by jpluimers on 2019/02/11

We look at the hard drive failure rates for the data drive models in operation in our data centers in 2018 (now totaling over 750 petabytes and 100,000 drives). In addition, we’ll see how the new hard drive models we added during the year did, including our 12 TB HGST and 14 TB Toshiba drives.

Full article: [WayBack] 2018 Hard Drive Reliability Stats by Manufacturer and Model

Statistical data at [WayBack] Backblaze Hard Drive Stats:  Hard Drive test data from the Backblaze data center. Backblaze is affordable, easy-to-use cloud storage.

Via [WayBack] 2018 Hard Drive Reliability Stats by Manufacturer and Model https://www.backblaze.com/blog/hard-drive-stats-for-2018/ – DoorToDoorGeek “Stephen McLaughlin” – Google+

–jeroen

Posted in Backup, Hardware, History, LifeHacker, Power User, Storage | Leave a Comment »

shared folders not visible · Issue #276 · stascorp/rdpwrap

Posted by jpluimers on 2019/02/11

Reminder to self: check if this rdpwrap issue [WayBack] shared folders not visible · Issue #276 · stascorp/rdpwrap has been resolved by now.

–jeroen

Posted in Power User, Remote Desktop Protocol/MSTSC/Terminal Services, Windows | Leave a Comment »

« Previous Entries
Next Entries »