The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,484 other followers

bash: converting numbers to human readable SI or IEC units

Posted by jpluimers on 2019/12/03

Many unix tools that report sizes in bytes can convert them to either IEC or SI readable formats.

For github.com/jpluimers/btrfs-du/blob/master/btrfs-du I wrote about last week, I also wanted that kind of behaviour. So I did some research and came up with the code and test cases below.

Note that depending on the bitness of your system, bash integer numeric values are limited in size; see [WayBack] What is the maximum value of a numeric bash shell variable? – Super User.

So I wrote a small bash script for that too, which needed also gave me the opportunity to show how a  perpetual while loop as explained by [WayBack] bash – “while :” vs. “while true” – Stack Overflow.

Two things that always bite me with these short scripts are expressions (done through [WayBack]Arithmetic Expansion) and comparisons (through[WayBack] Other Comparison Operators).

The IEC suffixes contain one extra i to indicate binary and – next to the ISO notation that were already ISO defined – made it into the ISO 80000 standard since 2008. Here is a comparison list from [WayBackBinary prefix – Wikipedia:

Prefixes for multiples of
bits (bit) or bytes (B)
Decimal
Value SI
1000 k kilo
10002 M mega
10003 G giga
10004 T tera
10005 P peta
10006 E exa
10007 Z zetta
10008 Y yotta
Binary
Value IEC JEDEC
1024 Ki kibi K kilo
10242 Mi mebi M mega
10243 Gi gibi G giga
10244 Ti tebi
10245 Pi pebi
10246 Ei exbi
10247 Zi zebi
10248 Yi yobi

Most tools nowadays default to binary IEC suffixes for byte sizes, though disk manufacturers still use SI suffixes because, well then they appear bigger but aren’t. Just for comparison, look at the numbers from [WayBack] File size – Wikipedia and [WayBack] IEC and SI Size Notations – AN!Wiki where I got the test cases from:

base2 Notation

IEC Notation   Size
-------------- -------------------------------------------------
KiB = kibibyte (2^10  / 1,024 bytes)
MiB = mebibyte (2^20  / 1,048,576 bytes)
GiB = gibibyte (2^30  / 1,073,741,824 bytes)
TiB = tebibyte (2^40  / 1,099,511,627,776 bytes)
PiB = pebibyte (2^50  / 1,125,899,906,842,624 bytes)
EiB = exbibyte (2^60  / 1,152,921,504,606,846,976 bytes)
ZiB = zebibyte (2^70  / 1,180,591,620,717,411,303,424 bytes)
YiB = yebibyte (2^80  / 1,208,925,819,614,629,174,706,176 bytes)

base10 Notation

SI Notation    Size
-------------- -------------------------------------------------
KB = kilobyte  (10^3  / 1,000 bytes)
MB = megabyte  (10^6  / 1,000,000 bytes)
GB = gigabyte  (10^9  / 1,000,000,000 bytes)
TB = terabyte  (10^12 / 1,000,000,000,000 bytes)
PB = petabyte  (10^15 / 1,000,000,000,000,000 bytes)
EB = exabyte   (10^18 / 1,000,000,000,000,000,000 bytes)
ZB = zettabyte (10^21 / 1,000,000,000,000,000,000,000 bytes)
YB = yottabyte (10^24 / 1,000,000,000,000,000,000,000,000 bytes)

https://en.wikipedia.org/wiki/Binary_prefix

–jeroen

#!/bin/bash
# related to bytesToHuman.bash
# 64-bibt checks at https://superuser.com/questions/1030122/what-is-the-maximum-value-of-a-numeric-bash-shell-variable/1030129#1030129
# perpetual while loop: https://stackoverflow.com/questions/10797835/while-vs-while-true
# expressions: http://tldp.org/LDP/abs/html/arithexp.html
# comparisons: http://tldp.org/LDP/abs/html/comparison-ops.html
b=1
n=1
while :
do
# loop infinitely
printf "%3s %40s\n" "$b" "$n"
b=$(( 1+$b ))
t=$n
n=$(( 2*$n ))
if (( $n < $t )); then
exit
fi
done
echo "the final number on the left is the largest bitness; on the right the largest power of 2 your system supports"

view raw

bitness.bash

hosted with ❤ by GitHub

#!/bin/bash
# bytesToHuman based on https://unix.stackexchange.com/questions/44040/a-standard-tool-to-convert-a-byte-count-into-human-kib-mib-etc-like-du-ls1/259254#259254
# units from wikipedia (note I had to re-order Z and Y):
#https://en.wikipedia.org/wiki/Binary_prefix
#https://en.wikipedia.org/wiki/File_size
#https://en.wikipedia.org/wiki/ISO/IEC_80000#Information_science_and_technology
# test cases from https://www.alteeve.com/w/IEC_and_SI_Size_Notations and https://en.wikipedia.org/wiki/File_size
# 64-bibt checks at https://superuser.com/questions/1030122/what-is-the-maximum-value-of-a-numeric-bash-shell-variable/1030129#1030129
bytesToHumanIEC() {
b=${1:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Z,Y}iB)
while ((b > 1024)); do
d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))"
b=$((b / 1024))
let s++
done
echo "$b$d ${S[$s]}"
}
bytesToHumanSI() {
b=${1:-0}; d=''; s=0; S=(Bytes {k,M,G,T,E,P,Z,Y}B)
while ((b > 1000)); do
d="$(printf ".%02d" $((b % 1000 * 100 / 1000)))"
b=$((b / 1000))
let s++
done
echo "$b$d ${S[$s]}"
}
# test cases
echo bytesToHumanIEC test cases:
bytesToHumanIEC 0
bytesToHumanIEC 1023
bytesToHumanIEC 1048575
bytesToHumanIEC 1073741823
bytesToHumanIEC 1099511627775
bytesToHumanIEC 1125899906842623
bytesToHumanIEC 1152921504606846975
## These are likekly not supported as they overflow on 64-bit systems:
bytesToHumanIEC 1180591620717411303423
bytesToHumanIEC 1208925819614629174706175
bytesToHumanIEC 1
bytesToHumanIEC 1024
bytesToHumanIEC 1048576
bytesToHumanIEC 1073741824
bytesToHumanIEC 1099511627776
bytesToHumanIEC 1125899906842624
bytesToHumanIEC 1152921504606846976
bytesToHumanIEC 1180591620717411303424
bytesToHumanIEC 1208925819614629174706176
bytesToHumanIEC 2
bytesToHumanIEC 1025
bytesToHumanIEC 1048577
bytesToHumanIEC 1073741825
bytesToHumanIEC 1099511627777
bytesToHumanIEC 1125899906842625
bytesToHumanIEC 1152921504606846977
bytesToHumanIEC 1180591620717411303425
bytesToHumanIEC 1208925819614629174706177
bytesToHumanIEC 0
bytesToHumanIEC 1999
bytesToHumanIEC 1999999
bytesToHumanIEC 1999999999
bytesToHumanIEC 1999999999999
bytesToHumanIEC 1999999999999999
bytesToHumanIEC 1999999999999999999
bytesToHumanIEC 1999999999999999999999
bytesToHumanIEC 1999999999999999999999999
bytesToHumanIEC 1
bytesToHumanIEC 1000
bytesToHumanIEC 1000000
bytesToHumanIEC 1000000000
bytesToHumanIEC 1000000000000
bytesToHumanIEC 1000000000000000
bytesToHumanIEC 1000000000000000000
## These are likekly not supported as they overflow on 64-bit systems:
bytesToHumanIEC 1000000000000000000000
bytesToHumanIEC 1000000000000000000000000
bytesToHumanIEC 1
bytesToHumanIEC 1001
bytesToHumanIEC 1000001
bytesToHumanIEC 1000000001
bytesToHumanIEC 1000000000001
bytesToHumanIEC 1000000000000001
bytesToHumanIEC 1000000000000000001
bytesToHumanIEC 1000000000000000000001
bytesToHumanIEC 1000000000000000000000001
echo bytesToHumanSI test cases:
bytesToHumanSI 1
bytesToHumanSI 1024
bytesToHumanSI 1048576
bytesToHumanSI 1073741824
bytesToHumanSI 1099511627776
bytesToHumanSI 1125899906842624
bytesToHumanSI 1152921504606846976
bytesToHumanSI 1180591620717411303424
bytesToHumanSI 1208925819614629174706176
bytesToHumanSI 1
bytesToHumanSI 1000
bytesToHumanSI 1000000
bytesToHumanSI 1000000000
bytesToHumanSI 1000000000000
bytesToHumanSI 1000000000000000
bytesToHumanSI 1000000000000000000
bytesToHumanSI 1000000000000000000000
bytesToHumanSI 1000000000000000000000000
bytesToHumanSI 0
bytesToHumanSI 1023
bytesToHumanSI 1048575
bytesToHumanSI 1073741823
bytesToHumanSI 1099511627775
bytesToHumanSI 1125899906842623
bytesToHumanSI 1152921504606846975
bytesToHumanSI 1180591620717411303423
bytesToHumanSI 1208925819614629174706175
bytesToHumanSI 1
bytesToHumanSI 1024
bytesToHumanSI 1048576
bytesToHumanSI 1073741824
bytesToHumanSI 1099511627776
bytesToHumanSI 1125899906842624
bytesToHumanSI 1152921504606846976
bytesToHumanSI 1180591620717411303424
bytesToHumanSI 1208925819614629174706176
bytesToHumanSI 2
bytesToHumanSI 1025
bytesToHumanSI 1048577
bytesToHumanSI 1073741825
bytesToHumanSI 1099511627777
bytesToHumanSI 1125899906842625
bytesToHumanSI 1152921504606846977
bytesToHumanSI 1180591620717411303425
bytesToHumanSI 1208925819614629174706177
bytesToHumanSI 0
bytesToHumanSI 1999
bytesToHumanSI 1999999
bytesToHumanSI 1999999999
bytesToHumanSI 1999999999999
bytesToHumanSI 1999999999999999
bytesToHumanSI 1999999999999999999
bytesToHumanSI 1999999999999999999999
bytesToHumanSI 1999999999999999999999999
bytesToHumanSI 1
bytesToHumanSI 1000
bytesToHumanSI 1000000
bytesToHumanSI 1000000000
bytesToHumanSI 1000000000000
bytesToHumanSI 1000000000000000
bytesToHumanSI 1000000000000000000
bytesToHumanSI 1000000000000000000000
bytesToHumanSI 1000000000000000000000000
bytesToHumanSI 1
bytesToHumanSI 1001
bytesToHumanSI 1000001
bytesToHumanSI 1000000001
bytesToHumanSI 1000000000001
bytesToHumanSI 1000000000000001
bytesToHumanSI 1000000000000000001
bytesToHumanSI 1000000000000000000001
bytesToHumanSI 1000000000000000000000001

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
%d bloggers like this: