bash: converting numbers to human readable SI or IEC units
Posted by jpluimers on 2019/12/03
Many unix tools that report sizes in bytes can convert them to either IEC or SI readable formats.
For github.com/jpluimers/btrfs-du/blob/master/btrfs-du I wrote about last week, I also wanted that kind of behaviour. So I did some research and came up with the code and test cases below.
Note that depending on the bitness of your system, bash integer numeric values are limited in size; see [WayBack] What is the maximum value of a numeric bash shell variable? – Super User.
So I wrote a small
bashscript for that too, which needed also gave me the opportunity to show how a perpetual while loop as explained by [WayBack] bash – “while :” vs. “while true” – Stack Overflow.Two things that always bite me with these short scripts are expressions (done through [WayBack]Arithmetic Expansion) and comparisons (through[WayBack] Other Comparison Operators).
The IEC suffixes contain one extra i to indicate binary and – next to the ISO notation that were already ISO defined – made it into the ISO 80000 standard since 2008. Here is a comparison list from [WayBack] Binary prefix – Wikipedia:
Most tools nowadays default to binary IEC suffixes for byte sizes, though disk manufacturers still use SI suffixes because, well then they appear bigger but aren’t. Just for comparison, look at the numbers from [WayBack] File size – Wikipedia and [WayBack] IEC and SI Size Notations – AN!Wiki where I got the test cases from:
base2 Notation
IEC Notation Size -------------- ------------------------------------------------- KiB = kibibyte (2^10 / 1,024 bytes) MiB = mebibyte (2^20 / 1,048,576 bytes) GiB = gibibyte (2^30 / 1,073,741,824 bytes) TiB = tebibyte (2^40 / 1,099,511,627,776 bytes) PiB = pebibyte (2^50 / 1,125,899,906,842,624 bytes) EiB = exbibyte (2^60 / 1,152,921,504,606,846,976 bytes) ZiB = zebibyte (2^70 / 1,180,591,620,717,411,303,424 bytes) YiB = yebibyte (2^80 / 1,208,925,819,614,629,174,706,176 bytes)base10 Notation
SI Notation Size -------------- ------------------------------------------------- KB = kilobyte (10^3 / 1,000 bytes) MB = megabyte (10^6 / 1,000,000 bytes) GB = gigabyte (10^9 / 1,000,000,000 bytes) TB = terabyte (10^12 / 1,000,000,000,000 bytes) PB = petabyte (10^15 / 1,000,000,000,000,000 bytes) EB = exabyte (10^18 / 1,000,000,000,000,000,000 bytes) ZB = zettabyte (10^21 / 1,000,000,000,000,000,000,000 bytes) YB = yottabyte (10^24 / 1,000,000,000,000,000,000,000,000 bytes)
– https://en.wikipedia.org/wiki/Binary_prefix
–
–jeroen
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # related to bytesToHuman.bash | |
| # 64-bibt checks at https://superuser.com/questions/1030122/what-is-the-maximum-value-of-a-numeric-bash-shell-variable/1030129#1030129 | |
| # perpetual while loop: https://stackoverflow.com/questions/10797835/while-vs-while-true | |
| # expressions: http://tldp.org/LDP/abs/html/arithexp.html | |
| # comparisons: http://tldp.org/LDP/abs/html/comparison-ops.html | |
| b=1 | |
| n=1 | |
| while : | |
| do | |
| # loop infinitely | |
| printf "%3s %40s\n" "$b" "$n" | |
| b=$(( 1+$b )) | |
| t=$n | |
| n=$(( 2*$n )) | |
| if (( $n < $t )); then | |
| exit | |
| fi | |
| done | |
| echo "the final number on the left is the largest bitness; on the right the largest power of 2 your system supports" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # bytesToHuman based on https://unix.stackexchange.com/questions/44040/a-standard-tool-to-convert-a-byte-count-into-human-kib-mib-etc-like-du-ls1/259254#259254 | |
| # units from wikipedia (note I had to re-order Z and Y): | |
| # – https://en.wikipedia.org/wiki/Binary_prefix | |
| # – https://en.wikipedia.org/wiki/File_size | |
| # – https://en.wikipedia.org/wiki/ISO/IEC_80000#Information_science_and_technology | |
| # test cases from https://www.alteeve.com/w/IEC_and_SI_Size_Notations and https://en.wikipedia.org/wiki/File_size | |
| # 64-bibt checks at https://superuser.com/questions/1030122/what-is-the-maximum-value-of-a-numeric-bash-shell-variable/1030129#1030129 | |
| bytesToHumanIEC() { | |
| b=${1:-0}; d=''; s=0; S=(Bytes {K,M,G,T,E,P,Z,Y}iB) | |
| while ((b > 1024)); do | |
| d="$(printf ".%02d" $((b % 1024 * 100 / 1024)))" | |
| b=$((b / 1024)) | |
| let s++ | |
| done | |
| echo "$b$d ${S[$s]}" | |
| } | |
| bytesToHumanSI() { | |
| b=${1:-0}; d=''; s=0; S=(Bytes {k,M,G,T,E,P,Z,Y}B) | |
| while ((b > 1000)); do | |
| d="$(printf ".%02d" $((b % 1000 * 100 / 1000)))" | |
| b=$((b / 1000)) | |
| let s++ | |
| done | |
| echo "$b$d ${S[$s]}" | |
| } | |
| # test cases | |
| echo bytesToHumanIEC test cases: | |
| bytesToHumanIEC 0 | |
| bytesToHumanIEC 1023 | |
| bytesToHumanIEC 1048575 | |
| bytesToHumanIEC 1073741823 | |
| bytesToHumanIEC 1099511627775 | |
| bytesToHumanIEC 1125899906842623 | |
| bytesToHumanIEC 1152921504606846975 | |
| ## These are likekly not supported as they overflow on 64-bit systems: | |
| bytesToHumanIEC 1180591620717411303423 | |
| bytesToHumanIEC 1208925819614629174706175 | |
| bytesToHumanIEC 1 | |
| bytesToHumanIEC 1024 | |
| bytesToHumanIEC 1048576 | |
| bytesToHumanIEC 1073741824 | |
| bytesToHumanIEC 1099511627776 | |
| bytesToHumanIEC 1125899906842624 | |
| bytesToHumanIEC 1152921504606846976 | |
| bytesToHumanIEC 1180591620717411303424 | |
| bytesToHumanIEC 1208925819614629174706176 | |
| bytesToHumanIEC 2 | |
| bytesToHumanIEC 1025 | |
| bytesToHumanIEC 1048577 | |
| bytesToHumanIEC 1073741825 | |
| bytesToHumanIEC 1099511627777 | |
| bytesToHumanIEC 1125899906842625 | |
| bytesToHumanIEC 1152921504606846977 | |
| bytesToHumanIEC 1180591620717411303425 | |
| bytesToHumanIEC 1208925819614629174706177 | |
| bytesToHumanIEC 0 | |
| bytesToHumanIEC 1999 | |
| bytesToHumanIEC 1999999 | |
| bytesToHumanIEC 1999999999 | |
| bytesToHumanIEC 1999999999999 | |
| bytesToHumanIEC 1999999999999999 | |
| bytesToHumanIEC 1999999999999999999 | |
| bytesToHumanIEC 1999999999999999999999 | |
| bytesToHumanIEC 1999999999999999999999999 | |
| bytesToHumanIEC 1 | |
| bytesToHumanIEC 1000 | |
| bytesToHumanIEC 1000000 | |
| bytesToHumanIEC 1000000000 | |
| bytesToHumanIEC 1000000000000 | |
| bytesToHumanIEC 1000000000000000 | |
| bytesToHumanIEC 1000000000000000000 | |
| ## These are likekly not supported as they overflow on 64-bit systems: | |
| bytesToHumanIEC 1000000000000000000000 | |
| bytesToHumanIEC 1000000000000000000000000 | |
| bytesToHumanIEC 1 | |
| bytesToHumanIEC 1001 | |
| bytesToHumanIEC 1000001 | |
| bytesToHumanIEC 1000000001 | |
| bytesToHumanIEC 1000000000001 | |
| bytesToHumanIEC 1000000000000001 | |
| bytesToHumanIEC 1000000000000000001 | |
| bytesToHumanIEC 1000000000000000000001 | |
| bytesToHumanIEC 1000000000000000000000001 | |
| echo bytesToHumanSI test cases: | |
| bytesToHumanSI 1 | |
| bytesToHumanSI 1024 | |
| bytesToHumanSI 1048576 | |
| bytesToHumanSI 1073741824 | |
| bytesToHumanSI 1099511627776 | |
| bytesToHumanSI 1125899906842624 | |
| bytesToHumanSI 1152921504606846976 | |
| bytesToHumanSI 1180591620717411303424 | |
| bytesToHumanSI 1208925819614629174706176 | |
| bytesToHumanSI 1 | |
| bytesToHumanSI 1000 | |
| bytesToHumanSI 1000000 | |
| bytesToHumanSI 1000000000 | |
| bytesToHumanSI 1000000000000 | |
| bytesToHumanSI 1000000000000000 | |
| bytesToHumanSI 1000000000000000000 | |
| bytesToHumanSI 1000000000000000000000 | |
| bytesToHumanSI 1000000000000000000000000 | |
| bytesToHumanSI 0 | |
| bytesToHumanSI 1023 | |
| bytesToHumanSI 1048575 | |
| bytesToHumanSI 1073741823 | |
| bytesToHumanSI 1099511627775 | |
| bytesToHumanSI 1125899906842623 | |
| bytesToHumanSI 1152921504606846975 | |
| bytesToHumanSI 1180591620717411303423 | |
| bytesToHumanSI 1208925819614629174706175 | |
| bytesToHumanSI 1 | |
| bytesToHumanSI 1024 | |
| bytesToHumanSI 1048576 | |
| bytesToHumanSI 1073741824 | |
| bytesToHumanSI 1099511627776 | |
| bytesToHumanSI 1125899906842624 | |
| bytesToHumanSI 1152921504606846976 | |
| bytesToHumanSI 1180591620717411303424 | |
| bytesToHumanSI 1208925819614629174706176 | |
| bytesToHumanSI 2 | |
| bytesToHumanSI 1025 | |
| bytesToHumanSI 1048577 | |
| bytesToHumanSI 1073741825 | |
| bytesToHumanSI 1099511627777 | |
| bytesToHumanSI 1125899906842625 | |
| bytesToHumanSI 1152921504606846977 | |
| bytesToHumanSI 1180591620717411303425 | |
| bytesToHumanSI 1208925819614629174706177 | |
| bytesToHumanSI 0 | |
| bytesToHumanSI 1999 | |
| bytesToHumanSI 1999999 | |
| bytesToHumanSI 1999999999 | |
| bytesToHumanSI 1999999999999 | |
| bytesToHumanSI 1999999999999999 | |
| bytesToHumanSI 1999999999999999999 | |
| bytesToHumanSI 1999999999999999999999 | |
| bytesToHumanSI 1999999999999999999999999 | |
| bytesToHumanSI 1 | |
| bytesToHumanSI 1000 | |
| bytesToHumanSI 1000000 | |
| bytesToHumanSI 1000000000 | |
| bytesToHumanSI 1000000000000 | |
| bytesToHumanSI 1000000000000000 | |
| bytesToHumanSI 1000000000000000000 | |
| bytesToHumanSI 1000000000000000000000 | |
| bytesToHumanSI 1000000000000000000000000 | |
| bytesToHumanSI 1 | |
| bytesToHumanSI 1001 | |
| bytesToHumanSI 1000001 | |
| bytesToHumanSI 1000000001 | |
| bytesToHumanSI 1000000000001 | |
| bytesToHumanSI 1000000000000001 | |
| bytesToHumanSI 1000000000000000001 | |
| bytesToHumanSI 1000000000000000000001 | |
| bytesToHumanSI 1000000000000000000000001 | |
The Wiert Corner - irregular stream of stuff 
Leave a comment