The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for the ‘bash’ Category

« Previous Entries
Next Entries »

Passing parameters to a Bash function – Stack Overflow

Posted by jpluimers on 2020/04/20

Since I tend to forget this:

  1. you can declare bash functions with parenthesis
  2. you pass parameters to functions space delimited (often people quote each parameter)
  3. within functions you can refer to parameters by number just like the parameters passed to a shell script

More info at:

I think for functions, you can apply what is linked from Some useful links on bash parameters: $1, $*, $@, quotes, etc., so a loop over all parameters in a function is the same as in a script, see [WayBack] shell – how to loop through arguments in a bash script – Unix & Linux Stack Exchange from Gilles:

There’s a special syntax for this:

for i do
  echo "$i"
done

More generally, the list of parameters of the current script or function is available through the special variable $@.

for i in "$@"; do
  echo "$i"
done

Note that you need the double quotes around $@, otherwise the parameters undergo wildcard expansion and field splitting. "$@" is magic: despite the double quotes, it expands into as many fields as there are parameters.

print_arguments () {
  for i in "$@"; do echo "$i"; done
}
print_arguments 'hello world' '*' 'special   !\characters'    # prints 3 lines
print_arguments ''                                            # prints one empty line
print_arguments                                               # prints nothing
 –jeroen

Posted in bash, Development, Scripting, Software Development | Leave a Comment »

bash – How to add a progress bar to a shell script? – Stack Overflow

Posted by jpluimers on 2020/03/17

From [WayBack] bash – How to add a progress bar to a shell script? – Stack Overflow (thanks Mitch Haile!):

You can implement this by overwriting a line. Use \r to go back to the beginning of the line without writing \n to the terminal.

Write \n when you’re done to advance the line.

Use echo -ne to:

  1. not print \n and
  2. to recognize escape sequences like \r.

Here’s a demo:

echo -ne '#####                     (33%)\r'
sleep 1
echo -ne '#############             (66%)\r'
sleep 1
echo -ne '#######################   (100%)\r'
echo -ne '\n'

–jeroen

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

linux – Bash: Command grouping (&&, ||, …) – Stack Overflow

Posted by jpluimers on 2020/02/18

Excellent answer at [WayBack] linux – Bash: Command grouping (&&, ||, …) – Stack Overflow by Charles Duffy:

Operator precedence for && and || is strictly left-to-right.

Thus:

pwd; (ls) || { cd .. && ls student/; }  && cd student || cd / && cd ;

…is equivalent to…

pwd; { { { (ls) || { cd .. && ls student/; }; } && cd student; } || cd /; } && cd ; }

…breaking that down graphically:

pwd; {                                      # 1
       {                                    # 2
         { (ls) ||                          # 3
                   { cd .. &&               # 4
                              ls student/;  # 5
                   };                       # 6
         } && cd student;                   # 7
       } || cd /;                           # 8
     } && cd ;                              # 9
  1. pwd happens unconditionally
  2. (Grouping only)
  3. ls happens (in a subshell) unconditionally.
  4. cd .. happens if (3) failed.
  5. ls student/ happens if (3) failed and (4) succeeded
  6. (Grouping only)
  7. cd student happens if either (3) succeeded or both (4) and (5) succeeded.
  8. cd / happens if either [both (3) and one of (4) or (5) failed], or [(7) failed].
  9. cd happens if (7) occurred and succeeded, or (7) occurred and succeeded.

Using explicit grouping operators is wise to avoid confusing yourself. Avoiding writing code as hard to read as this is even wiser.

–jeroen

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

bash: converting numbers to human readable SI or IEC units

Posted by jpluimers on 2019/12/03

Many unix tools that report sizes in bytes can convert them to either IEC or SI readable formats.

For github.com/jpluimers/btrfs-du/blob/master/btrfs-du I wrote about last week, I also wanted that kind of behaviour. So I did some research and came up with the code and test cases below.

Note that depending on the bitness of your system, bash integer numeric values are limited in size; see [WayBack] What is the maximum value of a numeric bash shell variable? – Super User.

So I wrote a small bash script for that too, which needed also gave me the opportunity to show how a  perpetual while loop as explained by [WayBack] bash – “while :” vs. “while true” – Stack Overflow.

Two things that always bite me with these short scripts are expressions (done through [WayBack]Arithmetic Expansion) and comparisons (through[WayBack] Other Comparison Operators).

The IEC suffixes contain one extra i to indicate binary and – next to the ISO notation that were already ISO defined – made it into the ISO 80000 standard since 2008. Here is a comparison list from [WayBack] Binary prefix – Wikipedia:

Prefixes for multiples of
bits (bit) or bytes (B)
Decimal
Value SI
1000 k kilo
10002 M mega
10003 G giga
10004 T tera
10005 P peta
10006 E exa
10007 Z zetta
10008 Y yotta
Binary
Value IEC JEDEC
1024 Ki kibi K kilo
10242 Mi mebi M mega
10243 Gi gibi G giga
10244 Ti tebi –
10245 Pi pebi –
10246 Ei exbi –
10247 Zi zebi –
10248 Yi yobi –

Most tools nowadays default to binary IEC suffixes for byte sizes, though disk manufacturers still use SI suffixes because, well then they appear bigger but aren’t. Just for comparison, look at the numbers from [WayBack] File size – Wikipedia and [WayBack] IEC and SI Size Notations – AN!Wiki where I got the test cases from:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

some links on bash and optional parameters

Posted by jpluimers on 2019/11/26

Hopefully I’ve been able to integrate some of the ideas in the links below in github.com/jpluimers/btrfs-du/blob/master/btrfs-du

One of the features I wanted there was to be able to add optional switches like --raw, --iec or --si to it similar to what as the btrfs qgroup show subcommand has.

It seems possible with bash, but it is not trivial, at least not for me as a non-frequent bash user, so here are some links to get me started:

In retrospect, other languages than bash might have been a better choice for a script like that (:

–jeroen

PS, some btrfs references:

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

How to Update All Your Ruby Gems At Once | Life, the Universe, and Everything

Posted by jpluimers on 2019/08/26

This looks smart

gem update `gem list | cut -d ' ' -f 1`

From: [WayBack] How to Update All Your Ruby Gems At Once | Life, the Universe, and Everything

Though on the bash prompt, it works fine on Mac OS X / OS X / macOS / …, it does not work nice as an alias.

You can get it to work with difficult escaping (or nesting).

But it is easier to escape this:

gem update $(gem list | cut -d ' ' -f 1)

Escaped, it comes down to:

alias "gem-update-all=gem update \$(gem list | cut -d ' ' -f 1)"

Based on:

–jeroen

Posted in Apple, bash, bash, Development, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development | Leave a Comment »

Show SCSI / HBA modules in ESXi 6.5 with file and version information

Posted by jpluimers on 2019/05/02

A small script I made: Show SCSI / HBA modules in ESXi 6.5 with file and version information:

MODULES=`esxcfg-scsidevs --hbas | awk 'FNR > 0 {print $2}'`
for MODULE in $MODULES ; do
    # echo "Probing $MODULE"
    vmkload_mod --showinfo $MODULE | grep 'file: \|Version'
done

The script is based on ideas from [WayBack] Determining Network/Storage firmware and driver version in ESXi 4.x and later (1027206) | VMware KB

It works in at least ESXi 6.5 where it shows this on one of my systems:

 input file: /usr/lib/vmware/vmkmod/lsi_msgpt3
 Version: 12.00.02.00-11vmw.650.0.0.4564106
 input file: /usr/lib/vmware/vmkmod/vmw_ahci
 Version: 1.0.0-39vmw.650.1.26.5969303
 input file: /usr/lib/vmware/vmkmod/vmw_ahci
 Version: 1.0.0-39vmw.650.1.26.5969303
 input file: /usr/lib/vmware/vmkmod/vmw_ahci
 Version: 1.0.0-39vmw.650.1.26.5969303
 input file: /usr/lib/vmware/vmkmod/lsi_mr3
 Version: 6.910.18.00-1vmw.650.0.0.4564106
 input file: /usr/lib/vmware/vmkmod/megaraid_sas
 Version: Version 6.603.55.00.2vmw, Build: 4564106, Interface: 9.2 Built on: Oct 26 2016
 input file: /usr/lib/vmware/vmkmod/vmkusb
 Version: 0.1-1vmw.650.1.26.5969303

–jeroen

Read the rest of this entry »

Posted in bash, Development, ESXi6.5, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

bash – aliasing cd to pushd – is it a good idea? – Unix & Linux Stack Exchange

Posted by jpluimers on 2019/04/30

On my research list: [WayBack] bash – aliasing cd to pushd – is it a good idea? – Unix & Linux Stack Exchange

It has a nice discussion on complements to pushd/popd/cd/dirs including a very nice set of navd scripts that eases the navigation of the directory stack.

I found it because the ESXi busybox does not have pushd and popd and a cd won’t work from inside a shell script: [WayBack] linux – Why doesn’t “cd” work in a bash shell script? – Stack Overflow

It also made me find out that the ESXi busybox does support cd - to go to the previous directory. More info on that cd syntax is at [WayBack] bash – Difference between “cd -” and “cd ~-” – Unix & Linux Stack Exchange

–jeroen

Posted in *nix, bash, Development, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

sed in a bash script: backslash escape anything that looks suspicious

Posted by jpluimers on 2019/02/26

Did I ever tell I dislike regular expressions and old-skool shells?

They’re not good for anything but basic commands, so if you try any scripts in them, you’re basically lost.

If you disagree, please read [WayBack] Don’t write Shell scripts. I would recommend Python, but I tried “pip search mysql”…. – Kristian Köhntopp – Google+) and [WayBack] How did this shit ever work? by the same author.

On the other hand: on many system, the baseline isn’t much more than a shell and a very limited tool set.

With nx like systems that usually comes down to sed and a shell like bash.

Since I wanted to modify an openssh hardening script to cover more permutations that was using sed in a bash script, I had not much choice but to bite the bullet.

TL;DR:

When you use any of the below characters, prepend them with a backslash as they have a bash meaning in addition to a sed meaning.

  • ? becomes \?
  • ( becomes \(
  • ) becomes \)
  • | becomes \|

The script

Hopefully by now it’s [Archive.is] been merged into https://github.com/comotion/gone/blob/github/modules/ssh. If not, it’s at https://github.com/jpluimers/gone/blob/jpluimers-ssh-hardening-patch/modules/ssh.

The diff: [Archive.is] https://github.com/jpluimers/gone/commit/329bf12a320704080e68eee90f4c099e92d8388d?diff=unified

The relevant portion (which also uses backslashes as line continuation and wrap a command over multiple lines [WayBack]):

sed -i \
-e 's/#\?MaxAuthTries *[0-9]*.*/MaxAuthTries 2/' \
-e 's/#\?PermitRootLogin *\(yes\|no\).*/PermitRootLogin no/' \
-e 's/#\?UsePrivilegeSeparation *\(yes\|no\|sandbox\).*/UsePrivilegeSeparation sandbox/' \
-e 's/#\?StrictModes *\(yes\|no\).*/StrictModes yes/' \
-e 's/#\?IgnoreRhosts *\(yes\|no\).*/IgnoreRhosts yes/' \
-e 's/#\?PermitEmptyPasswords *\(yes\|no\).*/PermitEmptyPasswords no/' \
-e 's/#\?ChallengeResponseAuthentication *\(yes\|no\).*/ChallengeResponseAuthentication yes/' \
-e 's/#\?KerberosAuthentication *\(yes\|no\).*/KerberosAuthentication no/' \
-e 's/#\?GSSAPIAuthentication *\(yes\|no\).*/GSSAPIAuthentication no/' \
-e 's/#\?GatewayPorts *\(yes\|no\).*/GatewayPorts no/' \
-e 's/#\?X11Forwarding *\(yes\|no\).*/X11Forwarding no/' \
-e 's/#\?PrintMotd *\(yes\|no\).*/PrintMotd no/' \
-e 's/#\?PrintLastLog *\(yes\|no\).*/PrintLastLog yes/' \
-e 's/#\?TCPKeepAlive *\(yes\|no\).*/TCPKeepAlive no/' \
-e 's/#\?PermitUserEnvironment *\(yes\|no\).*/PermitUserEnvironment no/' \
-e 's/^\(HostKey .*ssh_host_dsa_key\)/#\1/' \
sshd_config

More on sshd hardening

In case I have to revisit the script again, here are some more links on ssh and hardening from my blog posts:

–jeroen

 

 

 

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, sed, sed script, Software Development | Leave a Comment »

linux – Test if a port on a remote system is reachable (without telnet) – Super User

Posted by jpluimers on 2019/01/29

Just learned that bash can do TCP and UDP itself:

Bash has been able to access TCP and UDP ports for a while. From the man page:

/dev/tcp/host/port
    If host is a valid hostname or Internet address, and port is an integer port number
    or service name, bash attempts to open a TCP connection to the corresponding socket.
/dev/udp/host/port
    If host is a valid hostname or Internet address, and port is an integer port number
    or service name, bash attempts to open a UDP connection to the corresponding socket.

So you could use something like this:

xenon-lornix:~> cat < /dev/tcp/127.0.0.1/22
SSH-2.0-OpenSSH_6.2p2 Debian-6
^C pressed here

Taa Daa!

This for systems that do not have telnet installed (Windows stopped using this a long time ago, many Linux distributions followed suit) and you cannot to use nc (also known as netcat).

–jeroen: [WayBack] linux – Test if a port on a remote system is reachable (without telnet) – Super User

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

« Previous Entries
Next Entries »