The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for the ‘bash’ Category

“NAMED_CONF_INCLUDE_FILES” has been gone from /etc/sysconfig/named since OpenSuSE 15.4

Posted by jpluimers on 2025/04/28

In the past, I used to modify /etc/sysconfig/named and add entries to the NAMED_CONF_INCLUDE_FILES setting, then run /usr/share/bind/createNamedConfInclude
to generate /etc/named.conf.include.

As of OpenSuSE 15.4, /usr/share/bind/createNamedConfInclude has become an empty file and NAMED_CONF_INCLUDE_FILES got removed and NAMED_INITIALIZE_SCRIPTS introduced.

So now I changed my playbooks to manually generate /etc/named.conf.include and include it form /etc/sysconfig/named.

Since I hardly perform these new installations, it took a few years for me to find out about this change. Upgrading existing systems somehow kept the generated file and included it.

Related links with quotes as it was hard to find out what changed and how to work around and I wasn’t the only one bump into issues:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, bind-named, Development, DNS, LEAP, Linux, openSuSE, Power User, Scripting, Software Development, SuSE Linux | Tagged: | Leave a Comment »

Weather info in plain text or JSON

Posted by jpluimers on 2025/03/06

Two sites that can help you out getting weather info on the console:

wttr.in

wttr.in is developed by [Wayback/Archive] Igor Chubin (@igor_chubin) / X and looks at the request header to figure out what kind of output it sends.

It supports various output formats, so on my TODO list is to see how they do their mapping: always an opportunity to learn (it’s based on Python and Go so I am curious what libraries they use as well).

From the documentation:

wttr.in currently supports five output formats:

  • ANSI for the terminal;
  • Plain-text for the terminal and scripts;
  • HTML for the browser;
  • PNG for the graphical viewers;
  • JSON for scripts and APIs;
  • Prometheus metrics for scripts and APIs.

The ANSI and HTML formats are selected based on the User-Agent string.

There are more parameters in the documentation on the main page of the repository and through this command:

curl wttr.in/:help

The idea is derived from [Wayback/Archive] GitHub – schachmat/wego: weather app for the terminal.

Oh: Igor has more repositories at [Wayback/Archive] chubin (Igor Chubin) · GitHub (including [Wayback/Archive] GitHub – chubin/cheat.sh: the only cheat sheet you need which is hosted at [Wayback/Archive] cheat.sh; I thought I had blogged about that before, but found it only in a draft note mentioning that I got it via [WaybackSave/Archive] Nicolas Krassas on X: “The only cheat sheet you need cheat.sh)

7timer

A 7 timer JSON usage example is at [Wayback/Archive] Get Weather from 7Timer! · GitHub

It has documentation at

Output formats can be chosen from HTML, PNG, XML and JSON.

Via

[Wayback/Archive] Hacker Public Radio – hpr4266 :: What’s the weather?
Lee writes a script to check what the weather is like ~ The Technology Community Podcast

HPR is a great podcast series!

--jeroen

Posted in *nix, *nix-tools, ash/dash, bash, bash, Development, Go (golang), JavaScript/ECMAScript, JSON, Power User, Python, Scripting, Software Development, Web Development, XML/XSD | Leave a Comment »

ShellCheck – shell script analysis tool

Posted by jpluimers on 2024/02/22

Cool: [Wayback] ShellCheck – shell script analysis tool

ShellCheck finds bugs in your shell scripts

It needs a shebang at the start of a script (like #!/usr/bin/env bash) to recognise the kind of shell, then does amazing analysis.

It is open source at [Wayback/Archive.is] koalaman/shellcheck: ShellCheck, a static analysis tool for shell scripts with excellent documentation including explaining screenshots like

It’s not just available on-line or on the command-line, but also integrates with many code editors (like [Wayback/Archive.is] ShellCheck – Visual Studio Marketplace: Integrates ShellCheck into VS Code, a linter for Shell scripts.) and CI/CD pipelines.

Via: [Wayback] bash – error conditional binary operator expected in compound branch – Unix & Linux Stack Exchange (thanks [Wayback] Cyrus!)

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

GitHub – kellyjonbrazil/jc: CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

Posted by jpluimers on 2023/10/17

I already knew about jq and jo (output: the echo for JSON), but not yet about jc for JSON.

Like jq is for querying (the sed for JSON), jc tries to be the universal parser of common command-line tools into JSON:

[Wayback/Archive] kellyjonbrazil/jc: CLI tool and python library that converts the output of popular command-line tools and file-types to JSON or Dictionaries. This allows piping of output to tools like jq and simplifying automation scripts.

Via [Wayback/ArchiveKris on Twitter: ““Jc” verwandelt den Output von üblichen Linux Kommandozeilen Tools in JSON. “Jc” kann auch als Import in Python Programmen verwendet werden und mit subprocess kombiniert werden. Danke, ⁦@janwalzer⁩ und ⁦@the_mutax⁩ für den Tip. “

A few weeks later though, Kris discovered that command-line tools aren’t that portable in their output format: [Wayback/Archive] Kris on Twitter: “As much as I love the idea of @kellyjonbrazil’s jc, this is not a winnable game. sigh Probably still better than a self-cooked parser, but the slightest bit of extra makes it catch fire in multiple possible ways …”

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, JavaScript/ECMAScript, jo, jq, JSON, man/manual pages, mankier, Power User, Python, Scripting, Software Development | Leave a Comment »

Some threadreaderapp URLs

Posted by jpluimers on 2023/09/14

For my link archive so I can better automate archiving Tweet threads using bookmarklets written in JavaScript:

The base will likely be this:

javascript:void(open(`https://archive.is/?run=1&url=${encodeURIComponent(document.location)}`))

which for now I have modified into this:

javascript:void(open(`https://threadreaderapp.com/search?q=${document.location}`))

It works perfectly fine without URL encoding and demonstrates the JavaScript backtick feature for template literals for which you can find documentation at [WayBack/Archive] Template literals – JavaScript | MDN.

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Bookmarklet, Communications Development, cURL, Development, HTTP, https, Internet protocol suite, Power User, Scripting, Security, Software Development, TCP, Web Browsers | Leave a Comment »

5 days after the exploit publication of snowcra5h/CVE-2023-38408: Remote Code Execution in OpenSSH’s forwarded ssh-agent

Posted by jpluimers on 2023/07/26

TL;DR is at the bottom (;

5 days ago this exploit development got published: [Wayback/Archive] snowcra5h/CVE-2023-38408: CVE-2023-38408 Remote Code Execution in OpenSSH’s forwarded ssh-agent.

It is about [Wayback/Archive] NVD – CVE-2023-38408 which there at NIST isn’t rated (yet?), neither at [Wayback/Archive] CVE-2023-38408 : The PKCS#11 feature in ssh-agent in OpenSSH before 9.3p2 has an insufficiently trustworthy search path, leading to remot.

However at [Wayback/Archive] CVE-2023-38408- Red Hat Customer Portal it scores 7.3 and [Wayback/Archive] CVE-2023-38408 | SUSE it did get a rating of 7.5, so since I mainly use OpenSuSE I wondered what to do as the CVE is formulated densely at [Wayback/Archive] www.qualys.com/2023/07/19/cve-2023-38408/rce-openssh-forwarded-ssh-agent.txt: it mentions Alice, but no Bob or Mallory (see Alice and Bob – Wikipedia).

Luckily, others readly already did the fine reading and emphasised the important bits, especially at [Wayback/Archive] RCE Vulnerability in OpenSSH’s SSH-Agent Forwarding: CVE-2023-38408 (note that instead of Alex, they actually mean Alice)

“A system administrator (Alice) runs SSH-agent on her local workstation, connects to a remote server with ssh, and enables SSH-agent forwarding with the -A or ForwardAgent option, thus making her SSH-agent (which is running on her local workstation) reachable from the remote server.”

According to researchers from Qualys, a remote attacker who has control of the host, which Alex has connected to, can load (dlopen()) and immediately unload (dlclose()) any shared library in /usr/lib* on Alice’s workstation (via her forwarded SSH-agent if it is compiled with ENABLE_PKCS11, which is the default).

The vulnerability lies in how SSH-agent handles forwarded shared libraries. When SSH-agent is compiled with ENABLE_PKCS11 (the default configuration), it forwards shared libraries from the user’s local workstation to the remote server. These libraries are loaded (dlopen()) and immediately unloaded (dlclose()) on the user’s workstation. The problem arises because certain shared libraries have side effects when loaded and unloaded, which can be exploited by an attacker who gains access to the remote server where SSH-agent is forwarded to.

Mitigations for the SSH-Agent Forwarding RCE Vulnerability

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Communications Development, Development, Internet protocol suite, OpenSSH, Power User, PowerShell, Scripting, Security, Software Development, SSH | Leave a Comment »

Figuring out the open network connections for processes ran by python

Posted by jpluimers on 2023/07/11

TL;DR:

pidof python | tr " " "\n" | xargs -r -n 1 lsof -i -a -e /run/user/1001/gvfs -p

Breakdown:

  • Getting the process IDs of any python process using pidof (most of my systems do not have pgrep installed):
    # pidof python
    26128 12583
    
  • Given the above list is space separated, and xargs prefers line separated, lets replace spaces with newlines (I showed this before in Source: firewalld: show interfaces with their zone details and show zones in use):
    # pidof python | tr " " "\n"
    26128
    12583
    
  • By default, xargs squashes all input on one line:
    # pidof python | tr " " "\n" | xargs echo
    26128 12583
    
  • To work around that, you can either use the -L 1 or -n 1 argument to keep them on separate lines:
    # pidof python | tr " " "\n" | xargs -L 1 echo
    26128
    12583
    # pidof python | tr " " "\n" | xargs -n 1 echo
    26128
    12583
    
  • Now lsof can not only show open files, but also IP sockets (-i), and *only* those (-a), for a specific process ID (-p #). So by having the -p as last argument, xargs will append the process ID after it:
    # pidof python | tr " " "\n" | xargs -n 1 lsof -i -a -p
    lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1001/gvfs
          Output information may be incomplete.
    lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1001/gvfs
          Output information may be incomplete.
    COMMAND   PID    USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
    python  12583 jeroenp    7u  IPv4 8347396      0t0  TCP 192.168.124.38:54576->192.168.124.23:1012 (ESTABLISHED)
    python  12583 jeroenp    8u  IPv4 8345460      0t0  TCP 192.168.124.38:48250->192.168.124.23:http (CLOSE_WAIT)
  • The lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1001/gvfs is a warning not easy to workaround in a short manner as per [Wayback/Archive] privileges – lsof: WARNING: can’t stat() fuse.gvfsd-fuse file system – Unix & Linux Stack Exchange (thanks [Wayback/Archive] pabouk  and [Wayback/Archive] jmunsch):

    In your case lsof does not need to check the GVFS file systems so you can exclude the stat() calls on them using the -e option (or you can just ignore the waring):

    lsof -e /run/user/1000/gvfs

    (via: [Wayback/Archive] lsof: WARNING: can’t stat() fuse.gvfsd-fuse file system /run/user/1001/gvfs – Google Search)

    So you get this:

    # pidof python | tr " " "\n" | xargs -n 1 lsof -i -a -e /run/user/1001/gvfs -p
    COMMAND   PID    USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
    python  12583 jeroenp    7u  IPv4 8347396      0t0  TCP 192.168.124.38:54576->192.168.124.23:1012 (ESTABLISHED)
    python  12583 jeroenp    8u  IPv4 8345460      0t0  TCP 192.168.124.38:48250->192.168.124.23:http (CLOSE_WAIT)
  • When there are no process IDs, you do not want to run lsof, and xargs has an argument just for that: -r, see my earlier post Source: -r argument to pipe (no argument for MacOS)- If no input is given to xargs, don’t let xargs run the utility – Unix & Linux Stack Exchange, so you get this
    # pidof python | tr " " "\n" | xargs -r -n 1 lsof -i -a -e /run/user/1001/gvfs -p

Via:

–jeroen

Posted in *nix, *nix-tools, bash, bash, Development, lsof, Power User, Scripting, Software Development, xargs | Leave a Comment »

llamasoft/polyshell: A Bash/Batch/PowerShell polyglot!

Posted by jpluimers on 2023/03/16

PolyShell is a script that’s simultaneously valid in Bash, Windows Batch, and PowerShell (i.e. a polyglot).

[Wayback/Archive] llamasoft/polyshell: A Bash/Batch/PowerShell polyglot!

Need to check this out, as often I have scripts that have to go from one language to the other or vice versa.

Maybe it enables one language to bootstrap functionality in the other?

The quest

The above polyglot started with a quest to see if I can could include some PowerShell statements in a batch file with two goals:

  1. if the batch file started from the PowerShell command prompt, then execute the PowerShell code
  2. if the batch file started from the cmd.exe command prompt, then have it start PowerShell with the same command-line arguments

The reasoning is simple:

  1. PowerShell scripts will start from the PATH only when PowerShell is already running
  2. Batch files start from the path when either cmd.exe or PowerShell are running

Lots of users still live in the cmd.exe world, but PowerShell scripts are way more powerful, and since PowerShell is integrated in Windows since version 7, so having a batch file bootstrap PowerShell still makes sense.

Since my guess was about quoting parameters the right way, my initial search for the link below was [Wayback/Archive] powershell execute statement from batch file quoting – Google Search.

I have dug not yet into this, so there are still…

Many links to read

These should give me a good idea how to implement a polyglot batch file/PowerShell script.

–jeroen

Posted in *nix, *nix-tools, bash, bash, Batch-Files, Development, JavaScript/ECMAScript, Perl, Polyglot, Power User, PowerShell, Scripting, Software Development | Leave a Comment »

On my list of *n*x things to play with: script and ttyrec

Posted by jpluimers on 2023/01/26

Because of [Archive] PragmaticProgrammers on Twitter: “Helpful Unix trick: use script to log your session. …” / Twitter:

–jeroen

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, bash, bash, Batch-Files, Development, Power User, Scripting, Software Development | Leave a Comment »

Getting your public IP address from the command-line when http and https are blocked: use DNS

Posted by jpluimers on 2022/12/28

Years ago, I wrote Getting your public IP address from the command-line. All methods were http based, so were very easy to execute using cURL.

But then in autumn 2021, Chris Bensen wrote this cool little blog-post [Wayback/Archive] Chris Bensen: How do I find my router’s public IP Address from the command line?:

dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com

At first sight, I thought it was uncool, as the command was quite long and there was no explanation of the dig command trick.

But then, knowing that dig is a DNS client, it occurred to me: this perfectly works when http and https are disabled by your firewall, but the DNS protocol works and gives the correct result:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
"80.100.143.119"

This added the below commands and aliases to my tool chest for *nix based environments like Linux and MacOS (not sure yet about Windows yet :), but that still doesn’t explain why it worked. So I did some digging…

IPv4

  • command:
    dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
  • command removing outer double quotes:
    dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
  • alias:
    alias "whatismyipv4_dns=dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs"

IPv6

  • command:
    dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com
  • command removing outer double quotes:
    dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
  • alias:
    alias "whatismyipv6_dns=dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs"

How it works

Let’s stick to dig and IPv4 as that not having IPv6 (regrettably still) is the most common situation today:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
"80.100.143.119"

What it does is request the DNS TXT record of o-o.myaddr.l.google.com from the Google DNS server ns1.google.com and returns the WAN IPv4 address used in the DNS request, which is for instance explained in [Wayback/Archive] What is the mechanics behind “dig TXT o-o.myaddr.l.google.com @ns1.google.com” : linuxadmin.

Since these are TXT records, dig will automatically double quote them, which xargs can remove (see below how and why):

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
80.100.143.119

The DNS query will fail when requesting the Google Public DNS servers 8.8.8.8 or 8.8.4.4:

# dig -4 TXT +short o-o.myaddr.l.google.com @8.8.8.8
"2a00:1450:4013:c1a::103"
"edns0-client-subnet 80.101.239.0/24"

Or, with quotes removed (the -L 1 ensures that xargs performs the quote-pair removal action on each line):

# dig -4 TXT +short o-o.myaddr.l.google.com @8.8.8.8 | xargs -L 1
2a00:1450:4013:c1a::103
edns0-client-subnet 80.101.239.0/24

This request is both slower than requesting the ns1.google.com server and wrong.

The reason is that only ns1.google.com understands the special o-o.myaddr.l.google.com hostname which instructs it to return the IP address of the requesting dig DNS client.

That 8.8.8.8 returns a different IP address and an additional edns0-client-subnet with less accurate information is explained in an answer to [Wayback/Archive] linux – Getting the WAN IP: difference between HTTP and DNS – Stack Overflow by [Wayback/Archive] argaz referring to this cool post: [Wayback/Archive] Which CDNs support edns-client-subnet? – CDN Planet.

Not just ns1.google.com: any DNS server serving the google.com domain

Since o-o.myaddr.l.google.com is part of the google.com domain, the above works for any DNS server serving the google.com domain (more on that domain: [Wayback/Archive] General DNS overview  |  Google Cloud).

Getting the list of DNS servers is similar to getting the list of MX servers which I explained in Getting the IP addresses of gmail MX servers, replacing MX record type (main exchange) with the NS record type (name server) and the gmail.com domain with the google.com domain:

# dig @8.8.8.8 +short NS google.com
ns3.google.com.
ns1.google.com.
ns2.google.com.
ns4.google.com.

The ns1.google.com DNS server is a special one of the NS servers: it is the start of authority server, which you can query using the SOA record type that also gives slightly more details for this server:

# dig @8.8.8.8 +short SOA google.com
ns1.google.com. dns-admin.google.com. 410477869 900 900 1800 60

The difference between using NS and SOA records with dig are explained in the [Wayback] dns – How do I find the authoritative name-server for a domain name? – Stack Overflow answer by [Wayback/Archive] bortzmeyer who also explains how to help figuring out SOA and NS discrepancies (note to self: check out the check_soa tool originally by Michael Fuhr (I could not find recent content of him, so he might have passed away) of which source code is now at [Wayback/Archive] Net-DNS/check_soa at master · NLnetLabs/Net-DNS).

So this works splendid as well using ns4.google.com on my test system:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns4.google.com | xargs
80.100.143.119

The xargs removes outer quotes removal trick

[Wayback/Archive] string – Shell script – remove first and last quote (“) from a variable – Stack Overflow (thanks quite anonymous [Wayback/Archive] user1587520):

> echo '"quoted"' | xargs
quoted

xargs uses echo as the default command if no command is provided and strips quotes from the input.

More on https versus DNS requests

Some notes are in [Wayback/Archive] How to get public IP address from Linux shell, but note the telnet trick now fails as myip.gelma.net is gone (latest live version was archived in the Wayback Machine in august 2019).

Via

–jeroen

Posted in *nix, *nix-tools, Apple, bash, bash, Batch-Files, Communications Development, Development, DNS, Internet protocol suite, Linux, Mac, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development, TCP | Leave a Comment »