May 2026
M	T	W	T	F	S	S
	1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

Archive for the ‘*nix-tools’ Category

Find the TTL for a domain and subdomain by getting to the authoritative nameserver first

Posted by jpluimers on 2018/11/15

Lets find the authoritative name server and TTL (time to live) for the example.org domain and www.example.org subdomain.

Notes:

you can also do this using the web-page wrapper around dig [WayBack] Dig web interface

web-pages return most information in one go are [WayBack] intodns.com/example.org and [WayBack] dnsinspect.com: Report for example.org.

the steps below are partially based on a more in depth domain consistency check at [WayBack] Debugging nameservers using dig +norec

1a: get parents of name servers

First start with a root server (dig: getting the list of root servers) to get parents of the name servers for example.org (don’t you love indirection!):

$ dig +norecurse +noall +authority @f.root-servers.net. example.org.
org.            172800  IN  NS  a0.org.afilias-nst.info.
org.            172800  IN  NS  a2.org.afilias-nst.info.
org.            172800  IN  NS  b0.org.afilias-nst.org.
org.            172800  IN  NS  b2.org.afilias-nst.org.
org.            172800  IN  NS  c0.org.afilias-nst.info.
org.            172800  IN  NS  d0.org.afilias-nst.org.

You can repeat this query for 2 more root servers to ensure they are in sync.

1b: get authoritative name servers from the parents

Now repeat with at least 3 of these to ensure they give matching results for the name servers for example.org:

$ dig +norecurse +noall +authority @b0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  b.iana-servers.net.
example.org.        86400   IN  NS  a.iana-servers.net.
$ dig +norecurse +noall +authority @c0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.
$ dig +norecurse +noall +authority @a0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.

2a: getting the domain name servers from a public name server

A query to a public DNS server will also return a name server list, but then you would need to know that name server first. In addition, you can not ask for +authority; you have to ask for +answer NS in stead:

$ dig +norecurse +noall +answer NS @8.8.8.8 example.org.
example.org.        55312   IN  NS  a.iana-servers.net.
example.org.        55312   IN  NS  b.iana-servers.net.

The name servers on the list are not guaranteed to be authoritative, as this query returns an empty result:

$ dig +norecurse +noall +authority @8.8.8.8 example.org.

2b. ensuring the name servers are authoritative name servers

From the name servers returned, you can now check if the servers themselves return the same name servers. If so, then you are sure they are authoritative:

$ dig +norecurse +noall +authority @a.iana-servers.net. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.
$ dig +norecurse +noall +authority @b.iana-servers.net. example.org.
example.org.        86400   IN  NS  b.iana-servers.net.
example.org.        86400   IN  NS  a.iana-servers.net.

3: get the actual TTL

With the authoritative name servers, you can get the actual TTL:

$ dig +norecurse +noall +answer SOA @a.iana-servers.net. example.org.
example.org.        3600    IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +multiline +answer SOA @a.iana-servers.net. example.org.
example.org.        3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
                2017042729 ; serial
                7200       ; refresh (2 hours)
                3600       ; retry (1 hour)
                1209600    ; expire (2 weeks)
                3600       ; minimum (1 hour)
                )

I got the +multiline trick from [WayBack] HOWTO: Using dig(1) to Find DNS Time to Live (TTL) Values – A-Team Systems.

4: get the count down TTL from a local name server

You can repeat the above process with a non-authoritative name server a few times to see the TTL decrease:

$ dig +norecurse +noall +answer SOA example.org.
example.org.        322 IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA example.org.
example.org.        321 IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600

This is for instance what is returned by [WayBack] example.org DNS information – who.is example.org DNS information. DNS records such SOA, TTL, MX, TXT and more.

Public DNS servers having multiple servers per IP can even run disperse TTL numbers, for instance Google DNS at 8.8.8.8 does this:

$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        13  IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        1388    IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        10  IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600

Note that `+nssearch` does not work for me

Using +nssearch as per [WayBack] How to find what Authoritative Name Server provided the answer using dig? – Server Fault fails for me:

$ dig +nssearch example.org
SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600 from server 199.43.135.53 in 83 ms.
SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600 from server 199.43.133.53 in 144 ms.
;; connection timed out; no servers could be reached

This reveals this in the bold values:

The authoritative nameserver sns.dns.icann.org is not publicly accessible.
TTL 7200 (7200 seconds is 2 hours).

Future research

Authoritative answers might not be present in dig queries on some platforms. I need to dig deeper into [WayBack] terminal – Dig not returning authority section? – Ask Different to see why.

Glue records are always tricky to get right: [WayBack] How to check domain NS glue records using dig « Admins eHow

–jeroen

Posted in *nix, *nix-tools, bash, Development, dig, DNS, Internet, Power User, Scripting, Software Development | Leave a Comment »

PSBL: Passive Spam Block List powered by Spamikaze

Posted by jpluimers on 2018/11/01

On my research list: [WayBack] Passive Spam Block List:

PSBL is an easy-on, easy-off blacklist that does not rely on testing and should reduce false positives because any user can remove their ISP’s mail server from the list.

The idea is that 99% of the hosts that send me spam never send me legitimate email, but that people whose mail server was used by spammers should still be able to send me email.

This results in a simple listing policy: an IP address gets added to the PSBL when it sends email to a spamtrap, that email is not identified as non-spam and the IP address is not a known mail server.

Via: Hans Wolters commenting at [WayBack] For my research list: Source: Bruteforce login prevention… – Jeroen Wiert Pluimers – Google+

References:

–jeroen

Posted in *nix, *nix-tools, Development, Perl, Power User, Scripting, Software Development | Leave a Comment »

wget and curl: downloads that sometimes fail

Posted by jpluimers on 2018/10/19

For my archive somewhere between cURL 7.21.0 and 7.34.0 it does not like to be started from an RDP based tsclient share:

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.21.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 86465    0 86465    0     0  60805      0 --:--:--  0:00:01 --:--:-- 70012

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.34.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: web.archive.org

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.61.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.xs4all.nl

C:\Users\jeroen\Downloads>copy \\tsclient\bin\curl.7.61.0.exe
        1 file(s) copied.

C:\Users\jeroen\Downloads>curl.7.61.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    13    0    13    0     0     10      0 --:--:--  0:00:01 --:--:--    10

It fails the same way after net use B: \\tsclient\bin, so that does not matter.

The best link I could find until I got to the real problem was [WayBack] curl: (6) Could not resolve host: application – Stack Overflow which shows a different problem: properly quoting.

In addition to remote-name, you can also grab the file name from the headers using --remote-header-name, and --remote-time use the remote file time. The --location follows 302-redirects. You can see that in the example below which I build based on

[WayBack] unix – Curl to grab remote filename after following location – Stack Overflow: The remote side sends the filename using the Content-Disposition header.curl 7.21.2 or newer does this automatically if you specify –remote-header-name / -J.curl -O -J -L $url

C:\Users\jeroen\Downloads>b:\curl.7.21.0.exe --location --remote-name --remote-time --remote-header-name "https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 86465    0 86465    0     0  45748      0 --:--:--  0:00:01 --:--:-- 50772
curl: Saved to filename 'pkgWuppdiWP_DX102T_1-1-2.zip'

wget failed big time:

C:\Users\jeroen\Downloads>B:\wget.exe --no-check-certificate -v -v -v --content-disposition --restrict-file-names=windows "https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965"
wget: Cannot read b:/.wgetrc (No such file or directory).
--2018-07-12 09:55:23--  https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965
Resolving web.archive.org... 207.241.225.186
Connecting to web.archive.org|207.241.225.186|:443... failed: Invalid argument.
Retrying.

...

--2018-07-12 09:55:23--  (try:20)  https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965
Connecting to web.archive.org|207.241.225.186|:443... failed: Invalid argument.
Giving up.

This is not caused by the filename (Windows does not like the ? question mark in output file names, so – like & ampersand in file URLs – you have to quote the full URL, but also provide the --restrict-file-names=windows parameter; see [WayBack] wget – I can’t download files with “?” – Super User).

–jeroen

Posted in *nix, *nix-tools, cURL, Power User, wget | Leave a Comment »

when btrfs-size shows a snapshot as 16777216.00TB or btrfs qgroup as 16.00EiB

Posted by jpluimers on 2018/10/19

A long time ago I wrote about the btrfs-size tool: [WayBack] A bash script to btrfs snapshot details like disk sizes (requires btrfs quota to be enabled).

One day, it showed a ridiculously large size for /tmp:

# ./btrfs-size.sh 
=============================================================================================================================================================================================================================================================
Snapshot / Subvolume                                               ID   Total    Exclusive Data
=============================================================================================================================================================================================================================================================
257 gen 855182 top level 5 path .snapshots                         257  4.30MB   4.30MB   
258 gen 856438 top level 257 path .snapshots/1/snapshot            258  1.84GB   193.01MB 
...
262 gen 856438 top level 5 path srv                                262  1.83GB   1.83GB   
263 gen 856438 top level 5 path tmp                                263  16777216.00TB16777216.00TB
264 gen 856438 top level 5 path usr/local                          264  260.00KB 260.00KB 
...
990 gen 849192 top level 257 path .snapshots/583/snapshot          990  1.83GB   8.23MB   
991 gen 849224 top level 257 path .snapshots/584/snapshot          991  2.09GB   62.66MB  
=============================================================================================================================================================================================================================================================
                                                                Exclusive Total: 3.26GB

This tracks back to the output of this command, which I’ve shortened a bit:

# btrfs qgroup show /
qgroupid         rfer         excl
--------         ----         ----
0/5          16.00KiB     16.00KiB
0/257         4.30MiB      4.30MiB
...
0/262         1.83GiB      1.83GiB
0/263        16.00EiB     16.00EiB
0/264       260.00KiB    260.00KiB
...
255/274         0.00B        0.00B
255/797      16.00KiB     16.00KiB

This is a known issue as quotas in btrfs – though workable – aren’t fully stable yet: [WayBack] Linux BTRFS Storage: Re: During a btrfs balance nearly all quotas of the subvolumes became exceeded

It also provides this simple solution:

Read the rest of this entry »

Posted in *nix, *nix-tools, btrfs, File-Systems, Power User | Leave a Comment »

linux – dmesg time vs system time time isnt correct – Server Fault

Posted by jpluimers on 2018/10/10

[WayBack] linux – dmesg time vs system time time isnt correct – Server Fault helped me solve this problem with an Odroid C1+ running busybox:

[root@meye-062016b9 ~]# hwclock --show
Wed Apr  3 20:25:47 2013  0.000000 seconds
[root@meye-062016b9 ~]# date
Wed May 31 09:48:18 UTC 2018
[root@meye-062016b9 ~]# hwclock --systohc --utc
[root@meye-062016b9 ~]# hwclock --show
Wed May 31 09:48:29 2018  0.000000 seconds
[root@meye-062016b9 ~]# date
Wed May 31 09:48:35 UTC 2018
[root@meye-062016b9 ~]#
Note: If your logging clock in /var/log/dmesg.log is wrong by an exact couple of hours, then try [WayBack] syslog time wrong – but date returns the correct time? and edit [WayBack] /etc/sysconfig/clock.

The above involves looking if I can get MotionEyeOS working Giving up on the official Ubuntu for Odroid C1 image.

So far not much luck: the Ubuntu got hosed, but before it was stable as in that didn’t reboot suddenly.

Now the MotionEyeOS (which is busybox based) reboots itself without notice about every 3 minutes, despite no other hardware connected and trying 3 different power supplies.

The Odroid C1+ only draws 0.34 Ampère at 5.13 Volt which is well within specs.

I’m puzzled:

[Wed May 31 09:49:51 2018] Booting Linux on physical CPU 0x200
[Wed May 31 09:52:20 2018] Booting Linux on physical CPU 0x200
[Wed May 31 09:54:50 2018] Booting Linux on physical CPU 0x200
[Wed May 31 09:57:19 2018] Booting Linux on physical CPU 0x200
[Wed May 31 09:59:49 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:02:22 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:04:56 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:07:26 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:09:59 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:12:29 2018] Booting Linux on physical CPU 0x200
[Wed May 31 10:14:58 2018] Booting Linux on physical CPU 0x200

—jeroen

Posted in *nix, *nix-tools, Power User | Leave a Comment »

tcp – How can I trigger a script when a certain port becomes available for requests? – Unix & Linux Stack Exchange

Posted by jpluimers on 2018/10/09

Netcat to the rescue waiting for a Windows 10 upgrade to finish (which can take hours):

while ! nc -z 172.22.0.67 3389; do echo "sleeping"; sleep 10; done; echo 'The server is up!'

Via: [WayBack] tcp – How can I trigger a script when a certain port becomes available for requests? – Unix & Linux Stack Exchange, quoting from the answer:

nc is Netcat, “the Swiss-army knife for TCP/IP”,

-z means: do not send any data, just check if the port is open,

while ! nc -z …; do sleep 0.1; done: keep checking and sleeping for one tenth of a second until the port opens up, i.e. Netcat returns with a zero (success) status.

–jeroen

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, Power User, TCP, Windows | Leave a Comment »

Check If A Linux System Is Physical Or Virtual Machine

Posted by jpluimers on 2018/10/08

One day I am going to try to extend this for a few other virtualisation environments and Linux distributions: [WayBack] Check If A Linux System Is Physical Or Virtual Machine

Via: [WayBack] Check If A Linux System Is Physical Or Virtual Machine #Linux – Joe C. Hecht – Google+

–jeroen

Posted in *nix, *nix-tools, Fusion, Hyper-V, KVM Kernel-based Virtual Machine, Power User, Proxmox, View, VirtualBox, Virtualization, VMware, VMware ESXi, VMware Workstation | Leave a Comment »

aha (Ansi HTML Adapter) with clickable URIs

Posted by jpluimers on 2018/10/02

aha is great to generate HTML from ANSI text (i.e. the coloured output on a Linux console).

But it doesn’t generate clickable URIs (it can’t yet by itself as it only looks one character in the future).

The thread at https://github.com/theZiz/aha/issues/20 suggested a case-insensitive regex through sed but the exact suggestion failed for a few reasons I will explain below.

First the bash alias (requires both aha and perl):

	#!/usr/bin/env bash
	# based on https://github.com/theZiz/aha/issues/20#event-797466520
	aha-with-expanded-http-https-urls()
	{
	aha \| perl -C -Mutf8 -pe 's,([^"])((https?\|s?ftp\|ftps?\|file)://.*?)([\s]\|\"\s),$1<a href="$2">$2</a>$4,gi'
	}

view raw

testssh.sh-through-aha-with-expanded-http(s)-urls.bash

hosted with ❤ by GitHub

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Perl, Power User, RegEx, Scripting, Software Development | Leave a Comment »

On my todo list: experiment with having multiple sendmail queue directories under /var/spool/mqueue

Posted by jpluimers on 2018/09/27

Right now my sendmail configuration handling my domains have one queue directory /var/spool/mqueue which means that each round of the queue processing handles all the outgoing mail in succession.

This is getting less OK because of the increased mail volume over time both on mail that gets in and needs to be forwarded and mail that needs to be bounced for various reasons like SPAM.

So below are some links helping me to sort out various things including having multiple queues (as then each round can handle each queue in parallel).

The default sendmail configuration is one mail queue and I hope to find out for what reason that is.

Background info:

Read the rest of this entry »

Posted in *nix, *nix-tools, Development, Power User, Scripting, sendmail, Software Development | Leave a Comment »

pure-bash-bible/README.md – book for doing things in bash without external tools

Posted by jpluimers on 2018/09/21

[WayBack] pure-bash-bible/README.md at master · dylanaraps/pure-bash-bible · GitHub:

The goal of this book is to document known and unknown methods of doing various tasks using only built-in bash features. Using the snippets from this bible can help remove unneeded dependencies from scripts and in most cases make them faster. I came across these tips and discovered a few while developing neofetch, pxltrm and other smaller projects.

The snippets below are linted using shellcheck and tests have been written where applicable. Want to contribute? Read the CONTRIBUTING.md. It outlines how the unit tests work and what is required when adding snippets to the bible.

See something incorrectly described, buggy or outright wrong? Open an issue or send a pull request. If the bible is missing something, open an issue and a solution will be found.

Via:

—jeroen

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

« Previous Entries

Next Entries »

	Jeroen Wiert Pluimer… on Arjen Lentz Crystal Ball Vulne…
	Jeroen Wiert Pluimer… on Digitale toegankelijkheid als…
	Jeroen Wiert Pluimer… on Digitale toegankelijkheid als…
	Vereniging NLUUG on Digitale toegankelijkheid als…
	jpluimers on Sony STR-DE205 Receiver…

The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

Twitter Updates

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘*nix-tools’ Category

Find the TTL for a domain and subdomain by getting to the authoritative nameserver first

1a: get parents of name servers

1b: get authoritative name servers from the parents

2a: getting the domain name servers from a public name server

2b. ensuring the name servers are authoritative name servers

3: get the actual TTL

4: get the count down TTL from a local name server

Note that `+nssearch` does not work for me

Future research

PSBL: Passive Spam Block List powered by Spamikaze

wget and curl: downloads that sometimes fail

when btrfs-size shows a snapshot as 16777216.00TB or btrfs qgroup as 16.00EiB

linux – dmesg time vs system time time isnt correct – Server Fault

tcp – How can I trigger a script when a certain port becomes available for requests? – Unix & Linux Stack Exchange

Check If A Linux System Is Physical Or Virtual Machine

aha (Ansi HTML Adapter) with clickable URIs

On my todo list: experiment with having multiple sendmail queue directories under /var/spool/mqueue

pure-bash-bible/README.md – book for doing things in bash without external tools

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘*nix-tools’ Category

1a: get parents of name servers

1b: get authoritative name servers from the parents

2a: getting the domain name servers from a public name server

2b. ensuring the name servers are authoritative name servers

3: get the actual TTL

4: get the count down TTL from a local name server

Note that +nssearch does not work for me

Future research

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Note that `+nssearch` does not work for me