The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Email Subscription

    Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 717 other subscribers

Archive for the ‘DNS’ Category

« Previous Entries
Next Entries »

b0rk does fun things with DNS: CNAME records at the root of the domain; technically not allowed, definitely not recommended, but somehow work for web browsing

Posted by jpluimers on 2023/12/21

[Wayback/Archive] 🔎Julia Evans🔍 on Twitter: “I’ve always heard that you can’t create CNAME records at the root of the domain. But apparently you can? It seems to work fine as far as I can tell but I’m curious about the possible consequences. (yes, I registered cnameroot.com just to make this tweet) “

Read the rest of this entry »

Posted in Cloud, Cloudflare, DNS, Infrastructure, Internet, Power User | Leave a Comment »

notes on updating DNS info with bind DNS

Posted by jpluimers on 2023/12/01

Steps:

  1. Edit the DNS zone file in /var/lib/named/master
  2. Make sure you updated the SOA serial number
  3. rcnamed restart
  4. Watch /var/lib/named/log/general for errors (you can even increase logging, for instance by reading [Wayback/Archive] this serverfault entry and editing /etc/named.d/logging) and – on my system – these other logs for the actual zone transfers:
    • /var/lib/named/log/in-xfer.log for incoming zone transfers on the secondary server
    • /var/lib/named/log/notify.log for notifications of zone transfers on the primary and secondary servers
    • /var/lib/named/log/out-xfer.log for outgoing zone transfers on the primary server
  5. Check on [Wayback/Archive] mxtoolbox.com/SuperTool.aspx?action=dns:example.org&run=toolpage# to see if the SOA serials have been replicated (one day I will write a bookmarklet for this)

Alternatives for the last step are sites like these:

Read the rest of this entry »

Posted in DNS, Internet, Power User | Leave a Comment »

Smart idea by corbosman on Twitter: he wrote a home grown DynDNS for transIP DNS changes

Posted by jpluimers on 2023/11/02

[Wayback/Archive] corbosman on Twitter: “I created a container in my private kubernetes cluster that checks if my ziggo ip address changes and if so, notifies me and makes some DNS changes through the API of my DNS provider. Last night Ziggo changed my IP and it worked flawlessly. Win!” / Twitter

His ISP: [Wayback/Archive] corbosman on Twitter: “@jpluimers transip“.

The Domain API for his ISP TransIP is at [Wayback/Archive] TransIP API: Domains (via [Wayback/Archive] transip dns api – Google Search).

–jeroen

Posted in Development, DNS, Internet, Power User, Scripting, Software Development | Leave a Comment »

@b0rk (Julia Evans): SPF & DKIM records

Posted by jpluimers on 2023/07/17

A nice zine by @b0rk (Julia Evans) [Wayback/Archive.is] SPF & DKIM records

And a few days later [Archive] 🔎Julia Evans🔍 on Twitter: “… looks like a really nice way to learn about SPF/DKIM (h/t @CubicleApril) you send it an email and it walks you through everything the receiver does to verify the email …” / Twitter which referred to

[Wayback/Archive] Learn and Test DMARC.

Related:

Via: [Archive] 🔎Julia Evans🔍 on Twitter: “SPF & DKIM records …” / Twitter (the original got deleted as it had a [Archive] typo)

–jeroen

Posted in DNS, eMail, Internet, Power User, SocialMedia | Leave a Comment »

mess with dns: a cool site to directly mess around with subdomains of messwithdns.net.

Posted by jpluimers on 2023/03/17

[Wayback/Archive] mess with dns: a cool site at messwithdns.net that allows you directly mess around with subdomains of messwithdns.com (yup: these are on different top level domains by intent, see [Wayback/Archive] mess with dns: about/FAQ).

Her blog post [Wayback/Archive] New tool: Mess with DNS! has a cool insightful explanation with lots of gifs showing what is going on. The below gif is just one of them:

In addition, it has a cool [Wayback/Archive] mess with dns: DNS dictionary explaining common DNS related terms.

The backend is written in Go, and has a snapshot of the initial source code on github at [Wayback/Archive] jvns/mess-with-dns-backend (mainly so you can file issues or submit ideas).

Oh, and this is a cool video that shows you how DNS works assuming the protocol is phone based: [Archive] Max Meier👨🏻‍💻🛡⚔️ on Twitter: “DNS Server …” / Twitter.

Read the rest of this entry »

Posted in DNS, Internet, Power User | Leave a Comment »

Some notes on changing and checking bind DNS entries

Posted by jpluimers on 2023/01/09

DNS isn’t based on propagation, but on (cache) expiry. Which means it is all about TTL (Time to Live), and since humans are bad at coping with caching (remember the post There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors?), I needed some DNS refresh.

The time values in seconds of SOA (Start of Authority) and TTL record entries are always a pain, so hopefully this helps some:

Some TTL recommendations of the DNS SOA record via [Wayback/Archive] dns soa ttl best practice – Google Search and [Wayback/Archive] dns recommended ttl values – Google Search:

I’d rather have a good route for unplanned DNS changes (as in the past, quite a few were about those), so I settled for 3600 seconds (1 hour), but might go down to 600 seconds (10 minutes).

Checking DNS entries

Changing DNS entries

Some information on the DNS “master” files (which contain the RR or “resource records”) from [Wayback/Archive.is] RFC1035: domain names – implementation and specification and [Wayback/Archive] RFC2308: Negative Caching of DNS Queries (DNS NCACHE), as keep forgetting them.

Both RFCs together specify that these line entries are defined (i.e. can be valid):

    <blank>[<comment>]
    $ORIGIN <domain-name> [<comment>]
    $INCLUDE <file-name> [<domain-name>] [<comment>]
    <domain-name><rr> [<comment>]
    <blank><rr> [<comment>]

and that these are the valid <rr> or resource record formats:

    [<TTL>] [<class>] <type> <RDATA>
    [<class>] [<TTL>] <type> <RDATA>

If you do a DNS zone transfer – Wikipedia (often abbreviated to the DNS query type AXFR), then these entries always are single line with fields expanded, with each always having an absolute <domain name> entry ending in a dot (.), and read like this:

<domain name> TTL <class> <type> <RDATA>

From the specific RFCs:

  • RFC 1035 section 3.2.4: CLASS values
  • RFC 1035 section 5.1: master files – format
    • White space to separate items on a line can be spaces or tabs
    • Comments start with semi-colon (;), which means that hashes (#) or double-slashes (//) won’t work and are syntax errors (the above mentioned tool named-checkzone can help you figure out those syntax errors).
    • There are line entries for blank, $ORIGIN, $INCLUDE, and resource record.
    • A line entry starting with $ORIGIN defines the current origin to be used from then on. Without such entry, he default origin can be defined through a parameter when loading the master file (see the Zone file: localhost example file)
    • Resource record line entries can start with:
      • @ to use the current origin as domain name.
      • a specified <domain-name> to denote the domain name to be used
      • no <domain-name> indicating the previous domain name to be used
    • A <domain-name> can be absolute ending in a dot (.)), or relative (in which case the origin is appended).
    • No <TTL> or  no <class> value means to use the previous TTL or class values to be used.
    • One resource record entry per line, unless you put parenthesis, then you can wrap it over multiple lines, which can be convenient for SOA record like this:
      @               IN      SOA     ns1.4delphi.com.        hostmaster.4delphi.com. (
                        2021112201 ; serial
                        3600 ; refresh every 1 hour
                        120 ; retry every 2 minutes
                        1209600 ; expire in 2 weeks
                        10800 ; now is 3 hours (see RFC2308) used to be 2 days: 172800 ; default_ttl
                        )
  • RFC2308 section 4: SOA Minimum Field
    • The $TTL line entry defining the default TTL to be used from then on (similar to $ORIGIN )

There is more (like encoding of domain names, more resource record types, and domain name length limitations), but those go beyond this blog post.

Examples on Wikipedia

Zone file: Example file is great:

$ORIGIN example.com.     ; designates the start of this zone file in the namespace
$TTL 3600                ; default expiration time (in seconds) of all RRs without their own TTL value
example.com.  IN  SOA   ns.example.com. username.example.com. ( 2020091025 7200 3600 1209600 3600 )
example.com.  IN  NS    ns                    ; ns.example.com is a nameserver for example.com
example.com.  IN  NS    ns.somewhere.example. ; ns.somewhere.example is a backup nameserver for example.com
example.com.  IN  MX    10 mail.example.com.  ; mail.example.com is the mailserver for example.com
@             IN  MX    20 mail2.example.com. ; equivalent to above line, "@" represents zone origin
@             IN  MX    50 mail3              ; equivalent to above line, but using a relative host name
example.com.  IN  A     192.0.2.1             ; IPv4 address for example.com
              IN  AAAA  2001:db8:10::1        ; IPv6 address for example.com
ns            IN  A     192.0.2.2             ; IPv4 address for ns.example.com
              IN  AAAA  2001:db8:10::2        ; IPv6 address for ns.example.com
www           IN  CNAME example.com.          ; www.example.com is an alias for example.com
wwwtest       IN  CNAME www                   ; wwwtest.example.com is another alias for www.example.com
mail          IN  A     192.0.2.3             ; IPv4 address for mail.example.com
mail2         IN  A     192.0.2.4             ; IPv4 address for mail2.example.com
mail3         IN  A     192.0.2.5             ; IPv4 address for mail3.example.com

as well as the examples at Zone file: Localhost:

An example for manual configuration of the forward zone for localhost is the following:

$ORIGIN localhost.
@  86400  IN  SOA   @  root (
                  1999010100 ; serial
                       10800 ; refresh (3 hours)
                         900 ; retry (15 minutes)
                      604800 ; expire (1 week)
                       86400 ; minimum (1 day)
                    )
@  86400  IN  NS    @
@  86400  IN  A     127.0.0.1
@  86400  IN  AAAA  ::1

The corresponding reverse zone definition is:

;; reverse zone file for 127.0.0.1 and ::1
$TTL 1814400 ; 3 weeks
@  1814400  IN  SOA     localhost. root.localhost.  (
                      1999010100 ; serial
                           10800 ; refresh (3 hours)
                             900 ; retry (15 minutes)
                          604800 ; expire (1 week)
                           86400 ; minimum (1 day)
                        )
@  1814400  IN  NS      localhost.
1  1814400  IN  PTR     localhost.

This file does not specify the origin so that it may be used for both IPv4 and IPv6 with this configuration:

zone "0.0.127.in-addr.arpa"  IN {
                                type master;
                                file "r.local";
                                };
zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa"  IN {
                                type master;
                                file "r.local";
                                };

Similar zone master files may be created for the reverse resolution of the broadcast address and the null address. Such zone files prevent a DNS server from referring to other, possibly external DNS servers.

Related

OpenSuSE: the relation between /etc/var/named.d and /var/lib/named

—jeroen

Posted in DNS, Internet, Power User | Leave a Comment »

Getting your public IP address from the command-line when http and https are blocked: use DNS

Posted by jpluimers on 2022/12/28

Years ago, I wrote Getting your public IP address from the command-line. All methods were http based, so were very easy to execute using cURL.

But then in autumn 2021, Chris Bensen wrote this cool little blog-post [Wayback/Archive] Chris Bensen: How do I find my router’s public IP Address from the command line?:

dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com

At first sight, I thought it was uncool, as the command was quite long and there was no explanation of the dig command trick.

But then, knowing that dig is a DNS client, it occurred to me: this perfectly works when http and https are disabled by your firewall, but the DNS protocol works and gives the correct result:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
"80.100.143.119"

This added the below commands and aliases to my tool chest for *nix based environments like Linux and MacOS (not sure yet about Windows yet :), but that still doesn’t explain why it worked. So I did some digging…

IPv4

  • command:
    dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
  • command removing outer double quotes:
    dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
  • alias:
    alias "whatismyipv4_dns=dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs"

IPv6

  • command:
    dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com
  • command removing outer double quotes:
    dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
  • alias:
    alias "whatismyipv6_dns=dig -6 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs"

How it works

Let’s stick to dig and IPv4 as that not having IPv6 (regrettably still) is the most common situation today:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com
"80.100.143.119"

What it does is request the DNS TXT record of o-o.myaddr.l.google.com from the Google DNS server ns1.google.com and returns the WAN IPv4 address used in the DNS request, which is for instance explained in [Wayback/Archive] What is the mechanics behind “dig TXT o-o.myaddr.l.google.com @ns1.google.com” : linuxadmin.

Since these are TXT records, dig will automatically double quote them, which xargs can remove (see below how and why):

# dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com | xargs
80.100.143.119

The DNS query will fail when requesting the Google Public DNS servers 8.8.8.8 or 8.8.4.4:

# dig -4 TXT +short o-o.myaddr.l.google.com @8.8.8.8
"2a00:1450:4013:c1a::103"
"edns0-client-subnet 80.101.239.0/24"

Or, with quotes removed (the -L 1 ensures that xargs performs the quote-pair removal action on each line):

# dig -4 TXT +short o-o.myaddr.l.google.com @8.8.8.8 | xargs -L 1
2a00:1450:4013:c1a::103
edns0-client-subnet 80.101.239.0/24

This request is both slower than requesting the ns1.google.com server and wrong.

The reason is that only ns1.google.com understands the special o-o.myaddr.l.google.com hostname which instructs it to return the IP address of the requesting dig DNS client.

That 8.8.8.8 returns a different IP address and an additional edns0-client-subnet with less accurate information is explained in an answer to [Wayback/Archive] linux – Getting the WAN IP: difference between HTTP and DNS – Stack Overflow by [Wayback/Archive] argaz referring to this cool post: [Wayback/Archive] Which CDNs support edns-client-subnet? – CDN Planet.

Not just ns1.google.com: any DNS server serving the google.com domain

Since o-o.myaddr.l.google.com is part of the google.com domain, the above works for any DNS server serving the google.com domain (more on that domain: [Wayback/Archive] General DNS overview  |  Google Cloud).

Getting the list of DNS servers is similar to getting the list of MX servers which I explained in Getting the IP addresses of gmail MX servers, replacing MX record type (main exchange) with the NS record type (name server) and the gmail.com domain with the google.com domain:

# dig @8.8.8.8 +short NS google.com
ns3.google.com.
ns1.google.com.
ns2.google.com.
ns4.google.com.

The ns1.google.com DNS server is a special one of the NS servers: it is the start of authority server, which you can query using the SOA record type that also gives slightly more details for this server:

# dig @8.8.8.8 +short SOA google.com
ns1.google.com. dns-admin.google.com. 410477869 900 900 1800 60

The difference between using NS and SOA records with dig are explained in the [Wayback] dns – How do I find the authoritative name-server for a domain name? – Stack Overflow answer by [Wayback/Archive] bortzmeyer who also explains how to help figuring out SOA and NS discrepancies (note to self: check out the check_soa tool originally by Michael Fuhr (I could not find recent content of him, so he might have passed away) of which source code is now at [Wayback/Archive] Net-DNS/check_soa at master · NLnetLabs/Net-DNS).

So this works splendid as well using ns4.google.com on my test system:

# dig -4 TXT +short o-o.myaddr.l.google.com @ns4.google.com | xargs
80.100.143.119

The xargs removes outer quotes removal trick

[Wayback/Archive] string – Shell script – remove first and last quote (“) from a variable – Stack Overflow (thanks quite anonymous [Wayback/Archive] user1587520):

> echo '"quoted"' | xargs
quoted

xargs uses echo as the default command if no command is provided and strips quotes from the input.

More on https versus DNS requests

Some notes are in [Wayback/Archive] How to get public IP address from Linux shell, but note the telnet trick now fails as myip.gelma.net is gone (latest live version was archived in the Wayback Machine in august 2019).

Via

–jeroen

Posted in *nix, *nix-tools, Apple, bash, bash, Batch-Files, Communications Development, Development, DNS, Internet protocol suite, Linux, Mac, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development, TCP | Leave a Comment »

Viewing the most recent Windows DNS requests on the console

Posted by jpluimers on 2022/07/18

I missed that over time, ipconfig has been extended with some more commands.

The command ipconfig /displaydns does exaxctly what I want: Display the contents of the DNS Resolver Cache.

The ipconfig help:

USAGE:
    ipconfig [/allcompartments] [/? | /all |
                                 /renew [adapter] | /release [adapter] |
                                 /renew6 [adapter] | /release6 [adapter] |
                                 /flushdns | /displaydns | /registerdns |
                                 /showclassid adapter |
                                 /setclassid adapter [classid] |
                                 /showclassid6 adapter |
                                 /setclassid6 adapter [classid] ]

where
    adapter             Connection name
                       (wildcard characters * and ? allowed, see examples)

    Options:
       /?               Display this help message
       /all             Display full configuration information.
       /release         Release the IPv4 address for the specified adapter.
       /release6        Release the IPv6 address for the specified adapter.
       /renew           Renew the IPv4 address for the specified adapter.
       /renew6          Renew the IPv6 address for the specified adapter.
       /flushdns        Purges the DNS Resolver cache.
       /registerdns     Refreshes all DHCP leases and re-registers DNS names
       /displaydns      Display the contents of the DNS Resolver Cache.
       /showclassid     Displays all the dhcp class IDs allowed for adapter.
       /setclassid      Modifies the dhcp class id.
       /showclassid6    Displays all the IPv6 DHCP class IDs allowed for adapter.
       /setclassid6     Modifies the IPv6 DHCP class id.


The default is to display only the IP address, subnet mask and
default gateway for each adapter bound to TCP/IP.

For Release and Renew, if no adapter name is specified, then the IP address
leases for all adapters bound to TCP/IP will be released or renewed.

For Setclassid and Setclassid6, if no ClassId is specified, then the ClassId is removed.

Examples:
    > ipconfig                       ... Show information
    > ipconfig /all                  ... Show detailed information
    > ipconfig /renew                ... renew all adapters
    > ipconfig /renew EL*            ... renew any connection that has its
                                         name starting with EL
    > ipconfig /release *Con*        ... release all matching connections,
                                         eg. "Wired Ethernet Connection 1" or
                                             "Wired Ethernet Connection 2"
    > ipconfig /allcompartments      ... Show information about all
                                         compartments
    > ipconfig /allcompartments /all ... Show detailed information about all
                                         compartments

 

Via:

–jeroen

Posted in DNS, Internet, Power User, Windows, Windows 10 | Leave a Comment »

Setting up a GitLab project so it is served over https as a gitlab.io and a custom subdomain

Posted by jpluimers on 2022/05/05

Last week, I posted about Setting up a GitHub project so it is served over https as a custom github.io subdomain.

Today it’s the equivalent, but on GitLab.

Why GitLab? Two major reasons: unlike GitHub:

  1. it’s open source
  2. provides way more granular control over permissions
  3. allows a hierarchy of repositories on which you can specify that permission control

Already 2. and 3. combined are a huge advantage, though we will see that 3. also makes some of the subcases (hosting as user.gitlab.io from account gitlab.com/user where user is your username) is harder than the similar user.github.io, github.com/user combo.

So here we go, starting with a similar set of links:

The goal is to have

  1. page projects as or under wiert.gitlab.io (like wiert.gitlab.io/wiert)
  2. a gitlabstatus.wiert.me plain html (or maybe markdown) page project that eventually will show some status information (kind of like status.gitlab.com, but for different things).

The beauty of GitLab is that it supports hierarchies of repositories through groups and subgroups, so I already had these subgroups hoping they would cover both the first and second kind of page projects:

Steps I did

Since there are quite a few links above, here are the steps I took from my gitlab.com/wiert account and gitlab.com/wiert.me group.

Steps for wiert.gitlab.io/wiert

  1. For wiert.gitlab.io/wiert, try A (failed in part, and therefore interesting to understand why):
    1. Under leaf group gitlab.com/wiert.me/public/web/sites/gitlab.io, created a new GitLab repository
    2. Chose “Create from template”
    3. Chose the template “Pages/Plain HTML”
    4. Named the project “wiert” (with slug “wiert“) so it would appear at gitlab.com/wiert.me/public/web/sites/gitlab.io/wiert
    5. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    6. Now I got in a confusing situation as the page indicated “There are currently no pipelines.”, but an enabled blue “Run pipeline” button:
      By default there is no CI/CD pipeline, but there is an enabled blue "Run pipeline" button: confusing.

      By default there is no CI/CD pipeline, but there is an enabled blue “Run pipeline” button: confusing.

    7. Clicked the “Run pipeline” button nonetheless, and that created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    8. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job and deployed the page.
    9. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wiert.me.gitlab.io/public/web/sites/gitlab.io/wiert/ and https://wiert.me.gitlab.io/public/web/sites/gitlab.io/wiert/
      Warning: When using Pages under the general domain of a GitLab instance (gitlab.io), you cannot use HTTPS with sub-subdomains.

      Warning: When using Pages under the general domain of a GitLab instance (gitlab.io), you cannot use HTTPS with sub-subdomains.

      The sites do work (see the [Archive.is http version] and [Archive.is https version]), but the HTTPS fails because wiert.me.gitlab.io does not match the SANs (Subject Alternative Names) in the certificate: *.gitlab.io, gitlab.io

  2. For wiert.gitlab.io/wiert, try B (failed, and therefore interesting to understand why):
    1. In my my groups gitlab.com/dashboard/groups, added a new group wiert
    2. Added subgroups until the leaf gitlab.com/wiert/public/web/sites/gitlab.io which as URL is gitlab.com/wier1/public/web/sites/gitlab.io because user account wiert already occupies gitlab.com/wiert.
    3. Under leaf group gitlab.com/wier1/public/web/sites/gitlab.io, created a new GitLab repository
    4. Chose “Create from template”
    5. Chose the template “Pages/Plain HTML”
    6. Named the project “wiert” (with slug “wiert“) so it would appear at gitlab.com/wiert.me/public/web/sites/gitlab.io/wiert
    7. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    8. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    9. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    10. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    11. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wier1.gitlab.io/public/web/sites/gitlab.io/wiert and https://wier1.gitlab.io/public/web/sites/gitlab.io/wiert.
      Bummer: again not the wiert.gitlab.io/wiert I hoped for
      The sites do work (see the [Archive.is http version] and [Archive.is https version]). The HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

    12. If a user wiert exists and occupies gitlab.com/wiert, then a group named wiert cannot occupy gitlab.com/wiert, and therefore a project named wiert within that group won’t be deployed to wiert.gitlab.io/wiert.
      Maybe this can be shortened like “if there is a user wiert, then no group named wiert cannot be used to contain a project named wiert to host as wiert.gitlab.io/wiert“.
      Let’s find out!
  3. For  wiert.gitlab.io/wiert, try C (success, steps 1, 3, 4, 7 and 8 were the key ones):
    1. In my user gitlab.com/wiert, created a new GitLab repository
    2. Chose “Create from template”
    3. Chose the template “Pages/Plain HTML”
    4. Named the project “wiert” (with slug “wiert“) so it would appear at gitlab.com/wiert
    5. The odd but cool thing is that the actual project now ended up at gitlab.com/wiert/wiert:
    6. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    7. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    8. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    9. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    10. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wiert.gitlab.io/wiert/ and https://wiert.gitlab.io/wiert/.
      Success: finally the wiert.gitlab.io/wiert I hoped for:

      Success: published at https://wiert.gitlab.io/wiert/

      Success: published at https://wiert.gitlab.io/wiert/

      The sites do work fine (see the [Archive.is http version] and [Archive.is https version]). The HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

Steps for wiert.gitlab.io

  1. For wiert.gitlab.io, try A (failed, and therefore interesting to understand why):
    1. Under leaf group gitlab.com/wiert.me/public/web/sites/gitlab.io, created a new GitLab repository
    2. Chose “Create from template”
    3. Chose the template “Pages/Plain HTML”
    4. Named the project “wiert.gitlab.io” (with slug “wiert.gitlab.io“) so it would appear at gitlab.com/wiert.me/public/web/sites/gitlab.io/wiert.gitlab.io
    5. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    6. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    7. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    8. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    9. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wiert.me.gitlab.io/public/web/sites/gitlab.io/wiert.gitlab.io and https://wiert.me.gitlab.io/public/web/sites/gitlab.io/wiert.gitlab.io.
      Failure: not the wiert.gitlab.io I hoped for.

      The sites do work (see the [Archive.is http version] and [Archive.is https version]), but the HTTPS fails because wiert.me.gitlab.io does not match the SANs (Subject Alternative Names) in the certificate: *.gitlab.io, gitlab.io. The HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

  2. For wiert.gitlab.io, try B (failed, and therefore interesting to understand why):
    1. Under leaf group gitlab.com/wier1/public/web/sites/gitlab.io, created a new GitLab repository
    2. Chose “Create from template”
    3. Chose the template “Pages/Plain HTML”
    4. Named the project “wiert.gitlab.io” (with slug “wiert.gitlab.io“) so it would appear at gitlab.com/wier1/public/web/sites/gitlab.io/wiert.gitlab.io
    5. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    6. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    7. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    8. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    9. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wier1.gitlab.io/public/web/sites/gitlab.io/wiert.gitlab.io and https://wier1.gitlab.io/public/web/sites/gitlab.io/wiert.
      Bummer: again not the wiert.gitlab.io I hoped for
      The sites do work (see the [Archive.is http version] and [Archive.is https version]). The HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

    10. Try A and B were almost identical to wiert.gitlab.io/wiert try A and B, so let’s see if the solution C for that also works for us:
  3. For wiert.gitlab.io, try C (success, steps 1, 3, 4, 7 and 9 were the key ones)
    1. In my user gitlab.com/wiert, created a new GitLab repository
    2. Chose “Create from template”
    3. Chose the template “Pages/Plain HTML”
    4. Named the project “wiert.gitlab.io” (with slug “wiert.gitlab.io“) so it would appear at gitlab.com/wiert/wiert.gitlab.io.
    5. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    6. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    7. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    8. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    9. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wiert.gitlab.io/ and https://wiert.gitlab.io/.
      Success: finally the wiert.gitlab.io I hoped for with working sites (see the [Archive.is http version] and [Archive.is https version]).
    10. Note the HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

Steps for gitlabstatus.wiert.me

Having learned from the GitHub githubstatus.wiert.me procedure (where I had to wait a long time for the default *.wiert.me domain mapping timeout and the githubstatus.wiert.me DNS CNAME record to become effective), I started on the DNS CNAME record side which is documented at [Wayback] Custom domains and SSL/TLS certificates: Section 3. Set up DNS records for Pages: For subdomains | GitLab:

Subdomains (subdomain.example.com) require:

  • A DNS CNAME record pointing your subdomain to the Pages server.
  • A DNS TXT record to verify your domain’s ownership.
From DNS Record To
subdomain.example.com CNAME namespace.gitlab.io
_gitlab-pages-verification-code.subdomain.example.com TXT gitlab-pages-verification-code=00112233445566778899aabbccddeeff

Note that, whether it’s a user or a project website, the CNAME should point to your Pages domain (namespace.gitlab.io), without any /project-name.

DNS CNAME record pointing to GitLab.com project

The value for the TXT record is only known after you created the pages project, but the value for the CNAME record is known beforehand:

From DNS Record To
gitlabstatus.wiert.me CNAME namespace.gitlab.io

So let’s see if I can do this in one try, with these steps:

  1. For gitlabstatus.wiert.me, try A (success, steps 1, 3, 4, 7 and 9 were the key ones)
    1. In my DNS settings of the wiert.me domain, created a CNAME record from gitlabstatus.wiert.me to namespace.gitlab.io:

      gitlabstatus.wiert.me CNAME record pointing to namespace.gitlab.io

      gitlabstatus.wiert.me CNAME record pointing to namespace.gitlab.io

    2. Under leaf group gitlab.com/wiert.me/public/web/sites/wiert.me, created a new GitLab repository
    3. Chose “Create from template”
    4. Chose the template “Pages/Plain HTML”
    5. Named the project “gitlabstatus.wiert.me” (with slug “gitlabstatus.wiert.me“) so it would appear at gitlab.com/wiert.me/public/web/sites/wiert.me/gitlabstatus.wiert.me
    6. From the left sidebar, navigated to your project’s “CI/CD”, then “Pipelines”
    7. Again there was “There are currently no pipelines.”, but an enabled blue “Run pipeline” button, which I clicked
    8. That created [Wayback/Archive.is] a pipeline asking for parameters (that already had correct default values) and revealed a new blue “Run pipeline” button.
    9. Clicked that new “Run pipeline button” which created [Wayback/Archive.is] a job deployed the page.
    10. From the left sidebar, navigated to “Settings”, then “Pages” to get the links to the pages site: http://wiert.me.gitlab.io/public/web/sites/wiert.me/gitlabstatus.wiert.me and https://wiert.me.gitlab.io/public/web/sites/wiert.me/gitlabstatus.wiert.me.
      Intermediate success: working sites (see the [Archive.is http version] and [Archive.is https version]).
    11. Now it is time to get the DNS CNAME record from gitlabstatus.wiert.me to namespace.gitlab.io into operation by clicking the “New Domain” button:
      "New Domain" button in the "Pages" settings.

      “New Domain” button in the “Pages” settings.

    12. There I filled in the correct gitlabstatus.wiert.me domain name, then pressed the “Create New Domain” button:

      New domain becomes gitlabstatus.wiert.me

      New domain becomes gitlabstatus.wiert.me

    13. Then a page appeared voiding the DNS CNAME work I already did: the documentation is clearly wrong as these are the two DNS record entries to be made as shown by gitlab.com/wiert.me/public/web/sites/wiert.me/gitlabstatus.wiert.me/pages/domains/gitlabstatus.wiert.me:
      Correct instructions for the DNS records to get gitlabstatus.wiert.me working

      Correct instructions for the DNS records to get gitlabstatus.wiert.me working

      Subdomains (gitlabstatus.wiert.me) require:

      • A DNS CNAME record pointing your subdomain to the Pages server.
      • A DNS TXT record to verify your domain’s ownership.
      From DNS Record To
      gitlabstatus.wiert.me CNAME wiert.me.gitlab.io.
      _gitlab-pages-verification-code.gitlabstatus.wiert.me TXT gitlab-pages-verification-code=c5619988d386b1a36c253ce05db55dbb

      Basically the whole namespace.gitlab.io part of the documentation is a placeholder for the actual namespace that belongs to the leaf group the pages project is in (in my case wiert.me).

      So this is the new DNS entry, for which I had to wait until the DNS TTL to time out and effectuate:
      New DNS gitlabstatus.wiert.me CNAME record pointing to wiert.me.gitlab.io

      New DNS gitlabstatus.wiert.me CNAME record pointing to wiert.me.gitlab.io

      Note that this DNS administrative interface from WordPress.com does omit the final period of the CNAME destination (officially this would be wiert.me.gitlab.io.)

    14. After the CNAME DNS record, I also made the TXT DNS record:
      New DNS TXT record for verification of gitlabstatus.wiert.me

      New DNS TXT record for verification of gitlabstatus.wiert.me

      Then I waited a little for the DNS TXT record to be saved and try the verification of the TXT record.

    15. Even then, verification took some time. I had to click the refresh button a few times before verification succeeded:
      The DNS TXT record for gitlabstatus.wiert.me finally got verified

      The DNS TXT record for gitlabstatus.wiert.me finally got verified

    16. Now I could press blue “Save Changes” button below and waited for the CNAME record DNS TTL to expire so I could check the domain and – hopefully – the TLS certificate to be requested by Let’s Encrypt:
      After the gitlabstatus.wiert DNS TXT record got verified, I could save the domain information

      After the gitlabstatus.wiert DNS TXT record got verified, I could save the domain information

    17. After the old CNAME record DNS TTL expired and the new CNAME record came into effect, the domain became available as http://gitlabstatus.wiert.me/:
      Waiting for gitlabstatus.wiert.me to become active

      Waiting for gitlabstatus.wiert.me to become active

    18. After verification, the “Domains (1)” bit changed from this:
      Domain gitlabstatus.wiert.me information before verification

      Domain gitlabstatus.wiert.me information before verification

      to this:

      Domain gitlabstatus.wiert.me information after verification

      Domain gitlabstatus.wiert.me information after verification

    19. In the mean time, also the TLS certificate got issued by Let’s Encrypt, so the final sites now both worked: http://gitlabstatus.wiert.me/ and https://gitlabstatus.wiert.me/.
    20. Success: finally the gitlabstatus.wiert.me I hoped for with working sites (see the [Archive.is http version] and [Archive.is https version] for the wiert.me domain, and [Archive.is http version] and [Archive.is https version] for the wiert.me domain).
    21. Note the HTTP does not redirect to the HTTP version, as I did not tick the

      ☐ Force HTTPS (requires valid certificates)

In retrospect, this could have been shorter when I had done the DNS part later, which is contrary to how to do this with GitHub.

Conclusion

The conclusion seems this:

Gitlab Page repositories to be published as or under wiert.gitlab.io need to reside directly under user wiert. Having them reside under a different group like wiert or wiert.me won’t work.

Or in more generic terms:

When creating pages as user.gitlab.io you have to put your pages projects directly under your user account gitlab.com/user.

Putting them under groups or leaf groups fails, no matter if the (leaf) group is named user or otherwise.

In addition, you can add custom domains to any Gitlab repository (even one that never stated out as a GitLab Pages repository). It will work as soon as the domain DNS mapping is setup through both a CNAME mapping record and TXT verification record.

The steps for this in your GitLab repository are:

  1. Ensure you have a valid .gitlab-ci.yml file at the root of your repository; I used the [Wayback/Archive.is] one from [Wayback/Archive] GitLab Pages examples / plain-html · GitLab as my site is purely static
  2. Ensure you have a valid index.html file in the public directory of your repository, similar to [Wayback/Archive] GitLab Pages examples / plain-html · GitLab
  3. When both 1. and 2. are committed in your repository at GitLab, then it will automatically be deployed to a docker container on gitlab.io, which allows the outside world to visit your GitHub Pages sie, and the Let’s Encrypt Certificate to be generated (and prevents this error: [Wayback/Archive] GitLab Pages integration with Let’s Encrypt | GitLab: “Something went wrong while obtaining the Let’s Encrypt certificate”).
  4. Under “Settings” -> “Pages”, add a new domain name to the repository: now it automatically becomes a GitLab Pages repository.
  5. When adding the domain, the settings page will show both a DNS CNAME record and DNS TXT record; ensure both are applied on your primary DNS name server and replicated to all authoritative DNS name servers.
  6. Save the new page.
  7. Check if the page is available on the new domain you added.
  8. Optionally under “Settings” -> “Pages” enable the “Force HTTPS (requires valid certificates)” option and save.

TLS information

Note: I saved the TLS information – including certificates here:

More about the Let’s Encrypt certificates at [Wayback] Chain of Trust – Let’s Encrypt:

–jeroen

Read the rest of this entry »

Posted in Cloud, Communications Development, Development, DNS, Encryption, GitLab, Hosting, HTML, HTTPS/TLS security, Infrastructure, Internet, Internet protocol suite, Let's Encrypt (letsencrypt/certbot), Power User, Software Development, Source Code Management, TCP, TLS, Web Development | Leave a Comment »

Cryptosense Discovery

Posted by jpluimers on 2022/03/15

This is cool: [Wayback] Cryptosense Discovery:

Free tool that discovers security configuration errors in SSH and TLS servers and explains how to fix them. Supports STARTTLS and can also scan HTTPS, POP3, IMAP and SMTP servers.

It gives you a list of servers a target domain uses (for purposes like web, email, etc) that can have external encryption enabled, then allows you to test these.

The list by default has only servers within that target domain enabled, but you can optionally include other servers (for instance if a domain uses a third party for their SMTP handling).

Basically it is the web-counterpart of a tool like testssl.sh (which I have written about before).

Found while checking out how to test the MX security of a domain using [Wayback] testssl.sh as I forgot the syntax, which in retrospect is dead easy as per [Wayback] tls – How to use testssl.sh on an SMTP server? – Information Security Stack Exchange (thanks [Wayback] Z.T.!):

…

testssl.sh --mx <domain name>

works fine.

testssl.sh -t smtp <ip>:25

and

testssl.sh -t smtp <ip>:587

also work fine.

Note that not specifying the port assumes port 443, despite specifying protocol smtp. That doesn’t work.

…

Also, you might try discovery.cryptosense.com which does the same thing only better

That website is made by the cool people at [Wayback] Cryptosense.

Both are a lot easier than the alternatives described in [Wayback] Blog · How to test SMTP servers using the command-line · Halon MTA: using nslookup and dig for determining the affected hosts, using nc or telnet for testing basic connectivity, using [Wayback/Archive.is] openssl s_client to test TLS, and [Wayback/Archive.is] smtpping for measuring throughput.

In addition to the above tools mentioned in the blog, I’ve also used sendEmail (note case sensitivity), ehlo-size, and swaks.

This is what I tested:

–jeroen

Posted in *nix, *nix-tools, Awk, bash, bash, Communications Development, Development, DNS, Encryption, grep, HTTPS/TLS security, Internet, Internet protocol suite, Power User, Scripting, Security, SMTP, Software Development, SSH, ssh/sshd, TCP, testssl.sh, TLS | Leave a Comment »

« Previous Entries
Next Entries »