The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,909 other followers

In some countries @archiveis redirect their domains using http-302 which can have trouble with multi-WAN

Posted by jpluimers on 2022/08/12

From [Wayback Archive.is blog — Why has the URL “archive-li” changed to…:

Why has the URL “archive-li” changed to “archive-ph”, and will this affect saved bookmarks at any time in the future?

Anonymous

This is temporary and only for some countries. All 7 domains work, so you do not need to change the bookmarks.

In The Netherlands all Archive Today domains redirect to archive.ph using a HTTP 302 redirect.

This caused trouble at my home location, but not at my brother, so I searched for local issues.

In the end, it was because I have dual WAN as network load balancing at home.

TL;DR

Modifying the routing table so traffic for 54.37.18.234 goes to WAN1 was my solution.

Finding the destination address

So I looked up the IP address for the seven domains (which I got from [Wayback/Archive] archive.today – Wikipedia) using this PowerShell script:

$domainNamesString = 'today,is,ph,vn,fo,li,md'
$domainNamesList = $domainNamesString.Split(',')
foreach ($domainName in $domainNamesList) {
  $hostName = 'archive.' + $domainName
  $addresses = [System.Net.Dns]::GetHostAddresses($hostName)
  foreach($address in $addresses) {
    '{0} <- {1}' -f $address.IPAddressToString, $hostName
  }
}

I first tried a batch file using nslookup, but on Windows filtering and getting the right information out is a mess (in part because some output gets send to stderr instead of stdout) and tough to make multilingual (nslookup is localised for many locales).

Like batch files, PowerShell itself has no built-in DNS functionality, but can directly access the .NET core API [Wayback/Archive] Dns.GetHostAddresses Method (System.Net) | Microsoft Docs which can return more than one address per hostname.

You can also run this from a Batch file as one-liner:

@ 2>null PowerShell -Command "foreach ($d in 'today,is,ph,vn,fo,li,md'.Split(',')) { $h = 'archive.' + $d; foreach($a in [System.Net.Dns]::GetHostAddresses($h)) { '{0} <- {1}' -f $a.IPAddressToString, $h } }"

The output was far shorter than I expected, just a single IPv4 is used:

54.37.18.234 <- archive.today
54.37.18.234 <- archive.is
54.37.18.234 <- archive.ph
54.37.18.234 <- archive.vn
54.37.18.234 <- archive.fo
54.37.18.234 <- archive.li
54.37.18.234 <- archive.md

That led to the solution: directing all traffic for 54.37.18.234 goes to WAN1 in the routing table of the router.

Related

My Twitter thread here is related:

Knowing many people want to archive social media, especially with the Russian invasion in Ukraine, know that archival sites like [Wayback/Archive] @archiveis and [Wayback/Archive] @waybackmachine are extremely busy.

So be selective about what you archive.

Is it really needed now?
Is it already archived elsewhere?

[Wayback/Archive] Jeroen Wiert Pluimers on Twitter: “Archive.is blog: Twitter archival is slow, so limit the number of tweets you save in it ” / Twitter

Tweets saved at [Wayback/Archive] @waybackmachine now work better than the last couple of years (likely [Wayback/Archive] @Twitter understands archived document.location better), so can now be used next to [Wayback/Archive] @archiveis.

[Wayback/Archive] #Twitter however has strong rate limits and there are just limited archival nodes, so it is slow.

Do not even try to archive [Wayback/Archive] @Facebook or [Wayback/Archive] @LinkedIn: they want you to live in their ecosystem and the logon-restrictions make it impossible to archive.

Trying to archive in [Wayback/Archive] @archiveis or [Wayback/Archive] @waybackmachine just wastes archival node resources, so please don’t.

–jeroen


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
%d bloggers like this: