The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘cURL’ Category

« Previous Entries

Some links on non-official Ring API libraries and tools

Posted by jpluimers on 2026/03/26

There is no official Ring API. But there are libraries and tools around that can talk to a Ring ecosystem, mostly written in JavaScript or Python.

Some links I found:

Read the rest of this entry »

Posted in *nix, *nix-tools, cURL, Development, Hardware, IoT Internet of Things, JavaScript/ECMAScript, Network-and-equipment, Power User, Python, Ring Doorbell/Chime (Amazon), Scripting, Software Development | Leave a Comment »

Is this the ultimate XKCD “Dependency” derivative?

Posted by jpluimers on 2025/11/29

xkcd modern digital infrastructure torn down by a satisfied looking cat

Besides the August 2025 XKCD infrastructure dependency inspired cartoon on the right, the more recent and great [Wayback/Archive] XCKD: Dependency derivative below is a monumental piece as it combines the recent:

  • fiber outage of the Internet Archive
  • DoS of Cloudflare by itself
  • AWS us-east-1 dependencies outage
  • Crowdstrike DoS of Windows machines
  • framework-du-jour mentality in the JavaScript world
  • many more¹

Image [Wayback/Archive] 36247840bf294a9d.png (1080×1389) from [Wayback/Archive] xyla 🐀🪇: “someone pls alt text this shit…” – buy shitpost cheap:

Read the rest of this entry »

Posted in *nix, Amazon.com/.de/.fr/.uk/..., AWS Amazon Web Services, C, CDN (Content Delivery Network), Cloud, Cloudflare, cURL, Development, Fun, Hardware, Infrastructure, ISP, JavaScript/ECMAScript, Network-and-equipment, Node.js, npm, Power User, Rust, Scripting, Software Development, Web Development | Tagged: , | Leave a Comment »

linux – Get final URL after curl is redirected – Stack Overflow (plus some Twitter scraping tricks)

Posted by jpluimers on 2025/08/06

Sometimes I need [Wayback/Archive] Redirect Checker | Check your Statuscode 301 vs 302 on the command-line, so cURL to the rescue: [Wayback/Archive] linux – Get final URL after curl is redirected – Stack Overflow. The relevant portions of answers and comments further below.

TL;DR:

Since I prefer verbose command-line arguments (you can find them at the [Wayback/Archive] curl – How To Use on-line man page) especially in scripts this HTTP GET request is what works with Twitter:

% curl --location --silent --output /dev/null --write-out "%{url_effective}\n" https://twitter.com/anyuser/status/20
https://x.com/anyuser/status/20

This failed (twitter dislikes HTTP HEAD requests):

% curl --head --location --silent --output /dev/null --write-out "%{url_effective}\n" https://twitter.com/anyuser/status/20
https://twitter.com/anyuser/status/20

Notes

Given so many of my scripts now run on zsh, I added the new-line because of command line – Why does a cURL request return a percent sign (%) with every request in ZSH? – Stack Overflow. You can strip that bit.

Note that these do not perform client side redirects, so they do not return the ultimate originating URL https://x.com/jack/status/20 (which was the first ever Tweet on what was back then called twttr) as Twitter on the client-side overwrites window.location.href with the final URL. Similar behaviour for getting the Twitter user handle of a Twitter user ID, more on Twitter tricks below.

Tweet by TweetID trick via [Wayback/Archive] Accessing a tweet using only its ID (and without the Twitter API) – Bram.us.

Further reading (thanks [Wayback/Archive] vise, [Wayback/Archive] Daniel Stenberg, [Wayback/Archive] Ivan, [Wayback/Archive] AndrewF, [Wayback/Archive] Roger Campanera, and [Wayback/Archive] Dave Baird):

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, Batch-Files, Bookmarklet, Communications Development, Conference Topics, Conferences, CSS, cURL, Development, Event, HTTP, Internet protocol suite, JavaScript/ECMAScript, Power User, Scripting, SocialMedia, Software Development, TCP, Twitter, Web Browsers, Web Development | Tagged: | Leave a Comment »

Installing Chocolatey on Windows 10 and up

Posted by jpluimers on 2025/04/02

Steps for installing Chocolatey on Windows 11 and up or 10 version 1803 and up.

Since I often install Windows on machines where it is not easy to copy/paste longer install commands my steps are slightly different than the ones on [Wayback/Archive] Chocolatey Software | Installing Chocolatey:

  1. Start a regular command prompt
  2. Either these two (the options are equivalent, see [Wayback/Archive] curl: transfer a URL | curl Commands | Man Pages | ManKier for --remote-name and -O):
    • curl --remote-name https://community.chocolatey.org/install.ps1
    • curl -O https://community.chocolatey.org/install.ps1

    Note the cURL pre-installed on Windows 10 since at least 6 years*: release 1803 or insider build 17063 is good enough to download the Chocolatey install script

  3. Inspect the downloaded install.ps1 to check if you spot anything you dislike
  4. Start an elevated (administrator) command prompt
  5. Start PowerShell
  6. Execute this command
    • Set-ExecutionPolicy Bypass -Scope Process -Force
  7. Execute this command in the folder where you downloaded install.ps1
    • .\install.ps1
  • Yup, a custom build of cURL has been pre-installed on Windows 10 and up since more than 6 years:

Read the rest of this entry »

Posted in *nix, *nix-tools, Chocolatey, cURL, Development, Power User, Software Development, Windows, Windows 10, Windows 11, Windows Development | Leave a Comment »

Downloading a file from the Windows console without first installing a command-line tool

Posted by jpluimers on 2024/10/09

Note that the below methods likely will cause security warnings if a Windows machine has been properly configured, but in most cases at least one of them works.

  1. using cURL (Widows 10 and up)
    curl --url https://speed.hetzner.de/100MB.bin --output %TEMP%\100MB.bin
  2. using [Wayback/Archive] certutil | Microsoft Docs (at least Windows 7 and up; needs UAC elevation)
    certutil.exe -urlcache -split -f https://speed.hetzner.de/100MB.bin %TEMP%\100MB.bin
  3. using PowerShell (at least Windows Vista and up)
    powershell.exe -Command (New-Object System.Net.WebClient).DownloadFile('https://speed.hetzner.de/100MB.bin','%TEMP%\100MB.bin')

I think it works for all versions of curl, certutil, and PowerShell though I did not have anything older than up-to-date Windows 7 (having PowerShell version 3) and recent to test on.

Read the rest of this entry »

Posted in *nix, *nix-tools, .NET, Batch-Files, CommandLine, cURL, Development, Power User, PowerShell, PowerShell, Scripting, Software Development, Windows, Windows 10, Windows 11, Windows 7, Windows 8, Windows 8.1, Windows Development, Windows Vista | Leave a Comment »

Some threadreaderapp URLs

Posted by jpluimers on 2023/09/14

For my link archive so I can better automate archiving Tweet threads using bookmarklets written in JavaScript:

The base will likely be this:

javascript:void(open(`https://archive.is/?run=1&url=${encodeURIComponent(document.location)}`))

which for now I have modified into this:

javascript:void(open(`https://threadreaderapp.com/search?q=${document.location}`))

It works perfectly fine without URL encoding and demonstrates the JavaScript backtick feature for template literals for which you can find documentation at [WayBack/Archive] Template literals – JavaScript | MDN.

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Bookmarklet, Communications Development, cURL, Development, HTTP, https, Internet protocol suite, Power User, Scripting, Security, Software Development, TCP, Web Browsers | Leave a Comment »

When high SEO ranking fails to give you a reliable result: IsItDownRightNow.com failed to detect the WayBack Machine outage

Posted by jpluimers on 2022/02/11

A high SEO ranking does not automatically indicate a reliable result.

When the WayBack Machine was down a while ago (it responded to traceroute UDP requests, but would not establish TCP connections on ports 80 and 443), the first Google hit for detecting down status (searching for [Archive.is] waybackmachine down – Google Search) failed miserably because it redirected web.archive.org (which fails) to http://www.archive.org (which succeeds):

IsIdDownRightNow failing to detect web.archive.org downtime

IsIdDownRightNow failing to detect web.archive.org downtime

Luckily when asking around on Twitter:

  • others were experiencing the same problem, not just in The Netherlands, but also in other countries
  • after trying a few things, the WayBack machine got backup [Archive.is] before I could try cURL.
  • I got pointed at www.uptrends.com/tools/uptime which correctly does check the right subdomain and shows it is down from many locations:

Read the rest of this entry »

Posted in *nix, cURL, Infrastructure, Internet, InternetArchive, LifeHacker, Power User, WayBack machine | Leave a Comment »

Downloading files with wget on ESXi · random writes

Posted by jpluimers on 2021/05/31

Reminder to self to check if wget on ESXi now finally supports https downloading: [WayBack] Downloading files with wget on ESXi · random writes.

In the mean time, ESXi 6.7 Update 2 and up seems to support this; so the below workaround might only be needed for ESXi 6.7 update 1 and below.

[WayBack] VMware ESXi: help downloading large ISO – Server Fault

I will likely not do this, as by now all my ESXi boxes should have been recent enough.

I will keep the article because of the BusyBox commands section below.

If so, I might finally try and write a Python wrapper for this, as I know that Python 3 on ESXi supports https, but the ESXi BusyBox does not have a built-in cURL.

Some links and notes I might need by then:

BusyBox commands

Another cool thing in the above blog post is that it shows how to dump the BusyBox built in commands.

I ran it for ESXi 6.7 with a slight trick to get the full path (using back-ticks and escaped back-ticks) and content.

Since ESXi is BusyBox based, the commands that are in /bin are not actually binaries, but each command is a symlink to the BusyBox binary. BusyBox then knows the original name of the command, so it can deduct what part to execute. This makes for a very space efficient storage scheme.

The various bits of the tricks to get the location of the BusyBox binary, so the --list parameter can be passed to it:

  • The which wget gives the full path of wget.
  • The ls -l `which wget` shows the full path of wget and the symlink target (but there is no way for ls to only show the symlink target).
  • The readlink -f `which wget` shows the full path of where /bin/wget points to: the BusyBox binary.

The main trick consists of backtick evalution, and knowing that ls cannot get you just the symlink target, but readlink can:

Now the back-tick escapes, because you cannot nest back-ticks:

  • The `readlink -f \`which wget\`` executes the BusyBox binary without arguments.
  • The `readlink -f \`which wget\`` --list executes the BusyBox binary with the --list parameter.

Note I do not like the cat --help (see [WayBack] How do I check busybox version (from busybox)? – Unix & Linux Stack Exchange) way of getting the BusyBox version, as that gets echoed to stderr.

This is the output:

Read the rest of this entry »

Posted in *nix, *nix-tools, cURL, ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware ESXi, wget | Leave a Comment »

56 Linux Networking commands and scripts

Posted by jpluimers on 2021/01/25

Back in 2019, there were 56 commands and scripts covered. I wonder how many there are now.

An ongoing list of Linux Networking Commands and Scripts. These commands and scripts can be used to configure or troubleshoot your Linux network.

Source: [WayBack] 55 Linux Networking commands and scripts

List back then (which goes beyond just built-in commands: many commands from optional packages are here as well):

  1. arpwatch – Ethernet Activity Monitor.
  2. bmon – bandwidth monitor and rate estimator.
  3. bwm-ng – live network bandwidth monitor.
  4. curl – transferring data with URLs. (or try httpie)
  5. darkstat – captures network traffic, usage statistics.
  6. dhclient – Dynamic Host Configuration Protocol Client
  7. dig – query DNS servers for information.
  8. dstat – replacement for vmstat, iostat, mpstat, netstat and ifstat.
  9. ethtool – utility for controlling network drivers and hardware.
  10. gated – gateway routing daemon.
  11. host – DNS lookup utility.
  12. hping – TCP/IP packet assembler/analyzer.
  13. ibmonitor – shows bandwidth and total data transferred.
  14. ifstat –  report network interfaces bandwidth.
  15. iftop – display bandwidth usage.
  16. ip (PDF file) – a command with more features that ifconfig (net-tools).
  17. iperf3 – network bandwidth measurement tool. (above screenshot Stacklinux VPS)
  18. iproute2 – collection of utilities for controlling TCP/IP.
  19. iptables – take control of network traffic.
  20. IPTraf – An IP Network Monitor.
  21. iputils – set of small useful utilities for Linux networking.
  22. jwhois (whois) – client for the whois service.
  23. “lsof -i” – reveal information about your network sockets.
  24. mtr – network diagnostic tool.
  25. net-tools – utilities include: arp, hostname, ifconfig, netstat, rarp, route, plipconfig, slattach, mii-tool, iptunnel and ipmaddr.
  26. ncat – improved re-implementation of the venerable netcat.
  27. netcat – networking utility for reading/writing network connections.
  28. nethogs – a small ‘net top’ tool.
  29. Netperf – Network bandwidth Testing.
  30. netsniff-ng – Swiss army knife for daily Linux network plumbing.
  31. netstat – Print network connections, routing tables, statistics, etc.
  32. netwatch – monitoring Network Connections.
  33. ngrep – grep applied to the network layer.
  34. nload – display network usage.
  35. nmap – network discovery and security auditing.
  36. nslookup – query Internet name servers interactively.
  37. ping – send icmp echo_request to network hosts.
  38. route – show / manipulate the IP routing table.
  39. slurm – network load monitor.
  40. snort – Network Intrusion Detection and Prevention System.
  41. smokeping –  keeps track of your network latency.
  42. socat – establishes two bidirectional byte streams and transfers data between them.
  43. speedometer – Measure and display the rate of data across a network.
  44. speedtest-cli – test internet bandwidth using speedtest.net
  45. ss – utility to investigate sockets.
  46. ssh –  secure system administration and file transfers over insecure networks.
  47. tcpdump – command-line packet analyzer.
  48. tcptrack – Displays information about tcp connections on a network interface.
  49. telnet – user interface to the TELNET protocol.
  50. tracepath – very similar function to traceroute.
  51. traceroute – print the route packets trace to network host.
  52. vnStat – network traffic monitor.
  53. wget –  retrieving files using HTTP, HTTPS, FTP and FTPS.
  54. Wireless Tools for Linux – includes iwconfig, iwlist, iwspy, iwpriv and ifrename.
  55. Wireshark – network protocol analyzer.

Via:

–jeroen

Posted in *nix, *nix-tools, cURL, dig, Internet, nmap, Power User, SpeedTest, ssh/sshd, tcpdump, Wireshark | Leave a Comment »

curl/wget: use content-disposition for the file names

Posted by jpluimers on 2020/04/27

For me, on Windows, curl works better than wget, but on Linux/Mac OS X, curl tends to work better. Some people find wget easier for downloading multiple URLs at the same time. So here the parameter switches for both so they download to the file specified by the Content-Disposition http header:

  • curl --remote-name --remote-header-name
  • wget --content-disposition

My experience is that wget is better at this, especially when redirects are involved (by adding a [WayBack] --location parameter to  thecurl command line).

So for instance the first fails, but the second succeeds determining the download to be VSCodeUserSetup-x64-1.27.2.exe (so curl keeps the name stable):

curl.exe --location --remote-name --remote-time --remote-header-name https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable

wget.exe --content-disposition https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable

This takes into account the name after all followed redirects.

Via:

–jeroen

Posted in *nix, cURL, Power User, wget | Leave a Comment »

« Previous Entries