Archive for the ‘cURL’ Category
Posted by jpluimers on 2025/11/29
Besides the August 2025 XKCD infrastructure dependency inspired cartoon on the right, the more recent and great [Wayback/Archive] XCKD: Dependency derivative below is a monumental piece as it combines the recent:
- fiber outage of the Internet Archive
- DoS of Cloudflare by itself
- AWS us-east-1 dependencies outage
- Crowdstrike DoS of Windows machines
- framework-du-jour mentality in the JavaScript world
- many more¹
Image [Wayback/Archive] 36247840bf294a9d.png (1080×1389) from [Wayback/Archive] xyla 🐀🪇: “someone pls alt text this shit…” – buy shitpost cheap:
Read the rest of this entry »
Posted in *nix, Amazon.com/.de/.fr/.uk/..., AWS Amazon Web Services, C, CDN (Content Delivery Network), Cloud, Cloudflare, cURL, Development, Fun, Hardware, Infrastructure, ISP, JavaScript/ECMAScript, Network-and-equipment, Node.js, npm, Power User, Rust, Scripting, Software Development, Web Development | Tagged: Meme, ProgrammerHumor | Leave a Comment »
Posted by jpluimers on 2025/08/06
Sometimes I need [Wayback/Archive] Redirect Checker | Check your Statuscode 301 vs 302 on the command-line, so cURL to the rescue: [Wayback/Archive] linux – Get final URL after curl is redirected – Stack Overflow. The relevant portions of answers and comments further below.
TL;DR:
Since I prefer verbose command-line arguments (you can find them at the [Wayback/Archive] curl – How To Use on-line man page) especially in scripts this HTTP GET request is what works with Twitter:
% curl --location --silent --output /dev/null --write-out "%{url_effective}\n" https://twitter.com/anyuser/status/20
https://x.com/anyuser/status/20
This failed (twitter dislikes HTTP HEAD requests):
% curl --head --location --silent --output /dev/null --write-out "%{url_effective}\n" https://twitter.com/anyuser/status/20
https://twitter.com/anyuser/status/20
Notes
Given so many of my scripts now run on zsh, I added the new-line because of command line – Why does a cURL request return a percent sign (%) with every request in ZSH? – Stack Overflow. You can strip that bit.
Note that these do not perform client side redirects, so they do not return the ultimate originating URL https://x.com/jack/status/20 (which was the first ever Tweet on what was back then called twttr) as Twitter on the client-side overwrites window.location.href with the final URL. Similar behaviour for getting the Twitter user handle of a Twitter user ID, more on Twitter tricks below.
Tweet by TweetID trick via [Wayback/Archive] Accessing a tweet using only its ID (and without the Twitter API) – Bram.us.
Further reading (thanks [Wayback/Archive] vise, [Wayback/Archive] Daniel Stenberg, [Wayback/Archive] Ivan, [Wayback/Archive] AndrewF, [Wayback/Archive] Roger Campanera, and [Wayback/Archive] Dave Baird):
Read the rest of this entry »
Posted in *nix, *nix-tools, bash, Batch-Files, Bookmarklet, Communications Development, Conference Topics, Conferences, CSS, cURL, Development, Event, HTTP, Internet protocol suite, JavaScript/ECMAScript, Power User, Scripting, SocialMedia, Software Development, TCP, Twitter, Web Browsers, Web Development | Tagged: 76 | Leave a Comment »
Posted by jpluimers on 2025/04/02
Steps for installing Chocolatey on Windows 11 and up or 10 version 1803 and up.
Since I often install Windows on machines where it is not easy to copy/paste longer install commands my steps are slightly different than the ones on [Wayback/Archive] Chocolatey Software | Installing Chocolatey:
- Start a regular command prompt
- Either these two (the options are equivalent, see [Wayback/Archive] curl: transfer a URL | curl Commands | Man Pages | ManKier for
--remote-name and -O):
curl --remote-name https://community.chocolatey.org/install.ps1curl -O https://community.chocolatey.org/install.ps1
Note the cURL pre-installed on Windows 10 since at least 6 years*: release 1803 or insider build 17063 is good enough to download the Chocolatey install script
- Inspect the downloaded
install.ps1 to check if you spot anything you dislike
- Start an elevated (administrator) command prompt
- Start PowerShell
- Execute this command
Set-ExecutionPolicy Bypass -Scope Process -Force
- Execute this command in the folder where you downloaded
install.ps1
- Yup, a custom build of cURL has been pre-installed on Windows 10 and up since more than 6 years:
Read the rest of this entry »
Posted in *nix, *nix-tools, Chocolatey, cURL, Development, Power User, Software Development, Windows, Windows 10, Windows 11, Windows Development | Leave a Comment »
Posted by jpluimers on 2024/10/09
Note that the below methods likely will cause security warnings if a Windows machine has been properly configured, but in most cases at least one of them works.
- using cURL (Widows 10 and up)
curl --url https://speed.hetzner.de/100MB.bin --output %TEMP%\100MB.bin
- using [Wayback/Archive]
certutil | Microsoft Docs (at least Windows 7 and up; needs UAC elevation)
certutil.exe -urlcache -split -f https://speed.hetzner.de/100MB.bin %TEMP%\100MB.bin
- using PowerShell (at least Windows Vista and up)
powershell.exe -Command (New-Object System.Net.WebClient).DownloadFile('https://speed.hetzner.de/100MB.bin','%TEMP%\100MB.bin')
I think it works for all versions of curl, certutil, and PowerShell though I did not have anything older than up-to-date Windows 7 (having PowerShell version 3) and recent to test on.
Read the rest of this entry »
Posted in *nix, *nix-tools, .NET, Batch-Files, CommandLine, cURL, Development, Power User, PowerShell, PowerShell, Scripting, Software Development, Windows, Windows 10, Windows 11, Windows 7, Windows 8, Windows 8.1, Windows Development, Windows Vista | Leave a Comment »
Posted by jpluimers on 2023/09/14
For my link archive so I can better automate archiving Tweet threads using bookmarklets written in JavaScript:
The base will likely be this:
javascript:void(open(`https://archive.is/?run=1&url=${encodeURIComponent(document.location)}`))
which for now I have modified into this:
javascript:void(open(`https://threadreaderapp.com/search?q=${document.location}`))
It works perfectly fine without URL encoding and demonstrates the JavaScript backtick feature for template literals for which you can find documentation at [WayBack/Archive] Template literals – JavaScript | MDN.
Read the rest of this entry »
Posted in *nix, *nix-tools, bash, bash, Bookmarklet, Communications Development, cURL, Development, HTTP, https, Internet protocol suite, Power User, Scripting, Security, Software Development, TCP, Web Browsers | Leave a Comment »
Posted by jpluimers on 2022/02/11
A high SEO ranking does not automatically indicate a reliable result.
When the WayBack Machine was down a while ago (it responded to traceroute UDP requests, but would not establish TCP connections on ports 80 and 443), the first Google hit for detecting down status (searching for [Archive.is] waybackmachine down – Google Search) failed miserably because it redirected web.archive.org (which fails) to http://www.archive.org (which succeeds):
IsIdDownRightNow failing to detect web.archive.org downtime
Luckily when asking around on Twitter:
- others were experiencing the same problem, not just in The Netherlands, but also in other countries
- after trying a few things, the WayBack machine got backup [Archive.is] before I could try cURL.
- I got pointed at www.uptrends.com/tools/uptime which correctly does check the right subdomain and shows it is down from many locations:
Read the rest of this entry »
Posted in *nix, cURL, Infrastructure, Internet, InternetArchive, LifeHacker, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/05/31
Reminder to self to check if wget on ESXi now finally supports https downloading: [WayBack] Downloading files with wget on ESXi · random writes.
In the mean time, ESXi 6.7 Update 2 and up seems to support this; so the below workaround might only be needed for ESXi 6.7 update 1 and below.
[WayBack] VMware ESXi: help downloading large ISO – Server Fault
I will likely not do this, as by now all my ESXi boxes should have been recent enough.
I will keep the article because of the BusyBox commands section below.
If so, I might finally try and write a Python wrapper for this, as I know that Python 3 on ESXi supports https, but the ESXi BusyBox does not have a built-in cURL.
Some links and notes I might need by then:
BusyBox commands
Another cool thing in the above blog post is that it shows how to dump the BusyBox built in commands.
I ran it for ESXi 6.7 with a slight trick to get the full path (using back-ticks and escaped back-ticks) and content.
Since ESXi is BusyBox based, the commands that are in /bin are not actually binaries, but each command is a symlink to the BusyBox binary. BusyBox then knows the original name of the command, so it can deduct what part to execute. This makes for a very space efficient storage scheme.
The various bits of the tricks to get the location of the BusyBox binary, so the --list parameter can be passed to it:
- The
which wget gives the full path of wget.
- The
ls -l `which wget` shows the full path of wget and the symlink target (but there is no way for ls to only show the symlink target).
- The
readlink -f `which wget` shows the full path of where /bin/wget points to: the BusyBox binary.
The main trick consists of backtick evalution, and knowing that ls cannot get you just the symlink target, but readlink can:
Now the back-tick escapes, because you cannot nest back-ticks:
- The
`readlink -f \`which wget\`` executes the BusyBox binary without arguments.
- The
`readlink -f \`which wget\`` --list executes the BusyBox binary with the --list parameter.
Note I do not like the cat --help (see [WayBack] How do I check busybox version (from busybox)? – Unix & Linux Stack Exchange) way of getting the BusyBox version, as that gets echoed to stderr.
This is the output:
Read the rest of this entry »
Posted in *nix, *nix-tools, cURL, ESXi6, ESXi6.5, ESXi6.7, Power User, Virtualization, VMware ESXi, wget | Leave a Comment »
Posted by jpluimers on 2021/01/25
Back in 2019, there were 56 commands and scripts covered. I wonder how many there are now.
An ongoing list of Linux Networking Commands and Scripts. These commands and scripts can be used to configure or troubleshoot your Linux network.
Source: [WayBack] 55 Linux Networking commands and scripts
List back then (which goes beyond just built-in commands: many commands from optional packages are here as well):
- arpwatch – Ethernet Activity Monitor.
- bmon – bandwidth monitor and rate estimator.
- bwm-ng – live network bandwidth monitor.
- curl – transferring data with URLs. (or try httpie)
- darkstat – captures network traffic, usage statistics.
- dhclient – Dynamic Host Configuration Protocol Client
- dig – query DNS servers for information.
- dstat – replacement for vmstat, iostat, mpstat, netstat and ifstat.
- ethtool – utility for controlling network drivers and hardware.
- gated – gateway routing daemon.
- host – DNS lookup utility.
- hping – TCP/IP packet assembler/analyzer.
- ibmonitor – shows bandwidth and total data transferred.
- ifstat – report network interfaces bandwidth.
- iftop – display bandwidth usage.
- ip (PDF file) – a command with more features that ifconfig (net-tools).
- iperf3 – network bandwidth measurement tool. (above screenshot Stacklinux VPS)
- iproute2 – collection of utilities for controlling TCP/IP.
- iptables – take control of network traffic.
- IPTraf – An IP Network Monitor.
- iputils – set of small useful utilities for Linux networking.
- jwhois (whois) – client for the whois service.
- “lsof -i” – reveal information about your network sockets.
- mtr – network diagnostic tool.
- net-tools – utilities include: arp, hostname, ifconfig, netstat, rarp, route, plipconfig, slattach, mii-tool, iptunnel and ipmaddr.
- ncat – improved re-implementation of the venerable netcat.
- netcat – networking utility for reading/writing network connections.
- nethogs – a small ‘net top’ tool.
- Netperf – Network bandwidth Testing.
- netsniff-ng – Swiss army knife for daily Linux network plumbing.
- netstat – Print network connections, routing tables, statistics, etc.
- netwatch – monitoring Network Connections.
- ngrep – grep applied to the network layer.
- nload – display network usage.
- nmap – network discovery and security auditing.
- nslookup – query Internet name servers interactively.
- ping – send icmp echo_request to network hosts.
- route – show / manipulate the IP routing table.
- slurm – network load monitor.
- snort – Network Intrusion Detection and Prevention System.
- smokeping – keeps track of your network latency.
- socat – establishes two bidirectional byte streams and transfers data between them.
- speedometer – Measure and display the rate of data across a network.
- speedtest-cli – test internet bandwidth using speedtest.net
- ss – utility to investigate sockets.
- ssh – secure system administration and file transfers over insecure networks.
- tcpdump – command-line packet analyzer.
- tcptrack – Displays information about tcp connections on a network interface.
- telnet – user interface to the TELNET protocol.
- tracepath – very similar function to traceroute.
- traceroute – print the route packets trace to network host.
- vnStat – network traffic monitor.
- wget – retrieving files using HTTP, HTTPS, FTP and FTPS.
- Wireless Tools for Linux – includes iwconfig, iwlist, iwspy, iwpriv and ifrename.
- Wireshark – network protocol analyzer.
Via:
–jeroen
Posted in *nix, *nix-tools, cURL, dig, Internet, nmap, Power User, SpeedTest, ssh/sshd, tcpdump, Wireshark | Leave a Comment »
Posted by jpluimers on 2020/04/27
For me, on Windows, curl works better than wget, but on Linux/Mac OS X, curl tends to work better. Some people find wget easier for downloading multiple URLs at the same time. So here the parameter switches for both so they download to the file specified by the Content-Disposition http header:
curl --remote-name --remote-header-namewget --content-disposition
My experience is that wget is better at this, especially when redirects are involved (by adding a [WayBack] --location parameter to thecurl command line).
So for instance the first fails, but the second succeeds determining the download to be VSCodeUserSetup-x64-1.27.2.exe (so curl keeps the name stable):
curl.exe --location --remote-name --remote-time --remote-header-name https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable
wget.exe --content-disposition https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable
This takes into account the name after all followed redirects.
Via:
–jeroen
Posted in *nix, cURL, Power User, wget | Leave a Comment »
Posted by jpluimers on 2019/11/08
After all these years in the field, I still learned new tricks from [WayBack] 20 Cool Command Line Tricks for Windows and macOS which I have rephrased:
Windows
1) View installed drivers: [WayBack] driverquery
2) Watch Star Wars in ASCII: [WayBack] telnet towel.blinkenlights.nl
…
3) Save folder trees to disk: tree (I had totally forgotten about this, probably because it leaves out a lot of directories and files)
…
5) Show your Wi-Fi password [WayBack] netsh wlan show profile SSID key=clear (replace SSID with your network name; use nets wlan show profile to view the network names)
…
7) Check your laptop’s battery health: [WayBack] powercfg /batteryreport which will be in ” and hit Enter to generate the report, then %HOMEPATH%\battery-report.html
8) View your Windows license key: wmic path softwarelicensingservice get OA3xOriginalProductKey [WayBack]
…
Mac OS X / macOS / however it is called now
1) Change the default screenshot type: [WayBack] defaults write com.apple.screencapture type JPG (you can also use JP2 (for JPEG2000), PDF, PNG, TIFF and others)
2) Get your Mac to speak to you: use say
3) Add a message to the login screen: sudo defaults write /Library/Preferences/com.apple.loginwindow LoginwindowText "your new text on the logon window" [WayBack]
4) Play Tetris and other classics: start emacs, then press Esc followed by X, type in tetris, pong, snake or solitaire (to exit emacs, press Ctrl–X followed by Ctrl–C). There are [WayBack] more emacs games.
5) Get a dictionary definition: run curl dict://dict.org/d:word (where word is what you are after) which uses the [WayBack] dict protocol
6) Keep macOS awake: [WayBack] caffeinate optionally followed by a -t## parameter where ## is the number of seconds to not sleep.
7) Show hidden files: defaults write com.apple.finder AppleShowAllFiles -bool TRUE; killall Finder or use this AppleShowAllFiles script which I had forgotten about writing in the first place.
…
10) Add Spaces to the Dock: defaults write com.apple.dock persistent-apps -array-add ‘{”tile-type”=”spacer-tile”;}’; killall Dock running the command as many times as you want spaces. To get rid of a space you’ve added, just drag them to the Trash.
–jeroen
via: [WayBack] 20 Cool Command Line Tricks for Windows and macOS http://flip.it/SvcQlu – Joe C. Hecht – Google+
Posted in *nix, *nix-tools, Apple, cURL, Power User | Leave a Comment »
The Wiert Corner - irregular stream of stuff 