The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,091 other followers

Archive for the ‘cURL’ Category

curl/wget: use content-disposition for the file names

Posted by jpluimers on 2020/04/27

For me, on Windows, curl works better than wget, but on Linux/Mac OS X, curl tends to work better. Some people find wget easier for downloading multiple URLs at the same time. So here the parameter switches for both so they download to the file specified by the Content-Disposition http header:

  • curl --remote-name --remote-header-name
  • wget --content-disposition

My experience is that wget is better at this, especially when redirects are involved (by adding a [WayBack] --location parameter to  thecurl command line).

So for instance the first fails, but the second succeeds determining the download to be VSCodeUserSetup-x64-1.27.2.exe (so curl keeps the name stable):

curl.exe --location --remote-name --remote-time --remote-header-name https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable

wget.exe --content-disposition https://vscode-update.azurewebsites.net/latest/win32-x64-user/stable

This takes into account the name after all followed redirects.

Via:

–jeroen

Posted in *nix, cURL, Power User, wget | Leave a Comment »

20 Cool Command Line Tricks for Windows and macOS

Posted by jpluimers on 2019/11/08

After all these years in the field, I still learned new tricks from [WayBack20 Cool Command Line Tricks for Windows and macOS which I have rephrased:

Windows

1) View installed drivers: [WayBack] driverquery

2) Watch Star Wars in ASCII: [WayBacktelnet towel.blinkenlights.nl

3) Save folder trees to disk: tree (I had totally forgotten about this, probably because it leaves out a lot of directories and files)

5) Show your Wi-Fi password [WayBacknetsh wlan show profile SSID key=clear (replace SSID with your network name; use nets wlan show profile to view the network names)

7) Check your laptop’s battery health: [WayBack] powercfg /batteryreport which will be in ” and hit Enter to generate the report, then %HOMEPATH%\battery-report.html

8) View your Windows license key: wmic path softwarelicensingservice get OA3xOriginalProductKey [WayBack]

Mac OS X / macOS / however it is called now

1) Change the default screenshot type: [WayBackdefaults write com.apple.screencapture type JPG (you can also use JP2 (for JPEG2000), PDF, PNG, TIFF and others)

2) Get your Mac to speak to you: use say

3) Add a message to the login screen: sudo defaults write /Library/Preferences/com.apple.loginwindow LoginwindowText "your new text on the logon window" [WayBack]

4) Play Tetris and other classics: start emacs, then press Esc followed by X, type in tetris, pong, snake or solitaire (to exit emacs, press CtrlX followed by CtrlC). There are [WayBack] more emacs games.

5) Get a dictionary definition: run curl dict://dict.org/d:word (where word is what you are after) which uses the [WayBack] dict protocol

6) Keep macOS awake: [WayBack] caffeinate optionally followed by a -t## parameter where ## is the number of seconds to not sleep.

7) Show hidden files: defaults write com.apple.finder AppleShowAllFiles -bool TRUE; killall Finder or use this AppleShowAllFiles script which I had forgotten about writing in the first place.

10) Add Spaces to the Dock: defaults write com.apple.dock persistent-apps -array-add ‘{”tile-type”=”spacer-tile”;}’; killall Dock running the command as many times as you want spaces. To get rid of a space you’ve added, just drag them to the Trash.

–jeroen

via: [WayBack] 20 Cool Command Line Tricks for Windows and macOS http://flip.it/SvcQlu – Joe C. Hecht – Google+

Posted in *nix, *nix-tools, Apple, cURL, Power User | Leave a Comment »

Convert cURL command syntax to Python requests, Node.js code

Posted by jpluimers on 2019/07/26

Utility for converting curl commands to code

For my link archive: [WayBack] Convert cURL command syntax to Python requests, Node.js code

–jeroen

Posted in *nix, *nix-tools, cURL, Development, JavaScript/ECMAScript, Node.js, Power User, Python, Scripting, Software Development | Leave a Comment »

Transferring files from a Linux console: transfer.sh and anypaste.xyz

Posted by jpluimers on 2019/07/26

transfer.sh

anypaste.xyz

–jeroen

via: [WayBack] Interesting: Anypaste – Share And Upload Files To Compatible Hosting Sites Automatically… – DoorToDoorGeek “Stephen McLaughlin” – Google+

Posted in *nix, *nix-tools, bash, cURL, Power User | Leave a Comment »

wget and curl: downloads that sometimes fail

Posted by jpluimers on 2018/10/19

For my archive somewhere between cURL 7.21.0 and 7.34.0 it does not like to be started from an RDP based tsclient share:

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.21.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 86465    0 86465    0     0  60805      0 --:--:--  0:00:01 --:--:-- 70012

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.34.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: web.archive.org

C:\Users\jeroen\Downloads>\\tsclient\bin\curl.7.61.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: www.xs4all.nl

C:\Users\jeroen\Downloads>copy \\tsclient\bin\curl.7.61.0.exe
        1 file(s) copied.

C:\Users\jeroen\Downloads>curl.7.61.0.exe --remote-name https://www.xs4all.nl/index.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    13    0    13    0     0     10      0 --:--:--  0:00:01 --:--:--    10

It fails the same way after net use B: \\tsclient\bin, so that does not matter.

The best link I could find until I got to the real problem was [WayBack] curl: (6) Could not resolve host: application – Stack Overflow which shows a different problem: properly quoting.

In addition to remote-name, you can also grab the file name from the headers using --remote-header-name, and --remote-time use the remote file time. The --location follows 302-redirects. You can see that in the example below which I build based on

[WayBack] unix – Curl to grab remote filename after following location – Stack Overflow: The remote side sends the filename using the Content-Disposition header.curl 7.21.2 or newer does this automatically if you specify –remote-header-name / -J.curl -O -J -L $url

C:\Users\jeroen\Downloads>b:\curl.7.21.0.exe --location --remote-name --remote-time --remote-header-name "https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 86465    0 86465    0     0  45748      0 --:--:--  0:00:01 --:--:-- 50772
curl: Saved to filename 'pkgWuppdiWP_DX102T_1-1-2.zip'

wget failed big time:

C:\Users\jeroen\Downloads>B:\wget.exe --no-check-certificate -v -v -v --content-disposition --restrict-file-names=windows "https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965"
wget: Cannot read b:/.wgetrc (No such file or directory).
--2018-07-12 09:55:23--  https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965
Resolving web.archive.org... 207.241.225.186
Connecting to web.archive.org|207.241.225.186|:443... failed: Invalid argument.
Retrying.

...

--2018-07-12 09:55:23--  (try:20)  https://web.archive.org/web/20180712073755if_/https://www.danielwolf.eu/?wpdmdl=1965
Connecting to web.archive.org|207.241.225.186|:443... failed: Invalid argument.
Giving up.

This is not caused by the filename (Windows does not like the ? question mark in output file names, so  – like & ampersand in file URLs – you have to quote the full URL, but also provide the --restrict-file-names=windows parameter; see [WayBack] wget – I can’t download files with “?” – Super User).

–jeroen

Posted in *nix, *nix-tools, cURL, Power User, wget | Leave a Comment »

 
%d bloggers like this: