Instantly convert curl commands to Go code: curl-to-Go: Convert curl commands to Go code at https://mholt.github.io/curl-to-go/
via: Instantly convert curl commands to Go code: bit.ly/1PJprPV – nifty tool!
–jeroen
Archive for the ‘cURL’ Category
Instantly convert curl commands to Go code: https://mholt.github.io/curl-to-go/
Posted by jpluimers on 2017/01/31
Posted in *nix, cURL, Development, Go (golang), Power User, Software Development | Leave a Comment »
Some cURL links with tips I used doing some Fritz!Box scripting research
Posted by jpluimers on 2016/09/29
I needed to script a few things on my Fritz!Box. Here are the cURL links that I used to research some Fritz!Box scripting.
My first try was wget, but that didn’t do everything I need, so cURL came to the rescue.
In the end, I didn’t need cookies (a post request with an MD5 based handshake sufficed to get a session SID which is not stored in a Cookie), but that surely will come in useful another time.
- scripting – cURL and click a button in a website – Stack Overflow.
- linux – CURL to access a page that requires a login from a different page – Stack Overflow.
- How do you handle authentication via cookie with CURL? – php | Ask MetaFilter.
- How to get past the login page with Wget? – Stack Overflow.
- http – What is the cURL command-line syntax to do a POST request? – Super User.
- web standards – Is it a good practice to use an empty URL for a HTML form’s action attribute? (action=””) – Stack Overflow.
- Difference between id and name attributes in HTML – Stack Overflow.
- Curl: Re: How to POST to a page with multiple forms.
- FRITZ!Box Backup mit curl (Bash) [Daniels Braindump].
Curl man page entries:
The script is and docs are here: jpluimers/bash-fritzclient.
–jeroen
Share this:
Posted in *nix, bash, cURL, Development, Fritz!, Fritz!Box, Network-and-equipment, Power User, Scripting, Software Development, wget | Leave a Comment »
APC: getting ftp://ftp.apc.com/apc/public/software/pnetmib/mib/417/powernet417.mib turned out to be tricky
Posted by jpluimers on 2016/08/24
I tried updating my downloads for my APC7920 and APC7921 PDUs.
I knew the APC download site http://www.apc.com/nl/en/tools/download/index.cfm was slow and navigation unfriendly (lots of ERR_CACHE_MISS as you cannot ctrl-click on downloads), but it’s also buggy: Some of the ftp download URLs do not contain the authentication and one file would not download at all.
The solution for that is to prepend the credentials as username:password@ like these URLs where each first one is generated by the download site and each second one works:
ftp://ftp.apcc.com/restricted/hardware/nmcard/firmware/devipcfg_wiz/502/Device%20IP%20Configuration%20Wizard.exeftp://restrict:Kop$74!@ftp.apcc.com/restricted/hardware/nmcard/firmware/devipcfg_wiz/502/Device%20IP%20Configuration%20Wizard.exe
ftp://ftp.apc.com/restricted/hardware/nmcard/firmware/sec_wiz/104/SecWiz%201.04%20Install.exeftp://restrict:Kop$74!@ftp.apc.com/restricted/hardware/nmcard/firmware/sec_wiz/104/SecWiz%201.04%20Install.exe
The username is
restrictand the passwordKop$74!which requires single quotes on the command-line to prevent parameter and event expansion.Otherwise you will get
basherrors like these:event not foundfor the part starting with an exclamation mark andLogin incorrect.for the parts having a dollar.
One file would not download at all: ftp://ftp.apc.com/apc/public/software/pnetmib/mib/417/powernet417 as all download attempts would time out:
- Chrome with and without username:password@ (you will get a
ERR_FTP_FAILED) - wget with and without username:password@ (it will result in a
) - plain curl with and without username:password@ (it will result in a
curl: (28) Timeout was reached)
The only command that would work was this:
curl -G ftp://ftp.apc.com/apc/public/software/pnetmib/mib/417/powernet417.mib > powernet417.mib
via: SimplicityGuy/pynoc – Travis CI
The trick is to:
- leave username and password away
- specify the -G (or –get) parameter forcing GET behaviour (which should be the default).
I’m not sure why it works, but it does.
–jeroen
Share this:
Posted in *nix, APC Smart-UPS, cURL, Power User, UPS | Leave a Comment »
Use cURL to test local virtual hosts configuration – via: Stack Overflow
Posted by jpluimers on 2016/06/14
Quoting John Hart‘s brilliant answer:
Using
--resolveleverages all of the normal logic that applies, but simply pretends the DNS lookup returned the data in your command-line option. It works just like/etc/hostsshould.Note
--resolvetakes a port number, so for HTTPS you would useHttps:
curl --resolve 'yada.com:443:127.0.0.1' https://yada.com/somethingHttp:
curl --resolve 'yada.com:80:127.0.0.1' http://yada.com/something
It requires curl 7.21.3 or higher (from the end of 2010). Which by now everybody should have.
–jeroen
more via: Set cURL to use local virtual hosts – Stack Overflow.
Share this:
Posted in *nix, bash, cURL, Development, Power User, Scripting, Software Development | Leave a Comment »
cURL is not a wget clone (so cannot do mirror) – via: MacNN Forums
Posted by jpluimers on 2016/04/05
Even though cURL seems to handle https better than wget, it is not a wget replacement.
From the cURL FAQ :
1.3 What is cURL not?
Curl is not a wget clone. That is a common misconception. Never, during curl’s development, have we intended curl to replace wget or compete on its market. Curl is targeted at single-shot file transfers.
Curl is not a web site mirroring program. If you want to use curl to mirror something: fine, go ahead and write a script that wraps around curl to make it reality (like curlmirror.pl does).
Curl is not an FTP site mirroring program. Sure, get and send FTP with curl but if you want systematic and sequential behavior you should write a script (or write a new program that interfaces libcurl) and do it.
–jeroen
via:
- How to mirror a web site with curl or wget? – MacNN Forums.
- OpenSSL: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure.
Share this:
Posted in *nix, cURL, Power User, wget | Leave a Comment »
Getting your public IP address from the command-line
Posted by jpluimers on 2016/01/13
Many sites giving your public IP address return a web page with a bloat of html. From the command-line, you are usually only interested in the IP-address itself. Few services return exactly that.
Below are command-line examples to provide the public IP address mostly from a *nix perspective. Usually you can get similar commands to work with Windows binaries for wget and Windows binaries for curl.
In the end, I’ve opted for commands in this format, as I think akamai will last longer than the other sites (but does not include an end-of-line in the http result hence the echo on Mac/*nix):
- Mac / *nix:
- curl http://whatismyip.akamai.com && echo
- wget -q -O – http://whatismyip.akamai.com && echo
- Windows
- curl http://whatismyip.akamai.com
- wget -q -O – http://whatismyip.akamai.com
I’ve not tried aria2 yet, but might provide commands for that in the future.
These are the Linux permutations for akamai:
curl whatismyip.akamai.com && echo curl ipv4.whatismyip.akamai.com && echo curl ipv6.whatismyip.akamai.com && echo curl ipv4.whatismyip.akamai.com && echo && curl ipv6.whatismyip.akamai.com && echoThe last two are convenient when you have both IPv4 and IPv6 configured on “the outside”.
You can replace curl with wget -q -O – (which outputs to stdout) for each command. You can even ommit the http:// (as that is the default protocol for both curl and wget).
Share this:
Posted in *nix, *nix-tools, Apple, bash, bash, Batch-Files, cURL, Development, Linux, Mac, Mac OS X / OS X / MacOS, Mac OS X 10.4 Tiger, Mac OS X 10.5 Leopard, Mac OS X 10.6 Snow Leopard, Mac OS X 10.7 Lion, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, MacMini, OS X 10.10 Yosemite, OS X 10.8 Mountain Lion, OS X 10.9 Mavericks, Power User, Scripting, Software Development, SuSE Linux, wget | Leave a Comment »
Enabling GIT_CURL_VERBOSE to research “unable to get local issuer certificate”
Posted by jpluimers on 2015/05/28
A while ago, I was fighting a corporate web proxy playing Man-in-the-Middle on all https sessions.
Though playing MitM on your employees is a debatable thing to do (especially without informing the employees, and illegal in certain countries, I had to get a GIT connection to the outside world working.
This helped tracking it down: GIT_CURL_VERBOSE “unable to get local issuer certificate”.
What I finally did was this:
- obtain the CA certificate that issues the MitM certificate in base-64 CRT form (which is the same as the PEM form):
- added it at the top of either of these files:
"%ProgramFiles%\Git\bin\curl-ca-bundle.crt""%ProgramFiles(x86)%\Git\bin\curl-ca-bundle.crt"
- added it to the top of either of these files:
"%ProgramFiles%\Mercurial\cacert.pem""%ProgramFiles(x86)%\Mercurial\cacert.pem"
–jeroen
PS: These were the failures I was getting:
Share this:
Posted in *nix, cURL, Development, DVCS - Distributed Version Control, git, PKI, Power User, Security, Source Code Management | Leave a Comment »
Windows: authenticated command-line download from IIS server wget: no, cURL: yes.
Posted by jpluimers on 2014/10/03
Had to download a bunch of stuff over the command-line from an IIS server that was using authentication. Not basic authentication, but NTLM authentication.
wget kept failing, even wget 1.10 that usually does NTLM quite OK (but up to 1.10.2 has a security vulnerability so you should not use wget 1.10 any more).
So I installed a Windows x86 cURL binary, and downloaded+copied the root certificates, then did some reading on the command-line switches.
Without any, cURL does http basic authentication. But a Windows server usually expects NTLM authentication (hardly documented, but it uses the Negotiate protocol).
When not using NTLM, both would show (wget -d, or curl -v) this in the output, indicating you should use NTLM authentication: Read the rest of this entry »
Share this:
Posted in *nix, *nix-tools, cURL, Linux, Power User, SuSE Linux, wget, Windows, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2 | Leave a Comment »
The Wiert Corner - irregular stream of stuff 