May 2026
M	T	W	T	F	S	S
	1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30	31

Archive for the ‘Scripting’ Category

Mixed JScript/batch file hybrid to create Windows shortcuts

Posted by jpluimers on 2021/03/25

I bumped into this amazing JScript/batch file hybrid. It starts as a batch file, then continues as a JScript script, and creates/updates .lnk files: [WayBack] batch.scripts/shortcutJS.bat at master · npocmaka/batch.scripts · GitHub.

Wow. Just wow.

Not that I would want to be the one maintaining it (:

Via [WayBack] batch file – How do I create a shortcut via command-line in Windows? – Stack Overflow of which I like these answers most:

Check the shortcutJS.bat – it is a jscript/bat hybrid and should be used with .bat extension:
```
call shortcutJS.bat -linkfile "%~n0.lnk" -target  "%~f0" -linkarguments "some arguments"
```
With -help you can check the other options (you can set icon , admin permissions and etc.)
Nirsoft’s NirCMD can create shortcuts from a command line, too. (Along with a pile of other functions.) Free and available here:
http://www.nirsoft.net/utils/nircmd.html

Full instructions here: http://www.nirsoft.net/utils/nircmd2.html#using (Scroll down to the “shortcut” section.)

Yes, using nircmd does mean you are using another 3rd-party .exe, but it can do some functions not in (most of) the above solutions (e.g., pick a icon # in a dll with multiple icons, assign a hot-key, and set the shortcut target to be minimized or maximized).

Though it appears that the shortcutjs.bat solution above can do most of that, too, but you’ll need to dig more to find how to properly assign those settings. Nircmd is probably simpler.

–jeroen

Posted in Batch-Files, Development, Scripting, Software Development | Leave a Comment »

Mikrotik: script to set clock to Amsterdam, enable ntp, then show latest ntp correction

Posted by jpluimers on 2021/03/11

For my script list:

/system ntp client set enabled=yes server-dns-names=0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org
/delay 10
/system ntp client print

If the delay was long enough, you will see output like this:

             enabled: yes
         primary-ntp: 0.0.0.0
       secondary-ntp: 0.0.0.0
    server-dns-names: 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org
                mode: unicast
       poll-interval: 15m
       active-server: 149.210.230.59
    last-update-from: 149.210.230.59
  last-update-before: 29s290ms
     last-adjustment: 702us

If the delay was too short, the lines after mode will not be present.

–jeroen

Posted in Development, RouterOS, Scripting, Software Development | Leave a Comment »

GitHub – pastpages/savepagenow: A simple Python wrapper for archive.org’s “Save Page Now” capturing service

Posted by jpluimers on 2021/03/11

This makes it way easier to save WayBack content:

[WayBack] GitHub – pastpages/savepagenow: A simple Python wrapper for archive.org’s “Save Page Now” capturing service

A poor-mans alternative is the below bash script from [WayBack] Saving of public Google+ content at the Internet Archive’s Wayback Machine by the Archive Team has begun : plexodus:

For Linux, MacOS / OSX, BSD, and other Unix-like operating systems (including Android with Termux, or Windows, with a Unix/Linux environment), the following script (I’ve saved this as archive-url) will archive the requested URL:
#!/bin/bash
# archive-url
# Archive selected URL at the Internet Archive

curl -s -I -H "Accept: application/json" "https://web.archive.org/save/${1}" |
grep '^x-cache-key:' | sed "s,https,&://,; s,$${1}$.*$,\1,"
Save that to your execution path (I’ve chosen ~/bin, you might use /usr/local/bin or another location on your $PATH, and invoke as, say (again referring to the G+MM homepage):
$ archive-url https://plus.google.com/communities/112164273001338979772
If you have a list of URLs in a file (or pipelined from command output), you can request all of them to be archived in a simple bash loop. I’m using xargs here to run ten simultaneous requests from the file gplus-urllist:
cat gplus_urllist | while read url do xargs -I{} -P 10 archive-url {}; done
I’ve run this on over 10,000 URLs over a modest residential broadband connection in a hair over two hours.

Note that such requests trigger an archive by the Internet Archive from one of its archiving nodes, you’re not sending the page to the Archive yourself. In particular, archival from regions defaulting to another language may result in the Google+ site content (but not post or comments) being in a different language. I’ve frequently seen my pages turning up in Japanese, for instance.

–jeroen

Posted in bash, Development, Python, Scripting, Software Development | Leave a Comment »

Debouncing and Throttling Explained Through Examples | CSS-Tricks

Posted by jpluimers on 2021/03/10

TL;DR of https://css-tricks.com/debouncing-throttling-explained-examples/:

debounce: Grouping a sudden burst of events (like keystrokes) into a single one.

throttle: Guaranteeing a constant flow of executions every X milliseconds. Like checking every 200ms your scroll position to trigger a CSS animation.

requestAnimationFrame: a throttle alternative. When your function recalculates and renders elements on screen and you want to guarantee smooth changes or animations. Note: no IE9 support.

Full article [WayBack] Debouncing and Throttling Explained Through Examples | CSS-Tricks

Delphi implementations:

Throttling in [WayBack] TNotifyEvent debouncing in Delphi – Code Partners
- Explanation why the Code Partners are wrong: [WayBack] Nice article – but unfortunately a bit wrong… – Stefan Glienke – Google+

Both in https://bitbucket.org/sglienke/spring4d/src/fb18fd22613754cb98a7fda64b14627f2d8010a7/Source/Reactive

Example: a delayed search while the user is typing into an edit

  TObservable.FromEventPattern(Edit1, 'OnChange');
    .Throttle(1000)
    .ObserveOn(TScheduler.MainThread)
    .Subscribe(
    procedure(const args: TNotifyEventArgs)
    begin
      Caption := Format('searching for %s', [TEdit(args.Sender).Text]);
    end);

–jeroen

Posted in Algorithms, Delphi, Development, JavaScript/ECMAScript, Scripting, Software Development | Leave a Comment »

reg query and batch file for loop tricks that refreshes the cmd environment from the registry settings: choco/RefreshEnv.cmd at master · chocolatey/choco · GitHub

Posted by jpluimers on 2021/03/04

I bumped into a very interesting [WayBack] choco/RefreshEnv.cmd at master · chocolatey/choco · GitHub.

It allows you to refresh your cmd environment from new settings that were only applied to the registry using the SET command.

Note there is a PowerShell counterpart too: [WayBack] choco/Update-SessionEnvironment.ps1 at master · chocolatey/choco · GitHub

There are many cool tricks in it, most of which you can see in the [WayBack] new commit history, and a few you can find back in the [WayBack] old commit history of the previous repository (I have no idea why those histories have never been merged).

Intermediate batch files

The basic structure is to first create some intermediate batch files, then delete them afterwards:

"%TEMP%\_envget.tmp"
- is used in :GetRegEnv to get all environment variables for the MACHINE or USER level, then loop through them and call :SetFromReg during each iteration (except for the Path environment variable which is skipped).
"%TEMP%\_envset.tmp"
- is used in :SetFromReg to emit one line of SET code to "%TEMP%\_env.cmd".
"%TEMP%\_env.cmd"
- Contains the SET commands for the new environment variable values.

All the above methods use quoting to ensure that environment variables having names or values containing spaces are handled correctly.

Echo without newline

I like the echo | set /p trick to echo a string without a newline allows it to start as this:

C:\>RefreshEnv Refreshing environment variables from registry for cmd.exe. Please wait...

then finish like this by appending another string to it:

C:\>RefreshEnv Refreshing environment variables from registry for cmd.exe. Please wait...Finished..

It is explained in the old history at [WayBack] (GH-153)(GH-134) Update PATH on cmd.exe · chocolatey/chocolatey@a09e158 · GitHub.

There is an even more interesting example of this trick in [WayBack] windows – What does /p mean in set /p? – Stack Overflow:

<nul set /p=This will not generate a new line

Spaces, what spaces

One hard thing in scripting is taking into account that path names can contain spaces. This means you need to carefully quote path names, but not overdo the quotes, otherwise the quoting works against you.

Two commits from the commit history show there were two weak spots that had to be changed in [WayBack] (GH-1227) Fix: RefreshEnv doesn’t set path w/spaces · chocolatey/choco@fdfcd06 · GitHub.

The environment has a MACHINE and USER part

Environment variables can come from two places in the registry:

HKLM\System\CurrentControlSet\Control\Session Manager\Environment
HKCU\Environment

Normally, the second overrides the first.

This means they are grabbed from the registry MACHINE and USER order, then applied to the cmd environment.

Special case `PATH`

The PATH environment variable is special for two reasons:

In the registry it is called Path, but in the environment it is usually called PATH (this is true for both the MACHINE and USER parts of the registry). New values are applied with the Path environment variable name, so after executing RefreshEnv once, they are called Path in the cmd.exe environment too.
PATH is a combination from two PATH entries in the registry in the MACHINE and USER level, so it needs to be combined as you can see in [WayBack] choco/RefreshEnv.cmd at master · chocolatey/choco · GitHub.:

    :: Caution: do not insert space-chars before >> redirection sign
    echo/set "Path=%%Path_HKLM%%;%%Path_HKCU%%" >> "%TEMP%\_env.cmd"

I am not sure why there is a space before the >>, given there is a comment above it there should not be one.

The SET command however, puts the MACHINE PATH in front of the USER PATH.

Special case `USERNAME`, and collateral `PROCESSOR_ARCHITECTURE`

The USERNAME environment variable special too. In the registry, it is only in the MACHINE part, but with a value SYSTEM.

In cmd.exe, it is actually filled with the current username, so it should not be overwritten with the one in the MACHINE part.

Currently this is resolved by storing a copy of the old value of USERNAME and PROCESSOR_ARCHITECTURE in [WayBack] (GH-902) Fix: User changed to SYSTEM during env update · chocolatey/choco@cb6b92c · GitHub.

I am not sure why PROCESSOR_ARCHITECTURE is also stores.

In any case, this means that setting a USERNAME or PROCESSOR_ARCHITECTURE in the USER part of the registry, will not be reflected by RefreshEnv.

I am not sure yet when that can cause problems, so this is a reminder to myself that if ever it does, then this logic needs to be changed.

–jeroen

Posted in Batch-Files, Development, Scripting, Software Development | Leave a Comment »

Python: saving a web page to a jpeg image file by using the Google base64url encoded screenshot of it

Posted by jpluimers on 2021/02/19

As a follow-up on Still looking for base64url decoding tools, both on-line and for MacOS homebrew: this is in Python, works on MacOS, Linux and Windows, and can be integrated in a web page.

It is based on the ideas in [WayBack] Python-Twitter-Hacks/websiteScreenshot.py at master · edent/Python-Twitter-Hacks · GitHub, which was more like a code snippet with hard coded literals.

It downloads a jpeg web-site screenshot using the Google PageSpeed API V1, which generates the screenshot as a base64url encoded blob inside a JSON structure.

Python does not have native Python base64url support, but the concept of it is fairly straightforward: [WayBack] RFC 4648 – The Base16, Base32, and Base64 Data Encodings: Base 64 Encoding with URL and Filename Safe Alphabet, which allows data to be passed inside URLs without reverting to [WayBack] Percent-encoding – Wikipedia.

My changes work, but are by no means in canonical form or Idiomatic Python. I have a long way to go to reach that level of Python.

So I forked the repository, and fixed the script basing it on Python 3.

I might make it V2 compatible in the future. More information on V2 in [WayBack] Google APIs Explorer: Services > PageSpeed Insights API v2 > pagespeedonline.pagespeedapi.runpagespeed

Content is in the below gist.

–jeroen

Read the rest of this entry »

Posted in base64, base64url, Development, Encoding, Python, Scripting, Software Development | Leave a Comment »

PowerShell: avoid Write-Output, use Return only for ending execution, use $Output variable for returning additional output

Posted by jpluimers on 2021/02/18

Recently, I bumped into [WayBack] Write-Output confusion for the upteenth time.

Luckily I had the below links archived, basically invalidating the use of Write-Output, and invalidating the answer at [WayBack] powershell – What’s the difference between “Write-Host”, “Write-Output”, or “[console]::WriteLine”? – Stack Overflow.

Read the rest of this entry »

Posted in CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

explainshell.com: parse and explain just about any shell command

Posted by jpluimers on 2021/02/17

I bumped into the tremendously site [WayBack] explainshell.com – match command-line arguments to their help text only after documenting the relevant cURL options of yesterdays post on checking your CertBot domain expiration dates.

The site allows put in a shell command-line to see the help text that, including matches for each argument.

It works so well because it parses both the shell command-line and the man pages, then constructs a web-page linking the relevant man page content to the shell command-line in the correct shell command-line order.

The explainshell has a counterpart showthedocs (both are open source) for explaining other languages (on the one hand more extended as it goes much deeper into parsing for instance SQL, on the other hand more limited as it only supports a few languages). More on showthedocs later.

The links

The parsing results

The first bit below is just the text output, and the second bit the screenshot, of a relatively simple command like [WayBack] explainshell.com – curl -fsSL example.org:

curl(1) -fsSL example.org

transfer a URL

-f, --fail
       (HTTP)  Fail  silently  (no  output at all) on server errors. This is mostly done to better enable
       scripts etc to better deal with failed attempts. In normal cases  when  a  HTTP  server  fails  to
       deliver  a  document,  it  returns an HTML document stating so (which often also describes why and
       more). This flag will prevent curl from outputting that and return error 22.

       This method is not fail-safe and there are occasions where non-successful response codes will slip
       through, especially when authentication is involved (response codes 401 and 407).

-s, --silent
       Silent or quiet mode. Don't show progress meter or error messages.  Makes Curl mute.

-S, --show-error
       When used with -s it makes curl show an error message if it fails.

-L, --location
       (HTTP/HTTPS) If the server reports that the requested page  has  moved  to  a  different  location
       (indicated  with  a Location: header and a 3XX response code), this option will make curl redo the
       request on the new place. If used together with -i, --include or  -I,  --head,  headers  from  all
       requested pages will be shown. When authentication is used, curl only sends its credentials to the
       initial host. If a redirect takes curl to a different host, it won't  be  able  to  intercept  the
       user+password.  See  also  --location-trusted  on  how to change this. You can limit the amount of
       redirects to follow by using the --max-redirs option.

       When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will
       do  the  following  request  with a GET if the HTTP response was 301, 302, or 303. If the response
       code was any other 3xx code, curl will re-send the following request  using  the  same  unmodified
       method.

source manpages: curl

The screenshot is even more impressive:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

Checking expiration dates for your certbot certificates

Posted by jpluimers on 2021/02/16

I have these two little aliases in my toolbox:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

First a big fat warning

do not run just any script downloaded through curl. Plenty of reasons why this is dangerous:

[WayBack] Detecting the use of “curl | bash” server side | Application Security

[WayBack] One way “curl pipe sh” install scripts can be dangerous [proof of concept] / Jordan Eldredge: script content differs depending on user agent

[WayBack] sean cassidy : Don’t Pipe to your Shell: scripts having different behaviour when executed partially

[WayBack] Why using curl | sudo sh is not advised? – Stack Overflow:

You can proof your scripts against partial execution by putting the whole thing into the body of a function, and executing that function on the last line. If you know a script is defined like that, it’s exactly as secure as downloading and then executing some installer.

The first three can mostly prevented by using your own fork of the script repository, then checking each modification of the script, combined with ensuring your fork location does not throw tricks 1 or 2 on you.

That’s why I run the above alias only from a checkssl fork which I can inspect.

Back to the alias

The aliases use quite a few tricks:

Having curl download a command minimising
```
curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl
```
Via: [WayBack] The missing package manager for macOS (or Linux) — The missing package manager for macOS (or Linux)
Running that command through bash as if the download were a file by wrapping wrapping it in parenthesis and a less than sign .
```
bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl)
```
Via: [WayBack] linux – Execute bash script from URL – Stack Overflow

Passing arguments to the bash command by appending this to the command just like from the regular command-line:

bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live

An alias [WayBack] with double-quotes around the whole statement:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"

Either use the certbot configuration file or apache2 (via ISPconfig as the apache2 parameter value is not yet supported) domain configuration:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

Using `source` instead of `bash`

Note that an alternative alias is this one:

alias "certbot-check-all-by-file=(source <(curl -s https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live)"

However, that needs an extra set of parenthesis, otherwise you will get bumped out of your current shell.

The reason is that bash runs in a [WayBack] subshell, whereas [WayBack] source (and the equivalent [WayBack] “dot” command .) runs in the current shell, but the script performs a gracefull_exit or error_exit which end in an exit [WayBack] terminating the current shell.

The [WayBack] () parenthesis around the source command ensure it runs in a sub-shell.

In this case, you can still pass the --location /etc/letsencrypt/live parameters, as bash is the only shell allowing this: [WayBack] source – Passing variables to a bash script when sourcing it – Unix & Linux Stack Exchange.

References

Related [WayBack] Advanced Bash-Scripting Guide topics:

Related cURL options from [WayBack] curl – How To Use:

-f: [WayBack] -f, --fail

(HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).
-s: [WayBack] -s, --silent:

Silent or quiet mode. Don’t show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

Use -S, –show-error in addition to this option to disable progress meter but still show error messages.

See also -v, –verbose and –stderr.
-S: [WayBack] -S, --show-errors:

When used with -s, –silent, it makes curl show an error message if it fails.
-L: [WayBack] -L, --location:

(HTTP) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place. If used together with -i, –include or -I, –head, headers from all requested pages will be shown. When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won’t be able to intercept the user+password. See also –location-trusted on how to change this. You can limit the amount of redirects to follow by using the –max-redirs option.

When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.

You can tell curl to not change the non-GET request method to GET after a 30x response by using the dedicated options for that: –post301, –post302 and –post303.

Reminders

Reminder to self: see if JSON output is viable. This commit might help.

–jeroen

Posted in bash, Conference Topics, Conferences, Development, Event, Scripting, Software Development | Leave a Comment »

How do I pretty-print JSON in Delphi? – Stack Overflow

Posted by jpluimers on 2021/02/09

For my archive: the [WayBack] How do I pretty-print JSON in Delphi? – Stack Overflow answer by [WayBack] Bob:

If you do not want to use any external library, and you’re using a Delphi XE5 or newer, there is a very handy TJson.Format() function in the REST.Json unit.
uses json, REST.Json;

{ ... }    

function FormatJSON(json: String): String;
var
  tmpJson: TJsonObject;
begin
  tmpJson := TJSONObject.ParseJSONValue(json);
  Result := TJson.Format(tmpJson);

  FreeAndNil(tmpJson);
end;

–jeroen

Posted in Delphi, Development, JavaScript/ECMAScript, JSON, Scripting, Software Development | 2 Comments »

« Previous Entries

Next Entries »

	Jeroen Wiert Pluimer… on Digitale toegankelijkheid als…
	Jeroen Wiert Pluimer… on Digitale toegankelijkheid als…
	Vereniging NLUUG on Digitale toegankelijkheid als…
	jpluimers on Sony STR-DE205 Receiver…
	jpluimers on Position paper nachtsluiting D…

The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

Twitter Updates

Pages

All categories

Email Subscription

Archive for the ‘Scripting’ Category

Mixed JScript/batch file hybrid to create Windows shortcuts

Mikrotik: script to set clock to Amsterdam, enable ntp, then show latest ntp correction

GitHub – pastpages/savepagenow: A simple Python wrapper for archive.org’s “Save Page Now” capturing service

Debouncing and Throttling Explained Through Examples | CSS-Tricks

reg query and batch file for loop tricks that refreshes the cmd environment from the registry settings: choco/RefreshEnv.cmd at master · chocolatey/choco · GitHub

Intermediate batch files

Echo without newline

Spaces, what spaces

The environment has a MACHINE and USER part

Special case `PATH`

Special case `USERNAME`, and collateral `PROCESSOR_ARCHITECTURE`

Python: saving a web page to a jpeg image file by using the Google base64url encoded screenshot of it

PowerShell: avoid Write-Output, use Return only for ending execution, use $Output variable for returning additional output

explainshell.com: parse and explain just about any shell command

The links

The parsing results

Checking expiration dates for your certbot certificates

First a big fat warning

Back to the alias

Using `source` instead of `bash`

References

Reminders

How do I pretty-print JSON in Delphi? – Stack Overflow

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

Pages

All categories

Email Subscription

Archive for the ‘Scripting’ Category

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Intermediate batch files

Echo without newline

Spaces, what spaces

The environment has a MACHINE and USER part

Special case PATH

Special case USERNAME, and collateral PROCESSOR_ARCHITECTURE

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

The links

The parsing results

Rate this:

Share this:

First a big fat warning

Back to the alias

Using source instead of bash

References

Reminders

Rate this:

Share this:

Rate this:

Share this:

Special case `PATH`

Special case `USERNAME`, and collateral `PROCESSOR_ARCHITECTURE`

Using `source` instead of `bash`