The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Scripting’ Category

« Previous Entries
Next Entries »

explainshell.com: parse and explain just about any shell command

Posted by jpluimers on 2021/02/17

I bumped into the tremendously site [WayBack] explainshell.com – match command-line arguments to their help text only after documenting the relevant cURL options of yesterdays post on checking your CertBot domain expiration dates.

The site allows put in a shell command-line to see the help text that, including matches for each argument.

It works so well because it parses both the shell command-line and the man pages, then constructs a web-page linking the relevant man page content to the shell command-line in the correct shell command-line order.

The explainshell has a counterpart showthedocs (both are open source) for explaining other languages (on the one hand more extended as it goes much deeper into parsing for instance SQL, on the other hand more limited as it only supports a few languages). More on showthedocs later.

The links

The parsing results

The first bit below is just the text output, and the second bit the screenshot, of a relatively simple command like [WayBack] explainshell.com – curl -fsSL example.org:

curl(1) -fsSL example.org
transfer a URL
 
-f, --fail
       (HTTP)  Fail  silently  (no  output at all) on server errors. This is mostly done to better enable
       scripts etc to better deal with failed attempts. In normal cases  when  a  HTTP  server  fails  to
       deliver  a  document,  it  returns an HTML document stating so (which often also describes why and
       more). This flag will prevent curl from outputting that and return error 22.

       This method is not fail-safe and there are occasions where non-successful response codes will slip
       through, especially when authentication is involved (response codes 401 and 407).
 
-s, --silent
       Silent or quiet mode. Don't show progress meter or error messages.  Makes Curl mute.
 
-S, --show-error
       When used with -s it makes curl show an error message if it fails.
 
-L, --location
       (HTTP/HTTPS) If the server reports that the requested page  has  moved  to  a  different  location
       (indicated  with  a Location: header and a 3XX response code), this option will make curl redo the
       request on the new place. If used together with -i, --include or  -I,  --head,  headers  from  all
       requested pages will be shown. When authentication is used, curl only sends its credentials to the
       initial host. If a redirect takes curl to a different host, it won't  be  able  to  intercept  the
       user+password.  See  also  --location-trusted  on  how to change this. You can limit the amount of
       redirects to follow by using the --max-redirs option.

       When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will
       do  the  following  request  with a GET if the HTTP response was 301, 302, or 303. If the response
       code was any other 3xx code, curl will re-send the following request  using  the  same  unmodified
       method.
source manpages: curl

The screenshot is even more impressive:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

Checking expiration dates for your certbot certificates

Posted by jpluimers on 2021/02/16

I have these two little aliases in my toolbox:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

First a big fat warning

do not run just any script downloaded through curl. Plenty of reasons why this is dangerous:

  1. [WayBack] Detecting the use of “curl | bash” server side | Application Security
  2. [WayBack] One way “curl pipe sh” install scripts can be dangerous [proof of concept] / Jordan Eldredge: script content differs depending on user agent
  3. [WayBack] sean cassidy : Don’t Pipe to your Shell: scripts having different behaviour when executed partially
  4. [WayBack] Why using curl | sudo sh is not advised? – Stack Overflow:

    You can proof your scripts against partial execution by putting the whole thing into the body of a function, and executing that function on the last line. If you know a script is defined like that, it’s exactly as secure as downloading and then executing some installer.

The first three can mostly prevented by using your own fork of the script repository, then checking each modification of the script, combined with ensuring your fork location does not throw tricks 1 or 2 on you.

That’s why I run the above alias only from a checkssl fork which I can inspect.

Back to the alias

The aliases use quite a few tricks:

  1. Having curl download a command minimising
    curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl

    Via: [WayBack] The missing package manager for macOS (or Linux) — The missing package manager for macOS (or Linux)

  2. Running that command through bash as if the download were a file by wrapping wrapping it in parenthesis and a less than sign .
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl)

    Via: [WayBack] linux – Execute bash script from URL – Stack Overflow

  3. Passing arguments to the bash command by appending this to the command just like from the regular command-line:
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live
  4. An alias [WayBack] with double-quotes around the whole statement:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
  5. Either use the certbot configuration file or apache2 (via ISPconfig as the apache2 parameter value is not yet supported) domain configuration:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

Using source instead of bash

Note that an alternative alias is this one:

alias "certbot-check-all-by-file=(source <(curl -s https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live)"

However, that needs an extra set of parenthesis, otherwise you will get bumped out of your current shell.

The reason is that bash runs in a [WayBack] subshell, whereas [WayBack] source (and the equivalent [WayBack] “dot” command .) runs in the current shell, but the script performs a gracefull_exit or error_exit which end in an exit [WayBack] terminating the current shell.

The [WayBack] () parenthesis around the source command ensure it runs in a sub-shell.

In this case, you can still pass the --location /etc/letsencrypt/live parameters, as bash is the only shell allowing this: [WayBack] source – Passing variables to a bash script when sourcing it – Unix & Linux Stack Exchange.

References

Related [WayBack] Advanced Bash-Scripting Guide topics:

Related cURL options from [WayBack] curl – How To Use:

  • -f: [WayBack] -f, --fail

    (HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

    This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).

  • -s: [WayBack] -s, --silent:

    Silent or quiet mode. Don’t show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

    Use -S, –show-error in addition to this option to disable progress meter but still show error messages.

    See also -v, –verbose and –stderr.

  • -S: [WayBack] -S, --show-errors:

    When used with -s, –silent, it makes curl show an error message if it fails.

  • -L: [WayBack] -L, --location:

    (HTTP) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place. If used together with -i, –include or -I, –head, headers from all requested pages will be shown. When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won’t be able to intercept the user+password. See also –location-trusted on how to change this. You can limit the amount of redirects to follow by using the –max-redirs option.

    When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.

    You can tell curl to not change the non-GET request method to GET after a 30x response by using the dedicated options for that: –post301, –post302 and –post303.

Reminders

Reminder to self: see if JSON output is viable. This commit might help.

–jeroen

Posted in bash, Conference Topics, Conferences, Development, Event, Scripting, Software Development | Leave a Comment »

How do I pretty-print JSON in Delphi? – Stack Overflow

Posted by jpluimers on 2021/02/09

For my archive: the [WayBack] How do I pretty-print JSON in Delphi? – Stack Overflow answer by [WayBack] Bob:

If you do not want to use any external library, and you’re using a Delphi XE5 or newer, there is a very handy TJson.Format() function in the REST.Json unit.

uses json, REST.Json;

{ ... }    

function FormatJSON(json: String): String;
var
  tmpJson: TJsonObject;
begin
  tmpJson := TJSONObject.ParseJSONValue(json);
  Result := TJson.Format(tmpJson);

  FreeAndNil(tmpJson);
end;

–jeroen

Posted in Delphi, Development, JavaScript/ECMAScript, JSON, Scripting, Software Development | 2 Comments »

Forced routing of selective emails to ISP SMTP via Mikrotik Routing | Syed Jahanzaib Personal Blog to Share Knowledge !

Posted by jpluimers on 2021/01/14

For my link archive: [WayBack] Forced routing of selective emails to ISP SMTP via Mikrotik Routing | Syed Jahanzaib Personal Blog to Share Knowledge !

–jeroen

Posted in Development, MikroTik, Power User, RouterOS, routers, Scripting, Software Development | Leave a Comment »

Automating the closing of the Creative Cloud signing and ABBY FindReader for ScanSnap 5.0 dialogs

Posted by jpluimers on 2021/01/06

Every time my scan VM logs on I get the dialog on the right.

Every time I finish an OCR scan, I get the dialog below.

There are two reasons I want to close the ABBY dialog:

  1. While open, it will keep both the original PDF and OCR PDF files alive.When after a while, Windows updates auto-reboots the machine, before clicking the OK buttons I have to manually check if the conversion succeeded before removing the non-OCR PDF.This is time consuming.
  2. While open, it still consumes a lot of system resources: about 100 megabyte for a simple single monochrome A4 page. Much more for complex, multi-page or colour documents.When scanning a lot of document this causes the system to run out of memory, after becoming much much slower because the truckload of Window handles and underlying threads drags Windows down.

I do not want to fully get rid of these dialogs, as often being aware of the progress is important, and I always forget how to re-enable things. If you can do without the dialogs, then try these:

Finding the Windows and controls

I did use one nice feature of AutoHotKey: their Windows Spy utility, which is implemented as a AHK script: [WayBack] AutoHotKey-scripts/WindowSpy.ahk at master · elig0n/AutoHotKey-scripts · GitHub. In the past this was a separate executable, so do not start looking for that any more. You can get it either after a full install of the [WayBack] Releases · Lexikos/AutoHotkey_L · GitHub, or by extracting from the most current AutoHotKey.zip from [Archive.is] AutoHotkey Downloads.

Related:

This gets these for the Create Cloud and ABBY windows:

Automating the click

I contemplated about using AutoIt (freeware, but closed source) or AutoHotKey_L (the current active fork of AutoHotKey).

AutoIt is now closed source, forked in the past as AutoHotKey, which has a lot of half backed – usually poorly documented – scripts needing you to learn a new API wrapper around existing Windows API functionality.

So I reverted back to using the Windows API using Delphi: a simple repeat loop, to check for the existence of the underlying processes, windows and controls, plus some logic to terminate then the user stops the application (Ctrl-C, Ctrl-Break), logs off, or Windows shuts down.

Releated Windows API  keywords and posts:

 

I could have used AutoHotKey with these hints to get it working:

MacOS

Note that when you run on MacOS, you need an alternative like for instance the video below shows via [WayBack] Stop ScanSnap From Prompting You When You Scan.

–jeroen

Read the rest of this entry »

Posted in Development, Fujitsu ScanSnap, Hardware, ix100, ix500, Power User, Scanners, Scripting, Software Development, Windows, Windows 10, Windows 8.1 | Leave a Comment »

Batch files: deleting first/middle/ending parts of environment variables

Posted by jpluimers on 2021/01/06

Batch files are often a pain to write, but you cannot always rewrite them in PowerShell.

The pain below is about deleting parts of environment variables in batch files.

I’ll just redirect to and quote from posts that can way better describe this than I do:

  • [WayBack] Check if Batch variable starts with “…” – Stack Overflow made me find
  • [WayBack] windows – Batch – Delete Characters in a String – Super User
  • [WayBack] CMD Variable edit replace – Windows CMD – SS64.com

    The variable _test containing 12345abcabc is used for all the following examples:

    ::Replace '12345' with 'Hello '
   SET _test=12345abcabc
   SET _result=%_test:12345=Hello %
   ECHO %_result%          =Hello abcabc

::Replace the character string 'ab' with 'xy'
   SET _test=12345abcabc
   SET _result=%_test:ab=xy%
   ECHO %_result%          =12345xycxyc

::Delete the character string 'ab'
   SET _test=12345abcabc
   SET _result=%_test:ab=%
   ECHO %_result%          =12345cc

::Delete the character string 'ab' and everything before it
   SET _test=12345abcabc
   SET _result=%_test:*ab=% 
   ECHO %_result%          =cabc

::Replace the character string 'ab' and everything before it with 'XY'
   SET _test=12345abcabc
   SET _result=%_test:*ab=XY% 
   ECHO %_result%          =XYcabc


:: To remove characters from the right hand side of a string is 
:: a two step process and requires the use of a CALL statement
:: e.g.

   SET _test=The quick brown fox jumps over the lazy dog

   :: To delete everything after the string 'brown'  
   :: first delete 'brown' and everything before it
   SET _endbit=%_test:*brown=%
   Echo We dont want: [%_endbit%]

   ::Now remove this from the original string
   CALL SET _result=%%_test:%_endbit%=%%
   echo %_result%

    All the examples on this page assume the default Expansion of variables, if you are using DelayedExpansion then you can choose to change the variable references to !_variable! instead of %_variable%

    One advantage of DelayedExpansion is that it will allow you to replace the % character, it will still have to be escaped as %% but the replace action will then treat it like any other character:

    Replace the letter P with a percent symbol:
    Setlocal EnableDelayedExpansion
    _demo=somePdemoPtextP
    _demo=!_demo:P=%%!

    Remove spaces from a text string

    To delete space characters use the same syntax as above:

    SET _no_spaces=%_some_var: =%

–jeroen

Posted in Batch-Files, Development, Scripting, Software Development | Leave a Comment »

Batch file: check for (non-)existence of registry key

Posted by jpluimers on 2021/01/05

Small batch file that only deletes a registry key if it exists:

:DeleteKeyIfItExists
reg query %1 >nul 2>&1
if %errorlevel% equ 0 reg delete %1 /f
goto :eof

It is based on:

  • redirecting both stderr and stdout to nul (the >nul 2>&1 bit)
  • checking reg query with the appropriate errorlevel value for equality (equ operator) for 0 (existence); you can also use 1 for non-existence.

Based on:

–jeroen

Posted in Batch-Files, Development, Scripting, Software Development | Leave a Comment »

Windows command prompt: decrementing loop

Posted by jpluimers on 2020/12/30

I needed a decrementing loop on the Windows command prompt, but that seems very hard from batch files without programming your own kind of while loop:

PowerShell to the rescue to loop back from and including 463 down to and including 290:

PowerShell -Command "for ($i=463; $i -ge 290; $i--) { Write-Host "Value " $i}"

This outputs:

Value 463
Value 462
...
Value 291
Value 290

In a similar way, you can execute a cmd command, but then you need to be careful on how to pass parameters: the \" is important to you can have quotes within quoted strings..

PowerShell -Command "for ($i=463; $i -ge 290; $i--) { & echo \"Value $i\"}"

gives this:

Value 463
Value 462
...
Value 291
Value 290

Read the rest of this entry »

Posted in Batch-Files, CommandLine, Console (command prompt window), Development, PowerShell, PowerShell, Scripting, Software Development, Windows | 1 Comment »

Spinettaro’s Blog: Delphi Flux application architecture

Posted by jpluimers on 2020/12/16

On my list of things to try: [WayBack] Spinettaro’s Blog: Delphi Flux application architecture.

It is about a Delphi implementation of the Facebook Flux application architecture.

Related:

Via: [WayBack] Delphi Flux application architecture A good application architecture Finding a good application architecture is not easy, but defining an architecture f… – Daniele Spinetti – Google+

Note that by using the [WayBack] CodeRage 2018 Replay | Embarcadero Academy, you are disallowed using any of what you learn in a commercial way.

Embarcadero evangelists told the public that for the Community Edition, similar terms would not be upheld, but then the sales department started sending out nasty emails to people registering Community Edition using their work email address.

[WayBack] https://www.embarcaderoacademy.com/p/terms:

under this license you may not:

  1. modify or copy the materials;
  2. use the materials for any commercial purpose, or for any public display (commercial or non-commercial);
  3. attempt to decompile or reverse engineer any software contained on the School’s web site;
  4. remove any copyright or other proprietary notations from the materials; or
  5. transfer the materials to another person or ‘mirror’ the materials on any other server.

–jeroen

Read the rest of this entry »

Posted in Delphi, Development, JavaScript/ECMAScript, Scripting, Software Development, Systems Architecture, TypeScript | Leave a Comment »

On Windows 7 and 8.x too: Completely disable Windows 10 telemetry collection – twm’s blog

Posted by jpluimers on 2020/12/10

From [WayBack] Completely disable Windows 10 telemetry collection – twm’s blog:

So I don’t forget: According to an article in c’t magazine, disabling the “DiagTrack” service (“Connected User Experience and Telemetry”) will completely disable user tracking in Windows 10. They also say that they did not see any negative effects.

Source: [WayBack] Telefonierverbot in c’t 01/2019 page 172 (in German)

I saw at least one system where the service is not shown when you run Services.msc: it did not list DiagTrack, nor Connected User Experience and Telemetry. How awful is that!

The service can also be installed non older Windows versions: [WayBack] Just found DiagTrack running in Services – Tips and Tricks

Sometimes, it gets re-enabled. I think this happens during major Windows updates.

To inspect, stop and disable

Run all commands from the console the below bold commands. The non-bold text was the output on my system. If instead of the cmd.exe console, you run a PowerShell console, then remove the bits PowerShell -Command " and " at the start and end of each command.

The first command does not require an Administrative (UAC Elevated) command prompt; the last one does.

However, the first command, needs the | Select-Object * bit as otherwise most of the fields will not be displayed, excluding for instance StartType.

powershell -Command "Get-Service -Name DiagTrack | Select-Object *"


Name                : DiagTrack
RequiredServices    : {RpcSs}
CanPauseAndContinue : False
CanShutdown         : True
CanStop             : True
DisplayName         : Connected User Experiences and Telemetry
DependentServices   : {}
MachineName         : .
ServiceName         : DiagTrack
ServicesDependedOn  : {RpcSs}
ServiceHandle       :
Status              : Running
ServiceType         : Win32OwnProcess
StartType           : Automatic
Site                :
Container           :

On an Administrative command-prompt:

powershell -Command "Set-Service -Name DiagTrack -StartUpType Disabled"
powershell -Command "Get-Service -Name DiagTrack | Stop-Service"

Two notes:

Read the rest of this entry »

Posted in Batch-Files, CommandLine, Development, Power User, PowerShell, PowerShell, Scripting, Software Development, Windows | Leave a Comment »

« Previous Entries
Next Entries »