The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for February, 2021

showthedocs

Posted by jpluimers on 2021/02/18

[WayBack] showthedocs

is a documentation browser that finds the relevant docs for your code. It works by parsing the code and connecting parts of it to their explanation in the docs

, and supports these languages:

  • SQL
    • postgresql
    • mysql
  • Configuration
    • nginx
    • gitconfig

You can enter any language text, then click the language, followed by clicking the “SHOW ME THE DOCS!” button, for which an example is further below.

The site has an open architecture, allowing to plug in more languages and documentation:

 

gitconfig example

So for instance the below ./git/config file leads to this result [WayBack] where you can click on all the coloured areas for easy navigation through the documentation:

Read the rest of this entry »

Posted in *nix, *nix-tools, Database Development, Development, DVCS - Distributed Version Control, git, MySQL, nginx, PostgreSQL, Power User, Software Development | Leave a Comment »

Delphi: not all lists need to be generic

Posted by jpluimers on 2021/02/18

Lots of Delphi programmers made, or are making the move, of classic Delphi based containers like TObjectList into generic containers like TList<T>.

A while ago, I got into a project that needed to extend lifetime of some objects. Virtually all of them were interface based, and most of the code was from the non-Unicode era, and most of the developers there had a strong background in that era, so they started fiddling with TList, found it hard, then thought “maybe TList<IInterface>” where will help.

The problem however, is that Delphi has no IList<T>. For that, you have to go to the Spring4D library.

Then I sat down with them, and proposed to use an instance good old TInterfacedList of which the context was maintained in an IInterfacedList field.

Back in the days where Delphi did not support non-generic types, TInterfacedList was the only built-in way to store interface references, and the Collection Classes framework by Ray Lischner were the only ways to do that in a more structured way (as they were based on interfaces, an idiom that Embarcadero should have used for their generic collections as well; Spring4D did, so use those collection classes and interfaces whenever possible as they are way more versatile than the Delphi built-in ones)

Back to using TInferfacedList, as it can still be useful today in:

unit InterfacesHolderUnit;

interface

uses
  DebuggableInterfacedObjectUnit, System.Classes;

type
  IInterfacesHolder = interface
    procedure Add(const aReference: IInterface);
  end;

  TInterfacesHolder = class(TInterfacedObject, IInterfacesHolder)
  strict private
    FInterfaces: IInterfaceList;
  public
    constructor Create();
    procedure Add(const aReference: IInterface);
  end;

implementation

{ TInterfacesHolder }

procedure TInterfacesHolder.Add(const aReference: IInterface);
begin
   FInterfaces.Add(aReference);
end;

constructor TInterfacesHolder.Create();
begin
   inherited Create();
   FInterfaces := TInterfaceList.Create();
end;

end.

and some tests:

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | 1 Comment »

How not to do updates of your wiki site

Posted by jpluimers on 2021/02/17

If your company manages your own infrastructure, be sure you have monitoring on all levels.

It saves you from customers discovering issues like this: [WayBack] Thread by @jpluimers: “The @EmbarcaderoTech docwiki is down due to an error in duobook2.[…]”:

The @EmbarcaderoTech docwiki is down due to an error in duobook2. URLs pointing to wiki content fail, no matter the product. Examples for Rio and XE2 grabbed from docwiki.embarcadero.com/Libraries/Rio/… and docwiki.embarcadero.com/Libraries/XE2/…


This is the reason for allowing archival of all your product documentation web-content in the @internetarchive, even for non-current products, as now only parts that have been allowed to save in the past are available.

Apart from nobody noticing the outage yet, which is bad in it’s own way, I hope the cause is not somebody fiddling with duobook (3 year old and unmaintained) without testing the consequences. As that would make the cause of the outage embarrassing.

[WayBack] https://github.com/ElectricVersion/DuoBook

Finally it is rather odd to get a HTTP 200 SUCCESS code on a failure. A HTTP 500 or 503 would be far more appropriate.

I wonder if that is a @mediawiki thing; maybe they could shed some light on that.

References en.wikipedia.org/wiki/List_of_H… and en.wikipedia.org/wiki/List_of_H….

The cool thing is that the stack traces teach you a lot about how a framework is structured.

Related:

  • [Archive.is/WayBack] RAD Studio API Documentation: Rio
    Exception encountered, of type "ArgumentCountError"
    [6a5b64d3a502a9acff148fe1] /Libraries/Rio/en/Main_Page ArgumentCountError from line 420 of /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php: Too few arguments to function DuoBook2Template::displayPrefs(), 0 passed in /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php on line 99 and exactly 1 expected
    Backtrace:
    #0 /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php(99): DuoBook2Template->displayPrefs()
    #1 /var/www/html/shared/BaseWiki27/includes/skins/SkinTemplate.php(248): DuoBook2Template->execute()
    #2 /var/www/html/shared/BaseWiki27/includes/OutputPage.php(2335): SkinTemplate->outputPage()
    #3 /var/www/html/shared/BaseWiki27/includes/MediaWiki.php(743): OutputPage->output()
    #4 /var/www/html/shared/BaseWiki27/includes/MediaWiki.php(509): MediaWiki->main()
    #5 /var/www/html/shared/BaseWiki27/index.php(43): MediaWiki->run()
    #6 {main}
  • [Archive.is/WayBack] XE2 API Documentation
    Exception encountered, of type "ArgumentCountError"
    [d3d353581c3915881b976ab6] /Libraries/XE2/en/Main_Page ArgumentCountError from line 420 of /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php: Too few arguments to function DuoBook2Template::displayPrefs(), 0 passed in /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php on line 99 and exactly 1 expected
    Backtrace:
    #0 /var/www/html/shared/BaseWiki27/skins/DuoBook2/DuoBook2.php(99): DuoBook2Template->displayPrefs()
    #1 /var/www/html/shared/BaseWiki27/includes/skins/SkinTemplate.php(248): DuoBook2Template->execute()
    #2 /var/www/html/shared/BaseWiki27/includes/OutputPage.php(2335): SkinTemplate->outputPage()
    #3 /var/www/html/shared/BaseWiki27/includes/MediaWiki.php(743): OutputPage->output()
    #4 /var/www/html/shared/BaseWiki27/includes/MediaWiki.php(509): MediaWiki->main()
    #5 /var/www/html/shared/BaseWiki27/index.php(43): MediaWiki->run()
    #6 {main}

–jeroen

Read the rest of this entry »

Posted in Development, DevOps, Infrastructure, Power User, Software Development, Web Development | Leave a Comment »

explainshell.com: parse and explain just about any shell command

Posted by jpluimers on 2021/02/17

I bumped into the tremendously site [WayBack] explainshell.com – match command-line arguments to their help text only after documenting the relevant cURL options of yesterdays post on checking your CertBot domain expiration dates.

The site allows put in a shell command-line to see the help text that, including matches for each argument.

It works so well because it parses both the shell command-line and the man pages, then constructs a web-page linking the relevant man page content to the shell command-line in the correct shell command-line order.

The explainshell has a counterpart showthedocs (both are open source) for explaining other languages (on the one hand more extended as it goes much deeper into parsing for instance SQL, on the other hand more limited as it only supports a few languages). More on showthedocs later.

The links

The parsing results

The first bit below is just the text output, and the second bit the screenshot, of a relatively simple command like [WayBack] explainshell.com – curl -fsSL example.org:

curl(1) -fsSL example.org
transfer a URL
-f, --fail
       (HTTP)  Fail  silently  (no  output at all) on server errors. This is mostly done to better enable
       scripts etc to better deal with failed attempts. In normal cases  when  a  HTTP  server  fails  to
       deliver  a  document,  it  returns an HTML document stating so (which often also describes why and
       more). This flag will prevent curl from outputting that and return error 22.

       This method is not fail-safe and there are occasions where non-successful response codes will slip
       through, especially when authentication is involved (response codes 401 and 407).
-s, --silent
       Silent or quiet mode. Don't show progress meter or error messages.  Makes Curl mute.
-S, --show-error
       When used with -s it makes curl show an error message if it fails.
-L, --location
       (HTTP/HTTPS) If the server reports that the requested page  has  moved  to  a  different  location
       (indicated  with  a Location: header and a 3XX response code), this option will make curl redo the
       request on the new place. If used together with -i, --include or  -I,  --head,  headers  from  all
       requested pages will be shown. When authentication is used, curl only sends its credentials to the
       initial host. If a redirect takes curl to a different host, it won't  be  able  to  intercept  the
       user+password.  See  also  --location-trusted  on  how to change this. You can limit the amount of
       redirects to follow by using the --max-redirs option.

       When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will
       do  the  following  request  with a GET if the HTTP response was 301, 302, or 303. If the response
       code was any other 3xx code, curl will re-send the following request  using  the  same  unmodified
       method.
source manpages: curl

The screenshot is even more impressive:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

DUnit testing code that should raise a specific exception

Posted by jpluimers on 2021/02/17

A while back, I was writing some code to demonstrate a few inner workings of TInterfacedObject, interface reference counting and mixing object references with interface references.

One way to show this is through a test case that expects a certain exception to happen, but I forgot how to do that in DUnit. Luckily this pointed me on the right track: [WayBack] delphi – CheckException only accepts 0-parameter methods; how do I test that other methods throw exceptions? – Stack Overflow.

The solution shows that DUnit has had support for something similar as DUnitX: now has a WillRaiseAttribute to ease defining tests around code that should throw exceptions for a very long time (I think this was introduced around Delphi 2005).

You can do it in a property way:

unit InterfacedObjectTestCaseUnit;

interface

uses
  TestFramework;

type
  TDebuggableInterfacedObjectTestCase = class(TTestCase)
  published
    procedure System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
  end;

implementation

uses
  System.SysUtils,

procedure TInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  ExpectedException  := System.SysUtils.EInvalidPointer;
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  // the below is optional; should not be reached. If it is reached, it will fail earlier than the encompassing `RunTest` method would
  ExpectedException := nil; // or `StopExpectingException();`
end;

end.

or in a method way for an exception that happens in the current method:

procedure TInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  StartExpectingException(System.SysUtils.EInvalidPointer);
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  // the below is optional; should not be reached. If it is reached, it will fail earlier than the encompassing `RunTest` method would
  StopExpectingException();
end;

The alternative using CheckException that will raise earlier, but also tests the results of a complete method which also has to be parameterless:

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
end;

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer_TTestMethod_Based();
begin
  CheckException(System_TInterfacedObject_Free_Before_RefCount, System.SysUtils.EInvalidPointer);
end;

So I wrote a class helper based on TProc that allows you to test an anonymous method which usually has more fine grained testing potential.

Because of type compatibility, you have to call the inherited version of CheckException inside the new one:

unit TestCaseHelperUnit;

interface

uses
  System.SysUtils,
  TestFramework;

type
  TTestCaseHelper = class helper for TTestCase
  public
    procedure CheckException(const AProc: TProc; const AExceptionClass: TClass; const msg: string = '');
  end;

implementation

type
  TTestCaseInvoker = class
  strict private
    FProc: TProc;
  public
    constructor Create(const AProc: TProc);
    procedure Execute();
  end;

{ TTestCaseInvoker }

constructor TTestCaseInvoker.Create(const AProc: TProc);
begin
  inherited Create();
  FProc := AProc;
end;

procedure TTestCaseInvoker.Execute();
begin
  if Assigned(FProc) then
    FProc();
end;

procedure TTestCaseHelper.CheckException(const AProc: TProc; const AExceptionClass: TClass; const msg: string = '');
var
  TestCaseInvoker: TTestCaseInvoker;
begin
  TestCaseInvoker := TTestCaseInvoker.Create(AProc);
  try
    inherited CheckException(TestCaseInvoker.Execute, AExceptionClass, msg); // `inherited`, to avoid stack overflow because `TProc` is compatible with `TTestMethod`
  finally
    TestCaseInvoker.Free();
  end;
end;

end.

The test then becomes this:

procedure TDebuggableInterfacedObjectTestCase.System_TInterfacedObject_Free_Before_RefCount_Should_Raise_EInvalidPointer_TProc_Based();
var
  ObjectReference:    System.TInterfacedObject;
  InterfaceReference: IInterface;
begin
  ObjectReference    := System.TInterfacedObject.Create();
  InterfaceReference := ObjectReference;
  CheckException(procedure ()
  begin
    ObjectReference.Free(); // this should raise an exception in System.TInterfacedObject.BeforeDestruction, as it checks the RefCount to be zero
  end,
  System.SysUtils.EInvalidPointer);
end;

DUnit code snippets

Read the rest of this entry »

Posted in Agile, Conference Topics, Conferences, Delphi, Development, Event, Software Development, Unit Testing | Leave a Comment »

🔎Julia Evans🔍 on Twitter: “ngrep: grep your network!… “

Posted by jpluimers on 2021/02/16

[WayBack] 🔎Julia Evans🔍 on Twitter: “ngrep: grep your network!… “

So this taught me a new tool and other new things:

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, Internet protocol suite, Power User, Software Development, Wireshark | Leave a Comment »

Checking expiration dates for your certbot certificates

Posted by jpluimers on 2021/02/16

I have these two little aliases in my toolbox:

alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

First a big fat warning

do not run just any script downloaded through curl. Plenty of reasons why this is dangerous:

  1. [WayBack] Detecting the use of “curl | bash” server side | Application Security
  2. [WayBack] One way “curl pipe sh” install scripts can be dangerous [proof of concept] / Jordan Eldredge: script content differs depending on user agent
  3. [WayBack] sean cassidy : Don’t Pipe to your Shell: scripts having different behaviour when executed partially
  4. [WayBack] Why using curl | sudo sh is not advised? – Stack Overflow:

    You can proof your scripts against partial execution by putting the whole thing into the body of a function, and executing that function on the last line. If you know a script is defined like that, it’s exactly as secure as downloading and then executing some installer.

The first three can mostly prevented by using your own fork of the script repository, then checking each modification of the script, combined with ensuring your fork location does not throw tricks 1 or 2 on you.

That’s why I run the above alias only from a checkssl fork which I can inspect.

Back to the alias

The aliases use quite a few tricks:

  1. Having curl download a command minimising
    curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl

    Via: [WayBack] The missing package manager for macOS (or Linux) — The missing package manager for macOS (or Linux)

  2. Running that command through bash as if the download were a file by wrapping wrapping it in parenthesis and a less than sign .
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl)

    Via: [WayBack] linux – Execute bash script from URL – Stack Overflow

  3. Passing arguments to the bash command by appending this to the command just like from the regular command-line:
    bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live
  4. An alias [WayBack] with double-quotes around the whole statement:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
  5. Either use the certbot configuration file or apache2 (via ISPconfig as the apache2 parameter value is not yet supported) domain configuration:
    alias "certbot-check-all-by-file=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live"
    alias "certbot-check-all-by-config=bash <(curl -fsSL https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --server ISPconfig"

Using source instead of bash

Note that an alternative alias is this one:

alias "certbot-check-all-by-file=(source <(curl -s https://raw.githubusercontent.com/srvrco/checkssl/master/checkssl) --location /etc/letsencrypt/live)"

However, that needs an extra set of parenthesis, otherwise you will get bumped out of your current shell.

The reason is that bash runs in a [WayBack] subshell, whereas [WayBack] source (and the equivalent [WayBack] “dot” command .) runs in the current shell, but the script performs a gracefull_exit or error_exit which end in an exit [WayBack] terminating the current shell.

The [WayBack] () parenthesis around the source command ensure it runs in a sub-shell.

In this case, you can still pass the --location /etc/letsencrypt/live parameters, as bash is the only shell allowing this: [WayBack] source – Passing variables to a bash script when sourcing it – Unix & Linux Stack Exchange.

References

Related [WayBack] Advanced Bash-Scripting Guide topics:

Related cURL options from [WayBack] curl – How To Use:

  • -f: [WayBack] -f, --fail

    (HTTP) Fail silently (no output at all) on server errors. This is mostly done to better enable scripts etc to better deal with failed attempts. In normal cases when an HTTP server fails to deliver a document, it returns an HTML document stating so (which often also describes why and more). This flag will prevent curl from outputting that and return error 22.

    This method is not fail-safe and there are occasions where non-successful response codes will slip through, especially when authentication is involved (response codes 401 and 407).

  • -s: [WayBack] -s, --silent:

    Silent or quiet mode. Don’t show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.

    Use -S, –show-error in addition to this option to disable progress meter but still show error messages.

    See also -v, –verbose and –stderr.

  • -S: [WayBack] -S, --show-errors:

    When used with -s, –silent, it makes curl show an error message if it fails.

  • -L: [WayBack] -L, --location:

    (HTTP) If the server reports that the requested page has moved to a different location (indicated with a Location: header and a 3XX response code), this option will make curl redo the request on the new place. If used together with -i, –include or -I, –head, headers from all requested pages will be shown. When authentication is used, curl only sends its credentials to the initial host. If a redirect takes curl to a different host, it won’t be able to intercept the user+password. See also –location-trusted on how to change this. You can limit the amount of redirects to follow by using the –max-redirs option.

    When curl follows a redirect and the request is not a plain GET (for example POST or PUT), it will do the following request with a GET if the HTTP response was 301, 302, or 303. If the response code was any other 3xx code, curl will re-send the following request using the same unmodified method.

    You can tell curl to not change the non-GET request method to GET after a 30x response by using the dedicated options for that: –post301–post302 and –post303.

Reminders

Reminder to self: see if JSON output is viable. This commit might help.

–jeroen

Posted in bash, Conference Topics, Conferences, Development, Event, Scripting, Software Development | Leave a Comment »

The magic “procedure Touch(var …” construct

Posted by jpluimers on 2021/02/16

procedure Touch(var Argument);  
begin
end;

I included the above code in my blog a long time ago (2014 in fact: Delphi: always watch the compiler Warnings), but never got around to explain the why and how I got it, why it works and why it will likely work forever.

Background

Ever since the early Delphi days, there are three hints about “never used” of which the second often gets in the way during debugging:

(note that these %s only hold for non-managed types, which I also addressed in Why don’t I get the warning W1036 Variable “‘MyStrings’ might not have been initialized”… and Delphi 10.3 Rio got released; I’ll wait a while hoping to see more positive comments).

Usually the compiler is right, but sometimes it is not: [WayBack] Check your compiler warnings and hints. They may still be errors. | Shiftkey Software Blog

So once every while, you need this workaround:

Solution

The solution is to have a method with one untyped var parameter (a var parameter without any type: this way you can pass any field or variable to it) that just does nothing. Often I included only at the place I need it as this single line fragment: procedure Touch(var Argument); begin end;.

Former Delphi compiler engineer and Borland Chief Schientist Danny Thorpe handed this solution, I think it was during or around his famous BorCon99 in Philadelphi (and later BorCon2005 in San Jose) Reading Tea Leaves: The Fine Art of Debugging talk. The talk is not-online, but luckily there are notes and a StackOverflow post:

The session had seemingly simple things like this [WayBack] Shenoy At Work: Set Next Statement in Delphi? with the picture on the right.

Voiding the solution

I’ve seen teams making this method inline, but that voids it. Usually they do not see it as they already resolved the “never used” problem in another way.

Why it still works

Read the rest of this entry »

Posted in Conference Topics, Conferences, Delphi, Development, Event, Software Development | 1 Comment »

Eurotec 6-6 uit bedrijf nemen

Posted by jpluimers on 2021/02/15

Even wat linkjes en notities, want Eurotec 6-6 wordt al tijden niet meer gemaakt ([WayBack] Euromatec maakt andere alarmsystemen) en steeds meer worden vervangen door andere alarmapparatuur.

–jeroen

Posted in LifeHacker, Power User | Leave a Comment »

Research list: getting rid of the Windows 10 Delivery Content data and service

Posted by jpluimers on 2021/02/15

Not sure yet if this is still possible, but on my research list as it pollutes low-resource Windows 10 VMs and computers the Delivery Content:

–jeroen

Read the rest of this entry »

Posted in Power User, Windows, Windows 10 | Leave a Comment »