The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Scripting’ Category

How to Update All Your Ruby Gems At Once | Life, the Universe, and Everything

Posted by jpluimers on 2019/08/26

This looks smart

gem update `gem list | cut -d ' ' -f 1`

From: [WayBack] How to Update All Your Ruby Gems At Once | Life, the Universe, and Everything

Though on the bash prompt, it works fine on Mac OS X / OS X / macOS / …, it does not work nice as an alias.

You can get it to work with difficult escaping (or nesting).

But it is easier to escape this:

gem update $(gem list | cut -d ' ' -f 1)

Escaped, it comes down to:

alias "gem-update-all=gem update \$(gem list | cut -d ' ' -f 1)"

Based on:

–jeroen

Posted in Apple, bash, bash, Development, Mac OS X / OS X / MacOS, Power User, Scripting, Software Development | Leave a Comment »

Python line continuation: only use backslash if it gives cleaner code

Posted by jpluimers on 2019/08/20

Since Python is a [WayBack] line-oriented programming language, sometimes you want to wrap longer lines into more readable shorter ones.

Many people struggle with this, see for instance these questions (and excellent answers!):

This struggle is likely why it made it to the [WayBack] style guide. Relevant sections are below.

I had this struggle wile passing multiple parameters to a method creating a very long line, but found I did not need a line continuation as the Python language understands this construct perfectly fine:

    threadManager.append(
        UrlMonitorThread(monitor, "http://%s" % targetHost),
        SmtpMonitorThread(monitor, targetHost, 25),
        SmtpMonitorThread(monitor, targetHost, 587),
        SshMonitorThread(monitor, targetHost, 22))

You could use the line continuation backslash to do this, but often that is not needed or a better way exists (for instance wrapping an expression in parentheses), so here are are the relevant style guide sections:

Code lay-out

Indentation

Use 4 spaces per indentation level.

Continuation lines should align wrapped elements either vertically using Python’s implicit line joining inside parentheses, brackets and braces, or using a hanging indent[7]. When using a hanging indent the following should be considered; there should be no arguments on the first line and further indentation should be used to clearly distinguish itself as a continuation line.

Maximum Line Length

Limit all lines to a maximum of 79 characters.

For flowing long blocks of text with fewer structural restrictions (docstrings or comments), the line length should be limited to 72 characters.

Limiting the required editor window width makes it possible to have several files open side-by-side, and works well when using code review tools that present the two versions in adjacent columns.

The preferred way of wrapping long lines is by using Python’s implied line continuation inside parentheses, brackets and braces. Long lines can be broken over multiple lines by wrapping expressions in parentheses. These should be used in preference to using a backslash for line continuation.

Backslashes may still be appropriate at times. For example, long, multiple with-statements cannot use implicit continuation, so backslashes are acceptable:

with open('/path/to/some/file/you/want/to/read') as file_1, \
     open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

Make sure to indent the continued line appropriately.

Should a line break before or after a binary operator?

For decades the recommended style was to break after binary operators. But this can hurt readability in two ways: the operators tend to get scattered across different columns on the screen, and each operator is moved away from its operand and onto the previous line. Here, the eye has to do extra work to tell which items are added and which are subtracted:


To solve this readability problem, mathematicians and their publishers follow the opposite convention. Donald Knuth explains the traditional rule in his Computers and Typesetting series: “Although formulas within a paragraph always break after binary operations and relations, displayed formulas always break before binary operations” [3].

Following the tradition from mathematics usually results in more readable code:

# Yes: easy to match operators with operands
income = (gross_wages
          + taxable_interest
          + (dividends - qualified_dividends)
          - ira_deduction
          - student_loan_interest)

In Python code, it is permissible to break before or after a binary operator, as long as the convention is consistent locally. For new code Knuth’s style is suggested.

–jeroen

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

… compare two JSON structures and pin-point … the differences – – Nicholas Ring – Google+

Posted by jpluimers on 2019/08/20

I’ve added a few WayBack/Archive.is links to the interesting comments by Zoë Peterson from Scooter Software (of Beyond Compare fame) at [WayBack] … compare two JSON structures and pin-point … the differences – – Nicholas Ring – Google+:

Beyond Compare 4 has an optional “JSON sorted” file format that uses jq to pretty print and sort JSON data before comparing it. It’s not included out of the box yet, but you can get a copy here:

If you’re interested in an actual algorithm and not just an app, I don’t have a suggestion handy, but could dig one up. Tree alignment is more complicated than sequence alignment and we did do research into it, but it was quite a few years ago and didn’t get incorporated into BC. XML alignment algorithms were being actively researched back in the aughts and they should trivially transfer to JSON.

It looks like our research mostly ended around 2002, and I wasn’t personally involved in it, so I don’t know how helpful this will be, but here’s what I have:

The general idea in the thread is that JSON – though not as formalised as XML – does have structure, so if you can normalise it, then XML ways of differencing should work.

Normalisation also means that you need to normalise any floating point, date time, escaping, quoting, etc. Maybe not for the faint of heart.

–jeroen

Posted in *nix, *nix-tools, Beyond Compare, Development, diff, JavaScript/ECMAScript, jq, JSON, Power User, Scripting, Software Development, XML, XML/XSD | Leave a Comment »

When Powershell function won’t work: you define them with commas and parentheses, but call them with spaces and no parentheses

Posted by jpluimers on 2019/08/15

The function or command was called as if it were a method. 
Parameters should be separated by spaces. For information about 
parameters, see the about_Parameters Help topic.

Every now and then I bump into the above error. The reason is this:

  1. Functions are defined with commas between parameters and parentheses around them
  2. One-parameter functions can be called with one parameter surrounded by parentheses
  3. Multi-parameter functions need to be called with spaces between parameters and no parentheses surrounding them

Confused? #MeToo

The problem: [WayBackabout_Parameters_Default_Values | Microsoft Docs

Based on [WayBack] Powershell function won’t work.

–jeroen

Posted in CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

PowerShell: be careful using `-ReadCount` on `Get-Content`

Posted by jpluimers on 2019/08/14

I learned this the hard way: [WayBackDifferent result when using -ReadCount with Get-Content: because -ReadCount delivers data in chunks, the filter after [WayBack] Get-Content (Microsoft.PowerShell.Management) it will only filter on those chunks. If the filter isn’t prepared for that, it might only filter the last chunk.

So do not use for instance [WayBack] Select-String (Microsoft.PowerShell.Utility) on it, but perform your own [WayBack] ForEach-Object (Microsoft.PowerShell.Core) aliased as foreach like in [WayBack] Get all lines containing a string in a huge text file – as fast as possible?:

Get-Content myfile.txt -ReadCount 1000 |
  foreach { $_ -match "my_string" }

A more elaborate example is at [WayBack] How can I make this PowerShell script parse large files faster?.

–jeroen

Posted in CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

What is ‘if __name__ == “__main__”‘ for?

Posted by jpluimers on 2019/08/14

One of the things when I learned Python was that in some scripts I found a block starting with a statement like this:

if __name__ == '__main__':

It looked like an idiom to me, and indeed it is: [WayBack] What is ‘if name == “main“‘ for?.

It allows a file to be both used as “main” standalone program file and a module. That section of code will not be executed if it is loaded as a module.

Part of the idiom is also to put your code in a separate method so this block is as short as possible.

if __name__ == '__main__':
main()

Via: [WayBack] Why is Python running my module when I import it, and how do I stop it? (thanks user166390 and Jeremy Banks for the answers there)

–jeroen

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

Powershell: what kind of data type is [string[]] and when would you use it?

Posted by jpluimers on 2019/08/13

[WayBackIn Powershell, what kind of data type is [string[]] and when would you use it? (thanks cignul9 and arco444!): basically it forces an array of string.

It defines an array of strings. Consider the following ways of initialising an array:

[PS] > [string[]]$s1 = "foo","bar","one","two",3,4
[PS] > $s2 = "foo","bar","one","two",3,4

[PS] > $s1.gettype()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String[]                                 System.Array

[PS] > $s2.gettype()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Object[]                                 System.Array

By default, a powershell array is an array of objects that it will cast to a particular type if necessary. Look at how it’s decided what types the 5th element of each of these are:

[PS] > $s1[4].gettype()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     String                                   System.Object


[PS] > $s2[4].gettype()

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     True     Int32                                    System.ValueType


[PS] > $s1[4]
3
[PS] > $s2[4]
3

The use of [string[]] when creating $s1 has meant that a raw 3 passed to the array has been converted to a String type in contrast to an Int32 when stored in an Object array.

–jeroen

Posted in CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

When an installer errors out with “Please re-run this installer as a normal user instead of”…

Posted by jpluimers on 2019/08/12

Via [WayBack] Anyone with a hint on how to work around this: … “Please re-run this installer as a normal user instead of”… – Jeroen Wiert Pluimers – Google+

This happened for instance when trying to install Source Tree 2.x on Windows (1.9.x works fine):

[Window Title]
SourceTreeSetup-2.3.1.0.exe

[Main Instruction]
Installation has failed

[Content]
Please re-run this installer as a normal user instead of “Run as Administrator”.

[Close]

The problem was by accident the machine got in a state to run commands without UAC approval, so the run dialog would already look have “This task will be created with administrative privileges”:

It was odd, as the machine didn’t have it enabled in the security policy (secpo.msc):

So I did a bit more digging, bumped into [WayBack] Why does my Run dialog say that tasks will created with administrative privileges? – The Old New Thing and had one of those #facepalm moments: Explorer had crashed, and I had started it from Process Explorer, forgetting Process Explorer had an UAC token.

The solution is easy:

  1. Logoff / Logon
  2. Verify the Windows-R shows a “normal” run:

Then you can just run the installer:

–jeroen

Posted in Batch-Files, Console (command prompt window), Development, Power User, Scripting, Software Development, The Old New Thing, Windows, Windows Development | Leave a Comment »

The 14 JavaScript debugging tips you probably didn’t know | Raygun

Posted by jpluimers on 2019/08/08

For my link archive: [WayBackThe 14 JavaScript debugging tips you probably didn’t know | Raygun

–jeroen

Posted in Development, JavaScript/ECMAScript, Scripting, Software Development | Leave a Comment »

.NET and PowerShell: Getting proper version info from a PE file like EXE, DLL, assembly

Posted by jpluimers on 2019/08/01

I’ve learned the hard way that both .NET and PowerShell version information isn’t always accurate or usable for two reasons which I later found in various other blog and forum posts:

The easiest is to use these numbers to create a [WayBack] Version Class (System) instance using the [WayBack] Version Constructor (Int32, Int32, Int32, Int32) constructor. This has the added benefit that you directly compare versions with each other.

Sometimes it makes even sense to take the highest version from Product and File.

In PowerShell, this is the way to do that, assuming $imagePath points to a [WayBack] Portable Executable:

try {
  $VersionInfo = (Get-Item $imagePath).VersionInfo
  $FileVersion = [version]("{0}.{1}.{2}.{3}" -f $VersionInfo.FileMajorPart, $VersionInfo.FileMinorPart, $VersionInfo.FileBuildPart, $VersionInfo.FilePrivatePart)
  $ProductVersion = [version]("{0}.{1}.{2}.{3}" -f $VersionInfo.ProductMajorPart, $VersionInfo.ProductMinorPart, $VersionInfo.ProductBuildPart, $VersionInfo.ProductPrivatePart)
  $ActualVersion = $(if ($ProductVersion -gt $FileVersion) { $ProductVersion } else { $FileVersion })
}
catch {
  $ActualVersion = [version]("0.0.0.0")
}

Background information:

–jeroen

Posted in CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »