The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Scripting’ Category

PowerShell error in a script but not on the console: The string is missing the terminator: “.

Posted by jpluimers on 2021/09/29

The below one will fail in a script, both both work from the PowerShell prompt:

Success

Get-NetFirewallRule -DisplayGroup "File and Printer Sharing" | ForEach-Object { Write-Host $_.DisplayName ; Get-NetFirewallAddressFilter -AssociatedNetFirewallRule $_ }

Failure

Get-NetFirewallRule –DisplayGroup "File and Printer Sharing" | ForEach-Object { Write-Host $_.DisplayName ; Get-NetFirewallAddressFilter -AssociatedNetFirewallRule $_ }

The error you get this this:

At C:\bin\Show-File-and-Printer-Sharing-firewall-rules.ps1:5 char:52
+ ... -TCP-NoScope" | ForEach-Object { Write-Host $_.DisplayName ; Get-NetF ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The string is missing the terminator: ".
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : TerminatorExpectedAtEndOfString

Via [WayBack] script file ‘The string is missing the terminator: “.’ – Google Search, I quickly found these that stood out:

Cause and solution

Before DisplayGroup, the first line has a minus sign and the second an en-dash. You can see this via [WayBack] What Unicode character is this ?.

Apparently, when using Unicode on the console, it does not matter if you have a minus sign (-), en-dash (–), em-dash (—) or horizontal bar (―) as dash character. You can see this in [WayBack] tokenizer.cs at function [WayBack] NextToken and [WayBack] CharTraits.cs at function [WayBack] IsChar).

When saving to a non-Unicode file, it does matter, even though it does not display as garbage in the error message.

Similarly, PowerShell has support for these special characters:

    internal static class SpecialChars
    {
        // Uncommon whitespace
        internal const char NoBreakSpace = (char)0x00a0;
        internal const char NextLine = (char)0x0085;

        // Special dashes
        internal const char EnDash = (char)0x2013;
        internal const char EmDash = (char)0x2014;
        internal const char HorizontalBar = (char)0x2015;

        // Special quotes
        internal const char QuoteSingleLeft = (char)0x2018; // left single quotation mark
        internal const char QuoteSingleRight = (char)0x2019; // right single quotation mark
        internal const char QuoteSingleBase = (char)0x201a; // single low-9 quotation mark
        internal const char QuoteReversed = (char)0x201b; // single high-reversed-9 quotation mark
        internal const char QuoteDoubleLeft = (char)0x201c; // left double quotation mark
        internal const char QuoteDoubleRight = (char)0x201d; // right double quotation mark
        internal const char QuoteLowDoubleLeft = (char)0x201E; // low double left quote used in german.
    }

The easiest solution is to use minus signs everywhere.

Another solution is to save files as Unicode UTF-8 encoding (preferred) or UTF-16 encoding (which I dislike).

–jeroen

Posted in .NET, CommandLine, Development, Encoding, PowerShell, PowerShell, Scripting, Software Development, Unicode, UTF-16, UTF-8, UTF16, UTF8 | Leave a Comment »

Google Cloud Shell: connecting to cloud storage buckets

Posted by jpluimers on 2021/09/28

One of the drawbacks of Google Cloud Shell is that it will delete the home directory after 120 days of not using it: [WayBack] Deletion notice for my Google Cloud Shell home directory – Stack Overflow

This is documented [WayBack] here:

If you do not access Cloud Shell for 120 days, we will delete your home disk. You will receive an email notification before we do so and simply starting a session will prevent its removal.

This only applies to the home directory of your Cloud Shell instance (you may want to store it on Cloud Storage anyway if you want to keep it). Any other Google services you use will be unaffected.

I hardly use the cloud shell, as it is a last resort to shell out from overly protected networks. Fewer and fewer environments restrict so much, so I’ve bumped into the home directory deletion a few times now.

I might use it more in the future, as I recently discovered there is a URL trick so you can start a cloud shell with parameters like an initial git repository: [WayBack] Open in Cloud Shell  |  Google Cloud

The Open in Cloud Shell feature allows you to publish a link that opens the Cloud Console and either automatically clones a Git repository into Cloud Shell or starts Cloud Shell with a custom image. It also allows for instructions to be printed to the terminal to help users interact with the content.

The Open in Cloud Shell feature helps developers experiment with code samples and APIs without having to worry about downloading Cloud SDK, installing required dependencies, or searching for relevant source files. This page explains how to add this feature to your Git repository.

Currently, only GitHub and Bitbucket repositories are whitelisted. If you would like to add a different repository, send feedback with the repository type you’d like to use with Open in Cloud Shell.

Setting up the home directory with my scripts can be a curse, so I have contemplated on these kinds of solutions:

  • store scripts in Google Drive, and mount part of Google Drive into the Cloud Shell
  • store scripts in Google Cloud Storage
  • script the setup of the home directory via a bash script in a gist

Some links that will hopefully help me with that:

  • [WayBack] Use the Google Cloud Shell to Upload files to Google Drive : DataHoarder
  • [WayBack] Open Your Repository In Google Cloud Shell – Daisuke Maki – Medium
  • [WayBack] gsutil – Google cloud storage – Download file from web – Stack Overflow

    mount the bucket in your Cloud Shell using gcsfuse:

    Create a directory in your Cloud Shell user home

     mkdir ~/mybucket
    

    Now mount your bucket in that directory using gcsfuse:

     gcsfuse bucket_name ~/mybucket
    

    Change the current directory to mount point directory:

     cd mybucket
    

    (if you want to have some fun run “df -h .” to see how much space you got in that mount point)

    seems to work for all file sizes:

     curl http://speedtest.tele2.net/10GB.zip | gsutil cp - gs://YOUR_BUCKET_NAME/10GB.zip
    

    Basically curl “streams” the data directly to the bucket.

  • [WayBack] Limitations and restrictions  |  Cloud Shell  |  Google Cloud

    Usage limits

    Cloud Shell inactivity: If you do not access Cloud Shell for 120 days, your home disk will be deleted. You will receive an email notification before its deletion and simply starting a session will prevent its removal. Please consider a different solution on Google Cloud storage for sensitive data you wish to store long term.

    Non-interactive usage: Cloud Shell is intended for interactive use only. Non-interactive sessions will be ended automatically after a warning. Note that Cloud Shell sessions are capped at 12 hours, after which sessions are automatically terminated. You can use a new session immediately after.

    Weekly usage: Cloud Shell also has weekly usage limits. If you reach your usage limit, you’ll need to wait until the specified time (listed under Usage Quota, found under the three dots menu icon) before you can use Cloud Shell again.

    Restoring a session after a service limit violation: If your session is terminated or cannot be established because you exceeded a service limit, Cloud Shell will display an error with a link to a form that allows you to appeal the limit violation. Click the feedback link and submit the form with more information about the tasks you were performing before your session was terminated.

–jeroen

Posted in bash, Development, Google, GoogleCloudShell, GoogleDrive, Internet, Power User, Scripting, Software Development, SpeedTest | Leave a Comment »

Chocolatey: force install sysinternals after hash mismatch

Posted by jpluimers on 2021/09/28

Shortly after UltraVNC mismatching sha256 hash the chocolatey checksum check (Chocolatey: when upgrades or installs keep insisting the hash has changed, and over time the mismatch changes as well), I bumped into another occasion: now (because of a zero sized .nupkg file), I had to force reinstall sysinternals.

The problem however is that sysinternals chocolatey will always install the latest version as per [WayBack] Chocolatey Software | Sysinternals 2019.12.19

Notes

  • This package supports only latest version.
  • This package by default installs to tools directory which will create shims for all applications. When you install to different directory, shims are not created but directory is added to the PATH.
  • This package downloads the nano edition of sysinternals suite when installing it on a nano server.
  • To have GUI for the tools, install nirlauncher package and use /Sysinternals package parameter.

It means that when reinstalling an older version (in the process of fixing a broken chocolatey install), it is OK to ignore the error caused during forced reinstall:

C:\bin\bin>choco install --force --yes sysinternals
Chocolatey v0.10.15
Installing the following packages:
sysinternals
By installing you accept licenses for the packages.
sysinternals v2019.6.29 already installed. Forcing reinstall of version '2019.6.29'.
 Please use upgrade if you meant to upgrade to a new version.
Progress: Downloading sysinternals 2019.6.29... 100%

sysinternals v2019.6.29 (forced) [Approved]
sysinternals package files install completed. Performing other installation steps.
Sysinternals Suite is going to be installed in 'C:\ProgramData\chocolatey\lib\sysinternals\tools'
Downloading sysinternals
  from 'https://download.sysinternals.com/files/SysinternalsSuite.zip'
Progress: 100% - Completed download of C:\Users\jeroenp\AppData\Local\Temp\chocolatey\sysinternals\2019.6.29\SysinternalsSuite.zip (29 MB).
Download of SysinternalsSuite.zip (29 MB) completed.
Error - hashes do not match. Actual value was 'AE0AB906A61234D1ECCB027D04F5A920D78A31494372193EE944DD419842625C'.
ERROR: Checksum for 'C:\Users\jeroenp\AppData\Local\Temp\chocolatey\sysinternals\2019.6.29\SysinternalsSuite.zip' did not meet 'db59efe1739a2262104874347277f9faa0805a1a7a0acd9cc29e9544fb8040c5' for checksum type 'sha256'. Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary.
The install of sysinternals was NOT successful.
Error while running 'C:\ProgramData\chocolatey\lib\sysinternals\tools\chocolateyInstall.ps1'.
 See log for details.

Chocolatey installed 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - sysinternals (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\sysinternals\tools\chocolateyInstall.ps1'.
 See log for details.

So in this case, as always the most recent Sysinternals file is used, it is OK to follow the bold guideline above (and quoted below) use the checksum for that file. You might even want to ignore it, as the file is downloaded over https so tampering is virtually impossible:

Consider passing the actual checksums through with --checksum --checksum64 once you validate the checksums are appropriate. A less secure option is to pass --ignore-checksums if necessary.

For this checksum, the forced reinstall becomes choco install --force --yes sysinternals --checksum AE0AB906A61234D1ECCB027D04F5A920D78A31494372193EE944DD419842625C

Alternatively (with a slight chance of yet another checksum) would be choco install --force --yes sysinternals --ignore-checksums

Related:

Read the rest of this entry »

Posted in .NET, Chocolatey, CommandLine, Development, Power User, PowerShell, PowerShell, Scripting, Software Development, SysInternals, Windows | Leave a Comment »

Chocolatey: forcing a certain package version and pinning it at that version

Posted by jpluimers on 2021/09/23

For my future self.

Due to an issue with choco-cleaner versions [WayBack] 0.0.6 and [WayBack] 0.0.7, I needed to ensure it was installed as version [WayBack] 0.0.5.2 and keep it that version.

Not sure if this is the canonical way, but this worked:

choco uninstall --yes choco-cleaner
choco install --yes choco-cleaner --version 0.0.5.2
choco pin add --name=choco-cleaner --version 0.0.5.2
choco pin list

This worked to revert:

choco pin remove --name=choco-cleaner
choco pin list
choco upgrade --yes choco-cleaner

Aftere this upgrade, choco-cleaner version 0.0.7.1 shows a nice error message when the environment variable %ChocolateyToolsLocation% fails to exist.

In that case calling RefreshEnv.cmd will create that environment variable.

Related:

–jeroen

Read the rest of this entry »

Posted in .NET, Chocolatey, CommandLine, Development, Power User, PowerShell, PowerShell, Scripting, Software Development, Windows | Leave a Comment »

Select-Object versus Write-Output: “The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any o f the parameters that take pipeline input.”

Posted by jpluimers on 2021/09/23

I bumped in the error [WayBack] “The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input.” when using [WayBack] Write-Output where [WayBack] Select-Object worked just fine.

This happened when playing around with detecting empty Chocolatey .nupkg package files.

$LibPath = Join-Path $env:ChocolateyInstall 'lib'
$NupkgFilter = '*.nupkg'

Get-ChildItem -Path $LibPath -Recurse -Filter $NupkgFilter | 
    Where-Object {($_.Length -eq 0) -and ($_.BaseName -eq "hg")} | 
        Sort-Object LastWriteTime | 
            Select-Object BaseName

<#
Get-ChildItem -Path $LibPath -Recurse -Filter $NupkgFilter | 
    Where-Object {($_.Length -eq 0) -and ($_.BaseName -eq "hg")} | 
        Sort-Object LastWriteTime | 
            Write-Output BaseName
## Write-Output : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input.
#>

Get-ChildItem -Path $LibPath -Recurse -Filter $NupkgFilter | 
    Where-Object {($_.Length -eq 0) -and ($_.BaseName -eq "hg")} | 
        Sort-Object LastWriteTime | 
            ForEach-Object { Write-Output $_.BaseName }

The output is also slightly different, hinting on the root cause:

BaseName
--------
hg      
hg

The above shows that Select-Object selects a list of BaseName properties (italic part), whereas Write-Output shows a single BaseName property content (bold part).

Read the rest of this entry »

Posted in .NET, CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

GitHub – metasfresh/metasfresh: We do Open Source ERP – Fast, Flexible & Free Software to scale your Business.

Posted by jpluimers on 2021/09/22

For my link archive: [WayBack] GitHub – metasfresh/metasfresh: We do Open Source ERP – Fast, Flexible & Free Software to scale your Business.

It’s mainly written in JavaScript and Java with a PostgreSQL database and elasticsearch search.

There is plenty of [WayBack] metasfresh documentation (for users, administrators and developers), all with markdown source.

The developer documentation starts with a nice diagram of the [WayBack] metasfresh Architecture:

Markdown source of the architecture page: [WayBack] metasfresh-documentation/metasfresh_architecture.md at gh-pages · metasfresh/metasfresh-documentation · GitHub: Docs and Manuals for Devs, Users, Admins.

Via:

–jeroen

Posted in Development, Java, Java Platform, JavaScript/ECMAScript, Scripting, Software Development | Leave a Comment »

Twitter thread by thread by @0xdade; More unicode shit: zero width space and a zero width nonjoiner in filenames

Posted by jpluimers on 2021/09/22

[WayBack] Thread by @0xdade: Today I learned that you can put zero width spaces in file names on Linux. Have fun. I’m playing with this because punycode/IDN is fascinati…

Today I learned that you can put zero width spaces in file names on Linux. Have fun.

I’m playing with this because punycode/IDN is fascinating, and I wanted to know what happened when I started shoving unicode in the path portion of the url, which isn’t part of how browsers try to protect URLs, as far as I can tell

wiki.mozilla.org/IDN_Display_Al…

I think it’s more entertaining to have a file that is named *only* a zero width space, but I think using them throughout a filename is better to break tab completion and not stand out too much. A filename that is just blank looks strange in ls output.
Thank goodness adduser is looking out for our best interests.
Oooh this one is pretty subtle.
Just about pissed myself with this one.

Not related to the terminal fun, but related to zero width characters:

You can:
– Break url previews https://0xda​​​​​​.​de
– @​0xdade without tagging
– Make a word like system​d not searchable twitter.com/search?q=from%…

Okay but back to command line crap. I really like this one. Create a directory named .[ZWS]

One thing that is cool about using zero width spaces is that “ls” has a flag, “-b”, that is meant to escape non-graphic characters. Inserting a newline, for instance, would be escaped to \n. But the zero width space is technically a graphic character, so nothing happens.

Fun.

Have no fear, though. It’s not unbeatable. It’s only fun if the language and LC settings are set to support utf-8. If you set LC_ALL=C or whatever that isn’t utf-8, then it looks like this.

Putting a link to this tweet here so that I don’t lose it again in the future.

dade@0xdade

My god, it is beautiful. I mean except all the whitespace I can’t get rid of before the command lmao.

View image on Twitter
But on the other hand if you just have a search for the zws, then whatever you find is probably worth investigating. 
I guess I’ll start the hashtag before @QW5kcmV3 does for #irresponsibleutf8 🤭😏😂 

And these tweets:

[WayBack] Thread by @Plazmaz: @0xdade Was doing some real fucking around with urls recently: gist.github.com/Plazmaz/565a5c… (was gonna flesh it out more but didn’t find…:

mentions Was doing some real fucking around with urls recently:
mentions This one is my fave:
‘⁄’ (\u2044)
or
‘∕’ (\u2215)
Allow for this:
google.com⁄search⁄query⁄.example.com
google.com⁄search⁄query⁄@example.com 

[WayBack] url-screwiness.md · GitHub:

This is a list of methods for messing with urls. These are often useful for bypassing filters, SSRF, or creating convincing links that are difficult to differentiate from legitimate urls.

And a bit of documentation links:

–jeroen

 

Posted in *nix, .NET, C#, Development, NTFS, Power User, Python, Scripting, Software Development, Windows | Leave a Comment »

Code Layout and Formatting: Indentation · PowerShell Practice and Style

Posted by jpluimers on 2021/09/22

Since I switch a lot between languages, I tend to forget what indentation, spacing and termination to use.

So from the Indentation/Length/Spacing/Termination sections in [WayBack] Code Layout and Formatting · PowerShell Practice and Style:

Read the rest of this entry »

Posted in .NET, CommandLine, Development, PowerShell, PowerShell, Scripting, Software Development | Leave a Comment »

Overview of Client Libraries · Internet Archive

Posted by jpluimers on 2021/09/14

Besides manual upload at [Archive.is] Upload to Internet Archive, there are also automated ways of uploading content.

One day I need this to archive pages or sites into the WayBack machine: [WayBack] Overview of Client Libraries · Internet Archive (most of which is Python based):

Read the rest of this entry »

Posted in Bookmarklet, Development, Internet, InternetArchive, Power User, Python, Scripting, Software Development, WayBack machine, Web Browsers | Leave a Comment »

Run chrome in fullscreen mode on Windows – Stack Overflow

Posted by jpluimers on 2021/09/14

Since Chrome changes over time, the batch file below from [WayBack] Run chrome in fullscreen mode on Windows – Stack Overflow also changes:

@echo off
echo Countdown to application launch...
timeout /t 10
"C:\Program Files (x86)\chrome-win32\chrome.exe" --chrome --kiosk http://localhost/xxxx --incognito --disable-pinch --no-user-gesture-required --overscroll-history-navigation=0
exit

On most of my systems. Chrome is at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe, so I need to change the path anyway.

–jeroen

Posted in Batch-Files, Chrome, Development, Google, Power User, Scripting, Software Development, Windows | Leave a Comment »