The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,854 other subscribers

Archive for the ‘Scripting’ Category

« Previous Entries
Next Entries »

ShellCheck – shell script analysis tool

Posted by jpluimers on 2024/02/22

Cool: [Wayback] ShellCheck – shell script analysis tool

ShellCheck finds bugs in your shell scripts

It needs a shebang at the start of a script (like #!/usr/bin/env bash) to recognise the kind of shell, then does amazing analysis.

It is open source at [Wayback/Archive.is] koalaman/shellcheck: ShellCheck, a static analysis tool for shell scripts with excellent documentation including explaining screenshots like

It’s not just available on-line or on the command-line, but also integrates with many code editors (like [Wayback/Archive.is] ShellCheck – Visual Studio Marketplace: Integrates ShellCheck into VS Code, a linter for Shell scripts.) and CI/CD pipelines.

Via: [Wayback] bash – error conditional binary operator expected in compound branch – Unix & Linux Stack Exchange (thanks [Wayback] Cyrus!)

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »

Walls and Ladders when pasting e-mail on account sign-up forms: Paste It – Chrome Web Store

Posted by jpluimers on 2024/02/06

In a game of Walls and Ladders (similar to Arms Race), the Ladders usually win, see the references at the end of the post.

The actual “game” in this case is more and more sites trying to build walls prevent pasting credential related information like user IDs (often e-mail addresses) or passwords often citing “more safety” or “less security risks”, and users get taller ladders wanting to do just that because of their own security concerns:

[Wayback/Archive] Stef 🎈 on Twitter: “Dear mobile/web-apps, please never never disable copy and paste “due to security reasons”. -everybody with a password manager.”

The walls will always loose so it is better to invest the money for the walls into other security measures.

Given that most of the risks are web-sites getting that information exfiltrated, I wish they put more energy into bolting down that side of the security risk side than the hampering legitimate users entering that information in the first place.

Since so many of these sites have leaked my information in the past, any email address I use for activating an account is like 50 characters long. Something I am not going to type once (because of typing mistakes) and definitely not twice (to confirm I did not make typing mistakes).

Read the rest of this entry »

Posted in Authentication, Chrome, Clipboard, Development, Google, HTML, JavaScript/ECMAScript, Power User, Scripting, Security, Software Development, Web Development | Leave a Comment »

bloomberg/memray: Memray is a memory profiler for Python

Posted by jpluimers on 2024/01/22

For my link archive as one day I will need this:

Via this cool [Wayback/Archive] Thread by @1st1 on Thread Reader App that has these tweets:

  1. [Wayback/Archive] Yury Selivanov on Twitter: “WOW. Bloomberg finally opensourced memray—a new versatile memory profile for Python. Can’t way to use it. Thread 👇”

  2. [Wayback/Archive] Yury Selivanov on Twitter: “1. It’s fully open source (Apache 2), grab it here: …”

  3. [Wayback/Archive] Yury Selivanov on Twitter: “2. Until now you never could have such a deep insight in how your app allocates memory. The tool is a must for any long-running services implemented with Python. With memray you can generate flame charts or all allocations and trace absolutely everything.”

  4. [Wayback/Archive] Yury Selivanov on Twitter: “3. And this must be a Python snippet of the month:”

    Image

    @pytest.mark. limit_memory("24 MB")
def test_foobar():
    # do some stuff that allocates memory
  5. [Wayback/Archive] Yury Selivanov on Twitter: “4. It’s sophisticated enough to peek into native code. So you can profile your numpy and pandas code with it. And it has a live mode. You can just run your code and see how it allocates memory as it runs. 🤯”

  6. [Wayback/Archive] Yury Selivanov on Twitter: “5. A little birdie mentioned to me that @pyblogsal is one of the active contributors in memray. Let’s keep it a secret 🫢”
  7. [Wayback/Archive] Yury Selivanov on Twitter: “6. This is a Python client to @edgedatabase as it establishes a connection to the DB. cc @fantix @elprans”

  8. [Wayback/Archive] Yury Selivanov on Twitter: “7. Wow, this thread is blowing up. Here’s a link to my Spotify… err,”

–jeroen

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

For your next job interview: One-Line FizzBuzz Solution in Python 3 | by David Sanchez | CodeX | Medium

Posted by jpluimers on 2024/01/18

[Wayback/Archive] One-Line FizzBuzz Solution in Python 3 | by David Sanchez | CodeX | Medium

print('\n'.join(['fizzbuzz' if x%15 == 0 else 'buzz' if x%5 == 0 else 'fizz' if x%3 == 0 else str(x) for x in range(1,101)]))

(Yes, one can do similar list comprehension and conditional expression constructs in other languages to get one-liner solutions)

Via:

–jeroen

Posted in Development, Python, Scripting, Software Development | 1 Comment »

Troubleshooting Errors with winget… | FileWave KB

Posted by jpluimers on 2024/01/14

Need to figure out what is the cause here for [Wayback/Archive] Troubleshooting Errors… | FileWave KB

General Errors

Hex Decimal Symbol Description
…
0x8A15003B -1978335173 APPINSTALLER_CLI_ERROR_RESTAPI_INTERNAL_ERROR Rest API internal error
…

I got the error following the install steps at [Wayback/Archive] Download and install Google Chrome with winget

winget install -e --id Google.Chrome

Usually I don’t install through winget because it is often slow and during upgrades often fails to be silent (causing all kinds of popup Windows to appear), but Chocolatey had a history of hashing problems when installing [Wayback/Archive] Chocolatey Software | Google Chrome 130.0.6723.92.

This package always installs the latest version of Google Chrome, regardless of the version specified in the package. Google does not officially offer older versions of Chrome for download. Because of this you may get checksum mismatch between the time Google releases a new installer, and the package is automatically updated.

Yup the page contains the above warning, but often this happens a week at a time: not something I want to spend on installing a web-browser.

Error

Read the rest of this entry »

Posted in Chocolatey, Development, Scripting, Software Development, Windows, Windows Development, winget | Leave a Comment »

Script to rename a virtual machine in ESXi 6

Posted by jpluimers on 2024/01/10

I bumped into [Wayback/Archive] Script to rename a virtual machine in ESXi 6: gist.github.com/Gremgoll/8bd91258d71fe895c0d416e2543ca2dc.

Then I forked it with the intent to make usage more clear, as the current usage on my ESXi rig (which has mos VMs not in the root of data stores) is like the bold italic portion here:

[root@X9SRI3F-ESXi:/vmfs/volumes/608be754-f21556ad-1082-0025907d9d5c/VM] vm-rename.sh NVMe980PRO_1TB/VM X9SRI-3F-W10P-NL-OFFICE X9SRI-3F-W10P-NL X9SRI-3F-W10P-NL-OFFICE
VOLNAME=NVMe980PRO_1TB/VM
DIRNAME=X9SRI-3F-W10P-NL-OFFICE
OLDNAME=X9SRI-3F-W10P-NL
NEWNAME=X9SRI-3F-W10P-NL-OFFICE
VM_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
NW_DIRPATH=/vmfs/volumes/NVMe980PRO_1TB/VM/X9SRI-3F-W10P-NL-OFFICE
Failed to rename './X9SRI-3F-W10P-NL-flat.vmdk' to './X9SRI-3F-W10P-NL-OFFICE-flat.vmdk': The file specified is not a virtual disk (15)
renaming ./X9SRI-3F-W10P-NL-41260b40.vmem to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmem
renaming ./X9SRI-3F-W10P-NL-41260b40.vmss to ./X9SRI-3F-W10P-NL-OFFICE-41260b40.vmss
renaming ./X9SRI-3F-W10P-NL-6a4b8f29.hlog to ./X9SRI-3F-W10P-NL-OFFICE-6a4b8f29.hlog
renaming ./X9SRI-3F-W10P-NL.nvram to ./X9SRI-3F-W10P-NL-OFFICE.nvram
renaming ./X9SRI-3F-W10P-NL.vmsd to ./X9SRI-3F-W10P-NL-OFFICE.vmsd
renaming ./X9SRI-3F-W10P-NL.vmx to ./X9SRI-3F-W10P-NL-OFFICE.vmx
renaming ./X9SRI-3F-W10P-NL.vmx.backup to ./X9SRI-3F-W10P-NL-OFFICE.vmx.backup
renaming ./X9SRI-3F-W10P-NL.vmxf to ./X9SRI-3F-W10P-NL-OFFICE.vmxf
renaming ./vmx-X9SRI-3F-W10P-NL-1093012288-1.vswp to ./vmx-X9SRI-3F-W10P-NL-OFFICE-1093012288-1.vswp
All Done. You now need to register X9SRI-3F-W10P-NL-OFFICE to the inventory.

There also was a renaming bug (see the italic line).

I managed to fix both that and added more documentation plus output.

Repositories and code

  1. Original:
  2. Fork with added VSAN functionality (see [Wayback/Archive] Revisions · Script to rename a virtual machine in ESXi 6):
  3. My fork of 2:
  4. My fix:

Code before modifying: [Wayback/Archive] gist.githubusercontent.com/jpluimers/fcc601dd41ac89f601a5174be92c841c/raw/e3683fbb6bdf1e73d65d2b784027c70cf42a5512/vm-rename

Code after modifying: [Wayback/Archive] raw.githubusercontent.com/jpluimers/vm-rename/master/vm-rename.sh.

Yes, that is not in a gist any more (see below why), it is now part of [Wayback/Archive] jpluimers/vm-rename: Script to rename a virtual machine in ESXi 6; fork from https://gist.github.com/jpluimers/fcc601dd41ac89f601a5174be92c841c as [Wayback/Archive] vm-rename/vm-rename.sh at master · jpluimers/vm-rename.

The reason was

Yesterday I described the workaround in Fork Gist to Repo on GitHub – Stack Overflow.

Queries

I found the original via [Wayback/Archive] script esxi rename vmx vmxf and related files – Google Search.

My start was [Wayback/Archive] script esxi duplicate vm and rename vmdk – Google Search which found:

  1. [Wayback/Archive] Renaming a virtual machine and its files in VMware ESXi (1029513) which has a PowerCLI script (that requires Windows to run PowerCLI on)
  2. [Wayback/Archive] Solved: How to clone a VM on a ESXI 6.5 server? – VMware Technology Network VMTN which pointed me to the next Google Search result :
  3. [Wayback/Archive] VMware esxi – Script to clone a VM without vSphere or vCenter. | GrangerX which I put in this gist:
  4. [Wayback/Archive] Script to create Linked Clones on ESXi | RedNectar’s Blog which is great, but way to complicate for my use case. It references a few other cool scripts though, and shows how to use Resource Pools in ESXi (which the Web-UI does not seem to support):
  5. [Wayback/Archive] [script] Cloning VMs using ESXi shell/admin console – Virtualizing Unraid – Unraid
  6. [Wayback/Archive] Script to clone a VM with free VMware ESXi – Rob Pomeroy (again: PowerCLI)

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, PowerCLI, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

VMware ESXi: shell script to get uuid.bios and all Ethernet generatedAddress MAC address values for all VMs

Posted by jpluimers on 2023/12/28

This is a sort of follow-up on ESXi: listing virtual machines with their IP addresses where we ended with this:

I modified the above script to become this:

#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)s+((S.+S)?)s+([S+])s+(.+.vmx)s+(S+)s+(vmx-[[:digit:]]+)s*?((S.+)?)$/1/p"`
for vmid in ${vmids} ; do
    # powerState values:
    #   Powered off
    #   Powered on
    #   Suspended
    powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
    name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/(vim.vm.ConfigInfo) {/,/files = (vim.vm.FileInfo) {/ s/^ +name = "(.*)",.*?/1/p'`
    vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = (vim.vm.FileInfo) {/,/tools = (vim.vm.ToolsConfigInfo) {/ s/^ +vmPathName = "(.*)",.*?/1/p'`
    # For now, I choose to use only the IPv4 main address from ipAddress, which is in between (vim.vm.GuestInfo) { and net = (vim.vm.GuestInfo.NicInfo) [.
    ipAddress=`vim-cmd vmsvc/get.guest ${vmid} | sed -n -E -e '/(vim.vm.GuestInfo) {/,/net = (vim.vm.GuestInfo.NicInfo) [/ s/^ +ipAddress = "(.*)",.*?/1/p'`
    printf "VM with id %3s has power state %-11s and IPv4=%-15s (name = ${name}; vmPathName = ${vmPathName}).n" "${vmid}" "${powerState}" "${ipAddress}"
done

Now the script grew even larger in to vim-cmd-list-all-VMs-with-IPv4-MAC-uuid.sh:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Hello “SMTP Smuggling” information released days before the Holiday season to open source SMTP server teams

Posted by jpluimers on 2023/12/24

Jan Wildeboer was mad for good reasons, though the open source projects didn’t yet seem to publicly have show their real madness, just bits like [Wayback/Archive] oss-security – Re: Re: New SMTP smuggling attack:

I'm a little confused by sec-consult's process here. They identify a
problem affecting various pieces of software including some very widely
deployed open source software, go to the trouble of doing a coordinated
disclosure, but only do that with...looking at their timeline... gmx,
microsoft and cisco?

“SMTP Smuggling” is bad, and big open source SMTP server projects like exim, postfix and sendmail needed to assess and fix/prevent the issue on very short notice: effectively confronting them with a zero-day less than a week between the information got released and the Holiday season.

That gives “deploy on Fridays” a totally different dimension.

How bad? Well, it already managed to reach this Newline – Wikipedia entry:

The standard Internet Message Format[26] for email states: “CR and LF MUST only occur together as CRLF; they MUST NOT appear independently in the body”. Differences between SMTP implementations in how they treat bare LF and/or bare CF characters have led to so-called SMTP smuggling attacks[27].

The crux of the problem is very well described by the “Postfix: SMTP Smuggling” link below: recommended reading, and the middle of [Wayback/Archive] SMTP Smuggling – Spoofing Emails Worldwide | Hacker News

…

TLDR: In the SMTP protocol, the end of the payload (email message) is indicated by a line consisting of a single dot. The line endings normally have to be CRLF, but some MTAs also accept just LF before and/or after the dot. This allows SMTP commands that follow an LF-delimited dot line to be “tunneled” through a first MTA (which requires CRLF and thus considers the commands to be part of the email message) to a second MTA (which accepts LF and thus processes the commands as real commands). For the second MTA, the commands appear to come from the first MTA, hence this allows sending any email that the first MTA is authorized to send. That is, emails from arbitrary senders under the domains associated with the first MTA can be spoofed.

…

Here are some links to keep you busy the next hours/days/weeks:

And the toots linking to background information:

Read the rest of this entry »

Posted in *nix, *nix-tools, Communications Development, Development, exim mail, Internet protocol suite, postfix, Power User, Python, Scripting, sendmail, SMTP, Software Development | Leave a Comment »

A great source to learn about JavaScript element enumeration and modification: iamadamdev/bypass-paywalls-chrome

Posted by jpluimers on 2023/12/19

Sometimes one bumps into a Google Chrome extension that is both useful from a practical perspective as insightful on learning from how it is done.

This is one: [Wayback/Archive] iamadamdev/bypass-paywalls-chrome: Bypass Paywalls web browser extension for Chrome and Firefox.

It supports many sites (including more than a dozen Dutch ones) for which it is not easy to justify creating separate accounts for them (just the risk of them leaked into Have I been Pwned? is large, despite GDPR) and staying logged on for each of them. I have dozens of listings of my email addresses at haveibeenpwned.com, so I am a lot more careful making accounts than in the past despite assigning unique email addresses for each account (which is part of the burden).

Read the rest of this entry »

Posted in Chrome, Development, HTML, JavaScript/ECMAScript, Power User, Scripting, Software Development, Web Browsers, Web Development | Leave a Comment »

Reminder to self: pointers to recovering “The Great Suspender” suspended URLs (after in 2021 Google booted it from Chrome for being malware)

Posted by jpluimers on 2023/12/14

I was a long term user of “The Great Suspender”. It was a cool little Chrome Extension that would auto-suspend Chrome tabs that had not been used for a while and resume them when the tab did get accessed again thereby greatly reducing the horrible Chrome CPU and memory footprints.

During my year+ long treatment against metastasised rectum cancer I had suspended or hibernated most of my physical and virtual machines. So there was not just the surprised during the recovery of those that The Great Suspender had been kicked of the Chrome extensions, but also the problem of getting all the suspended tabs back of machines that eventually would be awoken out of sleep: I keep tabs open on stuff that I was working on or investigating for future blog posts, so these somehow could be important.

For now, I am not using anything as a replacement just to experience how well Chrome has evolved to suspend inactive tabs itself.

Now Chrome seems to do this well, as this post is based on an old VM that I have now unsuspended which had [Wayback/Archive] “the great suspender” “malware” – Google Search and the below links open in a mid-February 2021 state but not all archived in the Wayback Machine or Archive.is (some I did archived in February-May 2021).

The links are about why it got removed, how to recover lost suspended tabs and a possible alternative in case current Chrome suspend behaviour is not good enough.

Read the rest of this entry »

Posted in Bookmarklet, Chrome, Development, Google, HTML, JavaScript/ECMAScript, Power User, Scripting, Software Development, Web Development | Leave a Comment »

« Previous Entries
Next Entries »