The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,839 other subscribers

Archive for the ‘Software Development’ Category

« Previous Entries
Next Entries »

When floating point code suddenly becomes orders magnitudes slower (via C++ – Why does changing 0.1f to 0 slow down performance by 10x? – Stack Overflow)

Posted by jpluimers on 2022/01/26

When working with converging algorithms, sometimes floating code can become very slow. That is: orders of magnitude slower than you would expect.

A very interesting answer to [Wayback] c++ – Why does changing 0.1f to 0 slow down performance by 10x? – Stack Overflow.

I’ve only quoted a few bits, read the full question and answer for more background information.

Welcome to the world of denormalized floating-point! They can wreak havoc on performance!!!

Denormal (or subnormal) numbers are kind of a hack to get some extra values very close to zero out of the floating point representation. Operations on denormalized floating-point can be tens to hundreds of times slower than on normalized floating-point. This is because many processors can’t handle them directly and must trap and resolve them using microcode.

If you print out the numbers after 10,000 iterations, you will see that they have converged to different values depending on whether 0 or 0.1 is used.

Basically, the convergence uses some values closer to zero than a normal floating point representation dan store, so a trick is used called “denormal numbers or denormalized numbers (now often called subnormal numbers)” as described in Denormal number – Wikipedia:

…

In a normal floating-point value, there are no leading zeros in the significand; rather, leading zeros are removed by adjusting the exponent (for example, the number 0.0123 would be written as 1.23 × 10−2). Denormal numbers are numbers where this representation would result in an exponent that is below the smallest representable exponent (the exponent usually having a limited range). Such numbers are represented using leading zeros in the significand.

…

Since a denormal number is a boundary case, many processors do not optimise for this.

–jeroen

Posted in .NET, Algorithms, ARM, Assembly Language, C, C#, C++, Delphi, Development, Software Development, x64, x86 | Leave a Comment »

ESXi: storing an ISO 8601 time-stamped backup tarball locally

Posted by jpluimers on 2022/01/25

In Determining the ESXi installation type (2014558) | VMware KB, I also showed how to backup the configuration and download it.

Sometimes you want an ISO 8601 time-stamped local tarball just in case you want to revert to it at a later stage.

First a small recap on how to get the tarball, download location and temporary location in the first place (it will be automatically deleted from the temporary location):

# vim-cmd hostsvc/firmware/sync_config
# vim-cmd hostsvc/firmware/backup_config
Bundle can be downloaded at : http://*/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz
# find /scratch/downloads/ -name *.tgz
/scratch/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz

Goal is to get the download filename and save it to a different folder and embed the ISO 8601 timestamp in the filename.

Like many scripts, sed and regular expressions come to the rescue once more, just like in ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes (which we will need anyway because of the ISO 8601 time stamp, and a bit of fiddling at regex101.com/r/NyrzKF

# SCRATCH_CONFIG_BUNDLE_NAME=$(vim-cmd hostsvc/firmware/backup_config | sed -n -E -e "s/^(Bundle can be downloaded at : http://*)(/downloads/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/configBundle-.+?)(.tgz)$//scratch23/p")
# echo "SCRATCH_CONFIG_BUNDLE_NAME: '${SCRATCH_CONFIG_BUNDLE_NAME}'"
SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz'
# date --utc -I'seconds' --reference "${SCRATCH_CONFIG_BUNDLE_NAME}"
2021-05-09T17:44:42UTC

Note:

  • In the sed regular, expression, [[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12} matches a GUID.
  • In the date command, the --reference parameter must be last.
  • If you get the error below, then you ran too many backup_config commands in succession:
    (vim.fault.TooManyWrites) {
   faultCause = (vmodl.MethodFault) null, 
   faultMessage = 
   msg = "Received SOAP response fault from []: syncConfiguration
fault.TooManyWrites.summary"
}

    You can see this in the hostd.log, which on my system is in /scratch/log/hostd.log where it says I can retry in 2031 seconds (slightly more than half an hour):

    2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ASSIGN: latestEvent, ha-eventmgr. Applied change to temp map.
2021-05-09T19:34:13.420Z info hostd[A703B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=vim-cmd-27-2ab3] Event 196 : User root@127.0.0.1 logged in as VMware-client/6.5.0
2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ADD: sessionList["52f6d64e-0a35-c1d7-de97-624d234bc2a7"], ha-sessionmgr. Applied change to temp map.
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Created : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804
2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ADD: recentTask["haTask--vim.host.FirmwareSystem.syncConfiguration-114207804"], ha-taskmgr. Applied change to temp map.
2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
2021-05-09T19:34:13.423Z error hostd[9AC1B70] [Originator@6876 sub=Hostsvc.FirmwareSystem opID=vim-cmd-27-2ab5 user=root] Failed to sync configuration. Too many writes. Next sync possible in 2031 sec.
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Default opID=vim-cmd-27-2ab5 user=root] AdapterServer caught exception: vim.fault.TooManyWrites
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Completed : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804 Status error
2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Activation [N5Vmomi10ActivationE:0x0987e6e0] : Invoke done [syncConfiguration] on [vim.host.FirmwareSystem:ha-firmwareSystem]
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Throw vim.fault.TooManyWrites
2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Result:
--> (vim.fault.TooManyWrites) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = 
-->    msg = ""
--> }

Not few people have bumped into this, the only other I could find through [Wayback] “vim.fault.TooManyWrites” “syncConfiguration” – Google Search is [Archive.is] mal wieder purple Screen – VMware-Forum.

Figuring out the various parts of the SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz' is like at regex101.com/r/J4yU72, regex101.com/r/uID9xs and regex101.com/r/o8a4Am:

CONFIG_BUNDLE_DIRECTORY_NAME=$(echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\1/p")
CONFIG_BUNDLE_FILE_NAME=$(     echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\2/p")
CONFIG_BUNDLE_DOT_EXTENSION=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\3/p")
echo "CONFIG_BUNDLE_DIRECTORY_NAME: '${CONFIG_BUNDLE_DIRECTORY_NAME}'"
echo "CONFIG_BUNDLE_FILE_NAME:      '${CONFIG_BUNDLE_FILE_NAME}'"
echo "CONFIG_BUNDLE_DOT_EXTENSION:  '${CONFIG_BUNDLE_DOT_EXTENSION}'"

Output is like this:

SCRATCH_CONFIG_BUNDLE_NAME:   '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/configBundle-ESXi-X10SRH-CF.tgz'
CONFIG_BUNDLE_DIRECTORY_NAME: '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/'
CONFIG_BUNDLE_FILE_NAME:      'configBundle-ESXi-X10SRH-CF'
CONFIG_BUNDLE_DOT_EXTENSION:  '.tgz'

Full backup-config-to-ESXi_configuration_backup-directory.sh script:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, cron/crontab, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Some links on Excel format strings

Posted by jpluimers on 2022/01/25

In Get Formatted Value of Cell in Excel – Stack Overflow, I focused on the =TEXT function, then indicated I would look into Excel format strings later.

Below are just a few links and a very short description as hopefully later I will have more time to dig into this.

The basic format is this (where all bits other than Format1 are optional):

[Locale]Format1;Format2;Format3;Format4

For now this is for my link archive:

  • [Wayback] Excel number format strings (e.g. “@”, “$-409]d-mmm-yy;@”) – Complete reference availability? – Stack Overflow has two answers of which the second one is deleted, but very useful:
    1. Here’s a list of the Number Format Codes for Excel.
      • Second option for link: Ecma-376 4th Edition Part 1, Section 18.8.31
    2. If you are looking for the international number codes try at http://office.microsoft.com/en-gb/excel-help/creating-international-number-formats-HA001034635.aspx
  • The HA001034635 link has disappeared, which is a pity as it explains the localisation; luckily two forms of the URL have been saved where [Locale] is constructed like this example (all Locale digits are hexadecimal):

    [$-24050412]m/d/yy

    In the preceding example:

    24  is the numeral shape component (Korean 1).
    05  is the calendar type component (Korean (Tangun era)).
    0412  is the locale and language designation component (Korean).

    1. [Wayback] Archive.is: Creating international number formats – Excel … HA001034635.aspx
    2. [Wayback] Archive.is: Creating international number formats – Excel – Office.com … HA001034635.aspx?redir=0
  • More [Locale] information is explained in [Wayback] What does the 130000 in Excel locale code [$-130000] mean? – Stack Overflow by [Wayback] User IrwinAllen13 – Stack Overflow and has more calendar formats than the above Microsoft links.
  • [Wayback] Number format codes – Excel for Mac has the base and sort of explains what the @ is for: the content of the cell.

    First the base:

    When you create custom number formats, you can specify up to four sections of format code. These sections of code define the formats for

    1. positive numbers,
    2. negative numbers,
    3. zero values, and
    4. text,

    in that order. The sections of code must be separated by semicolons (;).

    The following example shows the four types of format code sections.

    Four types of format code sections

    Callout 1 Format for positive numbers

    Callout 2 Format for negative numbers

    Callout 3  Format for zeros

    Callout 4  Format for text

    Then on on specifying less than 4 sections:

    • If you specify only one section of format code, the code in that section is used for all numbers.
    • If you specify two sections of format code, the first section of code is used for positive numbers and zeros, and the second section of code is used for negative numbers.
    • When you skip code sections in your number format, you must include a semicolon for each of the missing sections of code.
    • You can use the ampersand (&) text operator to join, or concatenate, two values.

    Then on the @ sign (which is under “Text and spacing”):

    To create a number format that includes text that is typed in a cell, insert an “at” sign (@) in the text section of the number format code section at the point where you want the typed text to be displayed in the cell.

    …

    For example, to include text before the text that’s typed in the cell, enter “gross receipts for “@ in the text section of the number format code.

    Finally it explains how to colorise the formatting or hide particular values under “Decimal places, spaces, colors, and conditions”:

    The color code must be the first item in the code section.

    [Black] [Blue] [Cyan] [Green] [Magenta] [Red] [White] [Yellow]

    …

    Hiding various values:

    To hide Use this code
    Zero values 0;–0;;@
    All values ;;; (three semicolons)
  • [Wayback] ECMA-376 – Ecma International where Ecma-376 4th Edition Part 1, Section 18.8.31 is supposed to have format strings (will dig into this later)

    Office Open XML file formats – This Standard defines Office Open XML’s vocabularies and document representation and packaging

    1. [Wayback] ECMA-376, Part 1
    2. Not sure where Part 2 is
    3. [Wayback] ECMA-376, Part 3
    4. [Wayback] ECMA-376, Part 4
    5. Not sure where Part 5 is

Below a few screenshots while experimenting.

The first one shows the formulas, the second one the content. The Date value cells shows the values when entered as shown; the Date formatted cells are all formatted with yyyy-mm-dd;@ formatting.

I still need to figure out why using a =TEXT function shows #VALUE! whereas using cell formatting just a bunch of ############### (15 times a #, not sure if that is always the same number).

–jeroen

Posted in Development, Excel, Office, Office Development, Power User, Software Development | Leave a Comment »

RFC2606: Reserved Top Level DNS Names (RFC); draft-ellermann-idnabis-test-tlds-04: Reserved Top Level DNS Names (Internet-Draft, 2008)

Posted by jpluimers on 2022/01/20

Note

Though there are .example.edu and .example.info, though used in documentation and  registered by IANA, have a status is different from the official Reserved Top Level DNS Names:

This is not exactly the same situation as for say ".example.org", where IANA is the registrant *and* registrar.

Wikipedia links:

Read the rest of this entry »

Posted in Development, DNS, Documentation Development, Internet, Power User, Software Development, Testing | Leave a Comment »

Some links on Chrome not prompting to save passwords (when Firefox and Safari do)

Posted by jpluimers on 2022/01/20

For quite some time now, Chrome (think years) refuses to prompt for saving passwords whereas Firefox and Safari do prompt and save them, even for site types that it used to save passwords for in the past.

It has been annoying enough for too long now that I tried to do better than the Google searches I used back when I saw this happen first.

Below are some links based on new searches (starting with [Wayback] adding a password in chrome settings – Google Search); hopefully I can try them after I made a list of sites that Chrome does not show the password save prompt for.

Solutions I tried that failed (but maybe useful for others):

Solutions still to try:

Read the rest of this entry »

Posted in Chrome, Chrome, Communications Development, Development, Encryption, ESXi6, ESXi6.5, ESXi6.7, Firefox, Fritz!, Fritz!Box, Fritz!WLAN, Google, https, HTTPS/TLS security, Internet, Internet protocol suite, Let's Encrypt (letsencrypt/certbot), Power User, routers, Safari, Security, TCP, TLS, Virtualization, VMware, VMware ESXi, Web Browsers, Web Development | Leave a Comment »

The Seven Step Process to Creating an Amazing Demo | by Chris Bensen

Posted by jpluimers on 2022/01/20

On the reading list: [Wayback/Archive.is] The Seven Step Process to Creating an Amazing Demo | by Chris Bensen | Oracle Developers | May, 2021 | Medium

So you have some crazy idea and want to build a demo. This seven step process should help you turn that idea into something amazing

Chris created the World’s Largest 3D Printed Brick Computer and was the driving force behind the Raspberry Pi Mini Super Computer [Wayback].

World’s Largest 3D Printed Brick Computer

World’s Largest 3D Printed Brick Computer

Raspberry Pi Mini Super Computer

Raspberry Pi Mini Super Computer

–jeroen

Read the rest of this entry »

Posted in Development, Geeky, Hardware Development, Raspberry Pi, Software Development | Leave a Comment »

Some uptime monitoring tools that are still free and understand more than http/https

Posted by jpluimers on 2022/01/19

Since uptimerobot has slowly moving free features, part of my family uses it to monitor uptime of their personal stuff, and I don’t want to pay for the free monitoring I do on Embarcadero infrastructure (I know that they have been phasing out older useful sites for ages, but even their main sites were down for 2 days about a year ago).

Yes, I know the competitiveness of [Wayback] Website monitoring – Wikipedia, and I’m glad Uptimerobot has been there for a relatively long time, but alas: for personal use…

The list of of the pricing pages of each site, containing a one line summary blurb if it was available:

Via [Wayback] 16 Best Online Services to Monitor Sites Uptime

–jeroen

Posted in *nix, Delphi, Development, Monitoring, Power User, Software Development | Leave a Comment »

ESXi: getting and setting the host name, domain and fqdn

Posted by jpluimers on 2022/01/19

A few links and notes:

  1. [Wayback] Changing the hostname of an ESX or ESXi host (1010821)

    Run these commands to change the hostname in ESXi 5.x, ESXi 6.x,ESXi 7.x, using the command line:

    • esxcli system hostname set –host=hostname
    • esxcli system hostname set –fqdn=fqdn
  2. [Wayback] ESX Host appears as localhost.localdomain in VMware Infrastructure/vSphere client (2009720)

    Cause

    The name resolution parameters were not properly configured during the installation of the ESX host.
  3. [Wayback] Domain repoint for embedded vCenter Server fails with error: “domain_consolidator Failed to set machine id” (71020)

    This issue is caused by a mismatch between the FQDN that was configured as the PNID during the vCenter Server deployment and the hostname that is currently configured.

I had a mismatch happen because of the second entry: a host configured in a different domain than it was deployed to.

Here are the commands to list and change the hosts name, domain and fqdn:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

Get Formatted Value of Cell in Excel – Stack Overflow

Posted by jpluimers on 2022/01/18

It’s odd, but facing a potentially lot shorter life expectancy, I need to prepare to become more expendable in personal life.

This means that I need to document or/and automate a lot of duties.

In this case, it is administrative work as custodian for my brother that is based on scripts, Excel sheets and manual steps.

In order to lessen these steps, I wanted to get the formatted value of certain Excel WorkSheet cells so I could concatenate them in other places.

It appears that neither the CONCATENATE function (nor the more recently introduced CONCAT function) nor  & concatenation operator just take the unformatted value of the cell and put that in as text.

Note that the TEXTJOIN function is basically a CONCATENATE function with an extra delimiter parameter, so it does not format text.

One format I needed was YYYY-MM-DD, basically the ISO-8601 date format.

The Excel format string for this is yyyy-mm-dd;@, and the corresponding formula to apply it on cell L5 is =TEXT(J5, "yyyy-mm-dd;@")

Based on [Wayback] Get Formatted Value of Cell in Excel – Stack Overflow:

Use the TEXT() function:
TEXT(value, format_text)
So if the value is 23.5 and you pass =TEXT(A1, "$0.00") it will return $23.50
Source: http://office.microsoft.com/en-us/excel-help/text-function-HP010062580.aspx

At a later stage, I will look into the actual format strings.

References:

–jeroen

Posted in Development, Excel, Office, Power User, Software Development | Leave a Comment »

Need to revisit osquery: SQL powered operating system instrumentation, monitoring, and analytics supports more platforms and also aggregates to central log locations

Posted by jpluimers on 2022/01/18

Almost two years ago, GitHub – facebook/osquery: SQL powered operating system instrumentation, monitoring, and analytics published from the automatic blog queue.

It was in the midst of my rectum cancer treatment, so I was glad the blog queue back then was still about 18 months deep.

This meant I looked into osquery in 2018, which I remember because I needed it on MacOS as I did not want to remember the syntax for MacOS specific commands on getting system information. It also coincides with how much my repository fork was behind: [Wayback: jpluimers/osquery commits/Archive: jpluimers/osquery commits].

Fast forward to now, the breath of systems I’m involved with has widened, so I was glad to see that Kristian Köhntopp mentioned it:

So time to try it again (:

The links he mentioned:

  • [Wayback/Archive] Welcome to osquery – osquery

    osquery is an operating system instrumentation framework for Windows, OS X (macOS), Linux, and FreeBSD. The tools make low-level operating system analytics and monitoring both performant and intuitive.

  • [Wayback/Archive] Welcome to osquery – osquery: High Level Features
    The high-performance and low-footprint distributed host monitoring daemon, osqueryd, allows you to schedule queries to be executed across your entire infrastructure. The daemon takes care of aggregating the query results over time and generates logs which indicate state changes in your infrastructure. You can use this to maintain insight into the security, performance, configuration, and state of your entire infrastructure. osqueryd‘s logging can integrate into your internal log aggregation pipeline, regardless of your technology stack, via a robust plugin architecture.
    The interactive query console, osqueryi, gives you a SQL interface to try out new queries and explore your operating system. With the power of a complete SQL language and dozens of useful tables built-in, osqueryi is an invaluable tool when performing incident response, diagnosing a systems operations problem, troubleshooting a performance issue, etc.
  • [Wayback/Archive] osqueryd (daemon) – osquery
  • [Wayback/Archive] osqueryi (shell) – osquery
  • [Wayback/Archive] Aggregating Logs – osquery
  • [Wayback/Archive] AWS Logging – osquery

Main site: [Wayback/Archive] osquery | Easily ask questions about your Linux, Windows, and macOS infrastructure

Repository: [Wayback/Archive] osquery/osquery: SQL powered operating system instrumentation, monitoring, and analytics.

–jeroen

Posted in *nix, *nix-tools, Apple, Development, DevOps, Facebook, Infrastructure, Mac, Mac OS X / OS X / MacOS, Power User, SocialMedia, Software Development, Windows | Leave a Comment »

« Previous Entries
Next Entries »