The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

    Mastodon

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now20140424-VMware-Fusion-6.0.3.-no-reclaimable
    More Photos

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for the ‘UCS-2’ Category

UTF-8, Explained Simply – YouTube

Posted by jpluimers on 2026/03/04

Cool interesting video: [Wayback/Archive] UTF-8, Explained Simply – YouTube

It covers both history from the late 1800s Baudot Code (also known as ITA1) via 1930s ITA2 and 1950’s EBCDIC / FIELDATA ages through 7-bit ASCII in the 1970s  and incompatible UCS-2 (now UTF-16) of the 1990s to the current day and age of UTF-8 (which actually started out on a placemat in 1992).

Though mentioning 8-bit encoding, it skips details of extended ASCII encodings like ISO/IEC 8859 and Windows-1252.

It goes to quite some length on decoding UTF-8 and showing how forgiving the UTF-8 standard is. Yes, it is a self-synchronising code thanks to the venerable Ken Thompson.

Definitely worth watching as it also covers the Zero-width joiner which is not just important for combining Emoji, as it is used by many people nowadays, but got in fact implemented to support various scripts like Arabic script or any Indic script.

Oh, the placemat story: Read the rest of this entry »

Posted in ASCII, Development, EBCDIC, Encoding, ISO-8859, Software Development, UCS-2, Unicode, UTF-16, UTF-8, Windows-1252 | Leave a Comment »

Coping with UTF-16 / UCS-2 little endian in Batch files: numbers from WMIC

Posted by jpluimers on 2016/11/22

A while ago, I needed to get the various date, time and week values from WMIC to environment variables with pre-padded zeros. I thought: easy job, just write a batch file.

Tough luck: I couldn’t get the values to expand properly. Which in the end was caused by WMIC emitting UTF-16 and the command-interpreter not expecting double-byte character sets which messed up my original batch file.

What I wanted What I got 
wmic_Day=21
wmic_DayOfWeek=04
wmic_Hour=15
wmic_Milliseconds=00
wmic_Minute=02
wmic_Month=05
wmic_Quarter=02
wmic_Second=22
wmic_WeekInMonth=04
wmic_Year=2015

Day=21
wmic_DayOfWeek=4
wmic_Hour=15
wmic_Milliseconds=
wmic_Minute=4
wmic_Month=5
wmic_Quarter=2
wmic_Second=22
wmic_WeekInMonth=4
wmic_Year=2015

WMIC uses this encoding because the Wide versions of Windows API calls use UTF-16 (sometimes called UCS-2 as that is where UTF-16 evolved from).

As Windows uses little-endian encoding by default, the high byte (which is zero) of a UTF-16 code point with ASCII characters comes first. That messes up the command interpreter.

Lucikly rojo was of great help solving this.

His solution is centered around set /A, which:

  • handles integer numbers and calls them “numeric” (hinting floating point, but those are truncated to integer; one of the tricks rojo uses)
  • and (be careful with this as 08 and 09 are not octal numbers) uses these prefixes:
    • 0 for Octal
    • 0x for hexadecimal

Enjoy and shiver with the online help extract:
Read the rest of this entry »

Posted in Algorithms, Batch-Files, Development, Encoding, Floating point handling, Scripting, Software Development, UCS-2, UTF-16, UTF16 | Leave a Comment »