The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,860 other subscribers

Archive for the ‘Unicode’ Category

Some interesting encoding/Unicode/text articles on kunststube and links for test files of various encodings

Posted by jpluimers on 2016/08/17

After yesterdays post on Testing and static methods don’t go well together, I read around on Source (kunststube [WayBack]) a bit more and found these very nice articles on encoding,Unicode and text:

Related on those, some other nice readings:

–jeroen

Posted in Ansi, ASCII, CP437/OEM 437/PC-8, Development, EBCDIC, Encoding, ISO-8859, ISO8859, Shift JIS, Software Development, Unicode, UTF-16, UTF-8, UTF16, UTF8, Windows-1252 | Leave a Comment »

Graphical emoji are killing Unicode

Posted by jpluimers on 2016/08/05

Unicode is about Glyphs that are used in writing. Have you ever seen the emoji on the right being written like this?

This has been bothering me a while and gets worse over time.

According to: Microsoft just changed its toy gun emoji to a real pistol:

Looks like Microsoft and Apple may not be on the same page about firearm emojis afterall. Right after Apple changed its gun emoji to a water pistol in iOS 10, Microsoft replaced its toy pistol emoji with an actual revolver.

While Apple and Microsoft have gone back to edit their symbols, Google continues to use a pistol in Android keyboards and doesn’t appear to have plans to change this. None of the companies in question have adjusted their knife, sword, bomb, poison and coffin emojis, so… ¯\_(ツ)_/¯

When vendors start prescribing how emojis must look like (influenced by all sorts of emotions) without the user allowing to choose (via a font – that’s what fonts are for!) how they look then it invalidates the whole Unicode principle:

Unicode is a computing industry standard for the consistent encoding, representation, and handling of text expressed in most of the world’s writing systems.

These emoji aren’t text and should be gone from the Unicode standard before they can do more harm.

Will the next step be that vendors define their own colours for certain characters in fonts? For Windows Times New Roman A becomes red, B green, C yellow, but in Courier New we’ll permute these colours and all Operating Systems and Versions will do different random colour choices.

–jeroen

via:

Posted in Development, Encoding, Opinions, Software Development, Unicode | Leave a Comment »

ASCII and Unicode. Love and hate?

Posted by jpluimers on 2015/10/13

So I won’t forget:

Even though this does not work on most USA T-Shirt sites, it works on this Dutch one: T-Shirt Ontwerpen – t-shirt zelf ontwerpen | Spreadshirt.

–jeroen

PS:

Read the rest of this entry »

Posted in ASCII, Development, Encoding, Software Development, Unicode | Leave a Comment »

Jon Skeet’s speech “Back to basics” is really a good watch – via Jørn Einar Angeltveit G+

Posted by jpluimers on 2015/07/15

Thanks [Wayback] Jørn Einar Angeltveit for sharing this a while ago:

A session by Jon Skeet and Tony the Pony (which has strong teeth) presented during the Polish DevDay 2013 in Kraków, Poland.

[Wayback] +Jon Skeet’s speech [Wayback] “Back to basics” is really a good watch.

In a funny way, he explains why the simplest fundamentals of computer software text, dates and numbers can cause some real headache for the programmer…

In case you didn’t know: Jon Skeet is “Chuck Norris” on [Wayback] stackoverflow.com:

The subtitle is “the mess we’ve made of our fundamental data types”.

Some of the topics covered:

Read the rest of this entry »

Posted in .NET, C#, Conference Topics, Conferences, Delphi, Development, Encoding, Event, internatiolanization (i18n) and localization (l10), Java, Java Platform, Jon Skeet, Pascal, Scripting, Software Development, Unicode | 2 Comments »

20 resources on migrating to Unicode with Delphi | Software on a String

Posted by jpluimers on 2015/07/08

Great post by Marjan Venema when you need to migrate your old Delphi programs to the modern Delphi world: [Wayback] 20 resources on migrating to Unicode with Delphi | Software on a String.

I’m glad that some of the links overlap with what I posted and presented in the past at:

Well done Marjan!

–jeroen

Posted in Ansi, ASCII, Delphi, Delphi 2, Delphi 2005, Delphi 2006, Delphi 2007, Delphi 2009, Delphi 2010, Delphi 3, Delphi 4, Delphi 5, Delphi 6, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Delphi XE6, Delphi XE7, Development, Encoding, Software Development, Unicode | Leave a Comment »

[NL] encoding blijft moeilijk, waarom toch? (dit keer in een brief van @xs4all)

Posted by jpluimers on 2015/02/24

Hoe moeilijk kan het toch zijn om je encoding goed te doen.

Deze keer uit een brief van xs4all:

Mojibake encoding probleem

Mojibake encoding probleem

Als je een trema in een brief zet, dan controleer je toch even dat die ook goed op de brief wordt afgedrukt?

Read the rest of this entry »

Posted in Development, Encoding, ISO-8859, Mojibake, Software Development, Unicode, UTF-8, UTF8 | Leave a Comment »

Windows Alt Key Codes

Posted by jpluimers on 2015/02/16

Brillant!: Windows Alt Key Codes.

The page has both lists of four-digit ALT+#### codes, a how-to, and reference links:

  1. Letters with Accents – (e.g. ó, ò, ñ)
  2. Other Foreign Characters – (e.g. ç, ¿, ß)
  3. Currency Symbols – (e.g. ¢, £, ¥)
  4. Math Symbols – (e.g. ±, °, ÷)
  5. Other Punctuation – (e.g. &, ©, §)
  6. Using the Codes
  7. Other Accents and Symbols: Character Map Other Page
  8. Non-Numeric Accent Codes: Activate International Keyboard Other Page
  9. Links to Other References

–jeroen

Posted in Development, Encoding, Power User, Software Development, Unicode, Windows, Windows 7, Windows 8, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista, Windows XP | Leave a Comment »

Great Unicode presentation by

Posted by jpluimers on 2015/01/21

Stefan Heymann did a great presentation Character Sets and Unicode in Firebird at fbcon11. About 90% of it is not about Firebird, but about Unicode: a highly recommended presentation.

There is also a PDF version of the same presentation for easier reading/searching.

If you like Firebird, there is a whole bunch of Firebird related presentations from various authors shared by MindTheBird.

–jeroen

Posted in Ansi, Database Development, Development, Encoding, Firebird, ISO-8859, ISO8859, Software Development, Unicode, UTF-8, UTF8 | Leave a Comment »

Delphi: ZEROBASEDSTRINGS and maintaining cross-version Delphi libraries

Posted by jpluimers on 2015/01/14

One of the features that bites me over and over again is the ZEROBASEDSTRINGS that got introduced in Delphi XE3 and is by default ON in mobile compilers and OFF in Desktop compilers.

Back then, Mark Edington showed a small example of the effects:


procedure ZeroBasedTest;
const
S: string = '012';
begin
{$ZEROBASEDSTRINGS OFF}
Writeln(S[1]); // shows "0"
Writeln(S.Chars[1]); // shows "1"
{$ZEROBASEDSTRINGS ON}
Writeln(S[1]); // shows "1"
Writeln(S.Chars[1]); // shows "1"
end;

view raw

gistfile1.txt

hosted with ❤ by GitHub

and then explained:

The XE3 RTL source code has been refactored to be string index base agnostic. In most cases this is done by utilizing string helper functions which are always zero based.
When it is necessary to traverse a string, the Char[] property is often used to access the individual characters without concern for the current state of the compiler with respect to zero based strings.

In addition, the “Low” and “High” standard functions can now be passed a string variable to provide further flexibility as needed.
When zero based strings are enabled, Low(string) will return 0,  otherwise it will return 1. Likewise, High() returns a bounds adjusted length variation.

The problem is the non-existent forward compatibility of the other compilers (Delphi XE2 and lower).

So if you have library code that needs to work in Delphi versions, you cannot use the High and Low to make the code ZEROBASEDSTRINGS neutral.

Many Delphi developers regularly skip many Delphi versions, so these are still popular:

  • Delphi XE1 and XE2 (the last 2 compilers before Delphi really started to support mobile)
  • Delphi 2007 (the last non-Unicode Delphi compiler)
  • Delphi 7 (the last non-Galileo IDE)

The result is that library code is full of conditionan IF/IFDEF blocks like these:

Fix: this works only in XE3 or higher: “for Index := Low(input) to High(input) do // for ZEROBASEDSTRINGS”


{$ifdef GX_VER240_up}
for Index := Low(input) to High(input) do // for ZEROBASEDSTRINGS
{$else}
for Index := 1 to Length(input) do
{$endif GX_VER240_up}

view raw

gistfile1.txt

hosted with ❤ by GitHub

–jeroen

via: Mark Edington’s Delphi Blog : XE3 RTL Changes: A closer look at TStringHelper.

Posted in Ansi, Delphi, Delphi 2007, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Encoding, Software Development, Unicode | 8 Comments »

Default comparers in Delphi used by TArray.Sort (via: Stack Overflow)

Posted by jpluimers on 2014/11/26

A long while ago, I posted a detailed answer on what functions the default comparers actually were calling to get a feel for if they would apply or not answering delphi – What does the default TArray.Sort comparator actually do and when would you use it? – Stack Overflow.

I needed that information recently because of some sorting issues I bumped into (sorting generic records), so finally a blog post.

First some links to documentation for even more background information:

There is the answer I gave: Read the rest of this entry »

Posted in Algorithms, Ansi, Delphi, Delphi 2009, Delphi 2010, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Encoding, Floating point handling, Software Development, Unicode | 2 Comments »