The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,862 other subscribers

Archive for the ‘Encoding’ Category

Jon Skeet’s speech “Back to basics” is really a good watch – via Jørn Einar Angeltveit G+

Posted by jpluimers on 2015/07/15

Thanks [Wayback] Jørn Einar Angeltveit for sharing this a while ago:

A session by Jon Skeet and Tony the Pony (which has strong teeth) presented during the Polish DevDay 2013 in Kraków, Poland.

[Wayback] +Jon Skeet’s speech [Wayback] “Back to basics” is really a good watch.

In a funny way, he explains why the simplest fundamentals of computer software text, dates and numbers can cause some real headache for the programmer…

In case you didn’t know: Jon Skeet is “Chuck Norris” on [Wayback] stackoverflow.com:

The subtitle is “the mess we’ve made of our fundamental data types”.

Some of the topics covered:

Read the rest of this entry »

Posted in .NET, C#, Conference Topics, Conferences, Delphi, Development, Encoding, Event, internatiolanization (i18n) and localization (l10), Java, Java Platform, Jon Skeet, Pascal, Scripting, Software Development, Unicode | 2 Comments »

20 resources on migrating to Unicode with Delphi | Software on a String

Posted by jpluimers on 2015/07/08

Great post by Marjan Venema when you need to migrate your old Delphi programs to the modern Delphi world: [Wayback] 20 resources on migrating to Unicode with Delphi | Software on a String.

I’m glad that some of the links overlap with what I posted and presented in the past at:

Well done Marjan!

–jeroen

Posted in Ansi, ASCII, Delphi, Delphi 2, Delphi 2005, Delphi 2006, Delphi 2007, Delphi 2009, Delphi 2010, Delphi 3, Delphi 4, Delphi 5, Delphi 6, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Delphi XE6, Delphi XE7, Development, Encoding, Software Development, Unicode | Leave a Comment »

Some command-line tips for OpenSSL and file format (pfx, p12, cer, crt, key, etc.) conversion of certificates, keys

Posted by jpluimers on 2015/02/27

OpenSSL is really nice, but remembering all these command-line switches is difficult, especially when you do not use them often enough.

I don’t, and when I do there are a few common tasks I perform, and I was glad to find a few links with great information:

I’ve converted them to batch files that run fine when copied to the directory where you put the x86 or x64 Windows version of OpenSSL (they assume %~dp0openssl.exe for the location of the OpenSSL.exe binary, just in case it is not on the path, or you have various tools that scattered around incompatible copies of OpenSSL binaries).

OpenSSL defaults to PEM format (that has text base64 strings), so if you get DER format (binary) you need to convert them.

A few errors I got and what they mean

Error decrypting PKCS#7 structure

Error decrypting PKCS#7 structure
5216:error:21070073:PKCS7 routines:PKCS7_dataDecode:no recipient matches certificate:.\crypto\pkcs7\pk7_doit.c:538:
5216:error:21072077:PKCS7 routines:PKCS7_decrypt:decrypt error:.\crypto\pkcs7\pk7_smime.c:557:

This error means that the recipient of the email does not match the certificate you pass in. What happens is that OpenSSL tries to decrypt the mail, it cannot match the certificate to the mail, and barfs. It usually happens when you have From/To reversed by accident.

Error decrypting PKCS#7 structure

Error decrypting PKCS#7 structure
4948:error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch:.\crypto\x509\x509_cmp.c:330:
4948:error:2107207F:PKCS7 routines:PKCS7_decrypt:private key does not match certificate:.\crypto\pkcs7\pk7_smime.c:552:

This means somewhere you mixed up a private and public key in the certificate files.

Use something like the OpenSSL wrapper verify-private-key-matches-certificate-x509-pem-cer.bat to verify them.

Error reading S/MIME message

Error reading S/MIME message
6900:error:0D06B08E:asn1 encoding routines:ASN1_D2I_READ_BIO:not enough data:.\crypto\asn1\a_d2i_fp.c:251:
6900:error:0D0D106E:asn1 encoding routines:B64_READ_ASN1:decode error:.\crypto\asn1\asn_mime.c:193:
6900:error:0D0D40CB:asn1 encoding routines:SMIME_read_ASN1:asn1 parse error:.\crypto\asn1\asn_mime.c:528:

OpenSSL does not like .EML files to end with a period (. which SMTP needs to process when sending an .EML file).
See https://gist.github.com/anonymous/7233372 and https://gist.github.com/anonymous/7233329
The former throws this error, the latter not. This is not caused the width of the base64 encoding (not yet archived at the WayBack machine), which I initially thought, but the terminating period.

Verification failure

Verification failure
8228:error:21075075:PKCS7 routines:PKCS7_verify:certificate verify error:.\crypto\pkcs7\pk7_smime.c:342:Verify error:self signed certificate in certificate chai
n

–jeroen

Posted in base64, Development, Encoding, MIME, OpenSSL, Power User, Security, Software Development, Windows, Windows 7, Windows 8, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista, Windows XP | Leave a Comment »

[NL] encoding blijft moeilijk, waarom toch? (dit keer in een brief van @xs4all)

Posted by jpluimers on 2015/02/24

Hoe moeilijk kan het toch zijn om je encoding goed te doen.

Deze keer uit een brief van xs4all:

Mojibake encoding probleem

Mojibake encoding probleem

Als je een trema in een brief zet, dan controleer je toch even dat die ook goed op de brief wordt afgedrukt?

Read the rest of this entry »

Posted in Development, Encoding, ISO-8859, Mojibake, Software Development, Unicode, UTF-8, UTF8 | Leave a Comment »

Windows Alt Key Codes

Posted by jpluimers on 2015/02/16

Brillant!: Windows Alt Key Codes.

The page has both lists of four-digit ALT+#### codes, a how-to, and reference links:

  1. Letters with Accents – (e.g. ó, ò, ñ)
  2. Other Foreign Characters – (e.g. ç, ¿, ß)
  3. Currency Symbols – (e.g. ¢, £, ¥)
  4. Math Symbols – (e.g. ±, °, ÷)
  5. Other Punctuation – (e.g. &, ©, §)
  6. Using the Codes
  7. Other Accents and Symbols: Character Map Other Page
  8. Non-Numeric Accent Codes: Activate International Keyboard Other Page
  9. Links to Other References

–jeroen

Posted in Development, Encoding, Power User, Software Development, Unicode, Windows, Windows 7, Windows 8, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista, Windows XP | Leave a Comment »

ASCII is not just an RFC. It is an Internet Standard, but only recently.

Posted by jpluimers on 2015/01/29

When people tell you that ASCII is not an Internet Standard but an RFC. They are wrong. They used to be right though. Until 2015-01-12, when IETF declared the RFC 20 to be an Internet Standard: status-change-rfc20-ascii-format-to-standard-00.

So after more than 45 years (like many good things, the ASCII RFC is from 1969), it is not just an American Standard but an Internet Standard (:

Thanks Lauren Weinstein for sharing and Kristian Köhntopp for pointing to the reclassification.

–jeroen

via: ASCII – Wikipedia, the free encyclopedia.

Posted in ASCII, Development, Encoding, History, Software Development | Leave a Comment »

Great Unicode presentation by

Posted by jpluimers on 2015/01/21

Stefan Heymann did a great presentation Character Sets and Unicode in Firebird at fbcon11. About 90% of it is not about Firebird, but about Unicode: a highly recommended presentation.

There is also a PDF version of the same presentation for easier reading/searching.

If you like Firebird, there is a whole bunch of Firebird related presentations from various authors shared by MindTheBird.

–jeroen

Posted in Ansi, Database Development, Development, Encoding, Firebird, ISO-8859, ISO8859, Software Development, Unicode, UTF-8, UTF8 | Leave a Comment »

Delphi: ZEROBASEDSTRINGS and maintaining cross-version Delphi libraries

Posted by jpluimers on 2015/01/14

One of the features that bites me over and over again is the ZEROBASEDSTRINGS that got introduced in Delphi XE3 and is by default ON in mobile compilers and OFF in Desktop compilers.

Back then, Mark Edington showed a small example of the effects:


procedure ZeroBasedTest;
const
S: string = '012';
begin
{$ZEROBASEDSTRINGS OFF}
Writeln(S[1]); // shows "0"
Writeln(S.Chars[1]); // shows "1"
{$ZEROBASEDSTRINGS ON}
Writeln(S[1]); // shows "1"
Writeln(S.Chars[1]); // shows "1"
end;

view raw

gistfile1.txt

hosted with ❤ by GitHub

and then explained:

The XE3 RTL source code has been refactored to be string index base agnostic. In most cases this is done by utilizing string helper functions which are always zero based.
When it is necessary to traverse a string, the Char[] property is often used to access the individual characters without concern for the current state of the compiler with respect to zero based strings.

In addition, the “Low” and “High” standard functions can now be passed a string variable to provide further flexibility as needed.
When zero based strings are enabled, Low(string) will return 0,  otherwise it will return 1. Likewise, High() returns a bounds adjusted length variation.

The problem is the non-existent forward compatibility of the other compilers (Delphi XE2 and lower).

So if you have library code that needs to work in Delphi versions, you cannot use the High and Low to make the code ZEROBASEDSTRINGS neutral.

Many Delphi developers regularly skip many Delphi versions, so these are still popular:

  • Delphi XE1 and XE2 (the last 2 compilers before Delphi really started to support mobile)
  • Delphi 2007 (the last non-Unicode Delphi compiler)
  • Delphi 7 (the last non-Galileo IDE)

The result is that library code is full of conditionan IF/IFDEF blocks like these:

Fix: this works only in XE3 or higher: “for Index := Low(input) to High(input) do // for ZEROBASEDSTRINGS”


{$ifdef GX_VER240_up}
for Index := Low(input) to High(input) do // for ZEROBASEDSTRINGS
{$else}
for Index := 1 to Length(input) do
{$endif GX_VER240_up}

view raw

gistfile1.txt

hosted with ❤ by GitHub

–jeroen

via: Mark Edington’s Delphi Blog : XE3 RTL Changes: A closer look at TStringHelper.

Posted in Ansi, Delphi, Delphi 2007, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Encoding, Software Development, Unicode | 8 Comments »

Default comparers in Delphi used by TArray.Sort (via: Stack Overflow)

Posted by jpluimers on 2014/11/26

A long while ago, I posted a detailed answer on what functions the default comparers actually were calling to get a feel for if they would apply or not answering delphi – What does the default TArray.Sort comparator actually do and when would you use it? – Stack Overflow.

I needed that information recently because of some sorting issues I bumped into (sorting generic records), so finally a blog post.

First some links to documentation for even more background information:

There is the answer I gave: Read the rest of this entry »

Posted in Algorithms, Ansi, Delphi, Delphi 2009, Delphi 2010, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Development, Encoding, Floating point handling, Software Development, Unicode | 2 Comments »

Great answer on “windows – What encoding/code page is cmd.exe using” (via: Stack Overflow)

Posted by jpluimers on 2014/10/06

I just found this [Wayback] great answer (which by now regrettably is deleted; the previous Wayback link still has it) by [Wayback] Јοеу a.k.a. Johannes Rössel on [Wayback] What encoding/code page is cmd.exe using.

The whole answer is worth reading, so I won’t quote only some bits.

Edit 20210609: the answer now has been replaced by an even more detailed answer [Wayback] by [Wayback] andrewdotn. Also recommended reading. The summary of the new answer is this:

The moral of the story?

  • type can print UTF-16LE files with a BOM regardless of your current codepage
  • Win32 programs can be programmed to output Unicode to the console, using WriteConsoleW.
  • Other programs which set the codepage and adjust their output encoding accordingly can print Unicode on the console regardless of what the codepage was when the program started
  • For everything else you will have to mess around with chcp, and will probably still get weird output.

–jeroen

via:   windows – What encoding/code page is cmd.exe using – Stack Overflow.

Posted in Batch-Files, Development, Encoding, Power User, Scripting, Software Development, Windows, Windows 7, Windows 8, Windows Server 2000, Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista, Windows XP | Leave a Comment »