Posted by jpluimers on 2017/11/07
A well worth long rad:
We all recognize emoji. They’ve become the global pop stars of digital communication. But what are they, technically speaking? And what might we learn by taking a closer look at these images, characters, pictographs… whatever they are 🤔 (Thinking Face). We will dig deep to learn about how these thingamajigs work. Please note: Depending on your browser, you may not be able to see all emoji featured in this article (especially the Tifinagh characters). Also, different platforms vary in how they display emoji as well. That’s why the article always provides textual alternatives. Don’t let it discourage you from reading though! Now, let’s start with a seemingly simple question. What are emoji?
[WayBack] You, Me And The Emoji: Character Sets, Encoding And Emoji – Smashing Magazine
Via: [WayBack] Everything you ever wanted to know about characters, encodings, glyphs… and, oh yeah, emoji: bit.ly/2fNKeW3Long, rewarding read. – Ilya Grigorik – Google+
Here is just the ToC:
TABLE OF CONTENTS LINK
- Character Sets And Document Encoding: An Overview
- Characters
- Character Sets
- Coded Character Sets
- Encoding
- Declaring Character Sets And Document Encoding On The Web
- content-type HTTP Header Declaration
- Checking HTTP Headers Using A Browser’s Developer Tools
- Checking HTTP Headers Using Web-based Tools
- Using A Meta Element With charset Attribute
- An Encoding By Any Other Name
- What Were We Talking About Again? Oh Yeah, Emoji!
- So What Are Emoji?
- How Do We Use Emoji?
- Character References
- Glyphs
- How Do We Know If We Have These Symbols?
- The Great Emoji Proliferation Of 2016
- Emoji OS Support
- Emoji Support: Apple Platforms (macOS and iOS)
- Emoji Support: Windows
- Emoji Support: Linux
- Emoji Support: Android
- Emoji On The Web
- Emoji One
- Twemoji
- Conclusion
–jeroen
Like this:
Like Loading...
Posted in ASCII, Development, Encoding, ISO-8859, ISO8859, Shift JIS, Unicode, UTF-16, UTF-8, UTF16, UTF8, Windows-1252 | Leave a Comment »
Posted by jpluimers on 2017/09/25
At the end of April 2014, Roman Yankovsky started a nice discussion on Google+ trying to get upvotes for QualityCentral Report #: 124402: Compiler bug when comparing chars.
His report basically comes down to that when using Ansi character literals like #255, the compiler treats them as single-byte encoded characters in the current code page of your Windows context, translates them to Unicode, then processes them.
The QC report has been dismissed as “Test Case Error” (within 15 minutes of stating “need more info”) by one of the compiler engineers, directing to the UsingCharacterLiterals section of Delphi in a Unicode World Part III: Unicodifying Your Code where – heaven forbid – they suggest to replace #128 with the Euro-Sign literal.
I disagree, as the issue happens without any hint or warning whatsoever, and causes code that compiles fine in Delphi <= 2007 to fail in subtle ways on Delphi >= 2009.
The compiler should issue a hint or warning when you potentially can screw up. It doesn’t. Not here.
Quite a few knowledgeable Delphi people got involved in the discussion:
Read the rest of this entry »
Like this:
Like Loading...
Posted in Ansi, ASCII, CP437/OEM 437/PC-8, Delphi, Delphi 2006, Delphi 2007, Delphi 2009, Delphi 2010, Delphi 7, Delphi XE, Delphi XE2, Delphi XE3, Delphi XE4, Delphi XE5, Delphi XE6, Development, Encoding, ISO-8859, Missed Schedule, QC, SocialMedia, Software Development, Unicode, UTF-8, Windows-1252, WordPress | Leave a Comment »
Posted by jpluimers on 2017/05/31
A while ago I bumped into applications that write alternating UTF-16 and UTF-8 to files without checking what type of encoding the files were using.
So here are some notes to at least save some of the contents.
TODO: figure out how to strip the BOM.
–jeroen
Like this:
Like Loading...
Posted in Development, Encoding, Software Development, UTF-16, UTF-8, UTF16, UTF8 | Leave a Comment »
Posted by jpluimers on 2016/10/04
A while ago (in fact more than a year), I posted Encoding is hard… go G+ with the below picture.
ftfy (fixes text for you) fixes it, but:
How did the single quote become “’”?
Actually, because of a a common “beautification” of many Office suites (Microsoft and Open alike), the single quote was a special one: a Unicode Character ‘RIGHT SINGLE QUOTATION MARK’ (U+2019) which in UTF-8 is encoded as 0xE2 0x80 0x99.
Read the rest of this entry »
Like this:
Like Loading...
Posted in Development, Encoding, ISO-8859, ISO8859, Software Development, Unicode, UTF-8, UTF8, Windows-1252 | Leave a Comment »
Posted by jpluimers on 2016/09/06
Simple if you know it:
pip install ftfy
That installs it as a command which is a lot easier than using it from Github at https://github.com/LuminosoInsight/python-ftfy
It knows how to solve the encoding issues in  the future of publishing at W3C.
It didn’t solve my non-Unicode encoding issue: “v3/43/4r” -> “v¾¾r” -> “vóór”.
That was caused by an infamous Western Latin character set confusion issue, in this case ISO-8859–/Windows- versus CP850/CP858 encoding issue (so: no Unicode involved at all, nor CP437 as it doesn’t have ¾).
So I put in a suggestion for ftfy to support finding the above.
–jeroen
via
Like this:
Like Loading...
Posted in Development, Encoding, Software Development, Unicode, UTF-8, UTF8 | 4 Comments »
