David Cumps discovered that certain text files come up strange in Notepad. The reason is that Notepad has to edit files in a variety of encodings, and when its back against the wall, sometimes it’s forced to guess.
As a precursor to a post tomorrow showing that serving UTF8 does not mean organisations go without unicode problems, first some statistics.
The first Unicode ideas got drafted some 30 years ago in 1987. In 1991, more than 30 years ago, the Unicode Consortium saw the light. Nowadays more than 95% percent of the web-pages (close to 100% when you include plain ASCII) is served using the UTF-8 encoding.
It means that nowadays there is a very small chance you
will see mangled characters (what Japanese call mojibake) when you’re surfing the web.
Some nice graphs of unicode growth are at these locations are at these locations:
I think especially important are 2008 (when UTF-8 had outgrown all other individual encodings) and slightly after 2010, when UTF-8 alone covered more than 50% of the pages served. These exclude ASCII-only pages. Adding those would make the figures even larger.
Historical yearly trends in the usage statistics of character encodings for websites, June 2021
One of the most common upgrades for any Haswell (xx40) series ThinkPad is to replace the awful button-less touchpad (sometimes referred to as the ClunkPad) with a T450 touchpad that has the proper buttons for TrackPoint users. However, getting the buttons to work properly on xx40 hardware can be tricky – particularly if you are running Windows 10. In this video you will see how to get drivers installed that will allow you to use the TrackPoint as if this were a T450! As always thanks for watching! Some of the guides I used for this video:
I have edited the link so they show forum post titles, added way-back links, and added some crucial information:
PLEASE NOTE:
BEFORE YOU PUT IN THE NEW TOUCHPAD, INSTALL SOME EXTRA BUMPERS/RUBBERS ALONG THE TOP OF THE LID, LEFT AND RIGHT OF THE WEBCAM.
THESE SHOULD BE ABOUT TWICE AS THICK/HIGH AS THE SMALL BUMPERS THAT ARE THERE ALREADY.
Recommended size ~WxH: ~6 x 2 mm or ~1/4 x 5/64 inch.
PLACE THEM IMMEDIATELY LEFT AND RIGHT OF THE EXISTING BUMPERS.
FAILURE TO DO SO WILL RESULT IN SCRATCH MARKS ON THE SCREEN FROM THE NEW BUTTONS!
I needed to search for IBAN numbers in documents and used this regular expression: [a-zA-Z]{2}[0-9]{2} ?[a-zA-Z0-9]{4} ?[0-9]{4} ?[0-9]{4} ?[0-9]{2} which supports the usual optional whitespace like in NL12 INGB 0345 6789 01.
[set] ⇒ This indicates a set of characters, for example, [abc] means any of the literal characters a, b or c. You can also use ranges by doing a hyphen between characters, for example [a-z] for any character from a to z. You can use a collating sequence in character ranges, like in [[.ch.]-[.ll.]] (these are collating sequence in Spanish).
[^set] ⇒ The complement of the characters in the set. For example, [^A-Za-z] means any character except an alphabetic character. Care should be taken with a complement list, as regular expressions are always multi-line, and hence [^ABC]* will match until the first A, B or C (or a, b or c if match case is off), including any newline characters. To confine the search to a single line, include the newline characters in the exception list, e.g. [^ABC\r\n].
Please note that the complement of a character set is often many more characters than you expect: (?-s)[^x]+ will match 1 or more instances of any non-x character, including newlines: the (?-s)search modifier turns off “dot matches newlines”, but the [^x] is not a dot ., so that class is still allowed to match newlines.
[[:name:]] or [[:☒:]] ⇒ The whole character class named name. For many, there is also a single-letter “short” class name, ☒. Please note: the [:name:] and [:☒:] must be inside a character class [...] to have their special meaning.
short
full name
description
equivalent character class
alnum
letters and digits
alpha
letters
h
blank
spacing which is not a line terminator
[\t\x20\xA0]
cntrl
control characters
[\x00-\x1F\x7F\x81\x8D\x8F\x90\x9D]
d
digit
digits
graph
graphical character, so essentially any character except for control chars, \0x7F, \x80
l
lower
lowercase letters
print
printable characters
[\s[:graph:]]
punct
punctuation characters
[!"#$%&'()*+,\-./:;<=>?@\[\\\]^_{
s
space
whitespace (word or line separator)
[\t\n\x0B\f\r\x20\x85\xA0\x{2028}\x{2029}]
u
upper
uppercase letters
unicode
any character with code point above 255
[\x{0100}-\x{FFFF}]
w
word
word characters
[_\d\l\u]
xdigit
hexadecimal digits
[0-9A-Fa-f]
Note that letters include any unicode letters (ASCII letters, accented letters, and letters from a variety of other writing systems); digits include ASCII numeric digits, and anything else in Unicode that’s classified as a digit (like superscript numbers ¹²³…).
Note that those character class names may be written in upper or lower case without changing the results. So [[:alnum:]] is the same as [[:ALNUM:]] or the mixed-case [[:AlNuM:]].
As stated earlier, the [:name:] and [:☒:] (note the single brackets) must be a part of a surrounding character class. However, you may combine them inside one character class, such as [_[:d:]x[:upper:]=], which is a character class that would match any digit, any uppercase, the lowercase x, and the literal _ and = characters. These named classes won’t always appear with the double brackets, but they will always be inside of a character class.
If the [:name:] or [:☒:] are accidentally not contained inside a surrounding character class, they will lose their special meaning. For example, [:upper:] is the character class matching :, u, p, e, and r; whereas [[:upper:]] is similar to [A-Z] (plus other unicode uppercase letters)
[^[:name:]] or [^[:☒:]] ⇒ The complement of character class named name or ☒ (matching anything not in that named class). This uses the same long names, short names, and rules as mentioned in the previous description.
I wonder what kind of parser they use, as these printable special ASCII characters are forbidden:
\-/[\]^`{~
space (0x20)
tab (0x9)
line feed (0xa)
carriage return (0xb
vertical tab (0xb)
form feed (0xc)
Seems no JSON or SQL to me: there I would expect other limitations.
What would break if you use them in other fields or pass them in an HTML POST-request?
I mean: these passwords should be salted and hashed immediately when the HTML-POST request is received, so certainly they would not be stored somewhere or passed many layers into code, right?
Oh, in order to activate an account there, you need to accept some 40+ A4 sized pages of legal stuff. Brave Dutch judge that will put these all in favour of Hornbach.
Sometimes it is easier to have current and public CA signed TLS certificates for internal servers than to setup and maintain an internal CA and register it on all affected browsers (including mobile phones).
The Wiert Corner - irregular stream of stuff 