The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My work

  • My badges

  • Twitter Updates

  • My Flickr Stream

    20140508-Delphi-2007--Project-Options--Cannot-Edit-Application-Title-HelpFile-Icon-Theming

    20140430-Fiddler-Filter-Actions-Button-Run-Filterset-now

    20140424-Windows-7-free-disk-space

    More Photos
  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,724 other followers

Archive for the ‘*nix’ Category

`-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)` on fresh Raspbian on Raspberry

Posted by jpluimers on 2017/03/24

Every time I logged onto a freshly installed Rasbian system (Debian Jessie), I had this message:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Aug 27 19:52:33 2016 from 192.168.171.24
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Asking for the locale settings would give this:

jeroenp@raspberrypi:~ $ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_GB.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

Searching for raspbian jessie “-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)” I found fix locale issue #15 · scaleway/image-debian@543e9b4 [WayBack] that fixes Locale issue on Debian Jessie · Issue #15 · scaleway/image-debian · GitHub [WayBack]:

When logging into to a freshly booted debian jessie image:

-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
-bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Source: Locale issue on Debian Jessie · Issue #15 · scaleway/image-debian

I changed the crux of that solution to this one:

echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen && locale-gen

When running, it showed this:

# echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen && locale-gen
Generating locales (this might take a while)...
  en_GB.UTF-8... done
  en_US.UTF-8... done
Generation complete.

And logging this:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Aug 27 20:26:34 2016 from 192.168.171.24

Problem solved: 1 line of code!

–jeroen

Posted in *nix, Arduino, Debian, Development, Hardware Development, Linux, Power User, Raspberry Pi, Raspbian | Leave a Comment »

ducks: How Do I Find The Largest Top 10 Files and Directories On a Linux / UNIX / BSD?

Posted by jpluimers on 2017/03/20

Explains how to find out top 10 files and directories under Unix / Linux using sort and du command in human-readable format.

Interesting, especially

alias 'ducks=du -cks * | sort -rn | head'

Source: How Do I Find The Largest Top 10 Files and Directories On a Linux / UNIX / BSD?

via: Joe C. Hecht and nixCraft.

–jeroen

Posted in Power User, *nix, Apple, OS X, Mac, Linux, *nix-tools, BSD | Leave a Comment »

sed: convert Google Drive urls to direct download ones

Posted by jpluimers on 2017/03/14

RegEx Fu

RegEx Fu

One of the things after moving most of my things from copy.com to Google Drive was the direct (public) download URLs that copy.com provides. DropBox has them as well, but Google Drive lacks them in the UI.

There is a URL format that does allow for direct download though:

While Google aims for Drive to be a competent Dropbox competitor, there’s one small but key feature that isn’t easy: sharing direct download links. Fortunately, you can create your own.

Source: Share Direct Links to Files in Google Drive and Skip the Web Viewer

You can do a similar replacement for Google Doc URLs: How to Create Direct Download Links for Files on Google Drive

The Google Drive conversion seems straightforward as they convert from either of

https://drive.google.com/file/d/FILE_ID/edit?usp=sharing
https://drive.google.com/file/d/FILE_ID/view
https://drive.google.com/open?id=FILE_ID

to

https://drive.google.com/uc?export=download&id=FILE_ID

There are tons of RegEx examples for doing the first conversion at Regex to modify Google Drive shared file URL – Stack Overflow, but

  1. they don’t cover the two conversions
  2. they use the non-greedy (.*?) capturing groups which are tricky, introduce question mark escaping issues in hash and many sed implementations fail to implement non-greedy

Since I’m a command-line person, I’ve opted for a sed conversion that wasn’t in the above list. I choose sed because it allows you to convert either a line or a complete file at one time.

There are a few indispensable resources to get my regex expressions right:

So here it goes, starting with fixing https://drive.google.com/open?id=FILE_ID as it’s the most simple replacement because the FILE_ID is at the end.

First of all, these code fragments below are part of bash functions as bash functions remove the quoting hell you have with bash aliases.

Where bash aliases have no parameters (i.e. the arguments are put after the end of the expansion), functions have parameters. So if you want to pass all function parameters to a command inside a function, you have to use “$@” to pass all parameters.

This fragment fixes https://drive.google.com/open?id=FILE_ID printing each fix on one line using the p for printing command in sed:

sed -n 's@https://drive.google.com/open?id=@https://drive.google.com/uc?export=download\&id=@p' "$@"

A few remarks:

The second fragment fixes https://drive.google.com/file/d/FILE_ID/edit?usp=sharing and https://drive.google.com/file/d/FILE_ID/view again printing each fix:

sed -n 's@https://drive.google.com/file/d/\([^.]*\)/.*@https://drive.google.com/uc?export=download\&id=\1@p' "$@"

Some more remarks:

  • The FILE_ID is obtained from a capturing group during the match using \([^.]*\) and using the value in the replace with \1 as reference.
  • There is backslash escaping of the parentheses because that’s the sed way.
  • I’ve used a non-greedy \(.*?\) capturing group (sed can’t do that) but \([^.]*\)/ which matches any non-slash inside the capturing group until the first slash outside that group.

The final part is combing both replacement into one sed command:

sed 's@https://drive.google.com/open?id=@https://drive.google.com/uc?export=download\&id=@;s@https://drive.google.com/file/d/\([^.]*\)/.*@https://drive.google.com/uc?export=download\&id=\1@' "$@"

Final remarks:

–jeroen

Posted in Software Development, Development, Power User, *nix, Scripting, bash, sed script, *nix-tools, bash, sed | Leave a Comment »

Sniffers, Packet Capture – PFSenseDocs – cool, as it uses tcpdump/Wireshark format!

Posted by jpluimers on 2017/03/13

I hadn’t done a lot with pfSense in the past, which I regret a bit since I discovered this really cool feature: Sniffers, Packet Capture – PFSenseDocs.

The coolness isn’t so much that you can capture packets, but that it’s compatible with tcpdump and Wireshark (which has become available natively for Mac like 2 years ago).

Which means that you can download captures and open them in Wireshark.

So it’s as easy as 1,2,3:

  1. Set-up the capture on your router https://a.b.c.d/diag_packet_capture.php and start it
  2. Stop the capture and download the file
  3. Open the file in Wireshark or convert it to text using tshark

–jeroen

Posted in *nix, *nix-tools, Internet, Monitoring, pfSense, Power User, Routers, tcpdump, Wireshark | Leave a Comment »

node.js – a nightmare to get started. Did I try the wrong technology for my problem?

Posted by jpluimers on 2017/03/08

Most of my web-stuff is on Apache. Which works fine, has TLS/SSL enabled, etc.

But I wanted to do server-side JavaScript. Which somehow is a forrest without trees, or a nightmare to get started, especially on OpenSuSE.

First of all, virtually all examples explain how to run node as a script. But none explain where to save it, how to run it as a service (and restart when it crashes: it will crash) or how to run multiple sites under it. And the scripts seems to listen to a TCP port by themselves so they operate as a full server by themselves. Nice for a fully fledged portal, but not for some one-offs.

Some links below hopefully will get me re-started later on, but for now, I’ve given up: the out-of-the-box experience is totally non-intuitive.

Maybe what I really want is something else: I want JavaScript stuff that normally renders a page in the browser through the dom to run server side so I can run XMLHttpRequest to various places without bumping into CORS stuff but still render a page DOM.

If you know a better way to do what I want (serving small mostly single-page scripts written in an easy to debug/trace language) let me know.

So basically work around this:

XMLHttpRequest cannot load http://myApiUrl/login. No
'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'null' is therefore not allowed access.

Read the rest of this entry »

Posted in *nix, Apache2, Development, JavaScript/ECMAScript, Power User, Scripting, Software Development | 1 Comment »

 
%d bloggers like this: