The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,839 other subscribers

Archive for the ‘*nix-tools’ Category

OpenSuSE and logging: no more syslogd; journald is default, you can use rsyslog or syslog-NG as syslogd replacements

Posted by jpluimers on 2016/11/15

In the 1990s and early 2000s I did a lot of Unix-Like (Minix, SunOS, HP-UX, Xenox) and later Linux (mostly RedHat and SuSE) work. The internet and Linux weren’t as big as they are now and old stuff was still in use including syslogd.

So recently wanting to do more on the Linux side of things using OpenSuSE (as 15+ years ago, I spent most of my time with SuSE Linux) and assumed logging was still done using syslogd like Mac OS X does.

Boy, I was wrong. Like the internet and lots of other things, logging on OpenSuSE has fragmented in at least these three categories of which two syslog implementations (but syslogd is deprecated and – according to the URC #SUSE Channel – unmaintained):

  • journald (installed by default on my Tumbleweed text-only systems)
  • rsyslog (which is supposed to be default on modern OpenSuSE installs but somehow isn’t on my Tumbleweed but is on 13.1 and 13.2)
  • syslog-ng
  • proprietary logging (of many applications in /var/log like named, apache, etc)

There seems to be heated debates on what to use when, so I’ll try to stick with the defaults as much as possible.

A few things I need to sort out:

  1. where is journald persisted?
  2. how can journald being rotated?
  3. what to do with packages that require one form of syslog or the other?
    • not sure yet
  4. can I direct journald to a syslog implementation?
  5. how does this apply to other distros?
    • not sure yet

Tonu Su (TSu2) posted an elaborate answer on the above questions on the OpenSuSE forums.

–jeroen

via:

Posted in *nix, *nix-tools, About, Linux, openSuSE, Power User, RedHat, SuSE Linux, Tumbleweed | 2 Comments »

Can I connect to or view abandoned ssh sessions?

Posted by jpluimers on 2016/11/14

Boy  I wish I had known about screen and tmux years ago. Screen is such a generic term that I never bumped into it, but tmux is easier to find and I like it more. When on the road, I regularly loose SSH sessions, so I’ve been starting tmux ever since I discovered it and reattach to it whenever needed thereby getting the same exact she’ll I was connected to.
http://unix.stackexchange.com/q/598/69111

–jeroen

Posted in *nix, *nix-tools, Communications Development, Development, Linux, openSuSE, Power User, SSH, SuSE Linux, TCP | Leave a Comment »

openSuSE Tumbleweed: solution for `ImportError: No module named pkg_resources`

Posted by jpluimers on 2016/11/11

If ever on openSuSE Tumbleweed you get an error ImportError: No module named pkg_resources then check you have the installed the python-setuptools package it is different from python3-setuptools which was installed by default but is not the default python used.

This is how to install it:

zypper install python-setuptools

Tools like speedtest-cli require it.

The odd thing: on a Mac, the homebrew speedtest-cli installed and ran with no additional packages needed:

retinambpro1tb:tmp jeroenp$ brew install speedtest-cli
==> Downloading https://github.com/sivel/speedtest-cli/archive/v0.3.2.tar.gz
==> Downloading from https://codeload.github.com/sivel/speedtest-cli/tar.gz/v0.3.2
######################################################################## 100.0%
🍺 /usr/local/Cellar/speedtest_cli/0.3.2: 5 files, 52K, built in 2 seconds
retinambpro1tb:tmp jeroenp$ speedtest-cli
Retrieving speedtest.net configuration...
Retrieving speedtest.net server list...
Testing from Routit BV (37.153.243.246)...
Selecting best server based on latency...
Hosted by ExtraIP (Amersfoort) [3.99 km]: 6.488 ms
Testing download speed........................................
Download: 49.89 Mbit/s
Testing upload speed..................................................
Upload: 47.81 Mbit/s

(this is on one of my fiber connections back when it was 50/50 megabit).

Note that for both the web interface of speedtest.net and command-line versions (like Python based speedtest-cli) sometimes needs some fiddling with chosen servers and repeated measurements to get a consistent average as quite some factors can influence the measurements.

For my home location, this one gives me the most consistent results for my fiber connections (they’re so good and reliable that I don’t have ADSL or cable any more):

speedtest-cli --server 3629

You can get the list of servers ordered by increasing distance using this command:

speedtest-cli --list | head -n 20

–jeroen

Posted in *nix, *nix-tools, Internet, Linux, openSuSE, Power User, SpeedTest, SuSE Linux | 2 Comments »

parted “Error: Can’t have a partition outside the disk!” sometimes means you’ve a bogus DVD inserted

Posted by jpluimers on 2016/11/07

At first sight I thought I had a damaged partition table on the HDD, but then I realised it was a bogus DVD.

parted -l would give me this:

Error: Can't have a partition outside the disk!
Ignore/Cancel? i                                                          
Error: Can't have a partition outside the disk!
Ignore/Cancel? i                                                          
Model: NECVMWar VMware IDE CDR10 (scsi)
Disk /dev/sr0: 4647MB
Sector size (logical/physical): 2048B/2048B
Partition Table: msdos
Disk Flags: 

Number  Start   End     Size    Type     File system  Flags
 1      7340kB  23.9MB  16.5MB  primary               esp, type=ef
 2      23.9MB  18.6GB  18.6GB  primary               boot, hidden, type=17

A simple eject /dev/sr0 solved the issue.

Too bad there is no way to force parted to ignore errors (or specify a default answer).

–jeroen

Posted in *nix, *nix-tools, Linux, openSuSE, Power User, SuSE Linux, Tumbleweed | Leave a Comment »

What runs logrotate in OpenSUSE 13.2?

Posted by jpluimers on 2016/11/04

Historically, on many systems, logrotate is being ran from a daily cron job. Many tutorials still presume that, for instance HowTo: The Ultimate Logrotate Command Tutorial with 10 Examples.

I still thought it would and after writing On OpenSuSE, when adding Apache vhosts with their own log files don’t forget to update your logrotate configuration I was anxious to see when logrotate would run the second time.

So I tried finding it in the cron.daily and it wasn’t there.

OpenSuSE 13.2 changed how logrotate is inficated: from there on (including both Tumbleweed and LEAP) logrotate is ran from the systemd service:

logrotate is a systemd service in 13.2 (/usr/lib/systemd/system/logrotate.service) and it is run periodically by a systemd timer (not cron).

Have a look at /usr/lib/systemd/system/logrotate.timer and “man systemd.timer”.

You can view the status of the logrotate.timer that fires it every day:

systemctl status logrotate.timer

It triggers logrotate and reads the config in /etc/logrotate.conf for basic global settings and then files in /etc/logrotate.d/* for custom settings for specific files.

Which means you should not mess around with files in /etc/logrotate.d/ as each file there will be processed. So don’t leave around backup files ending in a tilde (~) or DEADJOE as it causes trouble:

Jul 07 00:00:02 revue logrotate[16121]: error: DEADJOE:5 lines must begin with a keyword or a filename (possibly in double quotes)
Jul 07 00:00:02 revue logrotate[16121]: error: DEADJOE:6 missing '{' after log files definition
Jul 07 00:00:02 revue logrotate[16121]: error: found error in file DEADJOE, skipping

There’s more you can do do debug logrotate behaviour

The below tips are all based on this thread: [Bug 913421] logrotate not running after update from 13.1 to 13.2

Show if the timer is there and counting:

# systemctl list-timers --all
NEXT                          LEFT          LAST                          PASSED    UNIT                         ACTIVATES
Fri 2016-07-08 00:00:00 CEST  5h 24min left Thu 2016-07-07 00:00:02 CEST  18h ago   logrotate.timer              logrotate.service
Fri 2016-07-08 17:51:53 CEST  23h left      Thu 2016-07-07 17:51:53 CEST  43min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service

2 timers listed.

Show the status of the logrotate service itself:

# systemctl status logrotate.service --full
● logrotate.service - Rotate log files
   Loaded: loaded (/usr/lib/systemd/system/logrotate.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2016-07-07 00:00:02 CEST; 18h ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 16121 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
 Main PID: 16121 (code=exited, status=1/FAILURE)

Jul 07 00:00:02 revue logrotate[16121]: error: DEADJOE:5 lines must begin with a keyword or a filename (possibly in double quotes)
Jul 07 00:00:02 revue logrotate[16121]: error: DEADJOE:6 missing '{' after log files definition
Jul 07 00:00:02 revue logrotate[16121]: error: found error in file DEADJOE, skipping
Jul 07 00:00:02 revue logrotate[16121]: error: skipping "/var/log/squidGuard/squidGuard.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
Jul 07 00:00:02 revue logrotate[16121]: compress_ext is /usr/bin/xz
Jul 07 00:00:02 revue logrotate[16121]: compress_ext was changed to .xz
Jul 07 00:00:02 revue systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Jul 07 00:00:02 revue systemd[1]: Failed to start Rotate log files.
Jul 07 00:00:02 revue systemd[1]: logrotate.service: Unit entered failed state.
Jul 07 00:00:02 revue systemd[1]: logrotate.service: Failed with result 'exit-code'.

The bottom lines are from journalctl -u logrotate which can show more information.

In this case, fixing both issues was easy: remove DEADJOE and correct the permissions on this empty directory:

# ls -al /var/log/squidGuard/
total 0
drwxrwx--- 1 squid squid   0 Jun 16 21:08 .
drwxr-xr-x 1 root  root  962 Jul  6 17:36 ..
# chmod 750 /var/log/squidGuard/
# ls -al /var/log/squidGuard/
total 0
drwxr-x--- 1 squid squid   0 Jun 16 21:08 .
drwxr-xr-x 1 root  root  962 Jul  6 17:36 ..

If you can’t wait for the timer to fire at midnight, you can invoke the logrotate service manually (after that wait until it is finished then do something like du -csh /var/log/* or list the files):

# systemctl start logrotate.service

–jeroen

via:

Posted in *nix, Linux, logrotate, openSuSE, Power User, SuSE Linux, Tumbleweed | Leave a Comment »

OpenSuSE fix “piix4_smbus 0000:00:07.3: SMBus Host Controller not enabled!”

Posted by jpluimers on 2016/11/01

If you see this in journalctl after boot in a VM, then you likely want to disable piix4 smbus device detection:

Jul 07 23:02:47 revue systemd-udevd[507]: maximum number (136) of children reached
Jul 07 23:02:47 revue systemd-udevd[507]: maximum number (136) of children reached
...
Jul 07 23:02:47 revue systemd-udevd[507]: maximum number (136) of children reached
Jul 07 23:02:47 revue systemd-udevd[507]: maximum number (136) of children reached
...
Jul 07 23:02:47 revue kernel: piix4_smbus 0000:00:07.3: SMBus Host Controller not enabled!

The solution is to add one line to /etc/modprobe.d/50-blacklist.conf  (well: maybe [WayBack] add a comment line as well):

blacklist i2c_piix4

via:

–jeroen

 

Posted in *nix, *nix-tools, bash, bash, Development, Linux, openSuSE, Power User, Scripting, Software Development, SuSE Linux, Tumbleweed | 2 Comments »

On OpenSuSE, when adding Apache vhosts with their own log files don’t forget to update your logrotate configuration

Posted by jpluimers on 2016/10/27

Sometimes you forget one crucial step…

When adding Apache vhosts on OpenSuSE and each vhost has it’s own set of log-files, then they will not be logrotated by default.

So you have to edit the configuration.

I’ve done it by copying the default apache2 logrotate configuration file for each vhost like this:

/etc/logrotate.d # cp apache2 apache2.vhost.##hostname##

Here ##hostname## is the name of the vhost.

Then I edited each file and replaced the generic log file names with the specific ones for each vhost.

There are only a few vhosts on my system so the manual job wasn’t so bad, but with a great number of vhosts you’d probably want to make this a template process beyond this:

function logrotate-add-apache2-vhost-file()
{
  # $1 is the vhost name
  ## http://stackoverflow.com/questions/16790793/how-to-replace-strings-containing-slashes-with-sed/16790877#16790877
  cat /etc/logrotate.d/apache2 | sed -r "s#/var/log/apache2/#/var/log/apache2/$1-#g" > /etc/logrotate.d/apache2.vhost.$1 
  git add /etc/logrotate.d/apache2.vhost.$1
}

This will then show in less what logrotate (which will output both to stderr and stdout, hence the 2>&1 redirect) would do on the next invocation:

logrotate -d /etc/logrotate.conf 2>&1 | less

And this is a very nice logrotate alias as well:

alias logrotate-show-status='echo "# systemctl list-timers --all" && systemctl list-timers --all && echo "# systemctl status logrotate.timer --full" && systemctl status logrotate.timer --full && echo "# journalctl -u logrotate" && journal

–jeroen

Posted in *nix, *nix-tools, Apache2, Development, Linux, logrotate, openSuSE, Power User, Scripting, Software Development, SuSE Linux, Tumbleweed | 1 Comment »

How to copy files from one machine to another using ssh – Unix & Linux Stack Exchange

Posted by jpluimers on 2016/10/25

I’m using Linux (centos) machine, I already connected to the other system using ssh. Now my question is how can I copy files from one system to another system?

Source: How to copy files from one machine to another using ssh – Unix & Linux Stack Exchange

Nice question, uh? In my opinion the best answer is “Use scp to avoid going through hoops with complex configurations to re-use your existing ssh connection” like this:

To copy a file from B to A while logged into B:

    scp /path/to/file username@A:/path/to/destination

To copy a file from B to A while logged into A:

    scp username@B:/path/to/file /path/to/destination

Source: DopeGhoti answering How to copy files from one machine to another using ssh – Unix & Linux Stack Exchange

Instead the question is marked duplicate of SSH easily copy file to local system – Unix & Linux Stack Exchange where (contrary to the ‘easily’ part of the question) go through hoops and loops with all kinds of fancy ssh settings and port forwards.

Recursive

For recursive, use the -r option, as per [WayBack] shell – How to copy a folder from remote to local using scp? – Stack Overflow:

scp -r user@your.server.example.com:/path/to/foo /home/user/Desktop/

From man scp (See online manual)

-r Recursively copy entire directories

Related:

Read the rest of this entry »

Posted in *nix, *nix-tools, bash, Communications Development, Development, Internet protocol suite, Power User, Scripting, Software Development, SSH, TCP | Leave a Comment »

Merging multiple commands and piping it to one output.

Posted by jpluimers on 2016/10/20

The unix shell is hard, but boy, sometimes it can work like magic, for instance piping two testssl.sh commands into one gist:

retinambpro1tb:testssl.sh jeroenp$ ( ./testssl.sh --version ; ./testssl.sh --local ) | gist -d "testsll version and local ciphers for Mac OS X Darwin binarries supporting zlib"
https://gist.github.com/701496d7fbf929967aa1

The source of this magic was this AskUbuntu answer: How to merge and pipe results from two different commands to single command? – Ask Ubuntu

–jeroen

via: openssl.Darwin.x86_64 lacks zlib support · Issue #164 · drwetter/testssl.sh

Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development, Uncategorized | Leave a Comment »

Wish ttystudio was available for OpenSuSE and Mac OS X…

Posted by jpluimers on 2016/10/10

Really interesting stuff: ttystudio. It allows to record an apng or gif of a terminal session (so it should work on headless systems).

Anyone knowing alternatives for OpenSuSE and Mac OS X?

(Cockos Incorporated | LICEcap might cut it on Mac OS X, but not on headless systems so GNOME/byzanz doesn’t cut it either)

Sources:

Handy as well:

–jeroen

Posted in *nix, *nix-tools, Apple, Linux, Mac, Mac OS X / OS X / MacOS, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, MacMini, openSuSE, Power User, SuSE Linux, Tumbleweed | Leave a Comment »