I love short and to the point examples. The list of permutations for scp is at [WayBack] Linux – How to Securely Copy Files Using SCP examples.
–jeroen
Archive for the ‘*nix-tools’ Category
Linux – How to Securely Copy Files Using SCP examples
Posted by jpluimers on 2020/03/16
Posted in *nix, *nix-tools, Development, Internet protocol suite, Power User, SSH, TCP | Leave a Comment »
Show the mmc and USB disks on a Raspberry Pi
Posted by jpluimers on 2020/03/13
Small trick:
find /dev | grep "sd\|mmc" && ls -al /dev/disk/by-id/
–jeroen
Share this:
Posted in *nix, *nix-tools, Power User | Leave a Comment »
command line – Recursive tar compression? – Ask Ubuntu
Posted by jpluimers on 2020/03/09
Since I always forget one-letter command-line options: [WayBack]command line – Recursive tar compression? – Ask Ubuntu (thanks andrew.46 for this very nice answer!):
Try:
tar -czvf directorios.tar.gz folderA few notes:
- Recursion is the default, from the
tarman pages:-c, --create Create a new archive. Arguments supply the names of the files to be archived. Directories are archived recursively, unless the --no-recursion option is given.Although this can be turned off by using the
--no-recursionoption…- You need the archive name immediately after the
-foption, the correct sequence being:tar -c [-f ARCHIVE] [OPTIONS] [FILE...] ^^^^^^^^^^- For a more flexible command line (particularly if you wanted to use other compression utilities apart from gzip with tar) you could omit the
-zoption and use-aoption to allow tar to automatically decide which compressor to use based on the archive suffix:-a, --auto-compress Use archive suffix to determine the compression program.Recognised suffixes are:
- .gz : gzip
- .tgz : gzip
- .taz : gzip
- .Z : compress
- .taZ : compress
- .bz2 : bzip2
- .tz2 : bzip2
- .tbz2 : bzip2
- .tbz : bzip2
- .lz : lzip
- .lzma : lzma
- .tlz : lzma
- .lzo : lzop
- .xz : xz
tar is pretty cool :)
–jeroen
Share this:
Posted in *nix, *nix-tools, Power User | Leave a Comment »
GitHub – facebook/osquery: SQL powered operating system instrumentation, monitoring, and analytics.
Posted by jpluimers on 2020/03/09
Boy, I wish I had found this years ago: [WayBack] GitHub – facebook/osquery: SQL powered operating system instrumentation, monitoring, and analytics.
No more remembering all those nifty configuration and log file details by heart, just install and query using SQL.
Now you need to learn a “database” schema, however that’s the same for all supported operating systems.
Example installation steps:
brew install osquery- Get the official downloads for your OS from the osquery.io site.
Run interactively through osqueryi, then perform .help or .schema to get an impression of what is possible.
There is lots of documentation at [WayBack] osquery.
–jeroen
Share this:
Posted in *nix, *nix-tools, Apple, Mac, Mac OS X / OS X / MacOS, Power User | Leave a Comment »
🔎Julia Evans🔍 on Twitter: “the ip command… “
Posted by jpluimers on 2020/03/06
[Archive.is] 🔎Julia Evans🔍 on Twitter: “the ip command… “
Larger image below the fold; thanks Julia!
Shortened transcript:
ip- lets you view and change network configuration
ip OBJECT COMMAND- object: addr, link, neigh, etc
- command: add, show, delete, etc
ip addr list- shows IP addresses
ip route list- displays the route table
ip route list table all
ip netsh- manage network namespaces
ip link- network devices (like
eth0)
- network devices (like
ip neigh- view/edit the ARP table
ip xfrm- for IPsec
ip -s linksis for statistics- shows transmitted/received packets for each device
Lots of additions in the Twitter Thread too, including (and some of them I still need to figure out):
Share this:
Posted in *nix, *nix-tools, Power User | Leave a Comment »
In Unix, what are some common dot files?
Posted by jpluimers on 2020/03/06
I came across some 20 year old Unix stuff a while ago, so I needed an historic reference of filenames starting with a dot (like .newsrc).
This is a pretty good one: [WayBack] In Unix, what are some common dot files?
–jeroen
Share this:
Posted in *nix, *nix-tools, History, Power User | Leave a Comment »
Mac OS X / macOS / …: quit screen or window when using it for a serial port
Posted by jpluimers on 2020/03/06
I wrote a bit on using screen from my Mac in The woods and trees of OpenSuSE on single-board computers – image abbreviations – and getting it installed using OS X.
The coverage was way too brief and also buried in large post.
Today, I want to focus on some things related to properly terminating a a screen window, session or screen by itself from a Mac which somehow is one of the harder things for me to do.
I’m talking about sessions and windows, because
screenis a terminal multiplexer, which can also put a serial link in a window on a session. So screen is not “just” a serial console terminal application by itself.
Part of being hard is probably that even on for instance 10.12 Sierra, Apple ships a fairly old version: screen --version returns Screen version 4.00.03 (FAU) 23-Oct-06 whereas after 2014 (before that, screen development was pretty stalled) many new versions appeared: [WayBack] screen.git.
It makes it less hard that it hardly reproduces, and seems to reproduce less with the screen version I installed through homebrew: Screen version 4.06.02 (GNU) 23-Oct-17.
Time to make some notes so I can hopefully amend them later with solutions.
The screen magic key
When screen gives you a session to a terminal, all but one key are being routed through on a 1-on-1 base except for a “magic” key combination: Ctrl–a (which the documentation abbreviates as C-a).
Following the Ctrl–a combination, you can type a character (sometimes that needs Ctrl too) for a lot screen functionality (for a start, see the “Getting help” below).
Listing existing screen sessions and windows
Each screen process has one session
You list screen sessions from the command-line. I usually combine the latter with getting a process list as well using this command because the process list will show you parameters passed to screen:
ps -ax | grep screen && screen -list
which gets you output like this:
31992 ttys019 0:00.01 screen
29040 ttys020 0:00.04 screen /dev/cu.usbserial 115200
31898 ttys021 0:00.01 screen
32503 ttys025 0:00.00 grep screen
There are screens on:
29041.ttys020.RMBPro1TBJWP (Attached)
31899.ttys021.RMBPro1TBJWP (Attached)
31993.ttys019.RMBPro1TBJWP (Attached)
3 Sockets in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.
Many people abbreviate screen -list as screen -ls, but I like descriptive commands over cryptic ones.
What you see is that:
- there are three screen sessions of which one is using a USB serial device.
- there is one session per screen session
Windows are within sessions
From within each session, you can use the Ctrl–* combination to list the Windows. For the both non-serial sessions – looked like this on my machine:
term-type size user interface window ---------- ------- ---------- ----------------- ---------- xterm-256c 84x28 jeroenp@/dev/ttys019 0(bash) rwx
and
term-type size user interface window ---------- ------- ---------- ----------------- ---------- xterm-256c 84x28 jeroenp@/dev/ttys021 1(bash) rwx
The first was for session 31899.ttys021.RMBPro1TBJWP, the second for 31993.ttys019.RMBPro1TBJWP.
Attaching (or re-attaching) to an existing session
The -x parameter can attach to both an Attached or a Detached screen session. This allows for:
- multiple Mac
terminaltabs to see the same content - getting access to a detached session (because you – maybe by accident – closed terminal, or detached the session)
Detached sessions keep their windows and the commands running in those windows. This allows you to have long-running scripts starting from the terminal but not terminated when the terminal closes.
So in my case, this command attaches to the second session not matter if it is attached or detached.
screen -x 31993.ttys019.RMBPro1TBJWP
Killing an existing window
When killing a window, all the processes in that window will be killed too.
Typing Ctrl–a followed by k or Ctrl–k will kill the current window.
Killing a session with all windows
When killing a session, it will kill all the windows with all the processes in those windows.
Typing Ctrl–a followed by Ctrl–\ will kill the current session with all the windows in it. On some Linux systems, you can replace Ctrl–\ with just \.
I’ve not run into dead sessions yet, but if I do, I should try the -wipe command line option as shown in [WayBack] How to kill a dead screen session? – Stack Overflow (which also shows this can fail if you run out of disk space, but then you have bigger issues).
TODO: Killing a session that seems attached but isn’t
I still need to research this further, as every now and then I run into this:
# ps -ax | grep -w screen && screen -list 29040 ?? 0:00.04 screen /dev/cu.usbserial 115200 35724 ttys025 0:00.00 grep -w screen There is a screen on: 29041.ttys020.RMBPro1TBJWP (Attached) 1 Socket in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.
Wiping it fails:
# screen -wipe 29041.ttys020.RMBPro1TBJWP There is a screen on: 29041.ttys020.RMBPro1TBJWP (Attached) 1 Socket in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.
Attaching however times out:
# screen -x 29041.ttys020.RMBPro1TBJWP
For now, the only option is to reboot my system.
I’ve seen this happening mostly with cables that present themselves as prolific PL2303HX chipset.
So I will get some TTL debug cables based on other chipsets.
These posts will help finding about the available serial cables:
Getting help
It is daunting, but the man screen page [WayBack] has a truckload of information, for instance on the command-line options, key bindings and much more.
If you like on-line lists of tips more, then follow these:
- [WayBack] screen Man Page – macOS – SS64.com
- [WayBack] GNU screen [quick_reference]
- [WayBack] Screen User’s Manual
- [WayBack] linux – How do I get out of a screen without typing ‘exit’? – Stack Overflow
- [WayBack] Using Screen on Mac OS X
The man page will get you most of that information:
Start screen
Screen version 4.00.03 (FAU) 23-Oct-06
Copyright (c) 1993-2002 Juergen Weigert, Michael Schroeder
Copyright (c) 1987 Oliver Laumann
This program is free software; you can redistribute it and/or modify it under the
terms of the GNU General Public License as published by the Free Software
Foundation; either version 2, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this
program (see the file COPYING); if not, write to the Free Software Foundation,
Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
Send bugreports, fixes, enhancements, t-shirts, money, beer & pizza to
screen@uni-erlangen.de
[Press Space or Return to end.]
Here you can press Ctrl-A followed by : to get a prompt where you can type help:
[Press Space or Return to end.]
:help
There you get one or two (depending on your terminal size) pages of help:
Screen key bindings, page 1 of 2. Command key: ^A Literal ^A: a break ^B b license , reset Z clear C lockscreen ^X x screen ^C c colon : log H select ' copy ^[ [ meta a silence _ detach ^D d monitor M split S digraph ^V next ^@ ^N sp n suspend ^Z z displays * number N time ^T t dumptermcap . only Q title A fit F other ^A vbell ^G flow ^F f pow_break B version v focus ^I pow_detach D width W hardcopy h prev ^H ^P p ^? windows ^W w help ? quit ^\ wrap ^R r history { } readbuf < writebuf > info i redisplay ^L l xoff ^S s kill ^K k remove X xon ^Q q lastmsg ^M m removebuf = [Press Space for next page; Return to end.]followed by:
Screen key bindings, page 2 of 2. ^] paste . " windowlist -b - select - 0 select 0 1 select 1 2 select 2 3 select 3 4 select 4 5 select 5 6 select 6 7 select 7 8 select 8 9 select 9 ] paste . [Press Space or Return to end.]
–jeroen
Partially based on: [WayBack] terminate screen monitoring serial port – Unix & Linux Stack Exchange.
Share this:
Posted in *nix, *nix-tools, Apple, Mac, Mac OS X / OS X / MacOS, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, MacMini, macOS 10.12 Sierra, macOS 10.13 High Sierra, OS X 10.10 Yosemite, OS X 10.11 El Capitan, Power User, screen | Leave a Comment »
Redirect Detective – A Free Tool To Trace Where Redirects End Up
Posted by jpluimers on 2020/02/27
Cool tool, but be aware it does aJavaScript:xmlhttpPost from , so your data can be logged [WayBack] Redirect Detective – A Free Tool To Trace Where Redirects End Up.
For http://www.xs4all.nl/~jp, it shows these redirects (where in bold I’ve listed the changes):
Oh and it runs this call: JavaScript:xmlhttpPost("/linkdetect.px")?word=www.xs4all.nl%2F%7Ejp from [WayBack] redirectdetective.com/ajax.js.
Note that this shows more redirects than the plain http ones, so wget from [WayBack] wget – How do I display all URLs in a redirect chain? – Unix & Linux Stack Exchange shows this:
$ wget http://www.xs4all.nl/~jp 2>&1 | grep Location:
Location: https://www.xs4all.nl/~jp [following]
Location: https://jp.home.xs4all.nl/ [following]
–jeroen
Share this:
Posted in *nix, *nix-tools, Communications Development, Development, HTTP, Internet protocol suite, Power User, TCP, wget | Leave a Comment »
linux – Bash: Command grouping (&&, ||, …) – Stack Overflow
Posted by jpluimers on 2020/02/18
Excellent answer at [WayBack] linux – Bash: Command grouping (&&, ||, …) – Stack Overflow by Charles Duffy:
Operator precedence for
&&and||is strictly left-to-right.Thus:
pwd; (ls) || { cd .. && ls student/; } && cd student || cd / && cd ;…is equivalent to…
pwd; { { { (ls) || { cd .. && ls student/; }; } && cd student; } || cd /; } && cd ; }…breaking that down graphically:
pwd; { # 1 { # 2 { (ls) || # 3 { cd .. && # 4 ls student/; # 5 }; # 6 } && cd student; # 7 } || cd /; # 8 } && cd ; # 9
pwdhappens unconditionally- (Grouping only)
lshappens (in a subshell) unconditionally.cd ..happens if (3) failed.ls student/happens if (3) failed and (4) succeeded- (Grouping only)
cd studenthappens if either (3) succeeded or both (4) and (5) succeeded.cd /happens if either [both (3) and one of (4) or (5) failed], or [(7) failed].cdhappens if (7) occurred and succeeded, or (7) occurred and succeeded.
Using explicit grouping operators is wise to avoid confusing yourself. Avoiding writing code as hard to read as this is even wiser.
–jeroen
Share this:
Posted in *nix, *nix-tools, bash, bash, Development, Power User, Scripting, Software Development | Leave a Comment »
Managing sendmail TLS authenticated users
Posted by jpluimers on 2020/02/03
A few notes for managing the users that should be allowed to send mail via sendmail using TLS authentication.
Most of it is derived/summarised for [WayBack] SMTP AUTH in sendmail 8.10-8.13 and [WayBack] Creating Users for a Postfix-Based Mail Relay – Scott’s Weblog – The weblog of an IT pro specializing in cloud computing, virtualization, and networking, all with an open source view
- Verify your sendmail allows TLS:
# sendmail -d0.1 -bv | grep SASL
NETUNIX NEWDB NIS NISPLUS PIPELINING SASLv2 SCANF SOCKETMAP - The list of TLS authentication users differs from the ones in
/etc/passwd - The tools and files manage if the output is
SASLv2or older. ForSASLv2they are:/etc/sasldb2has the users/passwordssasldblistusers2lists the userssaslpasswd2manages users
For instance, this commands creates a new user for use with sendmail:
# saslpasswd2 -c -u example.org firstname.lastname
Password:
Again (for verification):
sasldblistusers2
firstname.lastname@example.org: userPassword
cat /etc/sasldb2
....................firstname.lastname@example.orguserPassword
For future reading:
not found in sasldb- [WayBack] SMTP AUTH in sendmail 8.10-8.13: SMTP AUTH in sendmail 8.10: authenticate senders to allow relaying etc
- [WayBack] sendmail – How to generate entries for different mechanisms with saslpasswd2? – Server Fault which includes steps to test with [WayBack] CRAM-MD5 authentication online generator – busylog.net
- [WayBack] Sendmail-SMTP-AUTH-TLS-Howto (follow these if you do not yet have a TLS certificate for your sendmail server; technically you do not need TLS in order to provide AUTH, but sending challenge/responses unencrypted is not smart; see [WayBack] help diagnosing smtp auth problem | The FreeBSD Forums)
- Testing SMTP from the console on Linux, BSD and Mac OS: swaks and smtp-cli Perl script clients
I thought I needed this so I could add an alias @pluimers.com to my gmail box, as I read only the accepted answer at [WayBack] Add new alias to Gmail without SMTP (forwarding-only address) – Web Applications Stack Exchange pointing to:
[WayBack] External Addresses No Longer Use Gmail SMTP Servers- [Archive.is] Add Send mail as: email – Google Product Forums
- [WayBack] Send emails from a different address or alias – Gmail Help If you own another email address, you can send mail as that address. For example: Yahoo, Outlook, or other non-Gmail address Your work, school, or business domain or alias, like @yourschool.edu
I should have read the second answer at [WayBack] Add new alias to Gmail without SMTP (forwarding-only address) – Web Applications Stack Exchange:
As of writing, however, you can simply use the Gmail SMTP server, as long as you use [WayBack] Google two-step authentication.
So just for completeness, the full steps:
- Gmail settings, Accounts and Import tab.
- Add another email address you own
- Type name and email address to be added.
- For SMTP Server, put
smtp.gmail.com- For Username, your full Gmail address including
@gmail.com- For password, provide an App Password generated in Google Accounts at https://security.google.com/settings/security/apppasswords
- Leave Secured connection using TLS selected as is.
- Add Account
–jeroen
Share this:
Posted in *nix, *nix-tools, Power User, sendmail | Leave a Comment »
The Wiert Corner - irregular stream of stuff 