The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,867 other followers

Archive for the ‘Mac’ Category

Mounting a Time Machine backup under Linux so you can send it to Backblaze using Restic…

Posted by jpluimers on 2020/03/16

[WayBack1/WayBack2] Memo to self: Ich will ein Time Machine Backup unter Linux mounten, um das “Latest” Verzeichnis mit Restic an Backblaze senden zu können. Schritt 1: Sp… – Kristian Köhntopp – Google+:

Memo to self: Ich will ein Time Machine Backup unter Linux mounten, um das “Latest” Verzeichnis mit Restic an Backblaze senden zu können.

Schritt 1: Sparsebundle mounten

# ls -l /export/tm_kk/
total 8
drwx—— 3 kris users 4096 Oct 21 16:24 KK.sparsebundle

Geht mit

# git clone git://github.com/torarnv/sparsebundlefs.git
# cd sparsebundlefs; make
# mkdir -p /bundles/tm_kk
# sparsebundlefs /export/tm_kk/KK.sparsebundle /bundles/tm_kk
# ls -lh /bundles/tm_kk
total 0
-r——– 1 root nogroup 1.5T Oct 21 16:24 sparsebundle.dmg

Schritt 2: DMG loopmounten

# fdisk -l /bundles/tm_kk/sparsebundle.dmg
Disk /bundles/tm_kk/sparsebundle.dmg: 1.5 TiB, 1648462135296 bytes, 3219652608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 15FCCBBD-49E4-42BB-B359-EF662F9916CF

Device Start End Sectors Size Type
/bundles/tm_kk/sparsebundle.dmg1 40 409639 409600 200M EFI System
/bundles/tm_kk/sparsebundle.dmg2 409640 3219390423 3218980784 1.5T Apple HFS/HFS+

# kpartx -a -v /bundles/tm_kk/sparsebundle.dmg
add map loop8p1 (253:1): 0 409600 linear 7:8 40
add map loop8p2 (253:19): 0 3218980784 linear 7:8 409640

# mkdir -p /hfs/tm_kk
# mount -o ro -t hfsplus /dev/mapper/loop8p2 /hfs/tm_kk
# b=$(readlink /hfs/tm_kk/Backups.backupdb/KK/Latest)
# ls -l /hfs/tm_kk/Backups.backupdb/KK/$b
total 0
drwxr-xr-x 1 root root 30 Oct 21 14:26 ‘Macintosh HD’

Schritt 3: tmfs mount

Ein Time Machine Backup enthält doofe Hardlinks auf Verzeichnisse. Das kann Linux so nicht, und man muß das noch einmal mit tmfs fusemounten. Seufz.

# apt-get install tmfs
# mkdir -p /tmfs/tm_kk
# tmfs /hfs/tm_kk /tmfs/tm_kk
# ls -l /tmfs/tm_kk/KK/Latest/
total 0
drwxr-xr-x 1 root root 30 Oct 21 14:26 ‘Macintosh HD’

Das durchzulesen ist nicht mal langsam.

Schritt 4: Abbauen

# umount /tmfs/tm_kk
# umount /hfs/tm_kk/
# kpartx -d -v /bundles/tm_kk/sparsebundle.dmg
del devmap : loop8p1
del devmap : loop8p2
loop deleted : /dev/loop8
# umount /bundles/tm_kk

Schritt 5: Den Mist scripten

A few important comments from the WayBack2 link:

Jeroen Wiert Pluimers:

So your time machine data is on a Linux disk that you export from Linux to MacOS? (as otherwise, I don’t understand what /export/tm_kk/KK.sparsebundle is for)

Kristian Köhntopp:

Yes, I am running netatalk, SMB and NFS on an Ubuntu 18.04.

Jeroen Wiert Pluimers:

+Kristian Köhntopp Thanks. What do you envision as steps to restore a complete time machine?

Kristian Köhntopp:

We will see. I do not see that as normally necessary. Mostly I do not want to lose the work on my laptop, and am just to lazy to restrict the backup to that. In general, it should be possible to create this as a writeable setup so that I get the data back. I will probably never recreate a full runnable mac setup from this emergency offsite backup.

Tools used:

–jeroen

Posted in *nix, *nix-tools, Apple, Backup, Mac, Mac OS X / OS X / MacOS, MacBook, Power User | Leave a Comment »

MacOS: Checking a disk for bad blocks

Posted by jpluimers on 2020/03/13

Hardware fails, but most disk tools on MacOS only check logical disk structures, not bad blocks.

Luckily, fsck_hfs can, though Apple is a bit secretive on it: [WayBack] Page Not Found – Apple Developer: ManPages/man8/fsck_hfs.8.html is empty, but there is [WayBack] man page fsck_hfs section 8 and the gist below.

Disk volumes on MacOS use a successor of HFS called HFS Plus – Wikipedia, but the tooling never changed names.

I got at the below parameters through [

This is the disk check command:

# sudo fsck_hfs -dylS /dev/disk3s1
** /dev/rdisk3s1 (NO WRITE)
    Using cacheBlockSize=32K cacheTotalBlock=65536 cacheSize=2097152K.
Scanning entire disk for bad blocks
   Executing fsck_hfs (version hfs-407.50.6).
** Performing live verification.
** Checking Journaled HFS Plus volume.
   The volume name is SanDisk400GB
** Checking extents overflow file.
** Checking catalog file.
** Checking extended attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume SanDisk400GB appears to be OK.
    CheckHFS returned 0, fsmodified = 0

The italic part is the bad block scanning. The normal part the hfs scanning, which will continue even after finding bad blocks.

If bad blocks are found, output looks more like on the right. If it looks like that, basically you know a disk is toast.

It can be slow, as I did not specify a cache, so it defaults to 32 Kibibyte. You can increase that by adding for instance -c 512m  for 512 Mebibyte cache, just read the short help or man page below.

This tremendously helps checking volumes containing many files, for instance [WayBack] Checking Very Large Time Machine Volumes – Mac OS X Hints

Read the rest of this entry »

Posted in Apple, Mac, Mac OS X / OS X / MacOS, Power User | Leave a Comment »

MacOS: converting a man page to markdown

Posted by jpluimers on 2020/03/09

Converting a man page to markdown is a three step process:

  1. installing a tool that can convert the source of a man page to markdown
  2. finding the location of the man page source
  3. doing the actual conversion

Tool to convert man to markdown

The source format of man pages is troff, which is usually converted by man using groff, or a set of macros.

My initial thought for the first problem was to use pandoc, but as I found earlier in pandoc oneliner from reStructuredText to html, on MacOS, the pandoc can write groff format, but not read it.

Luckily doing a pandoc from groff to markdown – Google Search, I bumped into [WayBack] Convert groff to markdown · Issue #8 · neomutt/neomutt-docs · GitHub which lead to mandoc – Wikipedia.

Since I already had homebrew installed, getting mandoc was simple: brew install mandoc.

Finding the man page source

Earlier in the process when searching for pandoc based conversions, I found the solution for the second problem too: [WayBack] Man page with preserved text decorations, proportional text and fixed-width code – Unix & Linux Stack Exchange taught me about the -w option, but there is actually a -W option that works better if you have multiple pages for a keyword:

-w or --path
Don’t actually display the man pages, but do print the location(s) of the files that would be formatted or displayed. If no argument is given: display (on stdout) the list of directories that is searched by man for man pages. If manpath is a link to man, then “manpath” is equivalent to “man --path“.

-W Like -w, but print file names one per line, without additional information. This is useful in shell commands like man -aW man | xargs ls -l

Actual conversion for fsck_hfs

It all came down to a one-liner:

mandoc -T markdown `man -w fsck_hfs` > /tmp/fsck_hfs.8.md

Note the order here is important this will fail with an error:

mandoc `man -w fsck_hfs` -T markdown > /tmp/fsck_hfs.8.md

mandoc: -T: ERROR: No such file or directory
mandoc: markdown: ERROR: No such file or directory

–jeroen

Read the rest of this entry »

Posted in Apple, Mac, Mac OS X / OS X / MacOS, Power User | Leave a Comment »

GitHub – facebook/osquery: SQL powered operating system instrumentation, monitoring, and analytics.

Posted by jpluimers on 2020/03/09

Boy, I wish I had found this years ago: [WayBack] GitHub – facebook/osquery: SQL powered operating system instrumentation, monitoring, and analytics.

No more remembering all those nifty configuration and log file details by heart, just install and query using SQL.

Now you need to learn a “database” schema, however that’s the same for all supported operating systems.

Example installation steps:

Run interactively through osqueryi, then perform .help or .schema to get an impression of what is possible.

There is lots of documentation at [WayBack] osquery.

–jeroen

Posted in *nix, *nix-tools, Apple, Mac, Mac OS X / OS X / MacOS, Power User | Leave a Comment »

Mac OS X / macOS / …: quit screen or window when using it for a serial port

Posted by jpluimers on 2020/03/06

I wrote a bit on using screen from my Mac in The woods and trees of OpenSuSE on single-board computers – image abbreviations – and getting it installed using OS X.

The coverage was way too brief and also buried in large post.

Today, I want to focus on some things related to properly terminating a a screen window, session or screen by itself from a Mac which somehow is one of the harder things for me to do.

I’m talking about sessions and windows, because screen is a terminal multiplexer, which can also put a serial link in a window on a session. So screen is not “just” a serial console terminal application by itself.

Part of being hard is probably that even on for instance 10.12 Sierra, Apple ships a fairly old version: screen --version returns Screen version 4.00.03 (FAU) 23-Oct-06 whereas after 2014 (before that, screen development was pretty stalled) many new versions appeared: [WayBack] screen.git.

It makes it less hard that it hardly reproduces, and seems to reproduce less with the screen version I installed through homebrew: Screen version 4.06.02 (GNU) 23-Oct-17.

Time to make some notes so I can hopefully amend them later with solutions.

The screen magic key

When screen gives you a session to a terminal, all but one key are being routed through on a 1-on-1 base except for a “magic” key combination: Ctrla (which the documentation abbreviates as C-a).

Following the Ctrla combination, you can type a character (sometimes that needs Ctrl too) for a lot screen functionality (for a start, see the “Getting help” below).

Listing existing screen sessions and windows

Each screen process has one session

You list screen sessions from the command-line. I usually combine the latter with getting a process list as well using this command because the process list will show you parameters passed to screen:

ps -ax | grep screen && screen -list

which gets you output like this:

31992 ttys019    0:00.01 screen
29040 ttys020    0:00.04 screen /dev/cu.usbserial 115200
31898 ttys021    0:00.01 screen
32503 ttys025    0:00.00 grep screen
There are screens on:
    29041.ttys020.RMBPro1TBJWP  (Attached)
    31899.ttys021.RMBPro1TBJWP  (Attached)
    31993.ttys019.RMBPro1TBJWP  (Attached)
3 Sockets in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.

Many people abbreviate screen -list as screen -ls, but I like descriptive commands over cryptic ones.

What you see is that:

  • there are three screen sessions of which one is using a USB serial device.
  • there is one session per screen session

Windows are within sessions

From within each session, you can use the Ctrl* combination to list the Windows. For the both non-serial sessions – looked like this on my machine:

term-type   size         user interface           window
---------- ------- ---------- ----------------- ----------
xterm-256c  84x28     jeroenp@/dev/ttys019        0(bash)       rwx

and

term-type   size         user interface           window
---------- ------- ---------- ----------------- ----------
xterm-256c  84x28     jeroenp@/dev/ttys021        1(bash)       rwx

The first was for session 31899.ttys021.RMBPro1TBJWP, the second for 31993.ttys019.RMBPro1TBJWP.

Attaching (or re-attaching) to an existing session

The -x parameter can attach to both an Attached or a Detached screen session. This allows for:

  • multiple Mac terminal tabs to see the same content
  • getting access to a detached session (because you – maybe by accident – closed terminal,  or detached the session)

Detached sessions keep their windows and the commands running in those windows. This allows you to have long-running scripts starting from the terminal but not terminated when the terminal closes.

So in my case, this command attaches to the second session not matter if it is attached or detached.

screen -x 31993.ttys019.RMBPro1TBJWP

Killing an existing window

When killing a window, all the processes in that window will be killed too.

Typing Ctrla followed by k or Ctrlk will kill the current window.

Killing a session with all windows

When killing a session, it will kill all the windows with all the processes in those windows.

Typing Ctrla followed by Ctrl\ will kill the current session with all the windows in it. On some Linux systems, you can replace Ctrl\ with just \.

I’ve not run into dead sessions yet, but if I do, I should try the -wipe command line option as shown in [WayBack] How to kill a dead screen session? – Stack Overflow (which also shows this can fail if you run out of disk space, but then you have bigger issues).

TODO: Killing a session that seems attached but isn’t

I still need to research this further, as every now and then I run into this:

# ps -ax | grep -w screen && screen -list
29040 ?? 0:00.04 screen /dev/cu.usbserial 115200
35724 ttys025 0:00.00 grep -w screen
There is a screen on:
 29041.ttys020.RMBPro1TBJWP (Attached)
1 Socket in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.

Wiping it fails:

# screen -wipe 29041.ttys020.RMBPro1TBJWP
There is a screen on:
 29041.ttys020.RMBPro1TBJWP (Attached)
1 Socket in /var/folders/zr/dsp77fhs6zq179n72lykjrjw0000gq/T/.screen.

Attaching however times out:

# screen -x 29041.ttys020.RMBPro1TBJWP

For now, the only option is to reboot my system.

I’ve seen this happening mostly with cables that present themselves as prolific PL2303HX chipset.

So I will get some TTL debug cables based on other chipsets.

These posts will help finding about the available serial cables:

Getting help

It is daunting, but the man screen page [WayBack] has a truckload of information, for instance on the command-line options, key bindings and much more.

If you like on-line lists of tips more, then follow these:

The man page will get you most of that information:

Start screen

Screen version 4.00.03 (FAU) 23-Oct-06

Copyright (c) 1993-2002 Juergen Weigert, Michael Schroeder
Copyright (c) 1987 Oliver Laumann

This program is free software; you can redistribute it and/or modify it under the
terms of the GNU General Public License as published by the Free Software
Foundation; either version 2, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this
program (see the file COPYING); if not, write to the Free Software Foundation,
Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.

Send bugreports, fixes, enhancements, t-shirts, money, beer & pizza to
screen@uni-erlangen.de


                          [Press Space or Return to end.]

Here you can press Ctrl-A followed by : to get a prompt where you can type help:

[Press Space or Return to end.]
:help

There you get one or two (depending on your terminal size) pages of help:

                         Screen key bindings, page 1 of 2.

                         Command key:  ^A   Literal ^A:  a

   break       ^B b          license     ,             reset       Z         
   clear       C             lockscreen  ^X x          screen      ^C c      
   colon       :             log         H             select      '         
   copy        ^[ [          meta        a             silence     _         
   detach      ^D d          monitor     M             split       S         
   digraph     ^V            next        ^@ ^N sp n    suspend     ^Z z      
   displays    *             number      N             time        ^T t      
   dumptermcap .             only        Q             title       A         
   fit         F             other       ^A            vbell       ^G        
   flow        ^F f          pow_break   B             version     v         
   focus       ^I            pow_detach  D             width       W         
   hardcopy    h             prev        ^H ^P p ^?    windows     ^W w      
   help        ?             quit        ^\            wrap        ^R r      
   history     { }           readbuf     < writebuf >         
   info        i             redisplay   ^L l          xoff        ^S s      
   kill        ^K k          remove      X             xon         ^Q q      
   lastmsg     ^M m          removebuf   =         

                    [Press Space for next page; Return to end.]

followed by:

                         Screen key bindings, page 2 of 2.

^]  paste .
"   windowlist -b
-   select -
0   select 0
1   select 1
2   select 2
3   select 3
4   select 4
5   select 5
6   select 6
7   select 7
8   select 8
9   select 9
]   paste .






                          [Press Space or Return to end.]

–jeroen

Partially based on: [WayBack] terminate screen monitoring serial port – Unix & Linux Stack Exchange.

Posted in *nix, *nix-tools, Apple, Mac, Mac OS X / OS X / MacOS, MacBook, MacBook Retina, MacBook-Air, MacBook-Pro, MacMini, macOS 10.12 Sierra, macOS 10.13 High Sierra, OS X 10.10 Yosemite, OS X 10.11 El Capitan, Power User, screen | Leave a Comment »

 
%d bloggers like this: