The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,466 other followers

Archive for September, 2021

Unix and NTFS file systems, hardlinks, inodes, files, directories, dot directories, bugs and implementation details

Posted by jpluimers on 2021/09/21

Lots of interesting tidbits on unix and NTFS file systems.

If you want to blow up your tooling, try creating a recursive hardlink…, which is likely one of the reasons that nx file systems do not support them.

Covered and related topics:

The tweets (especially follow the train of thought in the various subtrees: a great way to learn new things!):

It is important to understand that the concept File IDs and inode/vnode has far reaching consequences, for instance from [WayBack] inode – Wikipedia

  • Files can have multiple names. If multiple names hard link to the same inode then the names are equivalent; i.e., the first to be created has no special status. This is unlike symbolic links, which depend on the original name, not the inode (number).
  • An inode may have no links. An unlinked file is removed from disk, and its resources are freed for reallocation but deletion must wait until all processes that have opened it finish accessing it. This includes executable files which are implicitly held open by the processes executing them.
  • It is typically not possible to map from an open file to the filename that was used to open it. The operating system immediately converts the filename to an inode number then discards the filename. This means that the getcwd() and getwd() library functions search the parent directory to find a file with an inode matching the working directory, then search that directory’s parent, and so on until reaching the root directorySVR4 and Linux systems maintain extra information to make this possible.
  • Historically, it was possible to hard link directories. This made the directory structure into an arbitrary directed graph contrary to a directed acyclic graph. It was even possible for a directory to be its own parent. Modern systems generally prohibit this confusing state, except that the parent of root is still defined as root. The most notable exception to this prohibition is found in Mac OS X (versions 10.5 and higher) which allows hard links of directories to be created by the superuser.[10]
  • A file’s inode number stays the same when it is moved to another directory on the same device, or when the disk is defragmented which may change its physical location. This also implies that completely conforming inode behavior is impossible to implement with many non-Unix file systems, such as FAT and its descendants, which don’t have a way of storing this invariance when both a file’s directory entry and its data are moved around.
  • Installation of new libraries is simple with inode file systems. A running process can access a library file while another process replaces that file, creating a new inode, and an all-new mapping will exist for the new file so that subsequent attempts to access the library get the new version. This facility eliminates the need to reboot to replace currently mapped libraries.
  • It is possible for a device to run out of inodes. When this happens, new files cannot be created on the device, even though there may be free space available. This is most common for use cases like mail servers which contain many small files. File systems (such as JFS or XFS) escape this limitation with extents or dynamic inode allocation, which can “grow” the file system or increase the number of inodes.

A very cool read in the midst of the tweet tree was this reference to former Google Plus by [WayBack] Rob Pike – Wikipedia (of Golang, Unix team and Plan 9 fame).

WayBack: A lesson in shortcuts.Long ago, as the design of the Unix file system was being worked out, the entries . and .. appeared, to make navigation easier. … – Rob Pike – Google+

A lesson in shortcuts.

Long ago, as the design of the Unix file system was being worked out, the entries . and .. appeared, to make navigation easier. I’m not sure but I believe .. went in during the Version 2 rewrite, when the file system became hierarchical (it had a very different structure early on).  When one typed ls, however, these files appeared, so either Ken or Dennis added a simple test to the program. It was in assembler then, but the code in question was equivalent to something like this:
if (name[0] == ‘.’) continue;
This statement was a little shorter than what it should have been, which is
if (strcmp(name, “.”) == 0 || strcmp(name, “..”) == 0) continue;
but hey, it was easy.

Two things resulted.

First, a bad precedent was set. A lot of other lazy programmers introduced bugs by making the same simplification. Actual files beginning with periods are often skipped when they should be counted.

Second, and much worse, the idea of a “hidden” or “dot” file was created. As a consequence, more lazy programmers started dropping files into everyone’s home directory. I don’t have all that much stuff installed on the machine I’m using to type this, but my home directory has about a hundred dot files and I don’t even know what most of them are or whether they’re still needed. Every file name evaluation that goes through my home directory is slowed down by this accumulated sludge.

I’m pretty sure the concept of a hidden file was an unintended consequence. It was certainly a mistake.

How many bugs and wasted CPU cycles and instances of human frustration (not to mention bad design) have resulted from that one small shortcut about  40 years ago?

Keep that in mind next time you want to cut a corner in your code.

(For those who object that dot files serve a purpose, I don’t dispute that but counter that it’s the files that serve the purpose, not the convention for their names. They could just as easily be in $HOME/cfg or $HOME/lib, which is what we did in Plan 9, which had no dot files. Lessons can be learned.)

–jeroen

Read the rest of this entry »

Posted in *nix, Development, File-Systems, History, NTFS, Power User, Software Development, Windows, Windows Development | Leave a Comment »

The spookback localghost address to resolve 👻 

Posted by jpluimers on 2021/09/21

“Spooky dev environment hack: add 127.0.0.1 xn--9q8h to /etc/hosts and then all your dev servers can be accessed at http://👻 It’s localghost!”

Via:

–jeroen

Posted in Communications Development, Development, HTTP, Internet protocol suite, Power User, Software Development, TCP | Leave a Comment »

Reggefiber NTU: convert blind cap to become pure fiber, so no ethernet media converter is needed

Posted by jpluimers on 2021/09/20

Some links:

 

–jeroen

Read the rest of this entry »

Posted in fiber, Internet, Power User | Leave a Comment »

Low cost remote IP KVM and control, is it possible?

Posted by jpluimers on 2021/09/20

A long time ago, I bought one [Wayback/Archive.is] SpiderDuo Local and Remote KVM Over IP | Lantronix, which – without power control unit – was already some USD 400 (while writing this in fall 2021, the price has increased to almost USD 600 [Archive.is]).

1PORT Local + Remote USB Securelinx Spiderduo KVM Over IP : Electronics - Amazon.com

It was about the only “affordable” remote KVM over IPv4 available and by now has a big drawback: it’s based on Java in the browser, which is a pain in the ass to keep working.

So I went looking for alternatives and found only two reasonable ones:

I will likely go for the Pi-KVM ; it’s on kickstarter right now

Not only that, but I found a few comparisons favouring PiKVM:

I found the Pi-KVM via [Archive.is] Solar Designer on Twitter: “PiKVM v3 HAT, “Raspberry Pi based open-source KVM over IP” by @mdevaev, is now funded on Kickstarter “

At USD 145 or less on kickstarter (excluding a Raspberry Pi 4 or power brick, so add some USD 50 for those), it is way cheaper than the SpiderDuo above which I bought some 5 years ago.

The kickstarter closes in about a week from now, so if you consider one: don’t be late! [Wayback/Archive.is] PiKVM v3 HAT by Maxim Devaev — Kickstarter shows what you get:

  • The PiKVM v3 HAT board for Raspberry Pi 4
  • USB-C bridge board – to connect the HAT with Pi over USB-C
  • ATX controller adapter board and wiring – to connect the HAT to the motherboard (if you want to manage power supply through hardware)
  • 2 flat CSI cables
  • Screws and brass standoffs

You will also need:

  • Raspberry Pi 4 with 2Gb RAM or more
  • MicroSD card
  • USB-C to USB-A cable
  • HDMI cable
  • Straight Ethernet cable (for the ATX expansion board connection)
  • Power supply unit (5.1V 3A USB-C, recommended by the Raspberry Pi)

You can use our free 3D printing case design to build a beautiful complete unit or wait a bit for the official PiKVM metal case we are working on!

–jeroen

Read the rest of this entry »

Posted in Hardware, KVM keyboard/video/mouse, Power User | Leave a Comment »

Mikrotik RouterOS “/ip ssh” setting not available from WinBox and defaulting to insecure?

Posted by jpluimers on 2021/09/20

Still need to research this further:

Somewhere around 6.44, when upgrading an existing RouterOS device, this snippet became part of the configuration:

/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote

A few remarks:

  • I could not find anything in WinBox that is equivalent.
  • This sounds very insecure, so I have run this script:
    /ip ssh
    set allow-none-crypto=no forwarding-enabled=no

    which makes the snippet to disappear (because they are default settings according to [WayBack] Manual:IP/SSH – MikroTik Wiki).

    Like usual, the on-line documentation is dense and insufficiently clear, hence my measure.

In the future, I need to decipher these posts (via [WayBack] winbox ssh allow none crypto – Google Search and [WayBack] winbox ssh forwarding enabled remote – Google Search):

–jeroen

Posted in Internet, MikroTik, Power User, Routers | Leave a Comment »

 
%d bloggers like this: