The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Internet’ Category

Highly esteemed science: An analysis of attitudes towards and perceived attributes of science in letters to the editor in two Dutch newspapers – Stefan P.L. de Jong, Elena Ketting, Leonie van Drooge, 2020

Posted by jpluimers on 2021/10/06

All my IPv4 addresses seem to be blocked with messages like this (note the odd, but allowed, leading zero in the IPv4 address [WayBack]):

Error

The IP you are accessing the site with (037.153.243.242) has been blocked because it has triggered one of our security measures. Please see the reason below:
Block reason: This IP was identified as infiltrated and is being used by sci-hub as a proxy.
To restore access, please contact onlinesupport@sagepub.com citing this message in full.

A quick [WayBack] “This IP was identified as infiltrated and is being used by sci-hub as a proxy.” – Google Search shows they also block the Google Bot.

I am not not even going to bother with companies that have bad infiltration detection.

Of course I ensured the paper has been archived:

[WayBack/Archive.is] Highly esteemed science: An analysis of attitudes towards and perceived attributes of science in letters to the editor in two Dutch newspapers – Stefan P.L. de Jong, Elena Ketting, Leonie van Drooge, 2020.

Note I do not run sci-hub, though it tempts me doing so. For more info: [WayBack] Sci-Hub – Wikipedia

I checked the router and web-proxy for any suspicious activity. There is none.

I do run the ArchiveBot by the ArchiveTeam to support the WayBackMachine of the InternetArchive and the great team Mark Graham has there providing some bandwidth and CPU/memory resources helping them archive public internet content for posterity.

It that triggers SAGE, too bad for them.

–jeroen

Read the rest of this entry »

Posted in Development, Internet, InternetArchive, LifeHacker, Power User, Software Development, WayBack machine, Web Development | Leave a Comment »

Google Cloud Shell: connecting to cloud storage buckets

Posted by jpluimers on 2021/09/28

One of the drawbacks of Google Cloud Shell is that it will delete the home directory after 120 days of not using it: [WayBack] Deletion notice for my Google Cloud Shell home directory – Stack Overflow

This is documented [WayBack] here:

If you do not access Cloud Shell for 120 days, we will delete your home disk. You will receive an email notification before we do so and simply starting a session will prevent its removal.

This only applies to the home directory of your Cloud Shell instance (you may want to store it on Cloud Storage anyway if you want to keep it). Any other Google services you use will be unaffected.

I hardly use the cloud shell, as it is a last resort to shell out from overly protected networks. Fewer and fewer environments restrict so much, so I’ve bumped into the home directory deletion a few times now.

I might use it more in the future, as I recently discovered there is a URL trick so you can start a cloud shell with parameters like an initial git repository: [WayBack] Open in Cloud Shell  |  Google Cloud

The Open in Cloud Shell feature allows you to publish a link that opens the Cloud Console and either automatically clones a Git repository into Cloud Shell or starts Cloud Shell with a custom image. It also allows for instructions to be printed to the terminal to help users interact with the content.

The Open in Cloud Shell feature helps developers experiment with code samples and APIs without having to worry about downloading Cloud SDK, installing required dependencies, or searching for relevant source files. This page explains how to add this feature to your Git repository.

Currently, only GitHub and Bitbucket repositories are whitelisted. If you would like to add a different repository, send feedback with the repository type you’d like to use with Open in Cloud Shell.

Setting up the home directory with my scripts can be a curse, so I have contemplated on these kinds of solutions:

  • store scripts in Google Drive, and mount part of Google Drive into the Cloud Shell
  • store scripts in Google Cloud Storage
  • script the setup of the home directory via a bash script in a gist

Some links that will hopefully help me with that:

  • [WayBack] Use the Google Cloud Shell to Upload files to Google Drive : DataHoarder
  • [WayBack] Open Your Repository In Google Cloud Shell – Daisuke Maki – Medium
  • [WayBack] gsutil – Google cloud storage – Download file from web – Stack Overflow

    mount the bucket in your Cloud Shell using gcsfuse:

    Create a directory in your Cloud Shell user home

     mkdir ~/mybucket
    

    Now mount your bucket in that directory using gcsfuse:

     gcsfuse bucket_name ~/mybucket
    

    Change the current directory to mount point directory:

     cd mybucket
    

    (if you want to have some fun run “df -h .” to see how much space you got in that mount point)

    seems to work for all file sizes:

     curl http://speedtest.tele2.net/10GB.zip | gsutil cp - gs://YOUR_BUCKET_NAME/10GB.zip
    

    Basically curl “streams” the data directly to the bucket.

  • [WayBack] Limitations and restrictions  |  Cloud Shell  |  Google Cloud

    Usage limits

    Cloud Shell inactivity: If you do not access Cloud Shell for 120 days, your home disk will be deleted. You will receive an email notification before its deletion and simply starting a session will prevent its removal. Please consider a different solution on Google Cloud storage for sensitive data you wish to store long term.

    Non-interactive usage: Cloud Shell is intended for interactive use only. Non-interactive sessions will be ended automatically after a warning. Note that Cloud Shell sessions are capped at 12 hours, after which sessions are automatically terminated. You can use a new session immediately after.

    Weekly usage: Cloud Shell also has weekly usage limits. If you reach your usage limit, you’ll need to wait until the specified time (listed under Usage Quota, found under the three dots menu icon) before you can use Cloud Shell again.

    Restoring a session after a service limit violation: If your session is terminated or cannot be established because you exceeded a service limit, Cloud Shell will display an error with a link to a form that allows you to appeal the limit violation. Click the feedback link and submit the form with more information about the tasks you were performing before your session was terminated.

–jeroen

Posted in bash, Development, Google, GoogleCloudShell, GoogleDrive, Internet, Power User, Scripting, Software Development, SpeedTest | Leave a Comment »

MikroTik PoE: automatically power cycle and reboot device when it becomes unresponsive.

Posted by jpluimers on 2021/09/24

In the past I had these manual scripts to power-cycle a hung RaaspberryPi device:

/interface ethernet poe set ether5 poe-out=off
/interface ethernet poe set ether5 poe-out=forced-on

or on one line:

/interface ethernet poe set ether5 poe-out=off; /interface ethernet poe set ether5 poe-out=forced-on

I am going to try this script for the port having a Raspberry Pi on it (note: this requires a 48V power brick for the Mikrotik!) on RouterOS version 6.48.3 (stable):

/interface ethernet
set [ find default-name=ether5 ] comment="RaspberryPi" poe-out=\
    forced-on power-cycle-ping-address=192.168.124.38 power-cycle-ping-enabled=\
    yes power-cycle-ping-timeout=2m

The above has not worked for a long time as per [Wayback] No POE Power Cycle @ hEX POE – MikroTik:

But it might be fixed as of [Wayback] RouterOS version v6.47.3[stable] as per [Wayback] MikroTik Routers and Wireless – Software: 6.47.3 (2020-Sep-01 05:24):

*) poe – fixed “power-cycle” functionality on RB960GSP;

Similar issues exist on RB760iGS/Hex S, and there the fix requires new hardware in addition to firmware as per [Wayback] POE OUT issue on ether5 rb760igs (no power) – MikroTik

Note that I did disassemble both of these routers for inspection and there are obvious changes to the hardware to correct the PoE problems – most notably a completely different relay, capacitor and some minor circuit design changes.

If it still fails, I might try

[Wayback] No POE Power Cycle @ hEX POE – MikroTik: workaround script

:local ipPing ("x.x.x.x")
:local pingip
#
# pingip below RUNS and sets the variable
# to number of successful pings ie 3 means 3 of 45 success
# can also use ($pingip > 1) or ($pingip >= 1) both TESTED
# ($pingip >= 1) means if only 1 or 0 pings do the IF, not the ELSE
#
:log info ("ping CHECK script IS RUNNING NOW")
# first delay 90 b4 ping test incase this is running at POWER UP
:delay 90
:set pingip [/ping $ipPing count=45]
:if ($pingip <= 3) do={ :log warning (">95% lost ping LOSS to isp GW IP x.x.x.x via ether5 so DO POE powerCYCLE")
  /interface ethernet poe set ether5 poe-out=off
  :delay 12
  /interface ethernet poe set ether5 poe-out=auto-on
  :delay 10
  :log warning ("ether5 POE HAS BEEN TURNED BACK ON")
  :delay 90
  /system script run emailPOEresult
} else={
  :log warning ("PoeCyclePINGcheck ELSE ran so no ping loss detected by script")
}

Based on:

Read the rest of this entry »

Posted in Development, Hardware Development, Internet, MikroTik, Power User, Raspberry Pi, routers | Leave a Comment »

Reggefiber NTU: convert blind cap to become pure fiber, so no ethernet media converter is needed

Posted by jpluimers on 2021/09/20

Some links:

Edit 20210922: added the below Twitter thread, added searches for the now unavailable videos

[Archive.is] Joel Haasnoot on Twitter: “Freedom laat je alles zelf doen zonder monteur, maar risico als het stuk gaat is 2000-6000 euro om nieuwe splice te doen. Maar er zijn dus ook geen filmpjes oid :)… “

Joel Haasnoot‏ @webguy· Sep 20

Ik moest mij hier een paar maanden gelden hier induiken, maar enigszins bizar hoe weinig documentatie of Google-baar spul is. Er wordt vanuit gegaan dat sysadmins dit maar leren…
Jeroen Wiert Pluimers‏ @jpluimers· Sep 20

Inderdaad. Helaas is het youtube filmpje inmiddels private. Wat heb jij uiteindelijk gedaan?
Joel Haasnoot‏ @webguy· Sep 21

Paar problemen, deels omdat ik een van de eerste 128 oid was bij Freedom en er geen verhuisproces was. Had eerst een patchcover nodig voor nieuwe TK-01 (https://www.kpnwebshop.com/modems-routers/producten/patchcover-kit-ftu-tk01/2 …) en daar zat geen patchkabel bij (geen NTU nodig bij Freedom), want dat was opeens een SC ipv LC eind.
Joel Haasnoot‏ @webguy· Sep 21

Uiteindelijk had KPNNetwerkNL 4 weken nodig om licht op de lijn te krijgen… Ik heb nu nog een patchkabel + optics liggen voor Mikrotik router, maar vooor nu nog maar even via router.
Replying to @webguy @jpluimers
Freedom laat je alles zelf doen zonder monteur, maar risico als het stuk gaat is 2000-6000 euro om nieuwe splice te doen. Maar er zijn dus ook geen filmpjes oid :)

Videos:

–jeroen

Read the rest of this entry »

Posted in fiber, Internet, Power User | Leave a Comment »

Mikrotik RouterOS “/ip ssh” setting not available from WinBox and defaulting to insecure?

Posted by jpluimers on 2021/09/20

Still need to research this further:

Somewhere around 6.44, when upgrading an existing RouterOS device, this snippet became part of the configuration:

/ip ssh
set allow-none-crypto=yes forwarding-enabled=remote

A few remarks:

  • I could not find anything in WinBox that is equivalent.
  • This sounds very insecure, so I have run this script:
    /ip ssh
    set allow-none-crypto=no forwarding-enabled=no

    which makes the snippet to disappear (because they are default settings according to [WayBack] Manual:IP/SSH – MikroTik Wiki).

    Like usual, the on-line documentation is dense and insufficiently clear, hence my measure.

In the future, I need to decipher these posts (via [WayBack] winbox ssh allow none crypto – Google Search and [WayBack] winbox ssh forwarding enabled remote – Google Search):

–jeroen

Posted in Internet, MikroTik, Power User, routers | Leave a Comment »

Windows and the current state of S.M.A.R.T. tooling that understands NVMe

Posted by jpluimers on 2021/09/16

I had trouble with two Intel 600p NVMe SSD devices: read-errors.

It appeared only few tools understand how to get S.M.A.R.T. health information from them, and even then they did not explain the read errors.

I’m going to RMA them, but in case anyone else needs to get health information from NVMe SSD devices, here is which tools do what:

So basically, CrystalDiskInfo and HD Tune are my first line of checking for drive issues, followed by smartmontools to get text output, then by vendor specific tools to assist with the RMA.

In the past, I used another smartmontools wrapper, but it was discontinued and had an even older version than GSmartControl: Source: Closed: HDD Guardian – Home.

On Intel 600p becoming locked in read-only mode after failure:

Start of Intel RMA procedure via [Wayback] Warranty Information.

My case looks remarkably similar to [Wayback] Full Diagnostic Scan always fails during Read Scan on my SSD 600p Series 256GB – Intel Community.

A few screenshots of the tools I used for health information:

Read the rest of this entry »

Posted in Hardware, NVMe, Power User, SSD, WayBack machine | Leave a Comment »

Overview of Client Libraries · Internet Archive

Posted by jpluimers on 2021/09/14

Besides manual upload at [Archive.is] Upload to Internet Archive, there are also automated ways of uploading content.

One day I need this to archive pages or sites into the WayBack machine: [WayBack] Overview of Client Libraries · Internet Archive (most of which is Python based):

Read the rest of this entry »

Posted in Bookmarklet, Development, Internet, InternetArchive, Power User, Python, Scripting, Software Development, WayBack machine, Web Browsers | Leave a Comment »

Tricks used by software developers to https://127.0.0.1

Posted by jpluimers on 2021/09/07

Long interesting thread at [WayBack] Thread by @sleevi_: “@SwiftOnSecurity So, some history: It used to be folks would get certs for “localhost”, just like they would from “webmail”, despite no CA e […]”

In  2019, applications were still using tricks (including shipping private keys!) to “securely” access https://127.0.0.1 on some port.

This should have stopped in 2015, but hadn’t. I wonder how bad it still is today.

Related:

Read the rest of this entry »

Posted in Communications Development, Development, DNS, HTTP, Internet, Power User, Software Development, TCP, TLS | Leave a Comment »

For my link archive: DNS over https

Posted by jpluimers on 2021/09/02

DNS over HTTPS

For my link archive:

JSON DNS output

Some DNS over HTTSP providers support dns-json, which Cloudflare delivers non-pretty printed.

Read the rest of this entry »

Posted in Cloud, Cloudflare, Communications Development, Development, DNS, Encryption, HTTP, https, HTTPS/TLS security, Infrastructure, Internet, Internet protocol suite, Power User, Security, Software Development, TCP, TLS | Leave a Comment »

MikroTik RB960PGS hEX PoE powering PoE devices: ensure you get a 48V power supply

Posted by jpluimers on 2021/08/24

By default, the [WayBack] MikroTik RB960PGS hEX PoE comes with a 24V power supply.

Most PoE capable devices cannot be powered by 24V but need 48V. I wrote about this before in the midst of the long post Linus Torvalds – Google+: Working gadgets: Ubiquiti UniFi collection (and a whole bunch of Unifi/Ubiquiti/Ubtn links)

So now I re-mention it in a much smaller post so it easier to find back, and a few links to Power over Ethernet – Wikipedia, where especially these bits are relevant:

  • The PoE Standard implementation for 802.3af (802.3at Type 1) “PoE” requires DC 44.0–57.0 V.
  • Of the PoE Non-standard_implementations, some common Passive specifications include:
    • 24VDC 0.5A 100 Mbit/s or 1 Gbit/s
    • 24VDC 1.0A 100 Mbit/s or 1 Gbit/s
    • 48VDC 1.0A 100 Mbit/s or 1 Gbit/s
    • 56VDC 1.0A and 2.0A 1 Gbit/s (used for 45W+ load point to point microwave and millimeter band radios

The 24V is what MikroTik sticks to with their default power supply.

Read the rest of this entry »

Posted in Internet, MikroTik, Network-and-equipment, Power User, routers, Unifi-Ubiquiti | Leave a Comment »