Archive for the ‘InternetArchive’ Category
Posted by jpluimers on 2022/02/01
I started my post ESXi: editing /etc/vmware/hostd/vmInventory.xml to fix the datastore UUID for unavailable VMs with
In case I ever need this on ESXi: Insights into the VMware inventory files (vmAutoStart.xml and vmInventory.xml on ESXi; inventory.vmls on VMware Workstation/Player)
Since almost all of my blog is about things I bumped into in real life, this post was a preparation because I kind of expected this to indeed happen, and it did.
Below are the screenshots and steps I took. Of course it is an N=1 experience, so your situation might differ, but I tried to be thorough and not miss any steps.
Read the rest of this entry »
Posted in ArchiveTeamWarrior, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Internet, InternetArchive, Power User, Virtualization, VMware, VMware ESXi, WayBack machine | Leave a Comment »
Posted by jpluimers on 2022/01/27
First the script that display messages for all virtual machines, vim-cmd-display-messages-for-all-VMs.sh:
#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)\s+((\S.+\S)?)\s+(\[\S+\])\s+(.+\.vmx)\s+(\S+)\s+(vmx-[[:digit:]]+)\s*?((\S.+)?)$/\1/p"`
for vmid in ${vmids} ; do
powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/\(vim.vm.ConfigInfo\) \{/,/files = \(vim.vm.FileInfo\) \{/ s/^ +name = "(.*)",.*?/\1/p'`
vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = \(vim.vm.FileInfo\) \{/,/tools = \(vim.vm.ToolsConfigInfo\) \{/ s/^ +vmPathName = "(.*)",.*?/\1/p'`
echo "Messages for VM with id ${vmid} which has power state ${powerState} (name = ${name}; vmPathName = ${vmPathName})."
vim-cmd vmsvc/message ${vmid}
done
exit 0
It is very similar to vim-cmd-reload-all-VM-vmx-configurations.sh from Source: ESXi: reloading all virtual machines from their (potentially) vmx files.
Messages I know either equal “No message” or are about “This virtual machine may have been moved or copied.“
If there is no available message, then you always get the stock message No message., so this is something you can use as a check in scripts.
Read the rest of this entry »
Posted in *nix, *nix-tools, ArchiveTeamWarrior, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »
Posted by jpluimers on 2021/10/24
Today, 20 years ago, the Wayback Machine started to unlock the archived content that the Internet Archive had been crawling since 1996 and make it accessible for the public at large.
Thanks Brewster Kahle for making all of this possible for such a long time!
Read the rest of this entry »
Posted in History, Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/10/06
All my IPv4 addresses seem to be blocked with messages like this (note the odd, but allowed, leading zero in the IPv4 address [WayBack]):
The IP you are accessing the site with (037.153.243.242) has been blocked because it has triggered one of our security measures. Please see the reason below:
Block reason: This IP was identified as infiltrated and is being used by sci-hub as a proxy.
To restore access, please contact onlinesupport@sagepub.com citing this message in full.
A quick [WayBack] “This IP was identified as infiltrated and is being used by sci-hub as a proxy.” – Google Search shows they also block the Google Bot.
I am not not even going to bother with companies that have bad infiltration detection.
Of course I ensured the paper has been archived:
[WayBack/Archive.is] Highly esteemed science: An analysis of attitudes towards and perceived attributes of science in letters to the editor in two Dutch newspapers – Stefan P.L. de Jong, Elena Ketting, Leonie van Drooge, 2020.
Note I do not run sci-hub, though it tempts me doing so. For more info: [WayBack] Sci-Hub – Wikipedia
I checked the router and web-proxy for any suspicious activity. There is none.
I do run the ArchiveBot by the ArchiveTeam to support the WayBackMachine of the InternetArchive and the great team Mark Graham has there providing some bandwidth and CPU/memory resources helping them archive public internet content for posterity.
It that triggers SAGE, too bad for them.
–jeroen
Read the rest of this entry »
Posted in Development, Internet, InternetArchive, LifeHacker, Power User, Software Development, WayBack machine, Web Development | Leave a Comment »
Posted by jpluimers on 2021/09/16
I had trouble with two Intel 600p NVMe SSD devices: read-errors.
It appeared only few tools understand how to get S.M.A.R.T. health information from them, and even then they did not explain the read errors.
I’m going to RMA them, but in case anyone else needs to get health information from NVMe SSD devices, here is which tools do what:
- [Wayback] CrystalDiskInfo version CrystalDiskInfo 8.12.7 ([Wayback/Archive.is] source code) can read the health data, and immediately showed one of the SSDs as bad, but not do a full read-scan of the device.
- [Wayback] HD Tune version 2.55, though old as in 2008 old, can do a full disk scan and shows the read errors perfectly.
- [Wayback/Archive.is] GSmartControl version 1.1.3 – despite being a few months old – cannot read the health data as it uses a far outdated smartmontools version 6.6:
smartctl 6.6 2017-11-05 r4594 [x86_64-w64-mingw32-w10-b19043] (sf-6.6-1)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org
- [Wayback] smartmontools version 7.2 (via command-line tool
smartctl) supports reading health data on Windows 10 via the Microsoft NVMe drivers (see [Wayback] NVMe_Support – smartmontools and [Wayback] Changeset 4348 – smartmontools); download via [Wayback/Archive.is] S.M.A.R.T. Monitoring Tools – Browse Files at SourceForge.net. I used this command for it:
smartctl.exe --xall /dev/sdc
smartctl 7.2 2020-12-30 r5155 [x86_64-w64-mingw32-w10-b19043] (sf-7.2-1)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org
...
Warning Comp. Temp. Threshold: 70 Celsius
...
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
- available spare has fallen below threshold
- media has been placed in read only mode
...
- [Wayback] SpeedFan version 4.52 can only read temperature and reference temperature (which seems to be the [Wayback] Intel NVMe Warning Composite Temperature Threshold) likely because it is already 5 years old.
- [Wayback] Intel Memory and Storage Tool (the successor of the – now deprecated, but only Windows < 10 supported – [Wayback] Intel SSD Toolbox) only gets the health status when doing a full diagnostic scan.
So basically, CrystalDiskInfo and HD Tune are my first line of checking for drive issues, followed by smartmontools to get text output, then by vendor specific tools to assist with the RMA.
In the past, I used another smartmontools wrapper, but it was discontinued and had an even older version than GSmartControl: Source: Closed: HDD Guardian – Home.
On Intel 600p becoming locked in read-only mode after failure:
Start of Intel RMA procedure via [Wayback] Warranty Information.
My case looks remarkably similar to [Wayback] Full Diagnostic Scan always fails during Read Scan on my SSD 600p Series 256GB – Intel Community.
A few screenshots of the tools I used for health information:
Read the rest of this entry »
Posted in Hardware, NVMe, Power User, SSD, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/09/14
Besides manual upload at [Archive.is] Upload to Internet Archive, there are also automated ways of uploading content.
One day I need this to archive pages or sites into the WayBack machine: [WayBack] Overview of Client Libraries · Internet Archive (most of which is Python based):
Read the rest of this entry »
Posted in Bookmarklet, Development, Internet, InternetArchive, Power User, Python, Scripting, Software Development, WayBack machine, Web Browsers | Leave a Comment »
Posted by jpluimers on 2021/06/16
On my list of things to play with: [WayBack] GitHub – jjjake/internetarchive: A Python and Command-Line Interface to Archive.org.
Via:
Related:
- [WayBack] The Internet Archive Python Library — Internet Archive item APIs 1.8.5 documentation
- [WayBack] Command-Line Interface — Internet Archive item APIs 1.8.5 documentation
- [WayBack] Quickstart — Internet Archive item APIs 1.8.5 documentation, including:
Configuring
Certain functionality of the internetarchive Python library requires your archive.org credentials. Your IA-S3 keys are required for uploading, searching, and modifying metadata, and your archive.org logged-in cookies are required for downloading access-restricted content and viewing your task history. To automatically create a config file with your archive.org credentials, you can use the ia command-line tool:
$ ia configure
Enter your archive.org credentials below to configure 'ia'.
Email address: user@example.com
Password:
Config saved to: /home/user/.config/ia.ini
Your config file will be saved to $HOME/.config/ia.ini, or $HOME/.ia if you do not have a .configdirectory in $HOME. Alternatively, you can specify your own path to save the config to via ia --config-file '~/.ia-custom-config' configure.
If you have a netc file with your archive.org credentials in it, you can simply run ia configure --netrc. Note that Python’s netrc library does not currently support passphrases, or passwords with spaces in them, and therefore not currently suported here.
–jeroen
Read the rest of this entry »
Posted in Development, Internet, InternetArchive, Power User, Python, Scripting, Software Development, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/06/07
For my link archive, some tweets. [WayBack] Mark Graham is the person to contact in case archiving a link in the WayBack machine fails.
These are the steps for my link archival:
- check if it saves and renders with the WayBack machine, if so, copy the saved URL and the original URL
- check if it saves and renders with archive.is, if so, copy the saved URL and the original URL
- if neither saved, then use the original URL and link text, but note it was unsavable; otherwise prepend the original URL and link text with [WayBack] or [Archive.is] containing the saved URL
Reporting history gist: https://gist.github.com/jpluimers/6115b3cd6dab568ebd1c10ebddfaf140
–jeroen
Read the rest of this entry »
Posted in Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/05/05
A while ago I wrote about Helping the WayBack ArchiveTeam team: running their Warrior virtual appliance on ESXi.
Since it was scheduled before my cancer treatment started and got posted when still recovering from it, I missed that version 3.2 of the [Wayback] ArchiveTeam Warrior appliance appeared in the [Wayback] Releases · ArchiveTeam/Ubuntu-Warrior at [Wayback] Release v3.2 · ArchiveTeam/Ubuntu-Warrior. You can download it form these places:
These two sites have not yet been updated, so they contain the older versions:
The source code now has been moved three times:
Read the rest of this entry »
Posted in *nix, *nix-tools, ArchiveTeamWarrior, Cloud, Containers, diff, Docker, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Infrastructure, Internet, InternetArchive, Kubernetes (k8n), KVM Kernel-based Virtual Machine, patch, Power User, VirtualBox, Virtualization, VMware, VMware ESXi, VMware Workstation, WayBack machine | Leave a Comment »
The Wiert Corner - irregular stream of stuff 