Archive for the ‘InternetArchive’ Category
Posted by jpluimers on 2021/06/07
For my link archive, some tweets. [WayBack] Mark Graham is the person to contact in case archiving a link in the WayBack machine fails.
These are the steps for my link archival:
- check if it saves and renders with the WayBack machine, if so, copy the saved URL and the original URL
- check if it saves and renders with archive.is, if so, copy the saved URL and the original URL
- if neither saved, then use the original URL and link text, but note it was unsavable; otherwise prepend the original URL and link text with [WayBack] or [Archive.is] containing the saved URL
Reporting history gist: https://gist.github.com/jpluimers/6115b3cd6dab568ebd1c10ebddfaf140
–jeroen
Read the rest of this entry »
Posted in Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/05/05
A while ago I wrote about Helping the WayBack ArchiveTeam team: running their Warrior virtual appliance on ESXi.
Since it was scheduled before my cancer treatment started and got posted when still recovering from it, I missed that version 3.2 of the [Wayback] ArchiveTeam Warrior appliance appeared in the [Wayback] Releases · ArchiveTeam/Ubuntu-Warrior at [Wayback] Release v3.2 · ArchiveTeam/Ubuntu-Warrior. You can download it form these places:
These two sites have not yet been updated, so they contain the older versions:
The source code now has been moved three times:
Read the rest of this entry »
Posted in *nix, *nix-tools, ArchiveTeamWarrior, Cloud, Containers, diff, Docker, ESXi5, ESXi5.1, ESXi5.5, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Infrastructure, Internet, InternetArchive, Kubernetes (k8n), KVM Kernel-based Virtual Machine, patch, Power User, VirtualBox, Virtualization, VMware, VMware ESXi, VMware Workstation, WayBack machine | Leave a Comment »
Posted by jpluimers on 2021/03/26
Learned about this in the fall of G+: [WayBack] GitHub – webrecorder/webrecorder-user-guide: webrecorder user guide and glossary
Back then, fully automated tools were easier.
So it is on my list of things to try one day for smaller projects.
From the glossary:
Quick Start
- Enter a URL in the box in the center of the screen labeled ‘URL to capture’.
- Press the ‘start’ button (look down and to the right of the box where you entered the URL).
- Interact with the web page that loads so Webrecorder can capture the content displayed on this page. To collect audio or video from a page be sure to press ‘play’ so the file will load into the browser.
- Continue to visit and browse the pages you would like to capture. Each page you view will be included in your capture session. Note: you will be capturing the contents of each page you visit but will not automatically obtain pages that are linked to on the pages you collect (hyperlinks).
- To end your capture session hover over the ‘Capture’ button in the upper left corner of the screen so it changes to read ‘Stop’ then click that button.
- Your capture will then be browsable. Note: the capture will not ‘replay’ like a linear recording but instead be an interactive copy of the pages you have collected.
- If you are a logged-in user, this session will be saved to your account automatically. If you are not logged in an account, you can sign up for an account or log in to your existing account to save the collection after you create it. If you do not log in you can still download your collection for a limited time (approximately 90 minutes from when you stop your recording session).
–jeroen
Posted in Internet, InternetArchive, Power User | Leave a Comment »
Posted by jpluimers on 2021/03/19
The [WayBack] Archiveteam helps the WayBack machine with feeding new content.
You can help that team by running one or more “warrior” virtual machine instances. The VM is distributed as a virtual appliance in an ova file according to the Open Virtualization Format.
That format sounds more generic than it actually is, so the (at the time of writing) archiveteam-warrior-v3-20171013.ova file at [WayBack] Index of /downloads/warrior3/ was created for VirtualBox.X
This meant running it on VMware ESXi or VMware vSphere takes a few steps for patching it, then uploading it to your VMware host.
Since I might want to run the appliance on multiple places or multiple instances, I wanted to have a ready-to-go solution, I created a git repository with both the patch instructions and the update at [WayBack] wiert.me / public / ova / archiveteam-warrior-v3-20171013.ESXi · GitLab.
Read the rest of this entry »
Posted in ArchiveTeamWarrior, Cloud, Containers, Docker, Infrastructure, Internet, InternetArchive, Kubernetes (k8n), Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2020/11/13
Archiving Google Product Forum URLs is a pain in the butt for a couple of reasons:
So the trick for saving is:
- Get from the
/forum/#!topic/ based URL to the /d/topic/ based one
- Put it after the
archive.is/?run=1&url=, then save
--jeroen
Posted in Conference Topics, Conferences, Event, Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2019/10/19
Got this a while ago while saving a bunch of links for my blog; unfortunately the email address did not respond for information
Too Many Requests
We are limiting the number of URLs you can submit to be Archived to the Wayback Machine, using the Save Page Now features, to no more than 15 per minute.
If you submit more than that we will block Save Page Now requests from your IP number for one day.
Please feel free to write to us at info@archive.org if you have questions about this. Please include your IP address and any URLs in the email so we can provide you with better service.
I wish there was a queue service that would make you wait longer, but does fulfill the request.
–jeroen
Posted in Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2019/08/16
When archiving pages in the WayBack machine, despite Privacy Badger having set to “save no cookies”, it still managed to set truckloads of cookies.
So I used the Chrome settings in chrome://settings/content/cookies to disable cookies and now everything is fine.
–jeroen
Read the rest of this entry »
Posted in Chrome, Google, Internet, InternetArchive, Power User, Privacy, WayBack machine | Leave a Comment »
Posted by jpluimers on 2019/05/27
When you get the response “web.archive.org unexpectedly closed the connection” without even returning an HTTP code, but:
- it works in anonymous mode
- it works with all extensions turned off
then likely there are too many cookies for archive.org or/and web.archive.org: in my case, I had 90 cookies.
Cleaning these cookies out resolved the problem (I used [WayBack] Awesome Cookie Manager for this).
Edit 20231230: Awesome Cookie Manager source repository at [Wayback/Archive] Phatsuo/awesome-cookie-manager: Awesome Cookie Manager.
--jeroen
Posted in Chrome, Google, Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
Posted by jpluimers on 2019/04/18
I still have to do this every few weeks on all my desktop machines: [WayBack] When +Google Nederland maps only fills none or part of the map tiles… – Jeroen Wiert Pluimers – Google+
When +Google Nederland maps only fills none or part of the map tiles at https://maps.google.nl, but https://maps.google.com works fine, then remove any gsScrollPos cookies from www.google.nl.
I need to do this every couple of days to keep maps.google.nl working.
Later I also found it can happen for YouTube, then did more digging for gsScrollPos and found a better workaround: [WayBack] Awesome Cookie Manager where you can just delete the gsScrollPos cookies from all sites in one go.
Even later I found out that this can be one of the causes for the WayBack machine giving an error 400 when archiving. A more common reason however is that many archived web-pages try to create cookies in the web.archive.com subdomain resulting in the same problem.
The cause seems to be the Great Suspender plugin which should be fixed by now, but might not automatically update to the latest version. See:
Pending a new Great Suspender release, below is a quick way to manually remove them if you are into SQL scripting for sqlite. It basically comes down to executing the below statement when Chrome is closed:
delete from cookies where name like 'gsScrollPos-%'
Edit 20231230: Awesome Cookie Manager source repository at [Wayback/Archive] Phatsuo/awesome-cookie-manager: Awesome Cookie Manager.
--jeroen
Posted in Chrome, Google, GoogleMaps, Internet, InternetArchive, Power User, WayBack machine | Leave a Comment »
The Wiert Corner - irregular stream of stuff 