The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,861 other subscribers

Archive for the ‘Scripting’ Category

Google Open Source Insights (hopefully by now more than just npm/golang/maven)

Posted by jpluimers on 2022/02/02

Interesting project at [Wayback] Open Source Insights

Open Source Insights is an experimental project by Google.

Hopefully by now it is supporting more than just npm/golang/maven and by the time it sunsets, other projects take over.

The introduction was some 9 months ago: [Wayback] Introducing the Open Source Insights Project | Google Open Source Blog

Via:

–jeroen

Posted in Development, Go (golang), JavaScript/ECMAScript, Node.js, Power User, Scripting, Security, Software Development | Leave a Comment »

cd-to-file.bat for when you have a full filename that is too long to truncate by hand

Posted by jpluimers on 2022/01/31

Small cd-to-file.bat tip:

pushd %~dp1

–jeroen

Posted in Batch-Files, Power User, Scripting, Software Development, Windows | Leave a Comment »

ESXi: on the console/ssh, when a moved VM pauses during power-on: show which VMs have messages waiting, then answer them

Posted by jpluimers on 2022/01/27

First the script that display messages for all virtual machines, vim-cmd-display-messages-for-all-VMs.sh:

#!/bin/sh
vmids=`vim-cmd vmsvc/getallvms | sed -n -E -e "s/^([[:digit:]]+)\s+((\S.+\S)?)\s+(\[\S+\])\s+(.+\.vmx)\s+(\S+)\s+(vmx-[[:digit:]]+)\s*?((\S.+)?)$/\1/p"`
for vmid in ${vmids} ; do
    powerState=`vim-cmd vmsvc/power.getstate ${vmid} | sed '1d'`
    name=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/\(vim.vm.ConfigInfo\) \{/,/files = \(vim.vm.FileInfo\) \{/ s/^ +name = "(.*)",.*?/\1/p'`
    vmPathName=`vim-cmd vmsvc/get.config ${vmid} | sed -n -E -e '/files = \(vim.vm.FileInfo\) \{/,/tools = \(vim.vm.ToolsConfigInfo\) \{/ s/^ +vmPathName = "(.*)",.*?/\1/p'`
    echo "Messages for VM with id ${vmid} which has power state ${powerState} (name = ${name}; vmPathName = ${vmPathName})."
    vim-cmd vmsvc/message ${vmid}
done
exit 0

It is very similar to vim-cmd-reload-all-VM-vmx-configurations.sh from Source: ESXi: reloading all virtual machines from their (potentially) vmx files.

Messages I know either equal “No message” or are about “This virtual machine may have been moved or copied.

If there is no available message, then you always get the stock message No message., so this is something you can use as a check in scripts.

Read the rest of this entry »

Posted in *nix, *nix-tools, ArchiveTeamWarrior, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: storing an ISO 8601 time-stamped backup tarball locally

Posted by jpluimers on 2022/01/25

In Determining the ESXi installation type (2014558) | VMware KB, I also showed how to backup the configuration and download it.

Sometimes you want an ISO 8601 time-stamped local tarball just in case you want to revert to it at a later stage.

First a small recap on how to get the tarball, download location and temporary location in the first place (it will be automatically deleted from the temporary location):

# vim-cmd hostsvc/firmware/sync_config
# vim-cmd hostsvc/firmware/backup_config
Bundle can be downloaded at : http://*/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz
# find /scratch/downloads/ -name *.tgz
/scratch/downloads/52aa233b-5db4-2298-5e1b-f510b2cd149f/configBundle-ESXi-X10SRH-CF.tgz

Goal is to get the download filename and save it to a different folder and embed the ISO 8601 timestamp in the filename.

Like many scripts, sed and regular expressions come to the rescue once more, just like in ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes (which we will need anyway because of the ISO 8601 time stamp, and a bit of fiddling at regex101.com/r/NyrzKF

# SCRATCH_CONFIG_BUNDLE_NAME=$(vim-cmd hostsvc/firmware/backup_config | sed -n -E -e "s/^(Bundle can be downloaded at : http://*)(/downloads/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/configBundle-.+?)(.tgz)$//scratch23/p")
# echo "SCRATCH_CONFIG_BUNDLE_NAME: '${SCRATCH_CONFIG_BUNDLE_NAME}'"
SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz'
# date --utc -I'seconds' --reference "${SCRATCH_CONFIG_BUNDLE_NAME}"
2021-05-09T17:44:42UTC

Note:

  • In the sed regular, expression, [[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12} matches a GUID.
  • In the date command, the --reference parameter must be last.
  • If you get the error below, then you ran too many backup_config commands in succession:
    (vim.fault.TooManyWrites) {
       faultCause = (vmodl.MethodFault) null, 
       faultMessage = 
       msg = "Received SOAP response fault from []: syncConfiguration
    fault.TooManyWrites.summary"
    }

    You can see this in the hostd.log, which on my system is in /scratch/log/hostd.log where it says I can retry in 2031 seconds (slightly more than half an hour):

    2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ASSIGN: latestEvent, ha-eventmgr. Applied change to temp map.
    2021-05-09T19:34:13.420Z info hostd[A703B70] [Originator@6876 sub=Vimsvc.ha-eventmgr opID=vim-cmd-27-2ab3] Event 196 : User root@127.0.0.1 logged in as VMware-client/6.5.0
    2021-05-09T19:34:13.420Z verbose hostd[A703B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab3] RecordOp ADD: sessionList["52f6d64e-0a35-c1d7-de97-624d234bc2a7"], ha-sessionmgr. Applied change to temp map.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Created : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ADD: recentTask["haTask--vim.host.FirmwareSystem.syncConfiguration-114207804"], ha-taskmgr. Applied change to temp map.
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
    2021-05-09T19:34:13.423Z error hostd[9AC1B70] [Originator@6876 sub=Hostsvc.FirmwareSystem opID=vim-cmd-27-2ab5 user=root] Failed to sync configuration. Too many writes. Next sync possible in 2031 sec.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Default opID=vim-cmd-27-2ab5 user=root] AdapterServer caught exception: vim.fault.TooManyWrites
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Vimsvc.TaskManager opID=vim-cmd-27-2ab5 user=root] Task Completed : haTask--vim.host.FirmwareSystem.syncConfiguration-114207804 Status error
    2021-05-09T19:34:13.423Z verbose hostd[9AC1B70] [Originator@6876 sub=PropertyProvider opID=vim-cmd-27-2ab5 user=root] RecordOp ASSIGN: info, haTask--vim.host.FirmwareSystem.syncConfiguration-114207804. Applied change to temp map.
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Activation [N5Vmomi10ActivationE:0x0987e6e0] : Invoke done [syncConfiguration] on [vim.host.FirmwareSystem:ha-firmwareSystem]
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Throw vim.fault.TooManyWrites
    2021-05-09T19:34:13.423Z info hostd[9AC1B70] [Originator@6876 sub=Solo.Vmomi opID=vim-cmd-27-2ab5 user=root] Result:
    --> (vim.fault.TooManyWrites) {
    -->    faultCause = (vmodl.MethodFault) null,
    -->    faultMessage = 
    -->    msg = ""
    --> }
    

Not few people have bumped into this, the only other I could find through [Wayback] “vim.fault.TooManyWrites” “syncConfiguration” – Google Search is [Archive.is] mal wieder purple Screen – VMware-Forum.

Figuring out the various parts of the SCRATCH_CONFIG_BUNDLE_NAME: '/scratch/downloads/5271677d-97db-30dc-673d-b99e61bed251/configBundle-ESXi-X10SRH-CF.tgz' is like at regex101.com/r/J4yU72, regex101.com/r/uID9xs and regex101.com/r/o8a4Am:

CONFIG_BUNDLE_DIRECTORY_NAME=$(echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\1/p")
CONFIG_BUNDLE_FILE_NAME=$(     echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\2/p")
CONFIG_BUNDLE_DOT_EXTENSION=$( echo "${SCRATCH_CONFIG_BUNDLE_NAME}" | sed -n -E -e "s/(\/scratch\/downloads\/[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}\/)(configBundle-.+?)(.tgz)$/\3/p")
echo "CONFIG_BUNDLE_DIRECTORY_NAME: '${CONFIG_BUNDLE_DIRECTORY_NAME}'"
echo "CONFIG_BUNDLE_FILE_NAME:      '${CONFIG_BUNDLE_FILE_NAME}'"
echo "CONFIG_BUNDLE_DOT_EXTENSION:  '${CONFIG_BUNDLE_DOT_EXTENSION}'"

Output is like this:

SCRATCH_CONFIG_BUNDLE_NAME:   '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/configBundle-ESXi-X10SRH-CF.tgz'
CONFIG_BUNDLE_DIRECTORY_NAME: '/scratch/downloads/528f9f5a-0123-f022-2b4d-a5c2e595c51a/'
CONFIG_BUNDLE_FILE_NAME:      'configBundle-ESXi-X10SRH-CF'
CONFIG_BUNDLE_DOT_EXTENSION:  '.tgz'

Full backup-config-to-ESXi_configuration_backup-directory.sh script:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, cron/crontab, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: getting and setting the host name, domain and fqdn

Posted by jpluimers on 2022/01/19

A few links and notes:

  1. [Wayback] Changing the hostname of an ESX or ESXi host (1010821)

    Run these commands to change the hostname in ESXi 5.x, ESXi 6.x,ESXi 7.x, using the command line:

    • esxcli system hostname set –host=hostname
    • esxcli system hostname set –fqdn=fqdn
  2. [Wayback] ESX Host appears as localhost.localdomain in VMware Infrastructure/vSphere client (2009720)

    Cause

    The name resolution parameters were not properly configured during the installation of the ESX host.
  3. [Wayback] Domain repoint for embedded vCenter Server fails with error: “domain_consolidator Failed to set machine id” (71020)

    This issue is caused by a mismatch between the FQDN that was configured as the PNID during the vCenter Server deployment and the hostname that is currently configured.

I had a mismatch happen because of the second entry: a host configured in a different domain than it was deployed to.

Here are the commands to list and change the hosts name, domain and fqdn:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

ESXi: persistent files you can edit to apply settings during boot

Posted by jpluimers on 2022/01/18

Since ESXi boots from RAM, most files in /etc are not persisted after modification.

The files that are persisted, are only persisted once every hour by auto-backup.sh, so better run auto-backup.sh by hand if you want to reboot after changing them.

The auto-backup.sh script is ran every hour at 1 minute past the hour as per below crontab.

Default ESXi crontab in /var/spool/cron/crontabs/root:

#min hour day mon dow command
1    1    *   *   *   /sbin/tmpwatch.py
1    *    *   *   *   /sbin/auto-backup.sh
0    *    *   *   *   /usr/lib/vmware/vmksummary/log-heartbeat.py
*/5  *    *   *   *   /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
00   1    *   *   *   localcli storage core device purge

Schedules deciphered via [Wayback] Crontab.guru – The cron schedule expression editor:

  • [Wayback] Every day at 01:01: “This module removes stale temporary files”
    1    1    *   *   *   /sbin/tmpwatch.py
  • [Wayback] Every hour at *:01: saves backup to /bootbank/state.tgz.
    1    *    *   *   *   /sbin/auto-backup.sh
  • [Wayback] Every hour at *:00 logs heartbeat messages to /var/log/vmksummary.log like 2021-02-23T19:00:02Z heartbeat: up 577d2h37m16s, 9 VMs; [[2802426 vmx 4194304kB] [6176344 vmx 4194304kB] [68997 vmx 8388608kB]] [[2802426 vmx 0%max] [6176344 vmx 0%max] [68997 vmx 0%max]]
    0    *    *   *   *   /usr/lib/vmware/vmksummary/log-heartbeat.py
  • [Wayback] Every 5th minute logs to /var/log/hostd-probe.log.
    */5  *    *   *   *   /bin/hostd-probe.sh ++group=host/vim/vmvisor/hostd-probe/stats/sh
  • [Wayback] Every day at 01:00: Removes storage devices which have not been seen in some time interval.
    00   1    *   *   *   localcli storage core device purge

Note that localcli commands are the same as esxcli; for esxcli, a running hostd is required; localcli can run without hostd. See:

–jeroen

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, Software Development, Virtualization, VMware | Leave a Comment »

ESXi ash/dash/busybox shell getting current timestamp in UTC ISO8601 format without colons or dashes

Posted by jpluimers on 2022/01/17

On VMware ESXi, with the  Busybox ash/dash shell, I wanted to get the current UTC timestamp in ISO 8601 format without dashes (-) or especially colons (:) and plus-signs (+) you have to back-slash escape colons or double quote parameters, which is often can be a pain).

This is why we can’t have good things: Getting the UTC 8610 timestamp was far less easy than I hoped for.

First of all, Busybox only allows for a precision of seconds, not milliseconds, and the specification format needs better documentation as per [Wayback] embedded linux – How to get ISO8601 seconds format from “date” in busybox? – Stack Overflow:

Read the rest of this entry »

Posted in *nix, *nix-tools, ash/dash, ash/dash development, Development, ESXi6, ESXi6.5, ESXi6.7, ESXi7, Power User, Scripting, sed, sed script, Software Development, Virtualization, VMware, VMware ESXi | Leave a Comment »

On learning vi/vim

Posted by jpluimers on 2022/01/13

I saved below thread (if you cannot read German: try Google Translate on it [Archive.is]).

The part that – translated – strikes me most is this:

A mini-challenge where I want to spent 15 minutes per day on: when I need some functionality, instead of hacking around, research how it really is supposed to be done and internalise that way.

This totally aligns with me “learn new things every day” mantra.

Also some interesting vi/vim resources and approaches are mentioned:

If you are better at reading tables or lists for commands, I can recommend [Wayback] Basic vi Commands (Computer Science Department, Colorado State University).

One of the hardest things for me to remember somehow is the search/replace syntax, though it is very similar to that of sed. Useful examples of most permutations are at [Wayback] Vi For Smarties – Lesson Eight under “Advanced Search and Replace”:

  1. Use :s/foo/bar/ to replace the first occurrence of the word foo on the current line with the word bar.
  2. Use :s/foo/bar/g to replace all occurrences of the word foo on the current line with the word bar.
  3. Use :%s/foo/bar/g to replace all occurrences of the word foo in the current file with the word bar. Leaving off the g at the end only replaces the first occurrence of foo on each line of the current file.
  4. Use :%s/foo//g to delete all occurrences of the word foo in the current file. Leaving off the percent sign (%), of course, only does this for the current line.
  5. Use :%s/foo/bar/gc to have Vi query you before each attempt to replace the word foo with the word bar.

It is part of a series of tutorials starting at [Wayback] Vi For Smarties – Lesson One, and after the last lesson directs to the in-depth – as prose written – tutorial series [Archive.is] Wayback: The Vi/Ex Editor (which, despite marked by “[ Editor’s Note: . This tutorial series is a “work in progress” and will evolve over time. We’ll announce each new addition and expand this table of contents.]” got ditched shortly after January 2006)

https://web.archive.org/web/20160108053134/http://www.networkcomputin…

.

Note that like sed, you can use a different search/replace character than /: [Wayback] vim – / in vi Search and replace? – Stack Overflow

  • Escape the / with \ like this: \/.
  • Use different search character than /, for instance ., or #.

Back to the twitter thread

[Archive.is] Das königliche Wir on Twitter: “Ich lerne jetzt vi(m). Klingt beknackt, oder? Aber ich hab überlegt: ich möchte effizienter werden, meinen Kram stressfreier bewältigen. Und ich finde, es bietet sich an da an Baustellen anzusetzen, die täglich relevant sind. Und vi(m) nutze ich in der Tat täglich. 1/”

Ich lerne jetzt vi(m).

Klingt beknackt, oder? Aber ich hab überlegt: ich möchte effizienter werden, meinen Kram stressfreier bewältigen. Und ich finde, es bietet sich an da an Baustellen anzusetzen, die täglich relevant sind. Und vi(m) nutze ich in der Tat täglich.

Aber auch wenn ich ihn nutze und über den »Hilfe, wie komm ich aus dem Editor wieder raus?!«-Witz nur sehr müde die Augen rolle gehe ich davon aus, dass ich nicht mal einen Bruchteil der Möglichkeiten ausschöpfe, die er bietet (1. Release 1976, älter als ich!). Das ist spannend.

Das ist jetzt meine Mini-Challenge, auf die ich jeden Tag eine Viertelstunde verwenden will: wenn ich eine Funktion brauche mich nicht mehr drum herum zu hacken, sondern recherchieren wie es richtig geht und das dann gefälligst auch verinnerlichen. Mal sehen, ob das so klappt

In dem Zuge will ich auch wieder verstärkt (neo)mutt in Verbindung mit vim nutzen – das wäre ein wunderbares tägliches Training 😎 Mails schreiben muss man irgendwie immer.

neomutt bietet leider auch einen Eimer voll Funktionen, die ich noch nicht ordentlich nutze. Gnah.

An euren Replies erkenne ich, dass das mit dem »sich die Kürzel merken« echt heikel zu sein scheint 🤔 Wie handhabt ihr das im Alltag? Einfach ein paar Basics wie :u und CTRL+r und gut ist? Ich bin neugierig. Oder nutzt ihr alle nano? (Ich glaub, dann muss ich entfolgen) 😂

Nur als kleinen Zwischenstand: nach dem Lesen eurer Antworten schließe ich, dass ich mich mit meinem Kenntnisstand nicht verstecken muss 😂 Da hat mir der virtuelle Schulterblick schon weiter geholfen.

vi(m), weil ich Admin bin und dieses Tool auf jedem System und ohne X-Geraffels üblicherweise vorfinde (wenn auch bei neueren Installationen dieser absurde „visual mode“ der Default ist 😳). Auch unter (Open)Solaris, IRIX, was weiß ich.

Antworten:

Patrick Ben Koetter (@patrickbkoetter)

Ich habe diverse Plugins ausprobiert und lande immer wieder bei “normal“, weil ich einfach nicht die Disziplin aufbringe das Zeug richtig zu üben.

Marcel Weinberg (@winem_)

Denke so geht es vielen, die eben teilweise auch schon viele Jahre in der IT sind. Ist dir vimtutor bekannt? Den regelmäßig durchzutanzen hilft auch etwas, auch wenn man einiges immer mal vergisst oder einfach nicht nutzt.

Kristian Köhntopp (@isotopp)

vi movement Kommandos haben System. Erkenne und lerne das System. hjkl + prefix+hjkl, Marken, prefix+jump to mark und so weiter.

Und bleibe von den verblödeten Plugins weg. Die braucht kein Mensch und machen vim nur langsam im Start und kompliziert.

Boris (@k3rvyn)

Was mich so richtig weiter gebracht hat, war dieser Talk:
https://www.youtube.com/watch?v=5r6yzFEXajQ

winfreak (@winfreak)

Ich bilde mir kleine Merkphrasen für movements…
Aus „ciw“ wird „change inner word“, z.B.
Das kann ich mir deutlich besser merken.

–jeroen

Read the rest of this entry »

Posted in *nix, *nix-tools, Development, Power User, Scripting, Software Development, vi/vim | Leave a Comment »

A few tools on my list for research: fswatch, entr, ack and ag

Posted by jpluimers on 2022/01/12

Based on the below tweet and replies, these tools are not on my research list:

[Archive.is] Aaron Patterson on Twitter: “Here is my “autotest for lazy people”. It watches for changes in the lib or test directory, and runs rake test on change. There are definitely better, purpose built tools for this. But I am lazy…”

–jeroen

Posted in *nix, *nix-tools, Development, fgrep, find, grep, Power User, Scripting, Software Development | Leave a Comment »

Twitter: find my new-style retweets that have images

Posted by jpluimers on 2022/01/11

This gets the tweets I retweeted and have images in them:

from:@jpluimers filter:images filter:nativeretweets

Based on:

  • [Wayback] twitter – How do I find my retweets of a certain account? – Web Applications Stack Exchange
    from:@someone filter:nativeretweets [KEYWORD(s)]
    

    This shows all retweets of @someone (including the optional KEYWORD(s)). If you retweeted the same tweet you can use @yourtwittername instead of @someone.

  • This article gives you a robust overview of everything you need to know about advanced TweetDeck features.[Wayback] About advanced TweetDeck features

    To search for mentions of #space from verified accounts, excluding Retweets, type the following in the search box: #space filter:verified -filter:nativeretweets

  • [Wayback] Difference between -filter:retweet and -filter:nativeretweets in Twitter Search API 1.1 – Stack Overflow

    I believe per TweetDeck documentation (https://support.twitter.com/articles/20170322) this is the difference:

    filter:nativeretweets shows retweets from users who have hit the retweet button. filter:retweets shows old style retweets (“RT”) + quoted tweets.

    Those are filtering FOR those types of results, but as you’ve done, the – is necessary to filter them out -filter:nativeretweets or -filter:retweets

  • [Wayback/Archive.is] Twitter API 1.1 tweets / favorites (likes) / following / followers backup in web browser
    /* Twitter API 1.1 tweets / favorites (likes) / following / followers backup in web browser
     * Get your access keys to use Twitter API 1.1: https://dev.twitter.com/docs/auth/tokens-devtwittercom
     * You can change Twitter API URL and Twitter screen_name, then execute script from a trusted web page without CSP protection like about:blank in a web browser console (F12 or Ctrl+Shift+K shortcut)
     * A textarea will appear so you can copy/paste to save data as a CSV file or search tweets / users in your web browser (Ctrl+F shortcut)
     * You can then view your backup in a spreadsheet editor like LibreOffice Calc
     * You can also compare the backup with another one to see who unfollowed you, who changed their Twitter username by looking at the user ID or which tweet you retweeted / favorited was deleted (e.g. with the Linux diff command)
     * 
     * Note about the tweets backup:
     * Usually you will search tweets that you retweeted using Twitter web version (https://twitter.com/search) with a search like "from:your_username filter:nativeretweets keyword"
     * But it is limited to the retweets of the last 7 days, like for the free version of the search API (https://developer.twitter.com/en/docs/tweets/search/overview/standard)
     * An alternative is to search tweets in your user timeline with this script but it is limited to your last 3200 tweets (including retweets and replies)
     * This script can be combined with the Twitter feature to backup data, which is not limited to your last 3200 tweets but you can only request a backup every 30 days
     * To find tweets that you retweeted or favorited / liked from a specific person, you can open the CSV file with LibreOffice Calc, click on the column you want to search and press Ctrl+H to search a username
    */

–jeroen

Posted in Development, JavaScript/ECMAScript, Power User, Scripting, SocialMedia, Software Development, Twitter | Leave a Comment »