June 2026
M	T	W	T	F	S	S
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30

Archive for the ‘Scripting’ Category

About Blocks – bl.ocks.org

Posted by jpluimers on 2018/12/20

[WayBack] About Blocks – bl.ocks.org is so cool:

Bl.ocks (pronounced “Blocks”) is a simple viewer for sharing code examples hosted on GitHub Gist.

…

The main source for your example is in index.html. This file can contain relative links to other files in your Gist, such as images, scripts or stylesheets. And of course you can use absolute links, such as CDN-hosted D3, jQuery or Leaflet. To explain your example, add a README.md written in Markdown. (You can omit the index.html if you just want to write, too.)

[WayBack] Code-only-Blocks are cool too:

When your Gist is missing an index.html file, will hide the example iframe but continue to display everything else.

Just compare these:

–jeroen

Posted in Development, DVCS - Distributed Version Control, gist, GitHub, jQuery, Scripting, Software Development, Source Code Management, Web Development | Leave a Comment »

Python threading: logging exceptions during the `run`

Posted by jpluimers on 2018/12/19

In A few observations on Python while I made my first steps into it, I mentioned the standard threading idiom in Python by wrapping the thread in a function. This has the drawback of having to catch and handle any exceptions in that function.

The higher level [WayBack] threading module has a [WayBack] Thread class with a [WayBack] run() method does not handle exceptions either.

For investigation of threading issues, it’s very convenient to know about the exceptions in a thread and their context.

So I’ve made a small base class that automagically logs any exceptions during a run:

import threading

from log import Log

class ExceptionLoggingThread(threading.Thread):

    def __init__(self):
        threading.Thread.__init__(self)
        self.logger = Log().getLogger()
        self.logger.debug("ExceptionLoggingThread().")

    def run_logic(self):
        self.logger.debug("Thread started.")

    def run(self):
        try:
            self.run_logic()
        except:
            self.logger.exception('Exception in `run`')
            raise

–jeroen

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

Programmer humour: Working at PornHub

Posted by jpluimers on 2018/12/11

Programmer humour from a while back:

“I don’t think I would be comfortable telling my family that I develop PHP either.”

Via:

[WayBack] Working at PornHub – CodeProject – Google+
- If you say “I work with PHP” around here, people think you have a lab and you cook up drugs.
[WayBack] Working at PornHub – Jeroen Wiert Pluimers – Google+

Posted in Development, Fun, PHP, Scripting, Software Development | Leave a Comment »

Backing up ZFS snapshots

Posted by jpluimers on 2018/12/06

An interesting idea at [WayBack] I was getting concerned about a backup, which had exceeded 1GB, when the data was only about 400MB. Once the job finished, I realized: Ahh, ZFS compres… – Dan Langille – Google+:

Here’s that script I use for creating/destroy the snaphots for a particular long dataset name. Then I backup from /mnt

[WayBack] gist.github.com/dlangille/480dbca509562eb03e76c2e1b576c6d2 is in sh, not even bash.

–jeroen

Read the rest of this entry »

Posted in Development, Power User, Scripting, sh, ZFS | Leave a Comment »

Parsing simple html in Python

Posted by jpluimers on 2018/11/29

Was working to get fritzcap to emit a list of interfaces so I could specify which one to capture.

For that I needed to parse the output of http://fritz.box/capture.lua which consists of HTML fragments like below.

What I needed was for each consecutive entries of [WayBack] th and first [WayBack] button tags:

content of the th tag
content of the value attribute of the button tag having a type="submit" attribute and name=start attribute

So before starting to work on it, I created [WayBack] In order to fix #5, print a list of available interfaces to potentially capture from · Issue #6 · jpluimers/fritzcap

The goal was to get a series of key/value pairs:

4-138 = AP2 (2.4 + 5 GHz, ath1) - Interface 1 4-137 = AP2 (2.4 + 5 GHz, ath1) - Interface 0 4-132 = AP (2.4 GHz, ath0) - Interface 1 4-131 = AP (2.4 GHz, ath0) - Interface 0 4-129 = HW (2.4 GHz, wifi0) - Interface 0 4-128 = WLAN Management Traffic - Interface 0a

So I built a class descending from [WayBack] HTMLParser — Simple HTML and XHTML parser that ships with the [WayBack] Python standard libraries.

If in the future I need more complex HTML parsing, then these links will help me choosing more feature rich parsers:

Back to the HTMLParser descendant in interfaces_dumper.py which can basically be condensed down to the code below.

handle_data is called for both start tags and end tags. The th value in data is only present in the start tag (at the time of end tag the data is empty), so you need to keep track of both last_start_tag and last_end_tag.
handle_endtag maintains last_end_tag to help handle_data.
handle_starttag maintains last_start_tag to help handle_data and also handles the button behaviour.
- The buttonis only relevant if it has type="submit" and name="start" and a value attribute in that order.
- Output is in data which is an array of key/value pairs.

Read the rest of this entry »

Posted in Development, Fritz!, Fritz!Box, fritzcap, Internet, Power User, Python, Scripting, Software Development | Leave a Comment »

UptimeRobot is written in PHP and runs on IIS

Posted by jpluimers on 2018/11/28

In [WayBack] Jeroen Pluimers‏ @jpluimers: Every now and then editing @uptimerobot entries failed. Just “HTTP Error 503.4 – Service Unavailable The FastCGI pool queue is full” 1/2 I found out that UptimeRobot:

is written in PHP
runs on IIS
uses the FastCGI protocol

There is also a maintenance page at uptimerobot.com/maintenance.php#tvMode [Archive.is] and uptimerobot.com/maintenance.php?c-e [Archive.is]. If you get to those, then retry in ~10 minutes as sometimes it takes that long for an update to be processed.

Sometimes setting up multiple Android devices for the same uptimerobot account can be a bit of a hassle: [WayBack] Uptime Robot on Twitter: “Once logged in to the account from another Andriod device, that device will be added as an alert contact too.… “.

All in all it is still a nice tool (:

–jeroen

Read the rest of this entry »

Posted in *nix, Development, IIS, Monitoring, PHP, Power User, Scripting, Software Development, Uptimerobot, Web Development | Leave a Comment »

A few observations on Python while I made my first steps into it

Posted by jpluimers on 2018/11/28

A while back, I made my first steps into Python.

Coming from a mixed language back-ground (including Pascal, Delphi, C#, SQL, batch files, PowerShell, bash, C, Java) it was an interesting experience.

A few observations:

It’s a [WayBack] line oriented language.
I secretly think that’s the reason there are no multi-line comments in Python, but Guido never gave a reason, just that you can use multi-line “”” or ”’ strings for them: [WayBack] Why doesn’t Python have multiline comments? – Stack Overflow
Multi-line strings can end up in generated documentation or mess with your indenting. So putting a comment # octothorp in front of each line is recommended. [WayBack] Comment out a Python code block – Stack Overflow
Most code editors have a shortcut for multi-line comments. Atom.io has Ctrl-/ for it. [WayBack] Commenting out code blocks in Atom – Stack Overflow
The pass statement is your [WayBack] no-op friend (via [WayBack] What’s a standard way to do a no-op in python? – Stack Overflow).
Though Python supports the subclass-the-Thread class for multi-threading [WayBack] multithreading – Stopping a python thread running an Infinite Loop – Stack Overflow
- The idiom however seems to wrap the thread into a function instead of [WayBack] multithreading – Overriding python threading.Thread.run() – Stack Overflow
Though the built-in data type bool is lower-case, the False and True built-in constants start with a capital [WayBack] 4. Built-in Constants — Python 2.7.13 documentation
Lots of idioms are centered around key/value or name/value pairs.
It has two ways of formatting/aligning strings, which is OK as I hate re-inventing wheels [WayBack] Python spacing and aligning strings – Stack Overflow and [WayBack] 5. Built-in Types — String Formatting Operations
There is a huge number of [WayBack] Python standard libraries.

More observations likely to follow.

–jeroen

Read the rest of this entry »

Posted in Development, Python, Scripting, Software Development | Leave a Comment »

A 90-byte “whereis” program – The Old New Thing

Posted by jpluimers on 2018/11/23

I needed a “get only the first result” of WHERE (which is present after Windows 2000, so XP, Server 2003 and up), so based on [WayBack] A 90-byte “whereis” program – The Old New Thing I came up with this:

@echo off
:: based on https://blogs.msdn.microsoft.com/oldnewthing/20050120-00/?p=36653
::for %%f in (%1) do @echo.%%~$PATH:f
for %%e in (%PATHEXT%) do @for %%i in (%1 %~n1%%e) do (
  @if NOT "%%~$PATH:i"=="" (
    echo %%~$PATH:i
    goto :eof
  )
)
:: note: WHERE lists all occurrences of a file on the PATH in PATH order
goto :eof

Two changes:

it takes into account the extension if you specify it (unlike WHERE.EXE)
it bails out at the first match (like WHERE.EXE)

References:

–jeroen

Posted in Batch-Files, Development, Power User, Scripting, Software Development, The Old New Thing, Windows, Windows Development | Leave a Comment »

How is JavaScript used within the Spotify desktop application? Is it packaged up and run locally only retrieving the assets as and when needed? What JavaScript VM is used? – Quora

Posted by jpluimers on 2018/11/22

[Archive.is] How is JavaScript used within the Spotify desktop application? Is it packaged up and run locally only retrieving the assets as and when needed? What JavaScript VM is used? – Quora

For my archive via a private share.

–jeroen

Posted in Development, JavaScript/ECMAScript, Scripting, Software Development | Leave a Comment »

Find the TTL for a domain and subdomain by getting to the authoritative nameserver first

Posted by jpluimers on 2018/11/15

Lets find the authoritative name server and TTL (time to live) for the example.org domain and www.example.org subdomain.

Notes:

you can also do this using the web-page wrapper around dig [WayBack] Dig web interface

web-pages return most information in one go are [WayBack] intodns.com/example.org and [WayBack] dnsinspect.com: Report for example.org.

the steps below are partially based on a more in depth domain consistency check at [WayBack] Debugging nameservers using dig +norec

1a: get parents of name servers

First start with a root server (dig: getting the list of root servers) to get parents of the name servers for example.org (don’t you love indirection!):

$ dig +norecurse +noall +authority @f.root-servers.net. example.org.
org.            172800  IN  NS  a0.org.afilias-nst.info.
org.            172800  IN  NS  a2.org.afilias-nst.info.
org.            172800  IN  NS  b0.org.afilias-nst.org.
org.            172800  IN  NS  b2.org.afilias-nst.org.
org.            172800  IN  NS  c0.org.afilias-nst.info.
org.            172800  IN  NS  d0.org.afilias-nst.org.

You can repeat this query for 2 more root servers to ensure they are in sync.

1b: get authoritative name servers from the parents

Now repeat with at least 3 of these to ensure they give matching results for the name servers for example.org:

$ dig +norecurse +noall +authority @b0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  b.iana-servers.net.
example.org.        86400   IN  NS  a.iana-servers.net.
$ dig +norecurse +noall +authority @c0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.
$ dig +norecurse +noall +authority @a0.org.afilias-nst.info. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.

2a: getting the domain name servers from a public name server

A query to a public DNS server will also return a name server list, but then you would need to know that name server first. In addition, you can not ask for +authority; you have to ask for +answer NS in stead:

$ dig +norecurse +noall +answer NS @8.8.8.8 example.org.
example.org.        55312   IN  NS  a.iana-servers.net.
example.org.        55312   IN  NS  b.iana-servers.net.

The name servers on the list are not guaranteed to be authoritative, as this query returns an empty result:

$ dig +norecurse +noall +authority @8.8.8.8 example.org.

2b. ensuring the name servers are authoritative name servers

From the name servers returned, you can now check if the servers themselves return the same name servers. If so, then you are sure they are authoritative:

$ dig +norecurse +noall +authority @a.iana-servers.net. example.org.
example.org.        86400   IN  NS  a.iana-servers.net.
example.org.        86400   IN  NS  b.iana-servers.net.
$ dig +norecurse +noall +authority @b.iana-servers.net. example.org.
example.org.        86400   IN  NS  b.iana-servers.net.
example.org.        86400   IN  NS  a.iana-servers.net.

3: get the actual TTL

With the authoritative name servers, you can get the actual TTL:

$ dig +norecurse +noall +answer SOA @a.iana-servers.net. example.org.
example.org.        3600    IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +multiline +answer SOA @a.iana-servers.net. example.org.
example.org.        3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. (
                2017042729 ; serial
                7200       ; refresh (2 hours)
                3600       ; retry (1 hour)
                1209600    ; expire (2 weeks)
                3600       ; minimum (1 hour)
                )

I got the +multiline trick from [WayBack] HOWTO: Using dig(1) to Find DNS Time to Live (TTL) Values – A-Team Systems.

4: get the count down TTL from a local name server

You can repeat the above process with a non-authoritative name server a few times to see the TTL decrease:

$ dig +norecurse +noall +answer SOA example.org.
example.org.        322 IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA example.org.
example.org.        321 IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600

This is for instance what is returned by [WayBack] example.org DNS information – who.is example.org DNS information. DNS records such SOA, TTL, MX, TXT and more.

Public DNS servers having multiple servers per IP can even run disperse TTL numbers, for instance Google DNS at 8.8.8.8 does this:

$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        13  IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        1388    IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600
$ dig +norecurse +noall +answer SOA @8.8.8.8 example.org.
example.org.        10  IN  SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600

Note that `+nssearch` does not work for me

Using +nssearch as per [WayBack] How to find what Authoritative Name Server provided the answer using dig? – Server Fault fails for me:

$ dig +nssearch example.org
SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600 from server 199.43.135.53 in 83 ms.
SOA sns.dns.icann.org. noc.dns.icann.org. 2017042729 7200 3600 1209600 3600 from server 199.43.133.53 in 144 ms.
;; connection timed out; no servers could be reached

This reveals this in the bold values:

The authoritative nameserver sns.dns.icann.org is not publicly accessible.
TTL 7200 (7200 seconds is 2 hours).

Future research

Authoritative answers might not be present in dig queries on some platforms. I need to dig deeper into [WayBack] terminal – Dig not returning authority section? – Ask Different to see why.

Glue records are always tricky to get right: [WayBack] How to check domain NS glue records using dig « Admins eHow

–jeroen

Posted in *nix, *nix-tools, bash, Development, dig, DNS, Internet, Power User, Scripting, Software Development | Leave a Comment »

« Previous Entries

Next Entries »

	xyzzy, Relay Confere… on Sad and Useless about Competit…
	jpluimers on Windows warned me of disk full…
	jpluimers on Started making people walk me…
	jpluimers on Stack Overflow’s forum is dead…
	jpluimers on Some links on getting the most…

The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

Twitter Updates

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘Scripting’ Category

About Blocks – bl.ocks.org

Python threading: logging exceptions during the `run`

Programmer humour: Working at PornHub

Backing up ZFS snapshots

Parsing simple html in Python

UptimeRobot is written in PHP and runs on IIS

A few observations on Python while I made my first steps into it

A 90-byte “whereis” program – The Old New Thing

How is JavaScript used within the Spotify desktop application? Is it packaged up and run locally only retrieving the assets as and when needed? What JavaScript VM is used? – Quora

Find the TTL for a domain and subdomain by getting to the authoritative nameserver first

1a: get parents of name servers

1b: get authoritative name servers from the parents

2a: getting the domain name servers from a public name server

2b. ensuring the name servers are authoritative name servers

3: get the actual TTL

4: get the count down TTL from a local name server

Note that `+nssearch` does not work for me

Future research

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

Subscribe

Archives

Recent Comments

Recent Posts

Blog Stats

Meta title

Tag Cloud Title

Top Clicks

Top Posts

My badges

My Flickr Stream

Pages

All categories

Email Subscription

Archive for the ‘Scripting’ Category

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

Rate this:

Share this:

1a: get parents of name servers

1b: get authoritative name servers from the parents

2a: getting the domain name servers from a public name server

2b. ensuring the name servers are authoritative name servers

3: get the actual TTL

4: get the count down TTL from a local name server

Note that +nssearch does not work for me

Future research

Rate this:

Share this:

Note that `+nssearch` does not work for me