The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,528 other followers

Archive for the ‘DevOps’ Category

In operations, code is not your friend. Make things simple, make them boring …

Posted by jpluimers on 2018/11/21

Painful lesson learned a while ago: In operations, code is not your friend. Make things simple, make them boring and make them obvious, and keep an eye on the configuration complexity cloc… – Kristian Köhntopp – Google+

Read the rest of this entry »

Posted in Cloud, Development, DevOps, Infrastructure, Software Development | Leave a Comment »

GitHub – yandex/gixy: Nginx configuration static analyzer

Posted by jpluimers on 2018/10/26

[WayBack] GitHub – yandex/gixy: Nginx configuration static analyzer

Gixy is a tool to analyze Nginx configuration. The main goal of Gixy is to prevent security misconfiguration and automate flaw detection.

Right now Gixy can find:

You can find things that Gixy is learning to detect at Issues labeled with “new plugin”

This helps you prevent an nginx configuration issue that can server too many static content by using ../ in the web request which got a lot of attention last week, but was in fact already found during 2016 HCTF by Aklis, and presented by Orange Tsai (twitter/github/blog) various times in 2018, including [WayBack] hack.lu 2018.

.

Related:

–jeroen

Read the rest of this entry »

Posted in *nix, DevOps, nginx, Power User, Security | Leave a Comment »

Doing hardware upgrades, infrastructure rearrangements and software updates over this week. Expect some down-time…

Posted by jpluimers on 2018/04/16

This is upgrade/update week, so new disk space, and quite a bit of reorganisation going on.

Expect down-times in various portions of infrastatus.wiert.me

Secondaries should catch most, but some of the web-sites will be down for a while.

–jeroen

Posted in DevOps, Infrastructure, Power User | Leave a Comment »

EmbarcaderoMonitoring – monitoring the Embarcadero internet related services

Posted by jpluimers on 2018/03/15

Over time, there are lots of complaints about Embarcadero related internet services (like forums, QC, Appanalytics, docwiki, blogsweb site, maintenance) so to track uptime, I’ve created a set of EmbarcaderoMonitoring pages:

This is preliminary work based on my own lists of Embarcadero endpoints combined with some research like [WayBack] dnsdumpster embarcadero.com.png and [WayBack] IdentIPSpy

Underneath, they run on the uptimerobot.com infrastructure which has a limit of 50 free monitors.

It means I have to:

  • trim this down for relevancy
  • better document the endpoint
  • find correct endpoint targets for the black (disabled) and red (down) entries as a few of them might need tweaking
  • maybe split off an insecure and secure version (now most subdomains have both http and https monitored)

Any ideas on improving this are welcome: please post a comment here on on the resulting G+ thread.

Note it likely won’t show cases like when the website was hacked or TLS certificate issues like in SSLLabs security reports for some embarcadero subdomains. I need to think about a means for those, as it will certainly help monitoring my own infrastructure in a similar way.

–jeroen

Read the rest of this entry »

Posted in *nix, Cloud, Development, DevOps, Infrastructure, Monitoring, Power User, Software Development, Uptimerobot | Leave a Comment »

Happy “check your backups day”; does your restore process work? And how is the rest of your admin process doing?

Posted by jpluimers on 2018/02/01

Today is [WayBack] Check your backups Day! started by @CyberShambles in dedication of the @Gitlab outage on 20170201.

Please check your restoration process now. As people screw up and accidents happen (I know first hand from a client).

Why isn’t this date on January 31st? Long short story: the failure started that date, but restoration took most of 20170201. So February 1st it is.

Others will follow and GitLab wasn’t alone, as a few days before soup.io had to restore a 2015 database backup.

It all comes back to

Nobody wants backup.

Everybody wants restore.

which made it to the 2008 [WayBackadminzen.org – The Admin Zen and has been attributed to various people including [WayBackto Kristian Köhntopp and [WayBackto Martin Seeger who told Kristian Köhntopp that it was coined by Sun’s Michael Nagorsnik at one of the early [WayBackNuBIT. Martin was there; he knows (:

The oldest mention of the phrase I could find was in 2006 by Volker Bir at [WayBackSpy Sheriff – so how do people get infected w/ this thing?.

Keeping clients in the loop

Since soup.io hosts their updates blog on their own platform, the restore resulted in the post prior to [Archive.isUpdate after crash ;) – Soup Updates sort of ironically being the mid-2015 [WayBackGive us your money! – Soup Updates. Usually dogfooding is a good thing though.

During such a downtime, it is crucial to stay in touch through alternative channels. Soup.io didn’t do a good job on their twitter account: they only announced the “update after crash”, not being down, why or progress.

They also deny the WayBack machine access to updates.soup.io because of [WayBack] robots.txt because how they redirect through /remotes, but luckily Archive.is doesn’t care about that and has less old updates.soup.io archived as recent as end of 2015.

GitLab did a much better job on their GitLabStatus account.

Postmortems and organisation culture.

Everybody can screw up, and usually a severe outage happens even when everybody tries to do the right thing. The only way to learn from it is to have [WayBackBlameless PostMortems and a Just Culture – Code as Craft.

Read the rest of this entry »

Posted in DevOps, Power User | Leave a Comment »

 
%d bloggers like this: