The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My work

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 1,811 other followers

Consensus systems (Zookepper, etcd, consul) – what are they?

Posted by jpluimers on 2016/12/06

A while ago,  wrote these very nice G+ posts

The English text is about a year older, but the German text Google Translates pretty well.

The most important points in ot for me were these:

  • Consensus systems are distribute systems, so take at least the P (partitioned) from the CAP theorem.
  • In addidtion, Consensus systems also chose the C (consistent) from the CAP theorem.
  • Since in CAP you can only pick 2 out of 3, the A (available) isn’t guaranteed on Consensus systems.
  • Only three systems get this right: Zookeeper, etcd, Consul. All others shred data eventually.
  • Leader election algorithms Paxos and Raft.
  • Cluster a.k.a. Ensemble provide a consistent view of the data no matter to what member of the Cluster/Ensemble you talk to
  • The (set of) connection(s) from a client to the Cluster/Ensemble is called session
  • Cluster/Ensemble operations are on a tree with nodes that can have atomic operations on them
  • Nodes can be persistent or ephemeral (temporal)
  • All nodes can have data (keep it small enough ~4kilobyte max)
  • Directories in the tree are usually persistent; leaf nodes often ephemeral
  • Useful operations: load balancing, queueing, data availability
  • There are transactions so you can make atomic operations larger. Don’t make them too long.
  • Consistency takes time; expect at max ~1000s of write operations per second
  • Not being available is a feature (it means it still is P and C, just not reachable right now)
  • Clients must cope with the Cluster/Ensemble being temporarily being read-only or unavailable
  • Applications should always re-create any persistent nodes they create (just in case – during non availability – from one consistent phase to another consistent phase) a persistent node is no more.

Some more keywords and links from the article:

–jeroen

2 Responses to “Consensus systems (Zookepper, etcd, consul) – what are they?”

  1. Imesha Sudasingha said

    What about hazlecast? As far as i know, hazlecast too do not do data sharding.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: