{"id":7968,"date":"2013-05-16T09:16:58","date_gmt":"2013-05-16T07:16:58","guid":{"rendered":"https:\/\/blog.trifork.com\/?p=7968"},"modified":"2013-05-16T09:16:58","modified_gmt":"2013-05-16T07:16:58","slug":"eventual-consistency","status":"publish","type":"post","link":"https:\/\/trifork.nl\/blog\/eventual-consistency\/","title":{"rendered":"Eventual Consistency"},"content":{"rendered":"<p>Once in a while, an idea emerges that is contrary to the way we have grown accustomed to doing things. Eventual consistency is such an idea, and the way we used to build datastores was with SQL and ACID transactions.<\/p>\n<p>So whats wrong with that?<\/p>\n<h2>Too many generals<\/h2>\n<p dir=\"ltr\">Information always flows as messages using a medium &#8211; not transactions. Atomic transactions spanning multiple systems is an abstraction that doesn\u2019t really exist in real life. It\u2019s impossible to guarantee the atomicity of a distributed transaction as proved by the <a href=\"http:\/\/en.wikipedia.org\/wiki\/Two_Generals%27_Problem\">Two_Generals%27_Problem<\/a>. The more generals &#8211; or partitions of a distributed system &#8211; the worse the problem. Not knowing the entire truth, is thus a fact of life that each partition must deal with, and therefore we need to ensure that the knowledge of all partitions converges when they send messages to eachother. This is eventual consistency.<\/p>\n<h2>Eventual Consistency in Real Life<\/h2>\n<p>Finance has always embraced eventual consistency. Money transfer is &#8211; contrary to popular belief &#8211; not done by transactions, but is an elaborate process where the money is first withdrawn from one account and after some time deposited at another account. Meanwhile the system is inconsistent for the outside observer, due to money being \u201cin movement\u201d. When the money arrives, eventually, the system will again be consistent.<\/p>\n<p><span style=\"font-size: 13px;line-height: 19px\">If I send a cheque by mail when I have enough money on my account, but it turns out I have insufficient funds, when the receiver tries to cash it, a conflict arises due to balance information being slow and inconsistent. The entire system still manages because we have processes for resolving conflicts. In this case the cheque bounces.<\/span><\/p>\n<p><span style=\"font-size: 13px;line-height: 19px\">It turns out, that in all cases of distributed updates to an object predating computerisation, eventual consistency is the norm, and conflict resolution processes exist. Medical records is another example where information from many sources about the same patient arrive out of order and is eventually reconciled.<\/span><\/p>\n<p>Early computerisation changed this by building monolithic systems to model the truth about something &#8211; and when the system was not available, users had to wait or go back to the old and robust processes without computers.<\/p>\n<h2>Physical limitations<\/h2>\n<p dir=\"ltr\">Consider the physics of sending information. Disregarding exotic theories involving multiple timelines, information cannot move faster than light. This would lead to a paradox, where you could make a phone call into the past with a so-called <a href=\"http:\/\/en.wikipedia.org\/wiki\/Tachyonic_antitelephone\">Tachyonic_antitelephone<\/a>.<\/p>\n<p><span style=\"font-size: 13px;line-height: 19px\">In the globally connected world, location and travel of information imposes very real limitations on the availability of up-to-date data. Connections drop, systems go down, sometimes entire data centers go offline for extended periods of times, and come back up using a restored, old version of data. When building highly distributed systems, these limitations cannot be sanely abstracted away in underlying distributed databases. The ugly head of reality will either cause distributed transactions to crumble write availability, or make cracks in the flawed illusion that all users always see up-to-date data.<\/span><\/p>\n<p dir=\"ltr\"><strong>Dealing with eventual consistency<\/strong><\/p>\n<p dir=\"ltr\">It is the gut instinct of every programmer to avoid complexity by abstraction. Since the old abstractions of transactions and a perfectly up-to-date database don\u2019t scale to distributed systems with high write availability, we need eventual consistency as theoretical basis and new abstractions and techniques for handling it. Luckily this has been the subject of academia for decades, and recently the NoSQL movement has pioneered numerous small and large-scale eventually consistency systems drawing many real-life experiences, which will be the topic of my next blog entry.<\/p>\n<address><span style=\"font-size: 13px;line-height: 19px\">Rune Skou Larsen<\/span><\/address>\n<address>NoSQL evangelist @Trifork<\/address>\n","protected":false},"excerpt":{"rendered":"<p>Once in a while, an idea emerges that is contrary to the way we have grown accustomed to doing things. Eventual consistency is such an idea, and the way we used to build datastores was with SQL and ACID transactions. So whats wrong with that? Too many generals Information always flows as messages using a [&hellip;]<\/p>\n","protected":false},"author":105,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":""},"categories":[125,10],"tags":[],"class_list":["post-7968","post","type-post","status-publish","format-standard","hentry","category-nosql-2","category-development"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Eventual Consistency - Trifork Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/trifork.nl\/blog\/eventual-consistency\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Eventual Consistency - Trifork Blog\" \/>\n<meta property=\"og:description\" content=\"Once in a while, an idea emerges that is contrary to the way we have grown accustomed to doing things. Eventual consistency is such an idea, and the way we used to build datastores was with SQL and ACID transactions. So whats wrong with that? Too many generals Information always flows as messages using a [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/trifork.nl\/blog\/eventual-consistency\/\" \/>\n<meta property=\"og:site_name\" content=\"Trifork Blog\" \/>\n<meta property=\"article:published_time\" content=\"2013-05-16T07:16:58+00:00\" \/>\n<meta name=\"author\" content=\"Rune Skou Larsen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rune Skou Larsen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/trifork.nl\/blog\/eventual-consistency\/\",\"url\":\"https:\/\/trifork.nl\/blog\/eventual-consistency\/\",\"name\":\"Eventual Consistency - Trifork Blog\",\"isPartOf\":{\"@id\":\"https:\/\/trifork.nl\/blog\/#website\"},\"datePublished\":\"2013-05-16T07:16:58+00:00\",\"author\":{\"@id\":\"https:\/\/trifork.nl\/blog\/#\/schema\/person\/67d6d5d4eb4f82ea8e1ac6b803b91380\"},\"breadcrumb\":{\"@id\":\"https:\/\/trifork.nl\/blog\/eventual-consistency\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/trifork.nl\/blog\/eventual-consistency\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/trifork.nl\/blog\/eventual-consistency\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/trifork.nl\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Eventual Consistency\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/trifork.nl\/blog\/#website\",\"url\":\"https:\/\/trifork.nl\/blog\/\",\"name\":\"Trifork Blog\",\"description\":\"Keep updated on the technical solutions Trifork is working on!\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/trifork.nl\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/trifork.nl\/blog\/#\/schema\/person\/67d6d5d4eb4f82ea8e1ac6b803b91380\",\"name\":\"Rune Skou Larsen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/trifork.nl\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/06f7df67fe9c51aee574f2128cdcaf95?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/06f7df67fe9c51aee574f2128cdcaf95?s=96&d=mm&r=g\",\"caption\":\"Rune Skou Larsen\"},\"url\":\"https:\/\/trifork.nl\/blog\/author\/runes\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Eventual Consistency - Trifork Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/trifork.nl\/blog\/eventual-consistency\/","og_locale":"en_US","og_type":"article","og_title":"Eventual Consistency - Trifork Blog","og_description":"Once in a while, an idea emerges that is contrary to the way we have grown accustomed to doing things. Eventual consistency is such an idea, and the way we used to build datastores was with SQL and ACID transactions. So whats wrong with that? Too many generals Information always flows as messages using a [&hellip;]","og_url":"https:\/\/trifork.nl\/blog\/eventual-consistency\/","og_site_name":"Trifork Blog","article_published_time":"2013-05-16T07:16:58+00:00","author":"Rune Skou Larsen","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Rune Skou Larsen","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/trifork.nl\/blog\/eventual-consistency\/","url":"https:\/\/trifork.nl\/blog\/eventual-consistency\/","name":"Eventual Consistency - Trifork Blog","isPartOf":{"@id":"https:\/\/trifork.nl\/blog\/#website"},"datePublished":"2013-05-16T07:16:58+00:00","author":{"@id":"https:\/\/trifork.nl\/blog\/#\/schema\/person\/67d6d5d4eb4f82ea8e1ac6b803b91380"},"breadcrumb":{"@id":"https:\/\/trifork.nl\/blog\/eventual-consistency\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/trifork.nl\/blog\/eventual-consistency\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/trifork.nl\/blog\/eventual-consistency\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/trifork.nl\/blog\/"},{"@type":"ListItem","position":2,"name":"Eventual Consistency"}]},{"@type":"WebSite","@id":"https:\/\/trifork.nl\/blog\/#website","url":"https:\/\/trifork.nl\/blog\/","name":"Trifork Blog","description":"Keep updated on the technical solutions Trifork is working on!","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/trifork.nl\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/trifork.nl\/blog\/#\/schema\/person\/67d6d5d4eb4f82ea8e1ac6b803b91380","name":"Rune Skou Larsen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trifork.nl\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/06f7df67fe9c51aee574f2128cdcaf95?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/06f7df67fe9c51aee574f2128cdcaf95?s=96&d=mm&r=g","caption":"Rune Skou Larsen"},"url":"https:\/\/trifork.nl\/blog\/author\/runes\/"}]}},"_links":{"self":[{"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/posts\/7968","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/comments?post=7968"}],"version-history":[{"count":0,"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/posts\/7968\/revisions"}],"wp:attachment":[{"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/media?parent=7968"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/categories?post=7968"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/trifork.nl\/blog\/wp-json\/wp\/v2\/tags?post=7968"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}