Skip to content
It Works Locally

Causality in collaborative editors · Part 1 of 2

No Effect Before Its Cause: From Einstein to Lamport

Einstein's moving train, Leslie Lamport's clocks and a 2,500-year-old idea explain the core problem of leaderless replication: ordering writes when no clock can be trusted.

Buddhika Gamage

· 16 min read

Every distributed systems article is legally required to feature Alice and Bob, so let's get it over with.

Alice and Bob are editing the same sentence: "The car is ". Alice types "fast". Bob is on a train (the universe, as you'll see, is a train enthusiast), and just as it enters a tunnel his internet drops. He doesn't notice, so a few minutes later he types "slow" in the same spot, and when the train comes out of the tunnel his edit syncs.

Now the system has to decide what the sentence says. The obvious rule is "newest timestamp wins": Bob's clock says he typed later, so "slow" stays and "fast" quietly disappears, even though Bob never saw it. That rule is wrong, and the reason is the central problem of leaderless replication, where every copy of the data accepts writes on its own and no boss server decides the order.

It turns out physics ran into this exact problem more than a century ago.

This article follows from Einstein's train to Lamport's clocks. It shows why no timestamp can settle who came first, and what leaderless systems trust instead: cause and effect. It ends at CRDTs (conflict-free replicated data types), the data structures that collaborative editors like Yjs use to merge Alice's and Bob's edits with no master clock and no boss server. (A 2,500-year-old Buddhist idea turns up along the way, saying much the same thing.) The rule they all share: no effect before its cause.

Why collaborative editors? Because that's where I ran into leaderless replication: trying to understand how a Notion-like editor lets several people type into the same document, even offline, without ever losing anyone's words.

What "leaderless" actually means

In practice, leaderless means there's no single source of truth everyone has to bow to. Every replica (each copy of the data, on each server or device) accepts writes on its own, online or offline, and the replicas swap updates afterwards. Databases like Amazon's Dynamo, Riak and Cassandra work this way, and so do many collaborative editors. Even when an editor has a sync server in the middle, it's just a messenger relaying updates, not a judge. The goal is that once every replica has received the same updates, they all hold exactly the same data.

That sounds easy until two replicas accept conflicting writes and something has to decide which came first. Physicists have lived without a universal clock for over a century, so let's see how they cope.

Einstein's train: one lightning storm, two correct answers

In 1905, a 26-year-old patent clerk named Albert Einstein realised something unsettling: two events can be simultaneous for one person and not simultaneous for another, and both people are right. In 1916, he explained it with a train.

Here's the setup. A train is zooming along a track. Lightning strikes both ends of the train. A woman is standing on the platform, exactly halfway between the two strikes. Both flashes reach her at the same moment, so she says: "Simultaneous. Obviously."

The catch is that light doesn't add up like a thrown ball. Throw a ball forward on a moving train and someone on the platform sees it go faster, because the train's speed adds to the throw. Light travels at the same speed for everyone, moving or not.

Now the man sitting in the middle of the train. While the light is travelling, the train carries him toward the front flash and away from the rear one, so the front flash reaches him first. Both flashes travel at the same speed, and he's sitting exactly in the middle of the train. So, as far as he's concerned, the flash that reaches him first must have happened first. He says: "The front strike happened first."

rear strike front strike train moving rear flash still coming front flash arrives first train observer platform platform observer: both flashes arrive together Train observer: the front strike came first Platform observer: the strikes were simultaneous
Same two lightning strikes, two honest answers

Who's right? Both of them. There is no master clock hanging over the universe to tell them who's wrong. "At the same time" is not a fact about the universe. It's a fact about whoever is watching. Einstein basically invented "it depends," and computer scientists have been billing for it ever since.

This is exactly the problem with comparing timestamps from two servers. Every server's clock is its own little train. Clocks drift, jump, and get corrected by NTP (the network time protocol that keeps server clocks roughly in sync) at the worst possible moment. Asking "whose edit came first?" by comparing clocks is like asking the woman on the platform and the man on the train to agree on which strike came first. Each server's clock is its own point of view, so they won't agree.

So if nobody agrees on when, does anyone agree on anything? Yes: cause and effect. If the first flash lit a fuse that set off the second, every observer, on any platform or any train, sees the cause before the effect. People can argue about timing all day, but an effect can never show up before its cause.

Take any event, like a firework going off. Light spreads out from it in every direction, and nothing in the universe is faster than light. Not even your project manager asking "any updates?" That growing bubble of light splits all of spacetime into three zones:

  • The future. Everything the firework can still reach. Like your inbox after a holiday.
  • The past. Everything that could have sent news to the firework in time. Like the friend who should have said, "Maybe don't light it."
  • The elsewhere. Everything else: too far away for any news to arrive in time, in either direction. Neither event can cause or affect the other. Their news might bump into each other later, somewhere else, but they can never be cause and effect.
time space path of light Future signals from the event can reach here Past could have sent a signal to the event Elsewhere no signal can connect it to the event Elsewhere order here depends on who is watching event
Only events inside the light cone can be causally linked

Here's the rough rule. Pick any one observer and measure two things: how long light needs to travel between the two events' locations, and how much time actually passed between them. If more time passed than light needs, a signal could have connected them, and every observer agrees on which came first. If less time passed, they're in each other's elsewhere: no signal could connect them, so neither can affect the other, and observers moving differently can disagree about their order. The best part: every observer agrees on which of the two cases it is, even when they disagree about the times.

Quick example: two points on opposite sides of the Earth are about 12,700 km apart in a straight line through the planet, so even light needs about 42 milliseconds to connect them (and about 67 ms if it had to follow the surface, the way a cable does). Two events there one second apart could be causally linked. Ten milliseconds apart? No signal could connect them, so asking which came first has no physical answer. It's like asking a friend which of his two exes was "the problem."

The elsewhere is where all the drama lives. Physics drama, software drama, all of it.

Lamport: "what if computers, but physics?"

In 1978, Leslie Lamport looked at a network of computers and thought, "This is basically a tiny universe." Every machine was an observer with its own clock, and there was no master clock to settle arguments. So he stopped asking "when did this happen?" and started asking "what caused this?" His paper, "Time, Clocks, and the Ordering of Events in a Distributed System," borrows its central idea straight from special relativity.

He draws the network as a simple picture. Each computer is a vertical line, with time running along it. When one computer sends a message to another, you draw an arrow from the sender's line to the receiver's line. The arrow slants because the message takes time to travel: it leaves at one moment and lands at a later one. Messages are the light rays of this universe. They are the only way one machine can find out what another machine did.

time Machine A Machine B Machine C message message a1 a2 b1 b2 c1 c2 a1 happens before c2: a chain of arrows connects them. c1 is concurrent with a1, a2, b1 and b2: no chain links them.
Happens-before: follow the arrows, ignore the clocks

Read it by following the arrows. Even though c1 sits between a1 and a2 on the page, no chain of arrows connects them, so they are concurrent, whatever any clock says.

From that picture, Lamport defined happens-before, and notice that it never mentions a clock.

  1. they're on the same machine and A came first, or
  2. A is sending a message and B is receiving it, or
  3. there's a chain of those steps from A to B.

If neither happens before the other, the events are concurrent. They're in each other's elsewhere. Martin Kleppmann sums it up in Designing Data-Intensive Applications: two operations are concurrent when neither knows about the other, no matter what the clock said. Concurrency isn't about doing things at the same time. It's about being clueless at the same time.

Where the analogy breaks: the internet has no speed limit, just vibes

In physics, the speed of light draws a clean, fixed line around what can affect what. Networks have no such line. What matters isn't whether information could have arrived. It's whether it actually did.

Light could cross between any two data centres on Earth in well under a tenth of a second. Yet two edits made an hour apart can still be concurrent, because the message never got there. A dropped connection, a crashed server, or a laptop that went to sleep mid-sentence stretches the elsewhere from milliseconds to hours.

It's like starlight. Look up at night and you see the Sun as it was 8 minutes ago, and Andromeda as it was 2.5 million years ago. Your screen showing a colleague's edits is old light too, except on a network the delay can be hours, not milliseconds.

Back to Alice and Bob

Now we can say exactly what went wrong in that tunnel. By the wall clock, Bob typed minutes after Alice. But wall-clock order is not cause-and-effect order, and only the second one matters. The key fact is this: Bob never saw "fast." His edit lives in Alice's elsewhere, so the two edits are concurrent. Neither came first, so neither should win just because of what a clock says.

Here's the whole drama, drawn the way Lamport would draw it. Each vertical line is a replica moving through time, and each arrow is a message.

time Alice Sync server Bob offline (tunnel) types “fast” “fast” never arrives types “slow” reconnects “slow” + state vector “slow” the missing “fast” All three replicas converge on the same text
Bob typed later by the clock, yet his edit is concurrent with Alice’s

No chain of arrows connects Alice's edit to Bob's, in either direction. They're strangers in the elsewhere. The minutes between them on the wall clock say nothing about cause and effect, like the "5 min away" on a food delivery app.

Three ways a leaderless system can handle Alice and Bob

This is the central problem of leaderless replication. When any replica can accept writes at any time, with no leader deciding the order, concurrent writes like Alice's and Bob's are guaranteed to happen, and every leaderless system has to decide what to do with them.

There are three common answers. The first trusts clocks. The other two stay on the relativity road and run on cause and effect.

  • Last write wins: trust the clock and keep the newest write.
  • Keep both: store concurrent writes side by side and let the application merge them.
  • CRDTs: build the data so the merge happens automatically.

1. Last write wins: trust the clock

Give every write a timestamp and keep the one with the latest. Cassandra does this by default. It's simple, and it's exactly the mistake the train taught us to avoid. Bob's clock says "later," so "slow" wins and "fast" silently disappears, even though Bob never saw it. Worse, if one replica's clock runs fast, its writes can beat writes that really came after them. Last write wins doesn't resolve conflicts. It hides them.

2. Keep both and let the application merge: the Dynamo style

This is the approach used by Amazon's Dynamo and databases like Riak. Each version of the data carries a version vector: a small set of counters, one per replica, recording which updates that version has already seen. If neither version has seen everything the other has, they're concurrent. The database then keeps both versions side by side (Riak calls them siblings) and lets the application merge them.

The most famous example is Amazon's shopping cart. In the Dynamo paper, Amazon wanted "add to cart" to always work, even when the network was having a bad day, so any replica could accept a write. That means two replicas can end up with different carts. One has a book. Another, which never heard about the book, has a phone. Dynamo used version vectors (it called them vector clocks) to notice that the two carts were concurrent, and it kept both. The application then merged them by taking the union: now the cart has the book and the phone. Nothing you added is ever lost. The price is that a deleted item can sometimes come back from the dead, which explains the shoes you removed three times.

time Client 1 Database Client 2 set cart = [book] version 1 [book] OK, version 1 set cart = [phone] (never saw the book) version 2 two siblings kept: [book] and [phone] OK, version 2 reads both, merges: set cart = [book, phone] version 3 [book, phone] OK, version 3 Neither add was lost: the application merges the siblings.
Concurrent adds are kept side by side, then merged

Read it top to bottom: the database never picks a winner. It keeps both carts as siblings, and the next write merges them.

3. Make the merge automatic: CRDTs

The third answer is the CRDT, short for conflict-free replicated data type. It's a data structure built so that replicas that have seen the same updates always end up with the same result, with nobody stepping in to decide. There are no siblings, and no merge code for you to write.

The trick is that a CRDT never stores "put this word at position 5," because position 5 means different things on different replicas once edits start flying around. Instead it relies on three ideas:

  • Unique IDs. Every insert gets an ID, usually the replica's ID plus a counter. Alice's third edit might be alice:3.
  • Neighbours, not positions. Each insert records what it was inserted next to rather than a numeric index. "My word goes right after bob:1" stays true no matter what else happens to the document.
  • Tombstones. Deletes don't actually remove anything. They mark the item as a tombstone, so later edits that point to it still know where they belong.

For Alice and Bob, that means both words survive. Their two inserts claim the same spot, so every replica falls back on a tie-break rule they all share, something deterministic like "lower replica ID goes first." Same data plus same rule gives the same order everywhere, with no one having to coordinate. And because of how the rules are built, updates can arrive in any order, arrive twice, or arrive late, and the result doesn't change: same set of updates in, same document out.

Collaborative editing libraries like Yjs and Automerge are built on CRDTs, and Riak offers CRDT data types too.

Part 2 of this article goes deep on this third approach using Yjs, a JavaScript library for collaborative editing, because it's the one I have actual experience with. That experience comes from the work of Tharaka Ariyarathna, an absolute legend, whose Notion-like collaborative editor for a customer success management (CSM) platform pulled me into this rabbit hole. Without his work, I'd have spent the weekend taking 400 more photos of my dog. If you'd like to say hi or work with him, he's on LinkedIn.

Read it here: Part 2: how Yjs pulls this off with a handful of bytes per character.

A 2,500-year-old footnote: Pratītyasamutpāda

It's a mouthful (roughly: pruh-TEE-tyuh-sum-oot-PAH-duh). It's a Buddhist idea usually translated as dependent origination, and its shortest form goes: when this exists, that comes to be; with the arising of this, that arises.

I'm not a Buddhist scholar, just as I'm not a physicist, so I'm borrowing the idea very loosely. In the suttas it's mostly about how suffering arises, not about ordering database writes. Please don't send either profession after me.

Still, the shape of it rhymes with this article. Einstein showed that observers can disagree about when something happened, but never about cause and effect. Lamport took that idea into distributed systems: forget agreeing on a universal clock; what matters is whether one event could have caused another.

That's exactly why Alice's and Bob's edits are concurrent: neither caused the other, so neither has to come first. But when one update does depend on another, that dependency matters. If an update arrives before the update it depends on, Yjs doesn't freak out and declare the universe broken. It waits. When this exists, that comes to be; until then, the update sits patiently in a queue.

You never see a glass shatter before it falls, and in Yjs an update never takes effect before its own cause, however confused the clocks are. Some things must come before other things, not because everyone agrees on the time, but because they are connected by cause and effect. In a strange way, that's the whole idea.

So, what did I learn with my free weekend?

Einstein took away the universal "now" and gave us something better: an order of cause and effect that everyone agrees on.

Lamport pointed that idea at computers. Instead of asking "what time did this happen?", he asked "what caused this?" One event comes before another only if a chain of causes connects them: earlier steps on the same machine, or messages sent from one machine to another. No chain, no order, whatever the clocks say.

Leaderless replication is that idea put to work. With no leader to set the order, replicas track what caused what. When two writes have no causal link, a good leaderless system treats them as concurrent and merges them, instead of letting a clock pick a winner.

That rhymes with an idea taught by Lord Buddha: nothing arises on its own; things arise from the conditions that came before them. Lamport, in his own way, drew those conditions as arrows.

Then Kevin Jahns, the creator of Yjs, turned it into something you can npm install. Not bad for four people spread across 2,500 years. And no, they're not concurrent: one of them had a 2,500-year head start, so he sits comfortably in everyone else's past light cone.

To be continued: how Yjs actually pulls this off, with origins, YATA and state vectors. Read Part 2: Yjs Internals →

Further reading (if you have nothing to do)

  • Albert Einstein, Relativity: The Special and the General Theory (1916), the source of the train thought experiment.
  • Leslie Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System" (1978).
  • Martin Kleppmann, Designing Data-Intensive Applications, Chapter 5, "Detecting Concurrent Writes."
  • Nicolaescu, Jahns, Derntl and Klamma, "Near Real-Time Peer-to-Peer Shared Editing on Extensible Data Types" (GROUP 2016), the YATA paper.
  • Yjs documentation and the Yjs source on GitHub.
  • The Dasabala Sutta (SN 12.21), which states the "when this exists, that comes to be" formula of dependent origination. For the full twelve links, see the Paṭiccasamuppāda Vibhaṅga Sutta (SN 12.2). Paṭiccasamuppāda is the Pali form of Pratītyasamutpāda.

Also published on Medium.

  • distributed-systems
  • leaderless-replication
  • crdt
  • physics
  • philosophy

Discussion

Comment as a guest with any name, or sign in / register to keep your name reserved.

Loading comments…

    1. Books

      Timshel: The One Word That Explains Your Entire Life

      On John Steinbeck's East of Eden and the meaning of 'Timshel': the idea that what makes us human is the power to choose.

      6 min read

    Newsletter

    Read the next one first.

    New posts in your inbox, about every two weeks. No spam, unsubscribe in one click.