Skip to content

17-Distributed-Concensus-and-Data-Replication.md

Latest commit

 

History

History
99 lines (73 loc) · 3.29 KB

17-Distributed-Concensus-and-Data-Replication.md

File metadata and controls

99 lines (73 loc) · 3.29 KB

Distributed Consensus and Data Replication on Servers

In distributed systems, ensuring data consistency, availability, and reliability is crucial. Key strategies like Master-Slave replication and distributed consensus help maintain these factors.

Key Concepts

  1. Master-Slave Replication:

    • Master handles writes, Slaves replicate data for reads and backup.
    • Pros: Redundancy, higher availability, and load distribution.
    • Cons: Master can become a bottleneck for writes.
    Master -> [Write Operation] -> Slave1, Slave2 (Replicating)

  2. Synchronous vs. Asynchronous Replication:

    • Synchronous: Write to Master and all Slaves, ensuring consistency. Higher latency.
    • Asynchronous: Master writes, slaves replicate later. Faster writes, but possible data inconsistency.
    Synchronous: Master -> Slave1, Slave2 (Wait for confirmation)
Asynchronous: Master -> Slave1, Slave2 (No wait)

  3. Peer-to-Peer Replication:

    • Every node is both Master and Slave. Data is replicated across all nodes.
    • Pros: No single point of failure, scalable.
    • Cons: Complexity increases.
    Node1 <-> Node2 <-> Node3 <-> Node4
(All nodes are equal)

  4. Split Brain Problem:

    • Occurs when network partitions cause two nodes to act as masters, leading to conflicts.
    • Solution: Distributed consensus protocols (e.g., Quorum).
    Node A <-> Node B  (Can Communicate)
Node C (isolated) -> Split Brain

Distributed Consensus Techniques

  1. Leader Election:

    • A leader coordinates tasks like data writes, ensuring no conflict and maintaining consistency.

  2. Techniques:

    • 2-Phase Commit (2PC): Ensures transaction consistency across all nodes. All nodes must agree to commit or abort.
    • MVCC (Multi-Version Concurrency Control): Allows multiple transactions to run concurrently by maintaining different versions of data.
    • SAGAs: Long-running transactions with compensatory actions in case of failure.
    • Quorum-based Voting: Majority of nodes must agree on a decision before committing an operation.

Summary

  • Master-Slave Replication ensures data redundancy and availability, but synchronous replication can introduce latency.
  • Peer-to-Peer Replication avoids a single point of failure, but is more complex.
  • Split Brain can be prevented by consensus protocols like Quorum.
  • Consensus Techniques (2PC, MVCC, Quorum) ensure data consistency across nodes, even in failure conditions.

Visualizations:

Master-Slave Replication:

     [Master] --> [Write] --> [Slave 1]
                     --> [Slave 2]

Synchronous vs. Asynchronous Replication:

  Synchronous:      Asynchronous:
   Master --> Slave1 (Wait)     Master --> Slave1 (No wait)

Peer-to-Peer Replication:

    Node1 <-> Node2 <-> Node3 <-> Node4
    (All nodes are equal, no central master)

Split Brain Scenario:

    Node A <-> Node B
    Node C (isolated)
    (Split Brain: Both Node A and Node B think they're the master)

With these strategies, systems can maintain high availability, handle failures, and ensure consistent data across distributed environments.