Efficient Synchronization of State-based CRDTs
share
Data consistency often needs to be sacrificed in order to ensure high-availability in large scale distributed systems. Conflict-free Replicated Data Types (CRDTs) relax consistency by enabling query and update operations to be performed locally at any replica without synchronization. Consistency is achieved by background synchronization operations. In state-based CRDTs replicas synchronize by periodically sending their local state to other replicas and merging the received remote states into the local state. This can be extremely costly as the local state grows. Delta-based CRDTs address this problem by defining delta-mutators, which produce small incremental states (deltas) to be used in synchronisation, instead of the full state. However, current synchronization algorithms induce redundant wasteful delta propagation, namely in the general case of a network graph with alternative synchronization paths (desirable to achieve fault-tolerance). In this paper we explore this problem and identify two sources of inefficiency in current synchronisation algorithms for delta-based CRDTs. We evolve the concept of join decomposition of a state-based CRDT and explain how it can be used to boost the efficiency of synchronization algorithms.
READ FULL TEXT