Chubby Lock Service
Chubby lock service is a distributed system meant for coarse-grained activities. Although it was initially designed for activities in Google’s distributed systems, it is currently used widely as a repository and name service for configuration information. It provides low-volume but reliable storage for loosely-coupled distributed systems made up of small machines that communicate via high-speed networks.
A Chubby cell may serve 10,000 four-processor machines connected by 1Gbit Ethernet. Most of the cells are confined to one machine room or data centre although it is possible to run a Chubby cell with replicas located thousands of kilometres away.
The service is meant to allow clients to synchronise activities and agree on basic details of their environment. It may be used in a variety of ways, including:
- appointing a master server
- electing a master
- letting the master discover the servers under its control
- allowing clients to find their master
- storing small amounts of meta-data
- partitioning work among several servers
Chubby’s interface is similar to that of simple distributed file systems that perform whole-file writes and reads and have advisory locks. However, it is designed to focus more on reliability and availability to many clients than high performance. The lock service focuses less on storage capacity and through-put. It uses simple semantics.
Prior to the deployment of Chubby lock service, most of Google’s distributed systems relied on ad hoc methods for primary election if they could duplicate work without harm. Alternatively, they needed operator intervention in case they had to be very precise. Chubby leads to less computing effort in the first case and notable improvement in availability in the second one where systems do not need human intervention.
The service uses well-established ideas, including:
- consistent client-side caching to use simple semantics while reducing server load
- distributed consensus among replicas to tolerate fault
Simple load adaptation and protocol-conversion servers allow each Chubby instance or cell to be scaled to tens of thousands of processes. The use of partitioning and proxies may scale it even further.
Chubby lock service is the main internal name service that Google uses. Systems like MapReduce rely on it as a rendezvous mechanism. The service provides a standard repository for access control lists and other files requiring high availability. It may also be used to choose a primary replica from redundant ones.
Burrows, M. The Chubby lock service for loosely-coupled distributed systems, http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/chubby-osdi06.pdf