Piper (source control system)
Piper is a centralized version control system used by Google for its internal software development. Originally designed for Linux, it supports Microsoft Windows and macOS since October 2012.[1]
Scale
[edit]Since its founding years Google used a central codebase shared by the developers.[2] For over 10 years Google relied on a single Perforce instance, using proprietary caching for scalability.[3] This mode of operation was kept as Google grew, the need for further scaling led to the development of Piper.[4] Currently, Google's version control "is an extreme case":[5] as of 2016, the repository was storing 86 terabytes of data comprising two billion lines of code in nine million files (two orders of magnitude more than in the Linux kernel repository). 25 thousand developers contributed 16 thousand changes daily, with additional 24 thousand commit operations by bots. Read requests each day are measured in billions.[6]
Architecture
[edit]Piper uses the standard Google storage infrastructure, Bigtable (now called Spanner), distributed across 10 data centers worldwide and replicated through Paxos protocol.[3]
Use
[edit]When using Piper, developers apply changes to a local copy of files, similar to a working copy of Subversion, local clone of Git, or a client of Perforce. Updates made by other developers can be pulled from the central repository and merged into the local code. The commits are only allowed after a code review.[7]
Typical use involves Clients in the Cloud (CitC). This system utilizes cloud backend and a local FUSE filesystem to create an illusion of changes overlaid on top of a full repository. This approach enables seamless browsing and use of standard Unix tools without explicit synchronization operations, thus keeping the local copy very small (average size of a local copy is less than ten files). All file writes are mapped to snapshots thus permitting restoration of the previous states of the code without explicit snapshotting. Due to the always-connected operation, CitC allows easy switching of the computers as well as sharing the modified code with other developers, the automated build system and testing tools.[7] As a result, the majority of Google developers practices trunk-based development with no personal branches; the branches are mostly used for releases.[8]
Security
[edit]Most of the codebase is visible to all developers, sensitive individual files (less than 1% as of 2016) are access-controlled. All operations with Piper are logged, accidentally committed files can be purged.[7]
Open-source clone
[edit]Piper is proprietary software. Mega, a Git-compatible open-source clone of Piper, is available on GitHub. It supports the trunk-based development, Conventional Commits and code owners.[9]
References
[edit]- ^ Potvin & Levenberg 2016, p. 79.
- ^ Potvin & Levenberg 2016, p. 78.
- ^ a b Potvin & Levenberg 2016, p. 80.
- ^ Potvin & Levenberg 2016, pp. 80–81.
- ^ Blank-Edelman 2018, p. 329.
- ^ Potvin & Levenberg 2016, pp. 78–79.
- ^ a b c Potvin & Levenberg 2016, p. 81.
- ^ Potvin & Levenberg 2016, p. 82.
- ^ "Mega - is an unofficial open source implementation of Google Piper". GitHub.
Sources
[edit]- Blank-Edelman, D.N. (2018). Seeking SRE: Conversations About Running Production Systems at Scale. O'Reilly Media. ISBN 978-1-4919-7881-8. Retrieved 2024-10-31.
- Potvin, Rachel; Levenberg, Josh (2016-06-24). "Why Google stores billions of lines of code in a single repository". Communications of the ACM. 59 (7). Association for Computing Machinery (ACM): 78–87. doi:10.1145/2854146. ISSN 0001-0782.
 | This Google-related article is a stub. You can help Wikipedia by expanding it. |