Community hub
0 subscribers7 pages, 0 posts
Recent from talks
All channels
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Welcome to the community hub built to collect knowledge and have discussions related to CockroachDB.
Nothing was collected or created yet.
CockroachDB
View on Wikipediafrom Wikipedia
 | This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these messages)
|
| CockroachDB | |
|---|---|
 | |
| Original authors | Spencer Kimball, Peter Mattis, Ben Darnell |
| Developer | Cockroach Labs |
| Initial release | 2017 |
| Stable release | 25.1.4
/ April 8, 2025 |
| Repository | |
| Written in | Go |
| Available in | English |
| Type | RDBMS |
| License | Proprietary[1] |
CockroachDB is a source-available distributed SQL database management system developed by Cockroach Labs.[2][3]
Key Information
The relational functionality is built on top of a distributed, transactional, consistent key-value store that can survive a variety of different underlying infrastructure failures, and is wire-compatible with PostgreSQL which means users can take advantage of a wide range of drivers and tools from the extensive PostgreSQL ecosystem. A CockroachDB cluster consists of a number of nodes that can be spread across failure domains such as data centres or public cloud regions. A cluster can be scaled both horizontally[4] (by adding nodes) and vertically (by increasing the resources allocated to the existing nodes). It can provide high levels of resilience and availability and can be run in a variety of environments such as bare metal, VMs, containers and Kubernetes, both in private data centers and in the cloud. CockroachDB gets its name from cockroaches, as they are known for being disaster-resistant.[5]
History
[edit]Cockroach Labs was founded in 2015 by ex-Google employees Spencer Kimball, Peter Mattis, and Ben Darnell. Kimball and Mattis had been key members of the Google File System team,[6] while Darnell was a key member of the Google Reader team.[7]
While at Google, all three had used Google-owned DBMS's Bigtable and its successor, Spanner.[3] After leaving Google, they wanted to design and build something similar.[8] Spencer Kimball wrote the first iteration of the design in January 2014, and began the open-source project on GitHub in February 2014, allowing outside access and contributions.[9]
Development on GitHub attracted substantial contributions, which earned the project the Open Source Rookie of the Year award by Black Duck Software.[10]
The co-founders supported the project with conferences, networking, meet-ups, and fund-raising financial rounds.
In June 2019, Cockroach Labs announced that CockroachDB would change its license from the free software license Apache License 2.0 to its source-available license, known as the Business Source License (BuSL), which forbids “offer[ing] a commercial version of CockroachDB as a service without buying a license,” while remaining free of charge for community use.[11][12]
Features
[edit]CockroachDB stores copies of data in multiple locations to deliver quick access.[8][13]
It is described as a scalable, consistently-replicated, transactional data store.[14] A single instance can scale from a single laptop to thousands of servers.[3]
CockroachDB is designed to run in the cloud and has a high fault tolerance. According to popular news outlets, it is described as “almost impossible” to take down.[15][16][13]
CockroachDB has a consistency model that is designed to match as closely as possible to the capabilities of Google Spanner, but without a dependence on specialized hardware for time synchronization. "No stale reads" is the simplest way to describe this consistency model which has deliberately made the trade-off of having non-linearizable transaction histories.[17] Transactions containing overlapping keys are guaranteed to have external consistency. And so, in practice, systems relying on CockroachDB are very unlikely to reproduce consistency issues because nodes with high variations in clock skew can be removed from clusters, applications can rely on external consistency provided by overlapping keys and writing to the same range, and writes propagate changes to followers' timestamp caches.[18]
See also
[edit]References
[edit]- ^ "Licensing FAQs".
- ^ Ovide, Shira (June 4, 2015). "CockroachDB Scampers Off With $6.3 Million to Tackle Database Shortcomings". The Wall Street Journal. Retrieved February 22, 2020.
- ^ a b c Finley, Klint (June 4, 2015). "Ex-Googlers Get Millions to Help You Build the Next Google". Wired. Retrieved February 22, 2020.
- ^ Heller, Martin (January 4, 2018). "CockroachDB review: A scale-out SQL database built for survival". InfoWorld. Retrieved March 19, 2024.
- ^ Pina, Eduardo; Sá, Filipe; Bernardino, Jorge (January 2023). "NewSQL Databases Assessment: CockroachDB, MariaDB Xpand, and VoltDB". Future Internet. 15 (1): 10. doi:10.3390/fi15010010. hdl:10316/114892. ISSN 1999-5903.
- ^ Metz, Cade (July 10, 2012). "Google Remakes Online Empire with 'Colossus'". Wired. Retrieved February 22, 2020.
- ^ Wauters, Robin (July 28, 2009). "Steal! Ben Darnell Leaves Google Reader Team, Joins FriendFeed". TechCrunch. Retrieved February 22, 2020.
- ^ a b Novet, Jordan (June 4, 2015). "Peter Fenton's latest investment is a database startup called Cockroach". VentureBeat. Retrieved February 22, 2020.
- ^ Ewbank, Kay (June 9, 2015). "CockroachDB Released". I Programmer. Retrieved February 22, 2020.
- ^ Finleey, Klint (January 28, 2015). "These are the hottest new open-source projects right now". Wired. Retrieved February 22, 2020.
- ^ Mattis, Peter; Darnell, Ben; Kimball, Spencer (June 4, 2019). "Why We're Relicensing CockroachDB". CockroachLabs.com. Retrieved February 22, 2020.
- ^ "CockroachDB Community License". Cockroach Labs. Retrieved July 28, 2020.
- ^ a b Babcock, Charles (June 4, 2015). "CockroachDB: Ultimate in Database Survival". InformationWeek. Retrieved February 22, 2020.
- ^ Darfler, Benjamin (August 29, 2014). "CockroachDB: A Scalable, Geo-Replicated, Transactional Datastore". InfoQ. Retrieved February 22, 2020.
- ^ Finley, Klint (July 22, 2014). "CockroachDB is the resilient cloud software built by ex-Googlers". Wired. Archived from the original on May 14, 2015. Retrieved February 22, 2020.
- ^ Finley, Klint (July 21, 2014). "Out in the Open: Ex-Googlers Building Cloud Software That's Almost Impossible to Take Down". Wired. Retrieved February 22, 2020.
- ^ Matei, Andrei (January 24, 2019). "CockroachDB's consistency model".
- ^ Cordell, Evan. "The One Crucial Difference Between Spanner and CockroachDB".
External links
[edit]CockroachDB
View on Grokipediafrom Grokipedia
CockroachDB is a source-available distributed SQL database designed for building resilient, scalable cloud-native applications, combining the consistency and ACID transaction support of traditional relational databases with the horizontal scalability and fault tolerance of NoSQL systems.[1] It provides a PostgreSQL-compatible interface, enabling developers to use familiar SQL syntax while benefiting from automated data distribution, replication, and recovery across clusters of nodes.[2] CockroachDB emphasizes high availability, surviving node failures and network partitions without data loss, making it suitable for global, data-intensive workloads.[3]
CockroachDB was developed by Cockroach Labs, a company founded in 2015 by Spencer Kimball, Peter Mattis, and Ben Darnell, all former Google engineers who drew inspiration from Google's Spanner distributed database to address challenges in scaling relational databases for cloud environments.[4] [5] The project originated as an initiative to create a "survivable" database resilient to the "apocalypse" of distributed system failures, with its name evoking the indestructibility of cockroaches.[6] Cockroach Labs has since grown to support enterprise adoption, offering both self-hosted and fully managed cloud deployments on platforms like AWS, Google Cloud, and Azure.[7]
Key features of CockroachDB include automatic sharding and rebalancing of data for effortless scaling, built-in geo-partitioning for low-latency global access, and advanced security measures such as role-based access control (RBAC) and encryption at rest and in transit.[1] It supports change data capture (CDC) for real-time integrations[8] and rolling upgrades to minimize downtime,[9] powering applications in sectors like finance, e-commerce, and telecommunications where reliability and performance under load are critical. In 2025, marking its 10th anniversary, CockroachDB released version 25.2 with major performance gains and enterprise advancements.[10] As of 2025, CockroachDB has over 800 contributors and maintains its position as a leading solution for distributed SQL in hybrid and multi-cloud architectures.[11]
History
Founding
CockroachDB was founded in 2015 by Spencer Kimball, Peter Mattis, and Ben Darnell, all former Google engineers who had previously collaborated on projects like the image editor GIMP and worked at startups such as Viewfinder and Square.[5][12] Their experience at Google exposed them to advanced distributed systems, particularly inspiring them to draw from the design principles of Google's Spanner, a globally distributed database that emphasized consistency and scalability across data centers.[13][12] The founders aimed to create an open-source alternative that could deliver similar capabilities without the proprietary constraints of cloud vendor lock-in, addressing the growing need for resilient databases in cloud-native applications.[5] The project's name, CockroachDB, reflects its core design philosophy: building a database resilient enough to survive catastrophic failures, much like the insect's legendary hardiness.[14] The initial goal was to develop a scalable, consistent distributed SQL database that maintains ACID (Atomicity, Consistency, Isolation, Durability) transactions while enabling horizontal scalability across clusters, even in the face of node, rack, or regional outages.[15] Early prototypes focused on this survivability, implementing a key-value store foundation with SQL semantics layered on top to ensure strong consistency without sacrificing availability.[5] In parallel with the technical development, the founders established Cockroach Labs as a private company in 2015 to commercialize and sustain the project.[12] The company prioritized open-source development from the outset, releasing the code under the Apache 2.0 license to encourage community contributions and widespread adoption.[16] Alpha releases began in late 2015, with initial binaries and documentation made available on GitHub, allowing early testers to experiment with the system's distributed architecture and fault-tolerant features.[15] By 2016, these efforts culminated in a beta release, marking the transition from prototype to a more stable offering that demonstrated the database's potential for production-like horizontal scaling and ACID compliance.[5]Key Milestones
CockroachDB achieved a significant milestone with the release of version 1.0 on May 10, 2017, which marked its production readiness as an open-source, cloud-native distributed SQL database with comprehensive SQL support and foundational multi-region capabilities inspired by Google's Spanner.[3] In October 2019, Cockroach Labs introduced CockroachDB Cloud as a fully managed service in beta, enabling self-service deployment of the database across multi-cloud environments to simplify scaling for global businesses.[17] That same year, in June 2019, Cockroach Labs shifted enterprise features of CockroachDB to the Business Source License (BSL) 1.1, a source-available model that restricted certain commercial uses while maintaining community access to core functionality. This approach was later fully transitioned in November 2024 with the v24.3 release on November 18, retiring the free Core edition and adopting the proprietary CockroachDB Enterprise license, which offers free access to qualifying organizations with under $10 million in annual revenue. Cockroach Labs marked its 10th anniversary on February 8, 2025, reflecting on a decade of advancements in distributed SQL technology.[18] The company celebrated this occasion with the v25.2 release on June 3, 2025, delivering over 41% performance improvements across key workloads and introducing vector indexing to support AI-driven applications.[19] Building on this momentum, the v25.3 release on August 4, 2025, added a new Kubernetes operator for automated deployment and scaling, alongside enhanced migration tools via the MOLT Fetch toolkit for smoother transitions from legacy databases.[20][21] Throughout its evolution, Cockroach Labs secured substantial funding to fuel development, culminating in a Series F round of $278 million in December 2021 that valued the company at $5 billion and brought total funding to $633 million.[22]Architecture
Core Principles
CockroachDB adheres to the CP (Consistency and Partition tolerance) model of the CAP theorem, prioritizing strong data consistency over availability in the event of network partitions.[23] This design choice ensures that all replicas maintain synchronized states using the Raft consensus algorithm, preventing divergent data copies even during failures, though it may temporarily pause operations until partitions resolve.[23] By focusing on consistency, CockroachDB avoids the risks of stale reads or lost updates common in availability-first systems.[24] At its foundation, CockroachDB operates as a distributed key-value store that supports a SQL interface, enabling relational queries while managing data across multiple nodes.[2] All user data, including tables and indexes, is encoded into a sorted map of key-value pairs, which the SQL layer translates into efficient distributed operations.[2] This architecture delivers full ACID (Atomicity, Consistency, Isolation, Durability) transactions spanning arbitrary keys and nodes, with serializable isolation to prevent anomalies like dirty reads or lost updates.[25] The transactional layer coordinates intents and timestamps to enforce these guarantees without requiring application-level locking.[26] Horizontal scalability is achieved through automatic sharding of the keyspace into ranges—contiguous units of data typically up to 512 MiB—and replication across nodes for fault tolerance.[27] As data grows, ranges split dynamically when they reach the configured maximum size, distributing load evenly without manual intervention.[28] By default, each range maintains three replicas on distinct nodes or failure domains, ensuring data survivability against the loss of any single node while leveraging Raft for consensus-driven replication.[23] This approach allows clusters to scale linearly by adding nodes, with automatic rebalancing to maintain balanced utilization.[2] In cloud deployments, CockroachDB employs a multi-tenant architecture to isolate resources among multiple users while enabling elastic scaling.[29] Tenants share underlying infrastructure but operate in virtualized clusters with strong isolation for security and performance, allowing compute and storage to scale independently based on demand.[30] This model supports thousands of concurrent databases on shared hardware without compromising per-tenant guarantees, optimizing costs for SaaS and cloud-native applications.[31] CockroachDB emphasizes geo-partitioning to minimize latency for global applications and enhance resilience against regional failures.[32] Data can be partitioned by locality—such as region or availability zone—at the row or table level, ensuring frequently accessed records remain close to users and reducing cross-region data travel.[33] For instance, regional tables replicate data within a specific geography for low-latency reads and writes, while global tables distribute replicas across regions for read-anywhere access, surviving data center outages through configurable survival goals.[34] This geo-aware distribution integrates with the replication layer to balance performance and fault tolerance in multi-region clusters.[35]Components and Data Management
CockroachDB operates as a distributed system composed of multiple nodes, each representing an individual server that runs the CockroachDB process and participates in the cluster. These nodes collectively manage data and transactions, communicating via gRPC with configurable network timeouts to ensure efficient coordination.[2] A node hosts one or more stores, which are local disk-based storage units containing replicas of data ranges; no two replicas of the same range reside on the same store or node to prevent single points of failure.[36] Data within stores is managed using the Pebble key-value storage engine, an LSM-tree implementation that persists key-value pairs in immutable SST files across multiple levels, with memtables and write-ahead logs ensuring durability before flushing to disk. As of v25.3 (October 2025), CockroachDB introduced value separation in Pebble to boost throughput for write-heavy workloads by up to 60%.[36][37] At the core of data management, CockroachDB shards its key-value store into ranges, which are contiguous, non-overlapping segments of the keyspace, typically sized up to 512 MiB before automatic splitting. Each table and its indexes initially occupy a single range, but as data volume grows, ranges split to distribute load. The entire dataset forms a monolithic sorted map, with keys structured as/<tableID>/<indexID>/<indexed columns> -> <row data>, enabling efficient lookups and scans. Metadata for range locations is maintained in special meta ranges—a two-level index (meta1 and meta2)—that tracks range descriptors, including RangeIDs, key spans, and replica node addresses, facilitating transparent data access from any node.[28][2]
Replication ensures data durability through the Raft consensus algorithm, which organizes replicas of each range into leader-follower groups with a configurable replication factor (defaulting to three). The leader replica, known as the leaseholder, coordinates writes by proposing commands to its Raft log, which followers replicate via heartbeats and quorum votes (requiring a majority, such as 2 out of 3), guaranteeing consistency and fault tolerance up to (replication factor - 1)/2 failures. Automatic failover occurs when liveness heartbeats detect a dead node, triggering a new leader election and lease transfer to a surviving replica, typically completing in seconds without data loss.[23][38] This mechanism, as detailed in CockroachDB's foundational design, supports geo-distributed resilience on commodity hardware.[39]
The transaction layer builds on this foundation by implementing Multi-Version Concurrency Control (MVCC) to provide serializable isolation without traditional locks, using hybrid logical clocks (HLC) to timestamp operations and maintain version histories. MVCC stores multiple versions of each key-value pair with timestamps, allowing reads to access consistent snapshots while writes place provisional "write intents" that serve as optimistic locks, resolved via a distributed commit protocol. Concurrency conflicts, such as write-write or write-read interferences, are detected using per-node lock tables and transaction records, with resolutions like timestamp pushing or transaction aborts ensuring serializability.[25][40] This lockless approach minimizes contention in distributed environments.[39]
System tables in the system and crdb_internal schemas manage cluster metadata, including topology details like node liveness (system.nodes), range leases (system.rangelog), and zone configurations that define replication rules per database, table, or index (e.g., num_replicas and locality constraints). Zone configs are hierarchical, inheriting from cluster defaults and overriding for specific objects via SQL statements like ALTER TABLE ... CONFIGURE ZONE. Schema changes, such as adding columns or indexes, are tracked in crdb_internal.schema_changes and propagated atomically across the cluster, updating range descriptors and metadata in meta ranges to reflect new topology without downtime.[41][42]
Rebalancing maintains even data distribution and high availability through automated processes like range splitting, which occurs when a range exceeds its size limit or based on load, creating two balanced sub-ranges whose descriptors update the meta index. Merging consolidates underutilized adjacent ranges to optimize storage and query efficiency. Lease transfers during rebalancing or recovery relocate leadership to nodes closer to read/write traffic, guided by metrics like QPS and latency, with operations scheduled periodically (default 10 minutes) or triggered by events like node additions/removals. Recovery from failures involves snapshotting replicas to new nodes and replaying Raft logs to catch up, ensuring minimal disruption.[23][28]
Features
SQL and Transactional Capabilities
CockroachDB provides high compatibility with the PostgreSQL wire protocol (version 3.0, or pgwire), enabling seamless integration with most PostgreSQL drivers, ORMs, and tools such as DBeaver and psql. This compatibility extends to the majority of PostgreSQL SQL syntax, allowing applications to migrate with minimal modifications while leveraging CockroachDB's distributed architecture.[43] The database ensures full ACID compliance for transactions, guaranteeing atomicity, consistency, isolation, and durability even across distributed nodes. By default, CockroachDB operates at the SERIALIZABLE isolation level—the strongest in the SQL standard—preventing anomalies like dirty reads or lost updates through multi-version concurrency control and timestamp ordering. Distributed transactions spanning multiple shards are coordinated via the transaction layer, which uses parallel commits to achieve atomicity without traditional two-phase commit blocking, ensuring data correctness in geo-distributed environments.[44][45][25] CockroachDB supports advanced SQL features, including joins (such as inner, outer, and cross joins), secondary indexes (including hash-sharded for sequential key optimization and generalized inverted indexes for JSONB and array queries), and stored procedures using PL/pgSQL or SQL logic callable via the CALL statement. Common Table Expressions (CTEs), available since version 2.0 with recursive and correlated support added in later releases like v20.1 and v21.2, enable complex subquery reuse for improved query readability and performance. These features align closely with PostgreSQL semantics, facilitating sophisticated analytical and operational workloads.[46][47][48][49][50] Schema management in CockroachDB uses DDL statements (e.g., CREATE TABLE, ALTER TABLE) that apply atomically across the entire cluster, ensuring consistent schema state without downtime through online schema changes. These operations are executed in implicit transactions, propagating modifications to all nodes via the distributed consensus mechanism for durability.[51][25] In version 25.2, CockroachDB introduced vector indexing in preview, supporting storage of high-dimensional embeddings and efficient similarity searches (e.g., Euclidean distance) via SQL queries, with cosine and inner product distances added in v25.3. This feature became generally available in v25.4. This integration enables AI and machine learning workloads, such as semantic search and recommendation systems, directly within the database using CockroachDB's C-SPANN-based indexing protocol.[52][19][20][53]Scalability and Resilience
CockroachDB achieves horizontal scalability by allowing users to add nodes dynamically to a cluster, which triggers automatic rebalancing of data across all nodes to maintain even distribution and performance. This process ensures that as data and workload volumes grow, the system can scale out seamlessly without manual intervention or downtime. Clusters can support hundreds of nodes, as demonstrated by customer deployments reaching up to 160 nodes while maintaining operational efficiency.[54][1][55] The database's resilience is enhanced by features designed to ensure high availability and fault tolerance, including a 99.999% uptime service level agreement (SLA) for multi-region clusters in CockroachDB Cloud. Multi-region replication enables zone and region survival goals, where data is synchronously replicated across multiple geographic locations to withstand outages in entire availability zones or regions. Fast recovery is facilitated through range leases, which allow sub-second leader elections and failovers in the underlying Raft consensus protocol, minimizing disruption during node or network failures.[56][57][23] Performance optimizations in versions 25.2 and later (as of 2025) deliver significant throughput improvements, with up to 50% gains across various workloads in v25.2 compared to v24.3, alongside further enhancements in v25.3 and v25.4 such as improved vector indexing and a new Kubernetes operator, and reductions in resource utilization for better efficiency on commodity hardware. The system also handles network partitions robustly, as validated through internal benchmarks that simulate adversity scenarios like partitions and regional outages, ensuring consistent performance without data loss. These enhancements contribute to ACID-compliant transactions that remain resilient under load, supporting reliable operations at scale.[19][58][20][53] CockroachDB provides comprehensive backup and recovery mechanisms, including full and incremental backups that capture changes efficiently to minimize storage and time costs. With revision history enabled, point-in-time recovery allows restoration to any timestamp within the backup window, preserving granular data states without full re-ingestion. Enterprise-grade change data capture (CDC) further supports resilience by streaming row-level changes to downstream systems for real-time replication and disaster recovery.[59][60][8] Monitoring and observability are integrated natively, with built-in metrics tracking key indicators such as latency, throughput, and resource usage across the cluster. SQL diagnostics tools provide insights into query performance and execution plans, aiding in troubleshooting and optimization. CockroachDB exposes these metrics for external tools like Prometheus and Grafana, enabling customizable dashboards and alerting for proactive resilience management.[61][62]Deployment and Licensing
Options for Deployment
CockroachDB offers flexible deployment options to suit different operational needs, ranging from self-managed setups to fully managed cloud services. Self-hosted deployments allow users to run CockroachDB on their own infrastructure, providing full control over the environment. These include single-node configurations ideal for development and testing, where a lightweight instance can be started locally without clustering overhead. For production environments, multi-node clusters can be deployed on-premises in private data centers or on virtual machines (VMs) across cloud providers, enabling horizontal scaling and high availability through manual replication and node management.[63] Orchestration is supported via Kubernetes, with the CockroachDB Operator—introduced and enhanced in version 25.3—facilitating automated deployment, scaling, and management of clusters in containerized environments.[64] CockroachDB Cloud provides managed alternatives, eliminating much of the operational burden. The serverless option, available as CockroachDB Standard, operates in a multi-tenant architecture with strong tenant isolation, automatically scaling resources based on demand and billing on a pay-per-use model (e.g., per request unit and storage).[65] Dedicated clusters under CockroachDB Advanced offer single-tenant isolation, configurable in single or multi-region setups across AWS, GCP, or Azure, with guarantees like 99.99% uptime SLA and node-based scaling for predictable performance.[66] These cloud deployments support multi-cloud strategies, allowing clusters to span providers for resilience and avoiding vendor lock-in.[67] Hybrid deployments bridge self-hosted and cloud environments, supporting seamless migrations from on-premises to cloud or multi-cloud configurations. Users can replicate data across on-premises setups and cloud regions on AWS, GCP, or Azure, leveraging CockroachDB's geo-partitioning for low-latency access and disaster recovery.[68] Installation for self-hosted options is straightforward, with binary downloads available for direct execution on supported operating systems like Linux, macOS, and Windows. Docker images enable containerized deployments for portability, while Helm charts simplify Kubernetes installations by packaging the necessary resources and configurations. Management is streamlined through built-in tools, including the CockroachDB Console—a web-based UI for monitoring cluster health, querying data, and administering settings—and thecockroach sql command-line shell for interactive SQL operations and scripting. These tools apply across self-hosted and cloud deployments, with additional cloud-specific features like single sign-on (SSO) in managed services.[30]