Amazon DynamoDB
View on Wikipedia| Amazon DynamoDB | |
|---|---|
![]() | |
| Developer | Amazon.com |
| Initial release | January 2012[1] |
| Written in | Java |
| Operating system | Cross-platform |
| Available in | English |
| Type | |
| License | Proprietary |
| Website | aws |
Amazon DynamoDB is a managed NoSQL database service provided by Amazon Web Services (AWS). It supports key-value and document data structures and is designed to handle a wide range of applications requiring scalability and performance.[2]
History
[edit]Werner Vogels, CTO at Amazon.com, provided a motivation for the project in his 2012 announcement.[3] Amazon began as a decentralized network of services. Originally, services had direct access to each other's databases. When this became a bottleneck on engineering operations, services moved away from this direct access pattern in favor of public-facing APIs. Still, third-party relational database management systems struggled to handle Amazon's client base. This culminated during the 2004[4][5] holiday season, when several technologies failed under high traffic.
Traditional databases often split data into smaller pieces to save space, but combining those pieces during searches can make queries slower. Many of Amazon's services demanded mostly primary-key reads on their data, and with speed a top priority, putting these pieces together was extremely taxing.[6]
Content with compromising storage efficiency, Amazon's response was Dynamo: a highly available key–value store built for internal use.[3] Dynamo, it seemed, was everything their engineers needed, but adoption lagged. Amazon's developers opted for "just works" design patterns with S3 and SimpleDB. While these systems had noticeable design flaws, they did not demand the overhead of provisioning hardware and scaling and re-partitioning data. Amazon's next iteration of NoSQL technology, DynamoDB, automated these database management operations.
Overview
[edit]
DynamoDB organizes data into tables, which are similar to spreadsheets. Each table contains items (rows), and each item is made up of attributes (columns). Each item has a unique identifier called a primary key, which helps locate it within the table.
DynamoDB Tables
[edit]A DynamoDB Table is a logical grouping of items, which represent the data stored in this Table. Given the NoSQL nature of DynamoDB, the Tables do not require that all items in a Table conform to some predefined schema.[7]
DynamoDB Items
[edit]An Item in DynamoDB is a set of attributes that can be uniquely identified in a Table. An Attribute is an atomic data entity that in itself is a Key-Value pair. The Key is always of String type, while the value can be of one of multiple data types.
An Item is uniquely identified in a Table using a subset of its attributes called Keys.[7]
Keys In DynamoDB
[edit]A Primary Key is a set of attributes that uniquely identifies items in a DynamoDB Table. Creation of a DynamoDB Table requires definition of a Primary Key. Each item in a DynamoDB Table is required to have all of the attributes that constitute the Primary Key, and no two items in a Table can have the same Primary Key. Primary Keys in Dynamo DB can consist of either one or two attributes.
When a Primary Key is made up of only one attribute, it is called a Partition Key. Partition Keys determine the physical location of the associated item. In this case, no two items in a table can have the same Partition Key.
When a Primary Key is made up of two attributes, the first one is called a "Partition Key" and the second is called a "Sort Key". As before, the Partition Key decides the physical Location of Data, but the Sort Key then decides the relative logical position of associated item's record inside that physical location. In this case, two items in a Table can have the same Partition Key, but no two items in a partition can have the same Sort Key. In other words, a given combination of Partition Key and Sort Key is guaranteed to have at most one item associated with it in a DynamoDB Table.[7]
DynamoDB Data Types
[edit]DynamoDB supports numerical, String, Boolean, Document, and Set Data Types.[8]
DynamoDB Indices
[edit]Primary Key of a Table is the Default or Primary Index of a DynamoDB Table.
In addition, a DynamoDB Table can have Secondary Indices. A Secondary Index is defined on an attribute that is different from Partition Key or Sort Key as the Primary Index.
When a Secondary Index has same Partition Key as Primary Index but a different Sort Key, it is called as the Local Secondary Index.
When Primary Index and Secondary Index have different Partition Key, the Secondary index is known as the Global Secondary Index.[7]
Architectural patterns in DynamoDB data modeling
[edit]DynamoDB data modeling patterns are architectural approaches used in Amazon DynamoDB, a NoSQL database service designed for distributed systems. These patterns address various data organization challenges and include "Single Table Design", which consolidates related data while adhering to DynamoDB's 400KB item size limit; "Multiple Table Design", which separates data into distinct tables based on access patterns and data model differences; and Hybrid Design, which blends both approaches to balance flexibility and efficiency.[9][10][11]
Additional patterns described in AWS documentation include "Event Sourcing", where data changes are stored as immutable events, enabling historical state reconstruction; "Materialized Views", which simplify analytical queries through pre-computed aggregations, often implemented via DynamoDB Streams, application-level processing, or periodic batch updates using Lambda functions. As well as "Time-Series Design", optimized for workloads like logging and metrics, typically using a partition key for entity identification and a sort key representing timestamps to efficiently query time-based datasets.[12][13][14]
Each pattern addresses specific technical requirements. "Single Table Design" can optimize query efficiency by co-locating related data under the same partition key to reduce access latency. "Multiple Table Design" enables separation of concerns by isolating data into purpose-specific tables with distinct access patterns. "Event Sourcing" preserves a historical log of state changes, often implemented with immutable data storage. "Materialized Views" simplify complex analytical queries through pre-aggregation strategies tailored to access patterns. "Time-Series Design" uses partitioning and sorting strategies to efficiently store and query large volumes of temporal data.[9][10][11][12][13][14]
Performance limitations of DynamoDB's latency claims
[edit]Amazon DynamoDB's claim of single-digit millisecond latency primarily applies to simple operations such as GetItem and PutItem, which retrieve or modify individual items using their primary keys. This reflects the average latency under ideal conditions, such as even partition distribution and sufficient throughput provisioning, and does not account for transport overhead incurred during communication with the DynamoDB endpoint. More complex operations, such as Query with filters, Scan, or those involving large datasets, may experience increased latency due to additional computation and data transfer requirements.[15][16]
Locking
[edit]Although DynamoDB does not natively support locking, different mechanisms exist. Optimistic locking may use a version number to detect conflicts that occur after updates, rather than preventing them in advance. Pessimistic locking, by contrast, may involve conditional updates with attributes such as lockTime and lockedBy. When combined with Time to Live (TTL), these attributes enable the automated removal of expired locks, potentially enhancing concurrency management in event-driven architectures.[15][17][18][19]
System architecture
[edit]
Data structures
[edit]DynamoDB uses hashing and B-trees to manage data. Upon entry, data is first distributed into different partitions by hashing on the partition key. Each partition can store up to 10 GB of data and handle by default 1,000 write capacity units (WCU) and 3,000 read capacity units (RCU).[20] One RCU represents one strongly consistent read per second or two eventually consistent reads per second for items up to 4 KB in size.[21] One WCU represents one write per second for an item up to 1 KB in size.
To prevent data loss, DynamoDB features a two-tier backup system of replication and long-term storage.[22] Each partition features three nodes, each of which contains a copy of that partition's data. Each node also contains two data structures: a B tree used to locate items, and a replication log that notes all changes made to the node. DynamoDB periodically takes snapshots of these two data structures and stores them for a month in S3 so that engineers can perform point-in-time restores of their databases.
Within each partition, one of the three nodes is designated the "leader node". All write operations travel first through the leader node before propagating, which makes writes consistent in DynamoDB. To maintain its status, the leader sends a "heartbeat" to each other node every 1.5 seconds. Should another node stop receiving heartbeats, it can initiate a new leader election. DynamoDB uses the Paxos algorithm to elect leaders.
Amazon engineers originally avoided Dynamo due to engineering overheads like provisioning and managing partitions and nodes.[6] In response, the DynamoDB team built a service it calls AutoAdmin to manage a database.[22] AutoAdmin replaces a node when it stops responding by copying data from another node. When a partition exceeds any of its three thresholds (RCU, WCU, or 10 GB), AutoAdmin will automatically add additional partitions to further segment the data.[20]
Just like indexing systems in the relational model, DynamoDB demands that any updates to a table be reflected in each of the table's indices. DynamoDB handles this using a service it calls the "log propagator", which subscribes to the replication logs in each node and sends additional Put, Update, and Delete requests to indices as necessary.[22] Because indices result in substantial performance hits for write requests, DynamoDB allows a user at most five of them on any given table.[23]
Query execution
[edit]This section contains promotional content. (October 2025) |
Suppose that a DynamoDB user issues a write operation (a Put, Update, or Delete). While a typical relational system would convert the SQL query to relational algebra and run optimization algorithms, DynamoDB skips both processes.[22] The request arrives at the DynamoDB request router, which authenticates—"Is the request coming from where/whom it claims to be?"—and checks for authorization—"Does the user submitting the request have the requisite permissions?" Assuming these checks pass, the system hashes the request's partition key to arrive in the appropriate partition. There are three nodes within, each with a copy of the partition's data. The system first writes to the leader node, then writes to a second node, then sends a "success" message, and finally continues propagating to the third node. Writes are consistent because they always travel first through the leader node.
Finally, the log propagator propagates the change to all indices. For each index, it grabs that index's primary key value from the item, then performs the same write on that index without log propagation. If the operation is an Update to a preexisting item, the updated attribute may serve as a primary key for an index, and thus the B tree for that index must update as well. B trees only handle insert, delete, and read operations, so in practice, when the log propagator receives an Update operation, it issues both a Delete operation and a Put operation to all indices.
Now suppose that a DynamoDB user issues a Get operation. The request router proceeds as before with authentication and authorization. Next, as above, we hash our partition key to arrive in the appropriate hash. Now, we encounter a problem: with three nodes in eventual consistency with one another, how can we decide which to investigate? DynamoDB offers the user two options when issuing a read: consistent and eventually consistent. A consistent read visits the leader node. But the consistency-availability trade-off rears its head again here: in read-heavy systems, always reading from the leader can overwhelm a single node and reduce availability.
The second option, an eventually consistent read, selects a random node. In practice, this is where DynamoDB trades consistency for availability. If we take this route, what are the odds of an inconsistency? We'd need a write operation to return "success" and begin propagating to the third node, but not finish. We'd also need our Get to target this third node. This means a 1-in-3 chance of inconsistency within the write operation's propagation window. How long is this window? Any number of catastrophes could cause a node to fall behind, but in the vast majority of cases, the third node is up-to-date within milliseconds of the leader.
Controversies
[edit]A race condition in a DynamoDB component triggered a 14 hour outage for Amazon's US-EAST-1 region on 19 October 2025 [24], resulting in widespread outages of global services.[25]
See also
[edit]References
[edit]- ^ "Amazon DynamoDB – a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications - All Things Distributed". www.allthingsdistributed.com. 18 January 2012.
- ^ "What is Amazon DynamoDB?".
- ^ a b Vogels, Werner (2012-01-18). "Amazon DynamoDB – a Fast and Scalable NoSQL Database Service Designed for Internet Scale Applications". All Things Distributed blog. Retrieved 2012-01-21.
- ^ "How Amazon's DynamoDB helped reinvent databases". Network World. Retrieved 2023-11-30.
- ^ brockmeier 1, joe (2012-01-18). "Amazon Takes Another Pass at NoSQL with DynamoDB". ReadWrite. Retrieved 2023-11-30.
{{cite web}}: CS1 maint: numeric names: authors list (link) - ^ a b DeCandia, Giuseppe; Hastorun, Deniz; Jampani, Madan; Kakulapati, Gunavardhan; Lakshman, Avinash; Pilchin, Alex; Sivasubramanian, Swaminathan; Vosshall, Peter; Vogels, Werner (October 2007). "Dynamo: Amazon's Highly Available Key–value Store". SIGOPS Oper. Syst. Rev. 41 (6): 205–220. doi:10.1145/1323293.1294281. ISSN 0163-5980.
- ^ a b c d "Core components of Amazon DynamoDB - Amazon DynamoDB". docs.aws.amazon.com. Retrieved 2023-05-28.
- ^ "Supported data types and naming rules in Amazon DynamoDB - Amazon DynamoDB". docs.aws.amazon.com. Retrieved 2023-05-28.
- ^ a b "Creating a single-table design with Amazon DynamoDB".
- ^ a b "Single-table vs. multi-table design in Amazon DynamoDB".
- ^ a b "Data Modeling foundations in DynamoDB".
- ^ a b "Best practices for handling time series data in DynamoDB".
- ^ a b "AWS Prescriptive Guidance Enabling data persistence in microservices".
- ^ a b "Build a CQRS event store with Amazon DynamoDB".
- ^ a b Dhingra, Aman; MacKay, Mike (30 August 2024). Amazon DynamoDB - The Definitive Guide: Explore enterprise-ready, serverless NoSQL with predictable, scalable performance. Packt Publishing. ISBN 9781803248325.
- ^ "Troubleshooting latency issues in Amazon DynamoDB".
- ^ "Using expressions in DynamoDB".
- ^ "Using time to live (TTL) in DynamoDB".
- ^ "DynamoDB and optimistic locking with version number".
- ^ a b Gunasekara, Archie (2016-06-27). "A Deep Dive into DynamoDB Partitions". Shine Solutions Group. Retrieved 2019-08-03.
- ^ "Amazon DynamoDB Developer Guide". AWS. August 10, 2012. Retrieved July 18, 2019.
- ^ a b c d AWS re:Invent 2018: Amazon DynamoDB Under the Hood: How We Built a Hyper-Scale Database (DAT321), 27 November 2018, retrieved 2019-08-03
- ^ "Service, account, and table quotas in Amazon DynamoDB - Amazon DynamoDB". docs.aws.amazon.com. Retrieved 2024-01-09.
- ^ "Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region". AWS. Retrieved 2025-10-28.
- ^ "A single point of failure triggered the Amazon outage affecting millions". Ars Technica. Retrieved 2025-10-28.
External links
[edit]Amazon DynamoDB
View on GrokipediaHistory
Origins and Development
The development of Amazon DynamoDB traces its roots to the internal challenges faced by Amazon in scaling its e-commerce infrastructure during the mid-2000s. In 2004-2005, Amazon engineers identified limitations in traditional relational databases, which struggled with peak holiday traffic and required complex sharding to handle growing data volumes. This led to the creation of Dynamo, an internal key-value storage system designed for high availability and scalability to support core services such as shopping carts, session management, and product catalogs.[6][7] Dynamo's design, detailed in a seminal 2007 paper presented at the SOSP conference, addressed critical challenges in distributed systems at massive scale. The system was engineered to manage tens of millions of customers across tens of thousands of servers without single points of failure, using techniques like consistent hashing for partitioning and replication across multiple data centers to ensure fault tolerance during disk failures, network partitions, or outages. It prioritized availability through an eventual consistency model, allowing tunable quorum levels for reads and writes while employing vector clocks for versioning and application-assisted conflict resolution, thus avoiding the downtime risks of strong consistency in high-traffic environments.[7][8] The project was led by a team of Amazon engineers, including Werner Vogels, then Vice President of Amazon and co-author of the Dynamo paper, along with contributors like Swaminathan Sivasubramanian. Early prototypes of Dynamo were deployed internally in production to power Amazon's e-commerce operations, validating its reliability under real-world loads such as over three million daily checkouts during peak seasons. This internal adoption refined the system before its evolution into a managed cloud service, DynamoDB, to meet broader scalability demands.[6][7][8]Key Releases and Updates
Amazon DynamoDB was launched on January 18, 2012, as a fully managed NoSQL database service on AWS, providing fast and predictable performance with seamless scalability.[9] In 2017, AWS introduced several enhancements to improve flexibility and global availability, including adaptive capacity to automatically adjust provisioned throughput across partitions and global tables on November 29 for multi-region, multi-master replication, enabling low-latency access to data across regions without manual intervention.[10] The following year, on March 26, 2018, point-in-time recovery (PITR) was added, allowing continuous backups with per-second granularity for restoration up to 35 days prior, enhancing data protection capabilities.[11] On November 28, 2018, on-demand capacity mode was released, eliminating the need for capacity planning by automatically scaling throughput based on traffic and charging only for actual usage.[12] Additionally, on November 27, 2018, transactions were introduced, providing ACID compliance for up to 25 items initially (expanded to 100 in September 2022), supporting atomic operations across multiple items and tables.[13] In June 2018, AWS established a 99.999% availability Service Level Agreement (SLA) for DynamoDB global tables, underscoring its high reliability for mission-critical applications.[14] Support for PartiQL, a SQL-compatible query language, was added in November 2020, simplifying data access for developers familiar with SQL while maintaining DynamoDB's performance.[15] Post-2022 updates focused on cost optimization, AI integration, and advanced features. In November 2024, pricing for on-demand throughput was reduced by 50%, and global tables costs dropped by up to 67%, making large-scale deployments more economical.[16] Integration with Amazon Bedrock began in 2023, enabling serverless access to foundational models for generative AI applications, such as building context-aware chatbots that leverage DynamoDB for persistent memory and data retrieval.[17] In January 2025, configurable PITR periods were introduced, allowing users to set recovery windows from 1 to 35 days for more granular control over backup retention and costs.[18] In June 2025, AWS launched multi-Region strong consistency for DynamoDB global tables, enabling a recovery point objective (RPO) of zero to build highly resilient multi-Region applications.[19] These updates, combined with ongoing SLA commitments of 99.99% for standard tables and 99.999% for global tables, continue to evolve DynamoDB for modern, scalable workloads.[4]Core Concepts
Tables and Capacity Management
In Amazon DynamoDB, a table serves as a collection of items, where each item represents a unique record without enforcing a fixed schema across all items; attributes and their data types can vary between items.[2] DynamoDB distributes data across multiple partitions to achieve scalability, using an internal hash function applied to the partition key of each item to determine its storage partition, which helps ensure even data distribution and parallel processing.[2][20] DynamoDB offers two capacity modes for managing throughput: provisioned mode, where users specify the desired read and write capacities in advance for predictable workloads, and on-demand mode, a serverless option that automatically scales to handle traffic variations without manual configuration, billing based on actual requests.[21][22] In provisioned mode, capacity is measured in read capacity units (RCUs) and write capacity units (WCUs), where 1 RCU allows one strongly consistent read per second (or two eventually consistent reads) of up to 4 KB of data, and 1 WCU permits one write per second of up to 1 KB of data; larger items consume multiple units proportionally.[23][20] Throughput in DynamoDB is subject to limits, including a maximum of 3,000 RCUs and 1,000 WCUs per partition to prevent hotspots, with overall table quotas enforced by AWS service limits; provisioned tables support burst capacity for short-term spikes up to 300 seconds using accumulated unused capacity, and auto-scaling policies can dynamically adjust provisioned throughput based on utilization targets.[20][21][23] Tables can be created through the AWS Management Console for a graphical interface, the AWS CLI for command-line automation, or various AWS SDKs for programmatic integration, typically requiring specification of the table name, primary key schema, and capacity mode during initial setup.[24] For example, using the AWS CLI in provisioned mode:aws dynamodb create-table --table-name MusicCollection --attribute-definitions AttributeName=Artist,AttributeType=S AttributeName=SongTitle,AttributeType=S --key-schema AttributeName=Artist,KeyType=HASH AttributeName=SongTitle,KeyType=RANGE --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5; in on-demand mode, replace the throughput parameter with --billing-mode PAY_PER_REQUEST.[24][22]
Items, Attributes, and Keys
In Amazon DynamoDB, an item represents a single record within a table, analogous to a row in a relational database, and consists of a collection of attributes that store the data.[2] Each attribute is a fundamental data element defined as a name-value pair, where the value can be a scalar type (such as a string or number), a set, or a document type, allowing flexible storage of structured or semi-structured information.[25] Items are uniquely identified by their primary key, which ensures no duplicates within the table and serves as the basis for all data access operations.[2] DynamoDB supports two types of primary keys to identify items: a simple primary key consisting solely of a partition key, or a composite primary key that combines a partition key with a sort key.[20] The partition key, a scalar attribute (string, number, or binary), is hashed by DynamoDB to determine the storage partition for the item, enabling efficient distribution across the system's partitions.[26] In a composite primary key, the sort key—an additional scalar attribute—organizes items with the same partition key value in sorted order, facilitating range queries and efficient retrieval of related data within a partition.[27] Effective primary key design is crucial for performance, emphasizing the selection of partition keys that promote even distribution of items and requests to prevent "hot partitions," where a single partition receives disproportionate traffic leading to throttling.[28] For instance, using a timestamp as the sole partition key can cause skew if many writes occur simultaneously, concentrating load on few partitions; instead, high-cardinality attributes or composite keys with random suffixes are recommended to spread workload uniformly.[20] Sort keys should be chosen to support common access patterns, such as sorting by date or status, while maintaining the overall key's cardinality for balanced partitioning.[27] Updates to item attributes are performed atomically, ensuring that operations like adding, modifying, or deleting attributes occur as a single, indivisible action without partial failures.[25] TheUpdateItem operation supports conditional expressions, which allow writes to succeed only if specified conditions are met—such as verifying an attribute's value for optimistic concurrency control to prevent overwrites in concurrent scenarios.[25]
Each item in DynamoDB is limited to a maximum size of 400 KB, encompassing the binary lengths of both attribute names (UTF-8 encoded) and attribute values, including any overhead from data types.[29] This constraint applies uniformly to all operations, ensuring predictable storage and performance.[30]
Item size directly impacts throughput consumption: reads are billed in 4 KB increments (1 RCU allows one strongly consistent read of up to 4 KB or two eventually consistent reads of up to 4 KB), and writes in 1 KB increments (1 WCU per 1 KB written). Exceeding these thresholds due to larger items (including long attribute names) increases capacity usage and costs. Use tools like the DynamoDB item size calculator for precise estimation.[31]
Supported Data Types
Amazon DynamoDB supports three main categories of data types: scalar types for simple values, set types for collections of unique scalar values, and document types for complex, nested structures. These types enable the storage of diverse data within item attributes, with all attributes in an item able to use any combination of supported types.[32] Scalar types form the foundation for basic attribute values. The String (S) type stores Unicode UTF-8 encoded text, with a minimum length of 0 bytes for non-key attributes and a maximum of 400 KB per item.[32] The Number (N) type accommodates positive, negative, or zero values up to 38 digits of precision, ranging from -9.9999999999999999999999999999999999999E+125 to 9.9999999999999999999999999999999999999E+125.[32] Binary (B) handles arbitrary binary data, also up to 400 KB, often represented in base64 encoding.[32] Boolean (BOOL) represents true or false values, while Null indicates an unknown or undefined state.[32] Set types allow storage of unique, unordered collections of scalar values without duplicates. StringSet (SS) holds unique strings, NumberSet (NS) unique numbers, and BinarySet (BS) unique binary values, each constrained to the 400 KB item size limit and requiring at least one element.[32] All elements in a set must share the same scalar type, and sets do not preserve insertion order.[32] Document types facilitate hierarchical data structures resembling JSON. List (L) provides an ordered collection of values, which can be scalars, sets, or other lists/maps, supporting nesting up to 32 levels deep.[32] Map (M) stores unordered key-value pairs where keys are strings and values can be any supported type, also nestable up to 32 levels.[32] These types enable flexible representations, such as nested maps for JSON-like objects; for instance, a user profile might use a map like{"name": "Alice", "preferences": {"theme": "dark", "notifications": true}}.[32]
Key constraints apply across types to ensure performance and storage efficiency. The overall item size, including all attributes and overhead, cannot exceed 400 KB.[32] While lists can directly hold primitive scalar values, homogeneous arrays of primitives are often modeled using lists of maps for more structured access, such as [{"value": 1}, {"value": 2}].[32] In PartiQL queries, DynamoDB handles type compatibility for operations like arithmetic on numbers or appending to lists, with implicit conversions where applicable, such as treating strings as numbers in expressions.[33] These types integrate into item attributes to define the schema-less structure of DynamoDB tables.[32]
Data Modeling and Access Patterns
Primary and Secondary Indexes
In Amazon DynamoDB, every table includes a primary index that is implicitly created upon table creation and is based on the table's primary key, which consists of a partition key and an optional sort key.[2] This primary index enables efficient querying and scanning of data using the designated key attributes, serving as the foundational structure for data organization and access within the table.[2] Secondary indexes extend the querying capabilities of DynamoDB by allowing alternative keys beyond the primary key, without requiring full duplication of the table's data.[34] They consist of a subset of attributes from the base table, including an alternate key, and support Query operations to enable diverse access patterns such as retrieving items by non-primary attributes.[34] There are two types of secondary indexes: global secondary indexes (GSIs) and local secondary indexes (LSIs), each designed for specific use cases while adhering to DynamoDB's NoSQL principles.[34] Global secondary indexes (GSIs) provide flexibility by using a different partition key and optional sort key from the base table, allowing queries on entirely new dimensions of the data.[35] GSIs are created either during table creation or added to existing tables via an update operation, during which DynamoDB performs backfilling to populate the index with existing data, a process that can take time proportional to the table's size.[35] They maintain separate provisioned read and write capacity from the base table, enabling independent scaling for index-specific workloads.[35] Projections in GSIs determine which attributes are included: KEYS_ONLY (only index and primary keys), INCLUDE (keys plus specified non-key attributes), or ALL (all table attributes). When using the INCLUDE projection type, the total number of non-key attributes across all secondary indexes on a table cannot exceed 100.[36] By default, GSIs offer eventual consistency for reads; they are particularly useful for ad-hoc queries on attributes not covered by the primary key, such as enabling access patterns in applications with multiple query requirements.[35] Local secondary indexes (LSIs), in contrast, share the same partition key as the base table but allow a different sort key, facilitating queries within the same partition using alternative sorting criteria.[37] LSIs must be defined at the time of table creation and cannot be added or removed afterward, limiting their use to scenarios anticipated during initial design.[34] They share the base table's read capacity units but consume write capacity from the table during updates, and projections follow the same options as GSIs (KEYS_ONLY, INCLUDE, or ALL). When using the INCLUDE projection type, the total number of non-key attributes across all secondary indexes on a table cannot exceed 100.[36] A table supports up to 5 LSIs, making them suitable for fine-grained access within partitions, such as querying by different timestamps or categories under the same partition key.[38] DynamoDB imposes key limitations on indexes to ensure scalability: a table can have up to 20 GSIs and 5 LSIs, with all secondary indexes requiring unique names and adhering to the same attribute naming rules as the base table.[38] These constraints, combined with projection efficiencies like sparse indexes (where items without projected attributes are omitted), help maintain performance while supporting complex data models without full table replication.[39]Common Architectural Patterns
In Amazon DynamoDB, common architectural patterns revolve around optimizing for query efficiency in a NoSQL environment by prioritizing access patterns and leveraging key structures to model relationships without traditional joins.[40] Denormalization is a foundational technique, where redundant data is intentionally duplicated across items to enable fast reads by co-locating related information, thereby avoiding the performance overhead of relational joins that DynamoDB does not natively support.[41] This approach trades off increased write amplification—requiring updates to multiple items when source data changes—for reduced read latency and throughput costs, as a single query can retrieve complete datasets.[41] A prevalent pattern is the single-table design, which consolidates multiple entity types into one DynamoDB table using composite primary keys to differentiate and access data efficiently.[42] By employing a partition key (PK) for entity grouping and a sort key (SK) for ordering or hierarchy, this pattern enhances data locality, minimizes cross-table queries, and simplifies table management while smoothing partition-level traffic.[42] For one-to-many relationships, such as a user and their orders, the design uses the user's ID as the PK and a composite SK (e.g., "ORDER#" followed by timestamp or order ID) to store child items adjacently, allowing selective queries via key condition expressions to retrieve related items without full scans.[43] This enables efficient pagination and filtering, though it requires careful SK prefixing to avoid hot partitions. Many-to-many relationships are modeled using adjacency lists within the single-table paradigm, where items represent connections between entities via SKs listing related IDs, often augmented by global secondary indexes (GSIs) for reverse lookups.[44] For instance, in a system linking invoices to bills, the PK might be the invoice ID with an SK listing bill IDs; a GSI on the SK then supports queries for all invoices per bill, projecting only necessary attributes to control costs.[44] The trade-off here is minimal duplication but potential need for additional processing, such as via DynamoDB Streams and AWS Lambda, to maintain consistency during updates.[44] Access pattern prioritization drives these designs, starting with identification of core queries (e.g., retrieving user orders by timestamp) to inform key structures, ensuring partition keys distribute load evenly and sort keys support range queries.[40] Anti-patterns include over-normalization, which fragments data across tables and forces inefficient scans or multiple requests, or embedding complex nested structures that inflate read capacity units (RCUs) without leveraging keys.[42] Schema evolution is handled flexibly using map attributes to add fields without altering the table, mitigating rigidity in evolving applications.[43] In an e-commerce scenario, orders can be modeled with a PK as the order ID and an SK combining entity types (e.g., "ITEM#productId" for line items, "SHIP#timestamp" for shipments), enabling queries like fetching all products in an order viaSK begins_with "ITEM#".[45] A GSI on product ID and order date supports pattern like retrieving recent orders per product, demonstrating how composite keys align with business queries while denormalizing details like customer info into order items to avoid extra reads.[45]
Operations
Queries and Scans
In Amazon DynamoDB, the primary read operations are Query and Scan, which enable retrieval of items from tables or secondary indexes based on specified criteria. The Query operation is designed for efficient access to items using primary keys or index keys, while the Scan operation performs a full table read, making it suitable for scenarios without precise key-based filtering but generally less performant due to higher capacity consumption. Both operations support optional filters, projections to limit returned attributes, and pagination for handling large result sets.[46][47] The Query operation retrieves items from a table or secondary index by specifying the partition key attribute name and value, which identifies all items sharing that partition key. Optionally, a sort key condition can be applied using operators such as equals (EQ), less than or equal to (LE), less than (LT), greater than or equal to (GE), greater than (GT), begins_with, or between, allowing refinement of results within the partition. For example, a KeyConditionExpression might specifypartition_key = :value AND sort_key BETWEEN :low AND :high to fetch a range of items. Queries can include a FilterExpression to further narrow results after the key-based retrieval, though filters do not reduce capacity consumption as they are applied post-query. The ProjectionExpression parameter allows selecting specific attributes to return, optimizing data transfer without affecting read capacity units (RCUs). Queries on global secondary indexes (GSIs) are eventually consistent only, while local secondary indexes (LSIs) and base tables support both consistency models.[48][46]
In contrast, the Scan operation examines every item in the entire table or index, returning all or a filtered subset without requiring key specifications, which makes it useful for ad hoc analysis or when access patterns do not align with keys. A FilterExpression can be applied to exclude items post-scan, using conditions on non-key attributes, but this does not lower the RCUs consumed since the full scan occurs regardless. Like Query, Scan supports ProjectionExpression for attribute selection and a Limit parameter to cap the number of items evaluated per request, with a maximum response size of 1 MB. Scans are less efficient than queries for targeted reads, as they can consume significant capacity—equivalent to reading the entire table—especially in large tables, and are best reserved for infrequent operations or small datasets. Parallel scans can be enabled by specifying Segment and TotalSegments parameters to distribute the workload across multiple workers.[49][47]
Both Query and Scan offer consistency options to balance data freshness and cost. Eventually consistent reads, the default, may reflect changes from recent writes after a brief propagation delay across replicas and consume 0.5 RCUs per 4 KB of data read. Strongly consistent reads, requested by setting ConsistentRead to true, guarantee the most recent data by querying a majority of replicas and double the RCU consumption to 1 RCU per 4 KB. Strongly consistent reads are unavailable for GSIs and streams but supported for tables and LSIs. Applications requiring immediate accuracy, such as financial systems, should use strongly consistent reads, while others can leverage eventual consistency for cost efficiency.[3][50]
Pagination is handled in both operations via the ExclusiveStartKey request parameter, which uses the LastEvaluatedKey from a previous response to resume from the next item, ensuring complete results across multiple requests without exceeding the 1 MB limit per call. The response includes Count for matching items and ScannedCount for evaluated items, helping assess efficiency—high ScannedCount relative to Count indicates potential for optimization via better indexing.[48][49]
Since 2020, DynamoDB has integrated PartiQL, a SQL-compatible query language that allows ad hoc, SQL-like statements across tables and indexes without needing to specify API operations directly. For instance, a SELECT statement like SELECT * FROM Table WHERE partition_key = ? performs a query equivalent, supporting joins, aggregations, and operations on multiple tables in a single request via the ExecuteStatement API. PartiQL simplifies development for users familiar with SQL while maintaining DynamoDB's performance characteristics, including the same consistency and capacity rules as native operations.[51][52]
Locking and Consistency Models
Amazon DynamoDB employs two primary read consistency models: eventually consistent reads, which are the default and provide lower latency at the cost of potential staleness, and strongly consistent reads, which guarantee the most up-to-date data but consume double the read capacity units.[3] Eventually consistent reads reflect changes from all prior successful writes but may not immediately show the latest updates across replicas, while strongly consistent reads query the primary replica directly to ensure accuracy, applicable to primary key lookups in tables and local secondary indexes but not global secondary indexes or streams, which remain eventually consistent.[3] For concurrency control, DynamoDB implements optimistic locking through conditional writes, where updates succeed only if specified attributes match expected values, often using a version number attribute to detect conflicts.[53] This approach avoids traditional locks by allowing concurrent reads and writes, with failures triggering retries; for instance, an application increments a version field on each update and conditions the write on the current version matching the expected one, preventing overwrites from out-of-date clients.[53][54] Introduced in late 2018, DynamoDB transactions provide atomicity and serializable isolation for multi-item operations across up to 100 actions in one or more tables, using APIs such as TransactWriteItems for writes and TransactGetItems for reads.[55] These operations execute as a single, all-or-nothing unit, ensuring that either all succeed or none do, with conflict detection via optimistic concurrency control; for example, TransactWriteItems supports Put, Update, Delete, and ConditionCheck actions, while TransactGetItems retrieves items atomically without consuming write capacity.[56][57] Serializable isolation prevents phenomena like dirty reads or lost updates, though transactions incur higher latency and capacity costs compared to individual operations.[55] DynamoDB Streams capture an ordered sequence of item-level changes for a table, enabling applications to maintain consistency through event-driven processing, with integration to services like Amazon Kinesis Data Streams for real-time data pipelines.[58] Each stream record includes the modified item's before-and-after images, new or old values, and metadata such as sequence numbers and timestamps, stored for up to 24 hours to support reliable replay and deduplication.[58] While streams provide eventual consistency for downstream applications—without built-in transactional guarantees—they facilitate use cases like cross-region synchronization or audit logging by triggering AWS Lambda functions or streaming to Kinesis for further processing.[59] In global tables, which replicate data across multiple AWS Regions for multi-active availability, conflict resolution defaults to a last-writer-wins strategy based on internal timestamps, ensuring eventual consistency by propagating the most recent update to all replicas. If simultaneous writes occur to the same item from different Regions, DynamoDB resolves the conflict in favor of the write with the latest timestamp, discarding others to avoid divergence.[60][61] As of June 30, 2025, global tables also support multi-Region strong consistency (MRSC), enabling applications to achieve strong consistency across regions with a zero recovery point objective (RPO) for enhanced resilience in mission-critical workloads.[62] For custom resolution, applications can leverage DynamoDB Streams to implement application-level logic, such as merging changes or notifying users of conflicts, rather than relying solely on the default mechanism.[63][64]Performance and Limitations
Latency and Throughput Claims
Amazon DynamoDB is designed to deliver single-digit millisecond latency for read and write operations at any scale, enabling predictable performance for applications handling varying workloads. This performance is achieved through its fully managed, serverless architecture that automatically scales resources without manual intervention. For instance, AWS documentation highlights that DynamoDB maintains low latency even as tables grow to millions of items, with typical response times under 10 milliseconds for standard operations like GetItem and PutItem.[1][65] Throughput in DynamoDB is measured in terms of Read Capacity Units (RCUs) and Write Capacity Units (WCUs), where one RCU supports one strongly consistent read per second for items up to 4 KB (or two eventually consistent reads), and one WCU supports one write per second for items up to 1 KB. The system offers unlimited throughput scalability via automatic data partitioning across multiple servers, allowing tables to handle millions of requests per second globally. Each partition supports a maximum of 3,000 RCUs per second for strongly consistent reads (or 6,000 for eventually consistent) and 1,000 WCUs per second. To manage traffic spikes, DynamoDB includes adaptive capacity, which dynamically allocates additional throughput to individual partitions experiencing hot spots, preventing throttling and sustaining higher workloads without reconfiguration.[23][21][66][20] AWS-provided benchmarks demonstrate DynamoDB's capability to achieve over 100,000 writes per second per table in provisioned capacity mode, with on-demand mode supporting similar scales after a brief warming period to exceed 100,000 reads per second. New on-demand tables initially support up to 4,000 writes per second and 12,000 reads per second, scaling higher based on usage patterns. These results underscore the service's ability to support high-volume applications, such as real-time analytics or e-commerce platforms, while maintaining consistent performance.[22][67] Several factors can influence latency in DynamoDB, including network distance between the client and the AWS region, item size (larger items up to the 400 KB limit increase processing time), and the chosen consistency level. Strongly consistent reads, which query a quorum of replicas for the most up-to-date data, typically incur approximately twice the latency of eventually consistent reads due to the additional coordination required. Additionally, DynamoDB commits to a 99.99% monthly uptime Service Level Agreement (SLA) for standard tables, with service credits issued if availability falls below this threshold, ensuring high reliability alongside low latency. For global tables, the SLA extends to 99.999% availability.[68][65][3][4]Known Limitations and Best Practices
Amazon DynamoDB imposes a maximum item size of 400 KB, which includes the lengths of attribute names and values in UTF-8 encoding.[29] This constraint necessitates strategies like compressing data or offloading large attributes to Amazon S3 with references stored in DynamoDB items.[30] Additionally, DynamoDB does not support server-side joins, requiring applications to denormalize data or perform client-side processing for related data retrieval.[69]Attribute Name Optimization
The total size of an item in DynamoDB is the sum of the UTF-8 byte lengths of all attribute names and values, plus overhead. Attribute names are treated as strings and fully contribute to this size. AWS recommends using shorter attribute names to reduce item sizes, which lowers storage costs and helps avoid crossing capacity boundaries that increase consumed Read Capacity Units (RCU, rounded up in 4 KB increments) and Write Capacity Units (WCU, rounded up in 1 KB increments). For example, long descriptive names can push an item from under 4 KB to over, doubling RCU for reads. Many applications use abbreviated names (e.g., "uid" instead of "userId") and map them to full names in application code for readability.[31][32] Eventual consistency, the default read model, can lead to risks of reading stale data shortly after writes, though strongly consistent reads mitigate this at double the cost in read capacity units. Hot partitions occur when uneven key distribution causes throttling on specific partitions exceeding provisioned throughput, potentially impacting application performance.[20] Scan operations are particularly costly, as they consume read capacity across the entire table or index, making them inefficient for large datasets compared to targeted queries.[70] Over-provisioning capacity in provisioned mode leads to unnecessary expenses, while attribute names can be up to 64 KB in length, but partition keys, sort keys, and names of projected attributes in secondary indexes are limited to 255 bytes (UTF-8 encoded); expression attribute names are limited to 255 bytes, which can complicate schema design if exceeded.[71][29] DynamoDB lacks native support for cross-region queries without using global tables, which enable multi-region replication but introduce eventual consistency across replicas.[72] To optimize usage, monitor tables using Amazon CloudWatch metrics for throughput, latency, and errors to detect issues like throttling early. As of August 2025, enhanced throttling observability includesThrottlingReasons fields in exceptions and new CloudWatch metrics for detailed analysis of throttling events.[73][74] Adopt single-table designs to minimize the number of tables, simplifying management and scaling while supporting multiple access patterns via secondary indexes.[40] Implement exponential backoff in retry logic to handle transient errors like provisioned throughput exceeded, starting with short delays and increasing progressively.[75]
Design operations for idempotency, especially in transactions, to safely retry without duplicating effects, using unique identifiers or conditional writes.[55] For scaling, ensure even distribution of partition keys to avoid hot partitions, and leverage global secondary indexes (GSIs) for fan-out scenarios where multiple query patterns are needed without overloading the base table.[20] Integrate with services like Amazon ElastiCache for caching frequently accessed data, reducing read latency and costs in read-heavy workloads.[76]

