Community hub
0 subscribers8 pages, 0 posts
Recent from talks
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Welcome to the community hub built to collect knowledge and have discussions related to Mnesia.
Nothing was collected or created yet.
Mnesia
View on Wikipediafrom Wikipedia
| Mnesia | |
|---|---|
| Original authors | Joe Armstrong, Robert Virding, Dan Gudmundsson and others |
| Developer | Ericsson |
| Stable release | 4.15.4
/ 19 June 2018 |
| Written in | Erlang |
| Operating system | Cross-platform |
| Platform | Cross-platform |
| Type | Relational Database Management System |
| License | Apache License 2.0 (since OTP 18.0) |
| Website | www |
Mnesia is a distributed, soft real-time database management system written in the Erlang programming language.[1] It is distributed as part of the Open Telecom Platform.
Description
[edit]As with Erlang, Mnesia was developed by Ericsson for soft real-time distributed and high-availability computing work related to telecoms. It was not intended as a general office-based data processing database management system, nor to replace SQL-based systems. Instead Mnesia exists to support Erlang, where DBMS-like persistence is required.[2] It has more in common with embeddable DBMS such as Berkeley DB than with any SQL database server.
Database model
[edit]"Rows" in tables are represented as records that contain a key value and a data field. This data field may in turn be a tuple containing an Erlang data structure of any complexity.
Backend types
[edit]Mnesia has three inbuilt table types: ram_copies, disc_copies and disc_only_copies.
Ram_copies
[edit]Data resides in memory and table size is limited by available memory and are backed by ETS (erlang term storage) table.
Disc_copies
[edit]Data resides in memory but is also persisted on disk backed by disk_log. Disc_copies were backed by Dets tables until 30 September 2001 with the release of Erlang R7B-4.[3]
Disc_only_copies
[edit]Data resides only on disc and are backed by Dets (disk version of ETS).[4] Dets file format uses signed 32-bit integers for file offsets and has a limit of 2 GB so do disc_only_copies.[3]
Backend plugins
[edit]Due to limits imposed by Dets, support for other backend plugins was suggested by Ulf Wiger[5] and these were added to Mnesia. Klarna added the LevelDB backend plugin[6] while Aeternity added the RocksDB backend plugin.[7]
Relational features
[edit]The database model is relational, but isn't what someone familiar with SQL might expect. A database contains tables. Relationships between them are modelled as other tables.[8]
A key feature of Mnesia's high-availability approach is that tables can be reconfigured within a schema and relocated between nodes, not only while the database is still running, but even while write operations are still going on.
Coding
[edit]The query language of Mnesia is Erlang itself, rather than SQL.[9] It permits easy representation of transactions as a natural feature of Erlang by allowing developers to utilize a single language throughout an application.
Transactions
[edit]Erlang is a functional language. Mnesia builds on this to obtain ACID transaction support. The functional block which is run as a transaction is a commonplace Erlang construct called a Functional Object (or Fun)[10] and is called by the single Mnesia statement mnesia:transaction(F). This can lead to clearer source code than the paired BEGIN / COMMIT syntax of SQL, and so avoids its problem of unclosed transactions within a procedure.
Again as a result of the functional nature of Erlang, nesting transactions is simple. It's also possible to distribute transactions across multiple nodes (i.e. separate servers). The semantics of using transactions in this way remains consistent, making it easy to write library code that works equally in either context.
General coding style for Mnesia will always use transactions. For performance reasons, it also supports deliberate "dirty operations" which avoid transactions. These compromise the atomicity and the isolation properties of ACID, but offer around 10× more throughput.[11] In addition there are also in-memory alternatives, although these lose the durability property of ACID.
Efficient execution
[edit]Mnesia forms part of the LYME web application stack. This is akin to LAMP, but based on Erlang. Implementation in Erlang confers an efficiency benefit because of the use of a single virtual machine throughout an application. LYME makes use of this, since the Yaws web server is also implemented in Erlang.
Address space is shared (although safely so, under Erlang) between code and data, including Mnesia's table data.[1][12]
Origins and licensing
[edit]Mnesia and Erlang were developed by the Ericsson Computer Science Laboratory. They have been released as open source software. Mnesia is released under a derivative of the Mozilla Public License.[13] Since OTP 18.0 they are published according to the open source Apache License 2.0. Versions before OTP 18.0 were published under the open source Erlang Public License.[14]
ejabberd
[edit]Mnesia is also an option embedded within the Erlang-based Jabber daemon, ejabberd.[15]
See also
[edit]
References
[edit]- ^ a b Håkan Mattsson; Hans Nilsson; Claes Wikstrom (1999). "Mnesia - A Distributed Robust DBMS for Telecommunications Applications" (PDF). Archived from the original (PDF) on 22 September 2021. Retrieved 25 September 2008.
- ^ "Purpose of Mnesia". Open-source Erlang project.
- ^ a b "Erlang -- Mnesia Database Questions". www.erlang.org. Retrieved 5 November 2021.
- ^ "Erlang -- Mnesia Reference Manual". www.erlang.org. Retrieved 5 November 2021.
- ^ mnesia_rocksdb, æternity, 18 October 2021, retrieved 5 November 2021
- ^ mnesia_eleveldb, Klarna AB, 3 November 2021, retrieved 5 November 2021
- ^ mnesia_rocksdb, æternity, 18 October 2021, retrieved 5 November 2021
- ^ "Mnesia Getting Started guide". Open-source Erlang project.
- ^ "Mnesia Reference Manual (v 4.4.5)". Open-source Erlang project.
- ^ "6.17 Fun Expressions". Open-source Erlang project.
- ^ Gavin Terrill (20 August 2007). "Erlang's Mnesia - a distributed DBMS for highly scalable apps". InfoQ.
- ^ "Mnesia performance basics". Archived from the original on 2 December 2012. Retrieved 25 September 2008.
- ^ "About Erlang". Open-source Erlang project.
- ^ "Erlang Public License, V 1.1". Open-source Erlang project.
- ^ "ejabberd project homepage".
Mnesia
View on Grokipediafrom Grokipedia
Mnesia is a distributed, soft real-time database management system (DBMS) written in Erlang and integrated into the Erlang/OTP platform, specifically designed for industrial-grade telecommunications applications requiring high fault tolerance, fast real-time operations, and dynamic reconfiguration.[1] It serves as a multiuser, distributed DBMS that runs in the same address space as Erlang applications, providing a tight coupling between the database and the programming environment to eliminate impedance mismatches common in traditional database systems.[1]
Data in Mnesia is organized into tables, each identified by a unique atom name and composed of user-defined Erlang records, supporting a hybrid relational/object data model with configurable types such as set (unique keys), bag (multiple records per key), or ordered_set.[2] Tables can be replicated across nodes for fault tolerance, with storage options including ram_copies (in-memory only), disc_copies (both memory and disk), or disc_only_copies (disk only), enabling location transparency and distributed transactions that span multiple nodes atomically.[2] Mnesia supports dynamic schema reconfiguration at runtime, allowing tables to be created, modified, or moved without downtime, which is particularly suited for nonstop, high-availability systems in telecommunications.[1]
Key operations in Mnesia include atomic transactions via functions like
Here,
This returns
[26] Deletions are handled by
This also returns
mnesia:transaction/1, which ensure consistency across distributed reads, writes, and deletes, as well as "dirty" operations for higher performance in soft real-time scenarios at the risk of temporary inconsistencies.[2] It incorporates the Query List Comprehension (QLC) module for efficient, complex queries expressed in Erlang syntax, supporting joins, selections, and aggregations over replicated data.[2] While optimized for replicated, telecom-oriented workloads with fast lookups and event handling, Mnesia is not intended for hard real-time constraints or large-scale plain text processing, instead excelling in environments demanding rapid reconfiguration and fault recovery.[1]
Overview
Introduction
Mnesia is a distributed, soft real-time, multiuser database management system (DBMS) written in Erlang, designed for telecommunications and other high-availability applications.[1] It provides fault-tolerant data management that eliminates the impedance mismatch between the database and the Erlang programming language, enabling seamless integration for industrial-grade systems.[1] The core design goals of Mnesia emphasize high availability, fault tolerance, scalability in distributed environments, and tight coupling with Erlang's lightweight process model for concurrency.[1] It supports relational-like tables with configurable replication and persistence, along with transactions and mechanisms for complex queries, facilitating fast real-time lookups and dynamic reconfiguration without downtime.[1] As a built-in application within the Erlang/OTP platform, Mnesia functions as a persistent, distributed data storage solution optimized for soft real-time, high-availability scenarios in clustered deployments.[1] Originating from Ericsson's telecommunications requirements, it has evolved to support robust data handling in fault-prone, scalable networks.[1]History and Development
Mnesia was developed by the Ericsson Computer Science Laboratory in the early 1990s as a distributed database management system tailored for high-availability telecommunications applications, addressing the need for real-time data management in concurrent, fault-tolerant systems built with Erlang.[2] Designed to integrate seamlessly with Erlang's concurrency model, Mnesia emerged from Ericsson's efforts to create robust tools for telecom infrastructure, where downtime and data consistency were critical challenges.[2] The initial public release of Mnesia occurred in 1996 as a core component of the Open Telecom Platform (OTP), Ericsson's framework for building scalable Erlang applications, marking its integration with underlying storage mechanisms like ETS (Erlang Term Storage) for in-memory operations and DETS (Disk ETS) for persistent storage to enable hybrid in-memory and disk-based functionality.[3][2] Initially proprietary software within Ericsson's ecosystem, Mnesia was open-sourced alongside Erlang in December 1998 under the Erlang Public License (EPL), a derivative of the Mozilla Public License, to foster broader community adoption and contributions. Over subsequent years, Mnesia evolved through iterative enhancements in OTP releases, with key improvements to replication protocols for better fault tolerance and support for "dirty" operations to optimize performance in high-throughput scenarios, as seen in updates across OTP versions from R9 (2003) onward.[4] A significant milestone came in 2015 with OTP 18, when the entire Erlang/OTP platform, including Mnesia, transitioned to the Apache License 2.0, promoting wider commercial use and compatibility with other open-source projects by removing EPL's copyleft restrictions.[5] This licensing shift, announced in March 2015, facilitated Mnesia's integration into diverse non-telecom applications while maintaining its core focus on distributed reliability.[6] Mnesia's development and maintenance have been led by Ericsson's Erlang/OTP team, with ongoing contributions ensuring compatibility and enhancements up to OTP 27 (released in June 2024) and OTP 28 (released in May 2025, including Mnesia version 4.24 with support for EEP-69 nominal types for improved type systems), reflecting its enduring role in the platform's evolution for real-time, distributed computing needs.[7][8][4]Architecture
Data Model
Mnesia employs an extended relational data model where data is organized into tables composed of Erlang records, which are essentially tagged tuples. Each record represents a row in the table, with the first element being the table name (an atom) and the second element serving as the primary key, forming a unique object identifier (OID) for the record. For instance, a record might appear as{employee, 104732, "klacke", 7, male, 98108, {221, 015}}, where employee is the table name and 104732 is the primary key. This structure allows Mnesia to store arbitrary Erlang terms, including complex nested structures like lists or tuples, rather than being limited to primitive data types found in traditional relational databases.[9]
Tables in Mnesia support three primary types: set, bag, and ordered_set. A set table enforces uniqueness, permitting only one record per primary key, with insertions overwriting existing records sharing the same key; this is the default type. In contrast, a bag table allows multiple records per key, enabling duplicates while ensuring individual records remain unique. The ordered_set type maintains one record per key but stores them sorted by the key value, facilitating range queries, though it is incompatible with certain persistent storage options. Primary keys are defined as the first attribute in the record definition, and secondary indexing can be applied to non-key attributes to optimize lookups, specified via the {index, [AttributeName]} option during table creation. For example, indexing on an address field would use {index, [address]}.[2]
While Mnesia incorporates relational concepts, it diverges from traditional RDBMS by lacking explicit foreign key support and full SQL semantics; instead, relationships are modeled through application logic or additional reference tables, and joins are performed using match specifications in functions like mnesia:select/3. These match specifications, written as Erlang terms, allow pattern matching and filtering akin to lightweight queries, such as [{#person.age= '$1', _ = '_'}, [{' > ', '$1', 30}], ['$1']}] to select ages greater than 30. Schema enforcement is not strict at runtime; attributes are declared at table creation using {attributes, [AtomList]}—often derived from record_info(fields, RecordName)—but dynamic handling permits flexibility without rigid validation, emphasizing Mnesia's suitability for distributed, real-time Erlang applications.[2][9]
Table schemas are defined programmatically using mnesia:create_table/2, which specifies the logical structure including type, attributes, and indexes. A representative example for an employee table might be:
erlang
mnesia:create_table(employee,
[{type, set},
{attributes, record_info(fields, employee)},
{index, [department]}]).
mnesia:create_table(employee,
[{type, set},
{attributes, record_info(fields, employee)},
{index, [department]}]).
employee is a set table with attributes matching the employee record fields (e.g., {id, name, department, salary}), and an index on the department attribute for efficient secondary lookups. This approach integrates seamlessly with Erlang's record system, promoting type safety at compile time while allowing runtime adaptability.[2]
Storage and Replication
Mnesia supports three primary storage backend types for tables, each balancing performance, persistence, and resource usage differently. Theram_copies type stores table data exclusively in memory on the specified nodes, offering the highest performance for read and write operations due to the absence of disk I/O, but it is volatile and loses all data upon node crash unless replicated to persistent copies on other nodes.[2] In contrast, disc_copies maintains data both in memory and on disk across the designated nodes, ensuring persistence through transactional logging to disk, though this introduces overhead from synchronous disk writes that can reduce throughput compared to pure in-memory storage.[2] The disc_only_copies type stores data solely on disk, providing persistence without consuming RAM for the table itself, but it incurs the highest latency for operations due to direct disk access without caching, making it suitable for archival or low-access scenarios.[2]
Replication in Mnesia allows tables to be copied across multiple nodes, with the storage type specified independently for each replica to enable hybrid configurations, such as combining ram_copies for speed on active nodes with disc_copies for durability elsewhere.[2] Replication operates in synchronous or asynchronous modes depending on the access context: full transactions ensure synchronous propagation using a two-phase commit protocol, guaranteeing atomic consistency across all replicas before committing, while sync_dirty operations perform synchronous updates to all active replicas without full locking, and async_dirty operations apply changes locally first with asynchronous replication to other nodes, prioritizing availability over immediate consistency.[10] The schema, which defines table structures and node memberships, is automatically replicated to all disk-resident nodes using ram_copies or disc_copies storage, ensuring cluster-wide awareness during startup or modifications.[2]
In the event of node failures, Mnesia's replication model maintains availability if a majority of replicas remain operational, particularly for synchronous transactions that require quorum for commits, though asynchronous modes allow continued local operations with potential for eventual consistency recovery upon node reconnection.[10] For custom storage integration, Mnesia supports an experimental external copies feature via the {external_copies, [Nodes]} option in mnesia:create_table/2, combined with {storage_properties, [{BackendModule, Props}]} where BackendModule implements the mnesia_external_copy behavior; backend modules (e.g., for RocksDB via mnesia_rocksdb:register/0) handle operations and enable high-performance key-value persistence without mixing with native storage types like ram_copies, disc_copies, or disc_only_copies. This API was introduced experimentally in OTP 19 (2016) and remains experimental as of OTP 27.[4][11]
Data durability is enhanced through mechanisms like checkpointing, which captures a transactionally consistent snapshot of multiple tables for backup while allowing ongoing access, though it increases memory usage and may slow updates on retainer nodes.[12] For ram_copies tables, periodic dumping via mnesia:dump_tables/1 saves content to .DCD files on disk with minimal disruption, enabling reload at startup to mitigate volatility.[12] Backups can be performed online using mnesia:backup_checkpoint/2 on an active checkpoint, supporting custom modules for export formats, while loading functions like mnesia:force_load_table/1 ensure table availability post-restart, albeit with risks of inconsistency if replicas are unavailable; overall, persistent storage types like disc_copies and disc_only_copies provide higher availability and faster recovery compared to volatile ram_copies, at the cost of performance.[12]
Distribution and Clustering
Mnesia operates as a distributed database management system by leveraging the Erlang runtime's built-in distribution mechanism, which enables transparent communication and data access across multiple nodes in a cluster. This integration allows processes on different nodes to interact as if they were local, facilitating replication and coordination without explicit location awareness in application code. The system's design ensures that tables can be replicated across nodes, providing scalability and resilience in fault-tolerant environments.[2] Clustering in Mnesia begins with the creation of a shared schema using themnesia:create_schema/1 function, which establishes a replicated system table on designated disc-resident nodes, each requiring a unique directory for storage. Nodes join the cluster by starting the Mnesia application via application:start(mnesia) or mnesia:start(), after which they synchronize schema information with existing cluster members. The configuration parameter extra_db_nodes can specify additional nodes to connect to beyond those defined in the schema, enabling flexible expansion. Full replication distributes all tables across all nodes, while partial replication allows selective table copies, with replication types such as ram_copies, disc_copies, or disc_only_copies determining storage behavior.[2][13]
Synchronization occurs in two phases: initial table loading upon node joining and ongoing updates through distributed transactions. When a node joins, it loads table replicas from other nodes based on the load_order attribute (default 0, with higher values loading first), using parallel loaders controlled by the no_table_loaders parameter (default 2). Ongoing consistency is maintained via transactional operations, where updates are logged and replicated to all relevant replicas synchronously or asynchronously, depending on the operation type. Enabling majority checking with {majority, true} in mnesia:create_table/2 ensures that transactions commit only if a majority of replicas acknowledge the update, enhancing reliability in partial connectivity scenarios.[2][2]
Fault tolerance is achieved through node monitoring, automatic failover mechanisms, and strategies for network partitions. Nodes monitor cluster membership using mnesia:subscribe/1 to receive events such as node up/down or table changes, with debug levels adjustable via mnesia:set_debug_level/1 for troubleshooting. Automatic failover is supported by designating master nodes with mnesia:set_master_nodes/2, allowing recovering nodes to load tables from these masters during restarts. Network partitions are managed by the max_wait_for_decision parameter (default infinity), which controls transaction resolution timeouts, though prolonged partitions may require manual intervention like mnesia:force_load_table/1 to resolve inconsistencies. Erlang's distribution protocol further aids by detecting node failures and reconnecting, with parameters like net_ticktime influencing heartbeat intervals for partition detection.[2][14]
The distributed schema is managed across disc nodes, with the schema_location parameter (options: disc, ram, opt_disc; default opt_disc) dictating its persistence. Schema mismatches during startup are resolved by Mnesia attempting to merge definitions from cluster nodes, or through fallback installation using mnesia:install_fallback/2 to recover from prior states. Deletion of the schema via mnesia:delete_schema/1 is irreversible and erases all associated data, necessitating careful use in production clusters. This schema-centric approach ensures uniform table structures across the distributed system.[2][2]
Core Features
Transactions and Concurrency
Mnesia employs a transaction model that ensures data integrity and consistency in distributed environments, adhering to the ACID properties: atomicity, consistency, isolation, and durability. Full transactions, initiated viamnesia:transaction/1 or mnesia:transaction/2, execute a series of database operations as a single atomic unit, with all changes committed across all replicas or rolled back entirely upon failure. This mechanism uses a two-phase commit protocol to coordinate updates spanning multiple tables and nodes, guaranteeing that either all nodes apply the changes or none do, even in the presence of network partitions or node failures.[10]
For scenarios requiring higher performance at the cost of strict ACID guarantees, Mnesia provides dirty operations such as mnesia:dirty_read/1, mnesia:dirty_write/1, and mnesia:dirty_delete/1, which bypass transaction locking and commit directly to the local storage without replication or rollback capabilities. These operations are non-atomic and do not ensure isolation, making them suitable for read-heavy or low-contention workloads where speed is prioritized over full consistency. To balance efficiency and control, developers can use mnesia:activity/4 for batched operations in contexts like async_dirty or sync_dirty, enabling asynchronous updates for high throughput while optionally synchronizing across nodes.[10]
Concurrency in Mnesia is managed through a sophisticated locking system that employs read (shared) locks for readers and write (exclusive) locks for writers on individual records or entire tables, preventing data races and ensuring serializable isolation by default—meaning concurrent transactions appear to execute sequentially without interference. Deadlocks are detected and resolved using a "wait-die" strategy, where transactions are assigned timestamps; younger transactions yield (wait) or abort (die) based on priority, with automatic retries up to a configurable limit to minimize disruption. In soft real-time systems, such as telecommunications applications, long-running transactions are mitigated by configurable lock acquisition timeouts and the encouragement of short transactions or dirty operations to avoid blocking other processes, though full transactions may abort and retry if locks cannot be acquired promptly.[10][1]
Queries and Indexing
Mnesia provides flexible query mechanisms primarily through pattern matching and iteration functions, enabling efficient data retrieval from distributed tables without native SQL support. Queries are typically performed within transactions to ensure consistency, though the focus here is on the retrieval patterns themselves. The core query function,mnesia:select/3, uses Erlang Term Storage (ETS) match specifications—lists of tuples defining patterns, guards, and result bindings—to filter records in a specified table, returning a list of matching objects up to a configurable limit.[15] For example, to retrieve names of individuals aged exactly 36 from a person table, a match specification might be [{#person{name='$1', age=36, _='_'}, [], ['$1']}], where '$1' binds to the name field and _ ignores others.[15]
For aggregating or processing entire tables, Mnesia offers folding functions like mnesia:foldl/4, which applies a user-defined function to each record sequentially, accumulating results while supporting read or write locks to control concurrency. This is particularly useful for computations such as summing values or building derived data structures, iterating from the table's start to end. A reverse-order variant, mnesia:foldr/4, is available for ordered sets to process from end to start. Additionally, mnesia:match_object/3 performs pattern matching similar to select/3 but without custom result bindings, directly returning full records that match a partial pattern like {person, '_', 36, '_', '_'}.[16][17]
Indexing in Mnesia enhances query performance by avoiding full table scans, which can be costly in large or distributed environments. Every table automatically includes a primary key index on the first attribute (or specified key position), facilitating fast lookups via functions like mnesia:read/2 or mnesia:read/3. For non-key attributes, secondary indexes can be added using mnesia:index/2 or declared during table creation with the {index, [AttrList]} option, where AttrList specifies attribute names or positions (e.g., [age] or [2]). Once indexed, queries leverage these via specialized functions: mnesia:index_read/3 retrieves records by indexed value (e.g., mnesia:index_read(person, 36, #person.age)), while mnesia:index_match_object/4 combines pattern matching with index traversal for bounded elements. Removing an index is done with mnesia:del_table_index/2, though this requires careful consideration of ongoing queries. Indexes increase storage overhead and slow insertions but significantly speed up selective retrievals.[18][19]
For more complex, relational-like operations such as joins across tables, Mnesia relies on multiple sequential selects or dirty operations like mnesia:dirty_match_object/1 for non-transactional efficiency, though these lack atomicity guarantees. A higher-level abstraction is provided through the Query List Comprehension (QLC) module in the Erlang standard library, which integrates seamlessly with Mnesia tables via mnesia:table/1 or mnesia:table/2. QLC allows SQL-inspired list comprehensions, such as qlc:q([P || P <- mnesia:table(person), P#person.age > 30, Q <- mnesia:table(query), Q#query.id =:= P#person.id]), to perform joins and filters while automatically optimizing for indexes when possible—using select-based traversal over raw scans. Options in mnesia:table/2, like {traverse, {select, MatchSpec}} or {n_objects, N}, further tune QLC for chunked reads or custom matching, reducing memory and network load in clustered setups.[20][21]
Performance in Mnesia queries benefits substantially from proper indexing, as unindexed select/3 or match_object/3 defaults to scanning all records, potentially leading to linear time complexity in table size. Guarded match specifications, such as [{'==', '$1', 42}] in a tuple like [{#example{key='$1', _='_'}, [{'==', '$1', 42}], ['$_']}], allow complex conditions (e.g., equality, ranges) evaluated efficiently when combined with indexes, minimizing data transfer across nodes. QLC further aids optimization by compiling comprehensions into indexed selects where feasible, though users must verify index usage via mnesia:table_info/2 to ensure scalability.[15][22]
Schema and Table Management
Mnesia requires the creation of a schema prior to defining any tables, as the schema serves as the foundational metadata repository for the entire database structure.[23] The primary function for schema initialization ismnesia:create_schema/1, which accepts a list of nodes to establish the schema across, ensuring a unique Mnesia directory per node on local disc devices.[2] This process fails if the specified nodes are not alive, if Mnesia is already running, or if a schema already exists on those nodes.[2]
In distributed environments, mnesia:create_schema/1 facilitates a shared schema among multiple nodes, promoting consistency in table definitions and replication strategies.[23] For single-node setups, a local schema is created by passing only the current node, while disc-less nodes automatically generate a minimal in-memory default schema upon Mnesia startup without explicit creation.[2] Only disc-resident nodes should be included in the schema list to avoid errors, and existing schemas can be removed using mnesia:delete_schema/1 if persistent data obsolescence is acceptable.[2]
Table management begins with mnesia:create_table/2, which takes a table name (an atom) and a list of options to configure properties such as structure, storage, and behavior.[2] Essential options include {attributes, [atom()]} to define record fields, {type, set | ordered_set | bag} to specify uniqueness and ordering semantics (with ordered_set unsupported for disc-only copies), and replica directives like {ram_copies, [node()]} for in-memory storage, {disc_copies, [node()]} for dual RAM-disc persistence with logging, or {disc_only_copies, [node()]} for slower disc-only access.[2] The {local_content, boolean()} option (default: false) allows tables with shared names but node-local contents, useful for non-replicated data.[2]
Additional configuration options in mnesia:create_table/2 encompass {majority, boolean()} (default: false) to enforce update acknowledgments from a majority of replicas for enhanced consistency, {index, [atom()]} for secondary attribute indexes that trade space and insertion time for query efficiency, and {load_order, non_neg_integer()} (default: 0) to prioritize table loading during Mnesia startup.[2] Storage can be further tuned via {storage_properties, [{Backend, [Prop]}]} where Backend can be ets, dets, or an external backend (introduced in OTP 27), passing backend-specific properties to optimize performance. Since OTP 27, Mnesia supports external storage backends, enabling integration with custom or third-party storage engines such as RocksDB for improved scalability.[2][24] Other settings include {access_mode, read_write | read_only} (default: read_write), {record_name, atom()} for custom record tags, and {snmp, term()} for monitoring integration.[2] Replication-related options, such as copy types and majority, are detailed further in the storage and replication mechanisms.[2]
Tables can be altered post-creation using functions like mnesia:add_table_index/2, which adds an index on a specified attribute to support faster lookups at the cost of increased storage and write overhead.[2] For changing storage types, mnesia:change_table_copy_type/3 converts replicas (e.g., from ram_copies to disc_copies) while maintaining data integrity.[2] Access modes are adjusted with mnesia:change_table_access_mode/2, load orders via mnesia:change_table_load_order/2, and majority settings through mnesia:change_table_majority/2.[2] Indexes can be removed using mnesia:del_table_index/2.[2]
The lifecycle of tables includes automatic loading at Mnesia startup based on load_order, with higher values loaded first to ensure dependencies are met.[2] To delete a table entirely, mnesia:delete_table/1 removes all replicas and metadata transactionally, rendering the table inaccessible.[2] For emptying contents without deletion, mnesia:clear_table/1 removes all records.[2] Replicas are added with mnesia:add_table_copy/3 or moved via mnesia:move_table_copy/3, and removed using mnesia:del_table_copy/2.[2]
Schema evolution, such as adapting to changing record formats, is handled by mnesia:transform_table/3 or mnesia:transform_table/4, which apply a user-defined transformation function to migrate data while updating attributes and optionally record names.[2] In cases of network partitions or restarts, mnesia:set_master_nodes/2 designates load sources, and mnesia:force_load_table/1 overrides default loading for availability, though at the risk of temporary inconsistencies.[2] Backups for recovery are installed via mnesia:install_fallback/2.[2] Potential inconsistencies in schema or table states can be diagnosed using mnesia:info/0, which outputs details on active tables, transactions, locks, and memory usage to identify and resolve issues.[23]
Implementation in Erlang
Basic Usage
To begin using Mnesia, the database must first be started on the Erlang node, typically after specifying a directory for storage via the-mnesia dir command-line option.[23] The function mnesia:start() initiates the local Mnesia system asynchronously, returning ok upon launch, while application:start(mnesia) provides an equivalent method integrated with the OTP application framework.[2] For a new installation, a schema must be created before starting, using mnesia:create_schema([node()]) to initialize an empty database structure on the current node; this returns ok on success or {error, Reason} on failure, such as if Mnesia is already running or the schema exists.[13] Once started, tables can be created programmatically with mnesia:create_table/2, specifying options such as {ram_copies, [node()]} for in-memory storage or {attributes, record_info(fields, person)} to define fields based on an Erlang record.[9]
Basic data operations in Mnesia revolve around create, read, update, and delete (CRUD) actions, which must be wrapped in transactions for atomicity and consistency.[23] Inserts and updates use mnesia:write/1, passing an Erlang record or tuple; for example, to add a person record:
erlang
mnesia:transaction(fun() ->
mnesia:write(#person{id = 1, name = "Alice", age = 30})
end).
mnesia:transaction(fun() ->
mnesia:write(#person{id = 1, name = "Alice", age = 30})
end).
{atomic, ok} on success.[25] Reads employ mnesia:read/2 or mnesia:read/3 (the latter allowing a lock kind like write), returning a list of matching records or an empty list if none found; for instance:
erlang
mnesia:transaction(fun() ->
mnesia:read(person, 1, read)
end).
mnesia:transaction(fun() ->
mnesia:read(person, 1, read)
end).
mnesia:delete/1 or mnesia:delete/2, specifying the table and key:
erlang
mnesia:transaction(fun() ->
mnesia:delete({person, 1})
end).
mnesia:transaction(fun() ->
mnesia:delete({person, 1})
end).
{atomic, [ok](/page/OK)} if successful.[27] Updates follow the same pattern as inserts: read the record, modify it, and write it back within a transaction.[23]
Error handling in basic usage focuses on transaction outcomes, which return {aborted, Reason} for failures such as {no_exists, Table} when a table is not found or {timeout, Tables} for lock timeouts during concurrent access.[28] Developers should check these results and retry transactions if needed, using mnesia:wait_for_tables/2 after startup to ensure tables are accessible within a specified timeout (e.g., 20 seconds).[9]
For reliable startup in production, Mnesia integrates with OTP supervision trees by declaring it as a required application in the .app file and starting it via the supervisor's boot process, ensuring it restarts on failure as part of the overall system fault tolerance. This approach leverages OTP's supervision principles to maintain database availability.[29]
Advanced Operations
Mnesia provides several mechanisms for performing high-performance operations that bypass the full transactional overhead, known as dirty operations. These are particularly useful in scenarios requiring low-latency access, such as real-time systems, where the trade-off of potential inconsistency is acceptable. Themnesia:dirty_write/1 function writes a record directly to a table without initiating a transaction, ensuring the operation is atomic at the local node but not guaranteed to be consistent across a distributed cluster without additional synchronization.[30] Similarly, mnesia:dirty_read/2 retrieves all records matching a given key from a specified table without locking or transaction context, returning a list of tuples or an empty list if no matches exist.[31] For deletions, mnesia:dirty_delete/2 removes records associated with a key from a table in a non-transactional manner, which can improve throughput but risks leaving the database in an inconsistent state if the node fails mid-operation.[32] To mitigate durability issues with these operations, mnesia:dirty_sync/0 forces an immediate synchronization of the transaction log to disk on the local node, ensuring that all pending dirty updates are persisted before proceeding, though it introduces a brief performance pause.[33]
For efficient traversal and processing of large datasets, Mnesia supports folding and asynchronous operations. The mnesia:foldl/4 function applies a user-defined folding function to each record in a table, accumulating results from left to right while acquiring a specified lock kind (such as read or write) to prevent concurrent modifications during iteration.[34] This is ideal for aggregate computations, like summing values or collecting statistics, without loading the entire table into memory. Complementing this, asynchronous dirty operations via mnesia:async_dirty/1 or mnesia:async_dirty/2 allow execution of arbitrary functions in the background without transaction protection, logging changes for replication but forgoing locks to enable parallelism; the function receives arguments and returns a result, making it suitable for non-critical background tasks like periodic cleanups.[35]
Performance tuning in Mnesia involves configuring storage and monitoring parameters to balance reliability and efficiency. Core dumps can be enabled by setting the -mnesia core_dir Directory option at startup, directing crash dumps to a specified directory for post-mortem analysis, which is disabled by default to conserve resources.[36] Storage properties are adjustable via command-line flags, such as -mnesia dc_dump_limit Number (defaulting to 4), which controls the frequency of automatic log dumps to disk based on log file size growth, preventing unbounded expansion; related thresholds like the dump log write threshold can be queried via mnesia:system_info(dump_log_write_threshold) to monitor write accumulation before dumping.[36] Monitor tuning is achieved through mnesia:set_debug_level/1, which sets verbosity levels from none (default, minimal output) to trace for detailed event logging, aiding in debugging distributed inconsistencies without impacting production performance.[37]
Backup and restore operations in Mnesia facilitate data management for large-scale deployments. The mnesia:backup/1 function creates a checkpoint-based backup of all tables to a destination (e.g., a file), ensuring maximum redundancy by including schema and log information, and supports custom modules for tailored formats.[38] For restoration, mnesia:restore/2 loads data from a backup source online, applying options like {clear_tables, [TableList]} to overwrite existing tables or {skip_tables, [TableList]} to preserve them, which is crucial for handling large datasets by minimizing downtime; for extremely large backups, the mnesia:traverse_backup/6 utility iterates incrementally, applying transformations to avoid memory exhaustion.[39]
Event-driven interactions are supported through mnesia:subscribe/1, which allows processes to receive notifications for changes to specific tables. Subscriptions can be for simple events (e.g., {table, Tab, simple}) that notify on inserts, updates, or deletes without full record details, or detailed ones including the affected records, delivering messages like {mnesia_table_event, {insert, Tab, Record}} to the subscribed process for reactive behaviors such as caching updates or auditing.[40] This mechanism integrates seamlessly with Erlang's actor model, enabling decoupled, scalable event handling across clustered nodes.
Applications and Use Cases
Notable Deployments
Mnesia has been widely adopted in messaging and real-time communication systems due to its distributed nature and integration with Erlang. One prominent example is Ejabberd, an open-source XMPP server, which utilizes Mnesia as its default backend for storing user sessions, rosters, and offline messages.[41] This setup allows Ejabberd to handle persistent data across clustered nodes efficiently, supporting high-availability chat services in deployments ranging from small teams to large-scale enterprise environments.[42] In the messaging broker domain, RabbitMQ's early versions relied on Mnesia to manage metadata such as queues, exchanges, bindings, users, and virtual hosts.[43] This choice leveraged Mnesia's in-memory performance and replication for coordinating distributed message routing, though scalability challenges with large datasets prompted a migration to the Khepri store in RabbitMQ 4.0 and later releases.[44] Mnesia's origins in telecommunications make it a staple in Ericsson's systems, where it supports real-time call data management in operations support systems (OSS) and switching platforms like derivatives of the AXE series.[1] Designed specifically for industrial-grade telecom applications, Mnesia enables fault-tolerant storage and replication of dynamic network data, ensuring low-latency operations in high-volume environments.[45] Early implementations of WhatsApp's backend also incorporated Mnesia for handling user data, routing tables, and offline message queues, capitalizing on its embedded efficiency within the Erlang runtime to scale to millions of concurrent users.[46] This allowed WhatsApp to process transient data without external database dependencies, though custom modifications like asynchronous dirty transactions were employed to optimize for massive scale.[47] In financial services, Klarna's KRED system, which tracks consumer debt and servicing operations, was initially powered by Mnesia starting around 2004, benefiting from its distributed transactions for real-time updates.[48] However, as data volumes grew to hundreds of gigabytes, scalability limits led to a zero-downtime migration to PostgreSQL in 2022, highlighting Mnesia's strengths in early-stage, low-to-medium scale deployments.[49] Beyond these, Mnesia finds use in custom IoT and telecom applications, such as EMQX, an MQTT broker for IoT messaging, which extends Mnesia with the Mria protocol to support clusters handling over 100 million connections for device data synchronization.[50] Its lightweight, embedded design makes it suitable for edge computing scenarios requiring distributed state management without heavy infrastructure.Advantages and Limitations
Mnesia provides seamless integration with the Erlang runtime, utilizing Erlang records for data representation and leveraging the language's actor-based concurrency model to treat the database as an extension of the application itself, which eliminates impedance mismatches common in external database integrations.[2] Its built-in distribution allows tables to be replicated across nodes using storage types such asram_copies for in-memory replication or disc_copies for persistent replication, enabling fault-tolerant operations without additional middleware.[51] Designed for soft real-time telecommunications applications, Mnesia delivers low-latency data access through dirty operations that bypass full transaction overhead, achieving higher efficiency than transactional reads for high-throughput scenarios.[52] Replication is facilitated via atomic transactions with ACID properties—atomicity, consistency, isolation, and durability—ensuring high availability by propagating changes synchronously or asynchronously across replicas while supporting majority quorum for critical data.[53]
Despite these strengths, Mnesia has notable limitations in scalability and functionality. It is not optimized for massive datasets, with disc_only_copies tables capped at 2 GB each due to underlying DETS storage constraints, and disc_copies limited by available RAM since the entire table must load into memory on startup; practical recommendations suggest keeping tables under 1-10 GB to avoid performance degradation.[54] Mnesia lacks full SQL support, instead relying on Query List Comprehensions (QLC) for querying, which offers Erlang-specific list-based operations but falls short in ad-hoc querying, joins, and complex analytics compared to relational standards.[55] Additionally, it enforces a single schema per cluster, stored as a special table with a unique magic cookie for node authentication, which restricts multi-schema deployments and requires careful planning for schema evolution in distributed environments.[56]
In comparisons, Mnesia outperforms key-value stores like Riak in relational capabilities and ACID compliance, supporting sets, ordered sets, and bags with low-latency microsecond reads for co-located data, but Riak provides superior scalability for key-value workloads, handling hundreds of nodes with automatic rebalancing and higher distributed throughput.[57] Against PostgreSQL, Mnesia achieves faster latency for small, distributed in-memory datasets due to its embedded nature and avoidance of network round-trips, making it ideal for caches or configuration stores, though PostgreSQL's mature SQL engine better handles complex queries and larger-scale persistence.[55] Performance-wise, Mnesia excels in concurrent reads and writes via two-phase locking and lightweight processes, enabling efficient isolation without lost updates, but it struggles with intricate analytical workloads where QLC's limitations hinder optimization and index utilization.[58]