Hubbry Logo
HAMMER2HAMMER2Main
Open search
HAMMER2
Community hub
HAMMER2
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
HAMMER2
HAMMER2
from Wikipedia
HAMMER2
Developer(s)Matthew Dillon
Full nameHAMMER2
IntroducedJune 4, 2014; 11 years ago (2014-06-04) with DragonFly BSD 3.8
Features
File system
permissions
UNIX permissions
Transparent
compression
Yes
Transparent
encryption
Planned
Data deduplicationLive
Other
Supported
operating systems
DragonFly BSD

HAMMER2 is a successor to the HAMMER filesystem, redesigned from the ground up to support enhanced clustering. HAMMER2 supports online and batched deduplication, snapshots, directory entry indexing, multiple mountable filesystem roots, mountable snapshots, a low memory footprint, compression, encryption, zero-detection, data and metadata checksumming, and synchronization to other filesystems or nodes. It lacks support for extended file attributes ("xattr").

History

[edit]
An example of file system layout of HAMMER2

The HAMMER2 file system was conceived by Matthew Dillon, who initially planned to bring it up to minimal working state by July 2012 and ship the final version in 2013.[1][2] During Google Summer of Code 2013 Daniel Flores implemented compression in HAMMER2 using LZ4 and zlib algorithms.[3][4] On June 4, 2014, DragonFly 3.8.0 was released featuring support for HAMMER2, although the file system was said to be not ready for use.[5] On October 16, 2017, DragonFly 5.0 was released with bootable support for HAMMER2, though file-system status was marked as experimental.[6]

HAMMER2 had a long incubation and development period before it officially entered production in April 2018, as the recommended root filesystem in the Dragonfly BSD 5.2 release.[7]

Dillon continues to actively develop and maintain HAMMER2 as of June 2020.

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
HAMMER2 is a modern, high-performance file system designed and implemented by Matthew Dillon for the DragonFly BSD operating system, serving as the successor to the original HAMMER file system and functioning as the default file system for DragonFly BSD. It employs a copy-on-write mechanism with a dynamic radix tree structure for efficient data management, supporting features such as instant snapshots, compression, deduplication, and extensive checksumming for data integrity. Development of HAMMER2 began in 2012, with initial experimental support introduced in DragonFly BSD version 5.0 in 2017, and it became the stable default in version 5.2 released in 2018. HAMMER2 continues to receive improvements, including experimental remote mounting in 6.4 (2022–2025). Key technical aspects include 64-bit inodes with a 1KB size, variable block sizes ranging from 1KB to 64KB in powers of two, and a logical limit of 2^63 bytes, enabling scalability for large storage environments. The also supports multiple mountable roots per disk image and online deduplication, making it suitable for advanced storage needs like mirroring and . Additionally, HAMMER2 provides immediate mountability after crashes without recovery delays and recursive check codes to detect corruption, enhancing reliability in production use.

Overview

Introduction

HAMMER2 is a 64-bit, file system designed to meet the demands of modern storage environments, utilizing a radix-tree for efficient and supporting file sizes up to 2^63 bytes. Developed by , the lead architect of , it emphasizes scalability, integrity, and performance for large-scale deployments. HAMMER2 was first integrated into with version 3.8, released on June 4, 2014, initially as a non-operational development component. It became the default file system for starting with version 5.2, released on April 10, 2018, serving as the primary platform for its deployment in production environments. Among its core capabilities, HAMMER2 maintains full compatibility with UNIX permissions and semantics, including support for hardlinks and softlinks. It features multiple mountable pseudo-file system roots (PFSs) for flexible organization, a low memory footprint achieved through an asynchronous buffer cache without locking, and scalability to exabyte-level storage volumes. As the successor to the original HAMMER file system, HAMMER2 was redesigned from the ground up to address limitations in clustering and performance.

Design Principles

HAMMER2 was designed with the primary goals of achieving instant recovery upon mount, dynamic adaptability to varying storage sizes, and efficient of large-scale without fragmentation. Instant recovery is facilitated by maintaining multiple headers—up to four for filesystems larger than 8GB—which allow the to select the most recent fully valid header during mount, ensuring rapid crash recovery without extensive scanning. The filesystem's supports scalability up to 16 exabytes through a multi-level freemap (1 to 6 levels), enabling seamless adaptation to storage growth while avoiding fragmentation via asynchronous block reference discarding. This design prioritizes robustness for petabyte-scale deployments, such as in clustered environments, by minimizing metadata overhead and ensuring consistent across diverse hardware configurations. Central to HAMMER2's principles is the use of block-level (COW) mechanisms to maintain filesystem consistency, where modifications create new blocks rather than altering existing ones, allowing for atomic updates and resilience against power failures. Variable block sizing, ranging from 1KB to 64KB, optimizes I/O by aligning smaller blocks (e.g., 512 bytes within inodes for tiny files) with access patterns while using larger sizes for bulk data, all under a fixed 64KB I/O boundary to simplify the allocator and reduce fragmentation risks. Fast lookups are enabled by indexing with 64-bit keys, providing logarithmic-time metadata access that scales dynamically without the need for rebalancing, thus supporting high-throughput operations on massive directories. From its inception, HAMMER2 incorporated native support for clustering, facilitating multi-node through quorum-based protocols and proxy streams for efficient data replication across distributed systems. To streamline the design and minimize overhead, it deliberately avoids extended attributes (xattrs), instead embedding such data directly into block references and inodes, which reduces complexity and improves performance in resource-constrained environments. Transparent is envisioned as a foundational tenet for future-proofing, implemented at the logical layer to secure data without impacting core operations, though it remains under development. These principles align with DragonFly BSD's emphasis on lightweight, high-performance storage integration.

Architecture

On-Disk Format

The on-disk format of HAMMER2 organizes data and metadata using a topology, with all blocks sized as powers of two ranging from 1 KB to 64 KB to optimize allocation and access efficiency. This structure enables a 64-bit capable of supporting volumes up to 1 exabyte, while ensuring alignment on 64-bit boundaries for all references. The format emphasizes integrity through embedded check codes and supports features like compression and deduplication at the block level. The superblock, referred to as the volume header, resides at the start of the volume and is replicated in up to four copies (one per 8 GB zone) for redundancy and recovery in larger filesystems. It includes the filesystem magic identifier (e.g., 0x48414d3205172011 for host byte order), a UUID for unique identification, creation and modification timestamps, and blockset pointers to the super-root inode and freemap. Additional fields cover transaction sequencing via mirror_tid for flush , copy configuration for up to 256 mirror targets, and CRC checksums for the header's eight 8 KB sectors to validate integrity on mount. The on-disk format supports copy-on-write by versioning these headers without overwriting prior copies. Central to the format is the blockref system, which uses 128-byte structures to every and metadata block with 64-bit aligned byte indexing for precise positioning. Each blockref contains a 64-bit key for , a 64-bit device offset (with low bits encoding block size from 10 to 16), three 64-bit transaction IDs (mirror_tid for versioning, modify_tid for changes, and update_tid for synchronization), a type field (e.g., 1 for inode, 3 for ), and a flexible check code field supporting up to 512 bits for algorithms like XXHash64 or SHA-256. These blockrefs form the backbone of trees, allowing indirect blocks to up to 512 child blockrefs per 64 KB indirect block. Inodes are fixed at 1 KB, comprising a 256-byte metadata section with attributes such as mode, user/group IDs, timestamps, file size, and generation counters; a 256-byte filename field; and a 512-byte union for either inline data (up to 512 bytes for small files) or four embedded blockrefs forming the blockset root for larger files and directories. The inode head portion embeds a 64-bit inode number (inum), object type, and compression method, while directories integrate small-file support by storing content directly in the inode. This design minimizes indirection for small objects while scaling to large files through the radix tree. Directory entries leverage hashing for constant-time lookups, with a 64-bit hash of the filename (using a collision-resistant ) indexing into leaf nodes. Entries for filenames up to 64 bytes are embedded within blockrefs, including the target inum, name (up to 255 bytes), and object type; longer names spill into dedicated 1 KB data blocks holding multiple entries. This hashed organization in leaf blocks (with the number of entries scaling with block size, up to hundreds or more) ensures efficient without linear scans. Volume organization centers on a single super-root inode that aggregates multiple pseudo-filesystem (PFS) roots, each functioning as an independent for features like snapshots and clones. The super-root's blockset points to PFS root inodes, supporting up to 256 volumes per filesystem (each up to 4 PB) via dynamic allocation without fixed partitions. The freemap, a specialized under the super-root, tracks free space at 1 KB granularity using approximately 4 MB per 2 GB of volume capacity. This hierarchical setup allows seamless multi-tenancy within a unified storage pool. Check code integration applies recursively to every block level, with each blockref's check field storing a hash or CRC computed over the block's contents post-compression and to detect corruption. The volume header's sector CRCs enable initial validation, propagating checks through the for full-path integrity verification during reads. Supported algorithms include 64-bit CRC for speed or longer hashes for security, configurable per filesystem.

In-Memory Data Structures

HAMMER2 employs a set of lightweight in-memory data structures to manage operations efficiently during runtime, prioritizing low overhead and fast access for indexing, caching, and metadata validation. These structures are designed to handle the nature of the while minimizing resource consumption, particularly on systems with constrained RAM. Central to this is the use of radix trees for indexing, a compact buffer cache for I/O operations, and per-pseudofile-system (PFS) inode caching to support concurrent mounts without excessive duplication. The implementation serves as the primary indexing mechanism for directories and indirect blocks, rooted at the super-root, PFS root, and individual inode boundaries. It dynamically splits into new indirect blocks when a node becomes full, ensuring balanced growth, and recombines by deleting empty blocks to reclaim memory during deletions or restructurings. This approach allows for efficient traversal and modification of the file system's topology in core, with the in-memory radix nodes directly mapping to on-disk block references for seamless persistence. HAMMER2's buffer cache adopts a low-footprint optimized for physical reads and writes, utilizing 64KB I/O clustering to reduce overhead and improve throughput. Buffers are managed asynchronously, enabling the operating system to retire dirty buffers directly to storage without blocking, which contributes to responsive system performance even under heavy workloads. This clustered approach minimizes the number of in-memory buffer objects required, keeping the cache size bounded and efficient. Inode caching in HAMMER2 is organized on a per-PFS basis, allowing multiple mounts of the same volume to share underlying structures while maintaining isolation. Lazy invalidation ensures that changes to one PFS do not immediately flush or invalidate caches in others, reducing overhead and supporting scalable multi-tenant environments. Each in-core inode, sized at 1KB, embeds up to four block references, enabling small files and directories to operate without additional indirect blocks for compact representation. Metadata handling relies on in-core representations of block references, each 128 bytes in size, which include space for up to 512-bit check codes to facilitate rapid validation without disk access. Directory entries, limited to under 64 bytes, are stored directly as these block references, permitting efficient random lookups and insertions in . This structure accelerates common operations like file lookups and attribute queries by keeping essential metadata resident. Overall, HAMMER2's in-memory design emphasizes efficiency for limited-RAM environments, avoiding the high-overhead structures found in file systems like , such as extensive ARC caching layers that can consume gigabytes. By bounding buffer cache overhead and using compact, dynamic indexing, it ensures the remains viable on embedded or low-memory servers without compromising functionality.

Features

Compression and Deduplication

HAMMER2 incorporates built-in compression and deduplication as core features for data reduction, applied transparently during write operations to optimize storage efficiency. Compression operates at the block level on up to 64KB logical blocks, using algorithms that reduce size only if the achieves a power-of-two savings, such as halving a block from 64KB to 32KB, thereby allocating smaller physical blocks. The supported algorithms include LZ4, which serves as the default for its high speed and low overhead, and zlib, which offers higher at the cost of increased CPU usage, configurable at levels from 1 to 9 (default 6 for zlib). These techniques are integrated into the write paths, where compression is applied automatically unless disabled, with a that skips uncompressible data to minimize overhead. Configuration occurs per pseudo-file system (PFS) via the hammer2 utility's set directive, specifying modes like lz4, zlib[:level], autozero (for treating zero blocks as holes), or none to bypass compression. Variable block sizes enabled by compression enhance efficiency for similar or compressible data, potentially yielding 25% to 400% more logical storage than physical capacity, particularly beneficial for text, logs, or . The streaming nature of the algorithms ensures minimal CPU cost, though double-buffering is required for caching compressed data at the device level and decompressed data at the file level. Deduplication in HAMMER2 is performed live and inline at the block level, detecting and avoiding duplicates by comparing hashes of pending write blocks against recently read on-media data. This process uses block reference (blockref) hashing to reference existing identical blocks instead of writing new ones, integrated seamlessly with the mechanism to reduce physical writes during operations like file cloning or updates. Enabled by default via the vfs.hammer2.dedup_enable , it requires either compression or check codes to be active and can be tuned with vfs.hammer2.always_compress for more consistent results, though the latter increases overhead. While live deduplication is fully operational, batch or "blind" deduplication for post-write scanning remains planned but unimplemented. These features contribute to integrity by ensuring consistent data representation through checksumming, as detailed elsewhere.

Snapshots and Cloning

HAMMER2 supports efficient point-in-time snapshots through its pseudo-file system (PFS) architecture, where each snapshot is implemented as a new PFS rooted at a copy of the original PFS's directory inode following a filesystem flush. This process is instantaneous and requires no quiescing of the filesystem, as the copy-on-write mechanism ensures that the snapshot captures the current state without duplicating data blocks. Snapshots are created manually using the hammer2 snap path [label] command on a mounted PFS, which flushes the mount to disk and duplicates the root inode, storing the resulting snapshot in the .snap/ directory of the PFS root. These snapshots are read-write by default, allowing modifications to diverge from the original filesystem while sharing unchanged blocks via copy-on-write. Cloning in HAMMER2 extends snapshot functionality by enabling the creation of writable copies from either a live PFS or an existing snapshot, leveraging the same inode duplication process to initialize a new PFS. This results in space-efficient clones that initially share all blocks with the source but allocate new blocks only upon modification, preserving the original . Clones are managed as independent PFS instances, mountable via the <device>@<pfs_name> syntax, and support operations like promotion to master or slave roles for clustering. For example, a clone from a snapshot can serve as a branch for testing or development without impacting the source. Snapshot and clone management includes both manual and automated options, with periodic snapshots configurable via jobs for daily, weekly, or monthly intervals using scripts that invoke the hammer2 snap command. Manual management involves commands like hammer2 pfs-list to enumerate PFSes (including snapshots and clones), hammer2 pfs-delete label to remove them, and mounting specific PFSes for access. HAMMER2's volume structure supports multiple independent PFSes per device, each capable of maintaining its own set of snapshots and clones without interference, facilitating hierarchical organization such as separate snapshots for user data, system logs, or backups. To maintain consistency across snapshots, HAMMER2 disables access time (atime) updates entirely, as the design prevents reliable atime tracking without compromising snapshot immutability for unchanged data. Theoretically, the filesystem supports up to 2^63 snapshots per PFS due to the 64-bit key space in its structures, though practical limits are governed by storage capacity and inode allocation. This enables extensive versioning while relying on the mechanism—detailed elsewhere—for efficient block sharing and recovery.

History and Development

Origins and Early Work

HAMMER2 was conceived by Matthew Dillon, the lead developer of DragonFly BSD, in early 2012 as a complete redesign of the original HAMMER file system to address key limitations and enable new capabilities such as advanced clustering and improved scalability. Unlike HAMMER1, which relied on a B-tree structure and struggled with multi-volume operations and inefficient handling of large storage volumes, HAMMER2 adopted a copy-on-write block-based design to support writable snapshots and better performance across distributed environments. Dillon had been working on initial concepts since 2011, with the project formally announced in 2012 to prioritize low memory usage, efficient I/O, and seamless integration for clustered setups. Development of early prototypes began in late 2012, when Dillon created a dedicated branch in the repository containing initial, non-compilable specifications and header files to outline the core architecture. By 2013, the first functional commits were integrated into the master branch, marking the start of internal prototyping focused on foundational elements like the on-disk format and basic mechanisms. These prototypes were tested within the kernel environment, emphasizing stability for core operations before expanding to advanced features. Community involvement during this phase was limited but targeted, with the project primarily driven by Dillon's vision and implementation. A notable contribution came through the in 2013, where student Daniel Flores developed the compression feature for HAMMER2, implementing LZ4 and zlib algorithms to compress file data blocks on write while preserving metadata integrity. This work, merged into the mainline later that year, represented one of the first external enhancements to the prototype, demonstrating early potential for collaborative refinement.

Key Releases and Milestones

HAMMER2 was initially integrated into as an experimental with the release of version 3.8.0 on June 4, 2014. At this stage, it was included in the system but deemed not ready for production use, serving primarily as a foundation for further development. Significant progress occurred with 5.0, released on October 16, 2017, which introduced the first bootable support for HAMMER2, enabling it to serve as the root . This milestone allowed users to install and boot from HAMMER2 volumes via the installer, though it remained experimental and required caution for critical data. HAMMER2 achieved production readiness in 5.2.0, released on April 10, 2018, where it was recommended as the default root for non-clustered operations. This recommendation followed numerous stability fixes and enhancements, such as KVABIO support for multi-core systems, marking a shift toward broader adoption within the DragonFly ecosystem. In 2023, ongoing refinements improved HAMMER2's CPU and introduced the "hammer2 recover" tool, which supports recovering or undoing changes to individual files and preliminary directory structures. These updates, committed by lead developer , enhanced reliability for data recovery scenarios. Additionally, ports of HAMMER2 to emerged, with experimental write support becoming available by November 2024, expanding its potential beyond . HAMMER2 remains actively developed as of 2025, with maintenance emphasizing replication for clustering and native encryption features, building on its foundation. The DragonFly BSD 6.4 series, with version 6.4.2 released on May 9, 2025, included various filesystem updates for HAMMER2 such as revamped VFS caching and other enhancements. Recent commits in November 2025 addressed debugging features and compatibility issues, with recovery capabilities seeing initial major support in October 2023 and ongoing refinements thereafter.

Implementation Details

Copy-on-Write Mechanism

The (COW) mechanism in HAMMER2 ensures data consistency by avoiding in-place modifications to existing blocks, instead allocating new storage for changes and updating metadata pointers atomically. When a file or directory is modified, HAMMER2 creates new blocks containing the updated and propagates the changes upward through the filesystem hierarchy by copying and updating parent pointers, culminating in an update to the blockset root that references the volume root. This process maintains the immutability of unmodified blocks, allowing concurrent reads to access consistent historical without interference from ongoing writes. Block allocation in HAMMER2's COW system relies on a dedicated freemap structure, which uses a to efficiently track free space across the volume. The freemap operates at a 1KB resolution and organizes storage into 2MB zoned segments to optimize I/O, while supporting dynamic block sizing from 1KB to 64KB to minimize fragmentation and adapt to varying data patterns. This approach allows HAMMER2 to allocate fresh blocks for COW operations without disrupting the layout of existing data, ensuring efficient space utilization over time. HAMMER2 batches all write operations into asynchronous transactions, which are committed periodically through a multi-phase sync process that flushes metadata to disk. Only the final update to the volume header—maintained in up to four redundant copies—is performed synchronously, providing crash consistency without the need for a separate journal, as the COW design inherently preserves prior states. This transaction model guarantees that filesystem operations like writes remain atomic even across sync points, snapshots, or unexpected crashes. The COW mechanism delivers key benefits, including the ability to create instant snapshots by simply copying the pseudo-filesystem (PFS) root inode after a sync, which requires minimal overhead and supports writable snapshots for branching. It also enables zero-downtime recovery, as the filesystem can roll back to the last committed root on , leveraging the pointer-based updates for rapid consistency checks. This integration with snapshots allows HAMMER2 to support space-efficient and versioning without halting operations. In handling edge cases, HAMMER2 uses 64-bit reference counters in inodes to manage hard links, allowing multiple paths to share underlying data blocks until a COW modification increments the count for the new version while decrementing the old one. Directory renames are performed efficiently without locking the entire directory tree, thanks to hashed entries stored in a structure that enables targeted updates to parent pointers. These features ensure scalability and concurrency in complex operations.

Checksumming and Integrity

HAMMER2 implements a robust checksumming system to safeguard against , utilizing 32-bit CRC check codes (the default) for each block, with logical block sizes ranging from 1KB to 64KB. These check codes are calculated on the compressed and potentially encrypted block contents and stored within the block reference (bref) structure, which is part of the filesystem's organization. The computation is recursive, propagating upward from leaf blocks through indirect blocks, inodes, and directory entries to the volume , ensuring end-to-end validation across the entire structure. This design allows HAMMER2 to maintain coherency even after power failures or crashes, as the check codes enable selection of the most recent valid during mounting. The checksumming covers all filesystem elements comprehensively, including user data blocks, metadata such as inodes and directory entries, and all block references in the . This broad coverage detects silent corruption—such as bit flips from hardware errors—by validating check codes on every read operation, triggering errors if mismatches occur. Inodes, for instance, embed up to four direct block references with associated check codes, while larger files use indirect blocks similarly protected, preventing undetected alterations at any level. Administrators can configure the check algorithm via commands like hammer2 setcrc32, which applies the CRC to new elements under specified paths, with options for alternatives like XXHash64 if needed. Verification is automatic and proactive: on mount, HAMMER2 scans the four redundant volume headers to identify and load the highest-sequenced one validated by its check code, followed by an incremental scan to synchronize the free map and detect inconsistencies. During runtime access, read operations implicitly verify check codes, halting on failures to prevent propagation of corrupt data. For deeper maintenance, the hammer2 utility supports scrub-like operations through directives such as cmd debug or cmd status, which can traverse and validate the filesystem structure offline or online. In cases of suspected post-crash, the hammer2 recover tool—introduced in 2023—enables targeted integrity checks and repairs, allowing recovery of single files or entire directory trees without a full rebuild, thus minimizing . Looking ahead, HAMMER2's features are poised for enhancement through integration, where check codes will be computed post- on logical subtrees, inodes, filenames, and data blocks to provide both and authenticated verification against tampering. This layered approach ensures that does not compromise the existing checksumming efficacy, aligning with the filesystem's commits for secure, verifiable updates.

Usage and Performance

System Integration

HAMMER2 file systems in are mounted using the mount_hammer2 utility, which attaches a pseudo file system (PFS) to a specified mount point. The command is mount_hammer2 [-o options] special node, where special refers to the block device and node is the mount directory; PFSes can be specified by using the @label for targeted mounting within a volume supporting multiple roots. Options include -o [local](/page/.local) for non-clustered operation and support for mounting multiple PFSes independently, enabling isolated environments like separate /home or /var partitions. Compression levels are not directly set at mount time but are configured per-PFS during creation. The hammer2 utility provides core management functions for HAMMER2 volumes, including snapshot for creating read-only copies, destroy for removing PFSes or snapshots, and prune for reclaiming space from historical data. For example, hammer2 snapshot <mountpoint> <snaplabel> generates an instantaneous snapshot, while hammer2 prune <mountpoint> removes old snapshots based on retention policies. Automatic snapshots are configured via /etc/periodic.conf, with variables like daily_snapshot_hammer2_enable="YES" and daily_snapshot_hammer2_dirs="/" enabling daily captures for specified directories or all mounted volumes by default. Weekly and monthly equivalents follow the same pattern, integrating with the periodic system for scheduled maintenance. Configuration occurs at the PFS level using hammer2 addpfs, which creates new pseudo file systems with options for compression (e.g., compression=lz4 or compression=zlib), deduplication enablement via nodeduplication=0, and quota enforcement through subhierarchy limits. These settings apply to the entire PFS, allowing tailored storage policies such as enabling deduplication for backup-heavy volumes or adjusting compression for trade-offs. HAMMER2 does not support extended attributes (xattrs), limiting its use in applications relying on metadata beyond standard file properties. Quotas track space and inode usage per directory without global enforcement. Porting efforts have brought experimental HAMMER2 support to via the filesystems/hammer2 port, achieving read/write functionality as of version 1.2.13 in August 2025. This implementation, sourced from a repository, allows mounting and basic operations but remains non-native and unsuitable for production without further integration. Similar experimental support exists for via pkgsrc. HAMMER2 is not natively available in other BSD variants like . Best practices for HAMMER2 deployment in include ensuring sufficient RAM for systems handling large volumes to support buffering and deduplication processes efficiently. It is recommended as the root file system (/) during installation, with separate PFSes for volatile directories like /tmp to optimize snapshot retention—defaulting to 60 days—and disable auto-snapshots on high-churn areas to minimize overhead.

Performance Characteristics

HAMMER2 employs variable logical block sizes ranging from 1KB to 64KB in powers of two, which minimizes seek overhead for small files and directories by embedding data directly in inodes or using smaller allocations where appropriate. This design enhances I/O efficiency, particularly for patterns, while 64KB physical clustering supports high-throughput sequential reads and writes, achieving competitive speeds on SSDs such as NVMe drives in benchmarks. The file system's memory footprint remains low by relying on the operating system's standard buffer cache without dedicated wiring, resulting in minimal overhead even for complex operations. HAMMER2 scales efficiently to volumes up to 16 exabytes via a multi-level freemap structure, maintaining performance without degradation as storage capacity grows. Inline compression using LZ4 or Zlib algorithms introduces modest CPU overhead during writes, primarily due to double-buffering for decompression, though this is mitigated by configurable limits on I/O and CPU utilization; read operations incur negligible additional cost. Deduplication further reduces write amplification with low runtime impact. In benchmarks, HAMMER2 outperforms HAMMER1 significantly in write-intensive workloads, such as database operations, and enables nearly instantaneous snapshot creation—requiring only a 1KB inode copy—compared to minutes for equivalent operations in its predecessor. It demonstrates competitiveness with for small-file handling in tests like and , while matching or exceeding performance on larger volumes in sequential and mixed workloads. Despite these strengths, HAMMER2 exhibits higher relative overhead on very small storage volumes under 1TB, where fixed block and metadata structures reduce efficiency compared to simpler file systems. Additionally, the absence of access time (atime) updates, due to the copy-on-write mechanism, can negatively impact workloads dependent on tracking file access patterns.

Comparisons

To HAMMER1

HAMMER2 represents a complete redesign of its predecessor, HAMMER1, with a focus on enhancing scalability, efficiency, and modern features while addressing limitations in the original implementation. Unlike HAMMER1, which relies on B-trees for its structure, HAMMER2 employs a dynamic radix tree and copy-on-write mechanism throughout, enabling a fully 64-bit architecture for all identifiers, block references, and inode numbers. This shift eliminates HAMMER1's pseudo-master/slave replication model for pseudo-file systems (PFSs), where masters handled writes and slaves provided read-only mirroring, in favor of a more flexible multi-master clustering approach that supports shared storage across nodes. Key improvements in HAMMER2 include support for writable snapshots, allowing users to mount and modify snapshots as independent PFSs, in contrast to HAMMER1's read-only snapshots that retain historical versions at 30-60 second intervals with a default 60-day retention. HAMMER2 also introduces inline, live deduplication to reduce physical writes during operations, a feature absent in HAMMER1, which only offered post-process data block deduplication. Additionally, HAMMER2 uses variable block sizes ranging from 1KB to 64KB for better efficiency with diverse workloads, whereas HAMMER1 employs fixed block sizes typically at 64KB for large allocations but with less flexibility for smaller data. These enhancements contribute to HAMMER2's superior performance in large-scale scenarios, with benchmarks showing it outperforming HAMMER1 in write-heavy tasks. However, HAMMER2 introduces trade-offs to achieve its design. It omits access time (atime) updates entirely, as modifying timestamps would conflict with the immutable block structure, whereas HAMMER1 supported atime for compatibility with traditional UNIX behaviors. Clustering in HAMMER2 adds through freemap domains and protocols for multi-node coordination, building on HAMMER1's simpler networked and slave-to-slave streaming without native multi-master support. Regarding migration, no direct in-place conversion tool exists for transforming HAMMER1 volumes to HAMMER2; users typically copy data using standard tools like or , with maintained via emulation layers for legacy HAMMER1 access on HAMMER2-enabled systems. In terms of scalability, HAMMER2 supports filesystems up to 16 exabytes across multiple volumes, exceeding HAMMER1's 1 exabyte limit (spanning 256 volumes of 4 petabytes each), making it better suited for exascale storage needs.

To Other Modern File Systems

HAMMER2 exhibits a lighter memory footprint compared to ZFS, primarily due to its asynchronous buffer cache design that avoids the resource-intensive Adaptive Replacement Cache (ARC) mechanism in ZFS, which can consume significant RAM for caching. This allows HAMMER2 to operate efficiently on systems with limited memory, as dirty buffers are retired directly to backing store without extensive filesystem interactions. In contrast, ZFS often requires substantial RAM allocation—typically 1GB per TB of storage—to maintain performance, potentially leading to memory pressure on lower-end hardware. HAMMER2 also provides native clustering support through its multi-node architecture, enabling shared filesystem access across machines with protocols for master/slave roles and quorum-based operations, whereas ZFS relies on external clustering solutions like GlusterFS or Pacemaker for similar multi-host capabilities. Both file systems offer snapshot functionality, but HAMMER2's copy-on-write snapshots are more resource-efficient, requiring only a 1KB inode copy after a sync, and do not necessitate integrated RAID levels like ZFS's RAID-Z, which adds complexity and overhead for redundancy. Compared to , HAMMER2 delivers more efficient deduplication via its live, online mechanism that detects and eliminates duplicates during writes using 512-bit block references, avoiding the post-process scanning required by , which can be computationally intensive and less seamless for real-time workloads. HAMMER2's pseudo-filesystem (PFS) structure simplifies management over Btrfs's subvolumes, which introduce additional complexity in handling independent mount points and of properties like compression. Furthermore, HAMMER2 has avoided the stability issues that plagued early implementations, such as RAID5/6 degradation handling and send/receive inconsistencies, establishing itself as production-ready for general use in since version 5.2. In terms of , HAMMER2 surpasses by providing comprehensive checksumming across both metadata and user data blocks using strong 512-bit CRC or cryptographic hashes, enabling detection and prevention of silent corruption at the filesystem level. , while supporting metadata checksums since kernel 3.18, lacks native data checksumming, relying instead on underlying hardware or journaling for basic protection against errors. HAMMER2 also includes built-in snapshot capabilities for , a feature entirely absent in , which prioritizes simplicity and speed for non-COW workloads. However, remains preferable for small-scale or embedded systems due to its minimal overhead and broad compatibility. A key strength of HAMMER2 lies in its exabyte-scale scalability, supporting up to 16 exabytes per filesystem without the need for storage pools, achieved through a dynamic 6-level and freemap structure that efficiently handles vast metadata volumes. This design also facilitates low-RAM operation, making it suitable for embedded environments where constraints are tight, as the filesystem's overhead remains bounded regardless of size. On the downside, HAMMER2's is less mature than that of or , with limited tooling and community support outside , and it lacks native integration in kernels, unlike Btrfs, restricting its adoption to BSD derivatives or experimental ports like the ongoing NetBSD effort. As of 2025, experimental ports continue for NetBSD and , with read-write support in development, while has an external port not yet integrated into the base system.

References

Add your contribution
Related Hubs
User Avatar
No comments yet.