Hubbry Logo
Sync (Unix)Sync (Unix)Main
Open search
Sync (Unix)
Community hub
Sync (Unix)
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Sync (Unix)
Sync (Unix)
Sync (Unix)
View on Wikipedia
from Wikipedia

sync is a standard system call in the Unix operating system, which commits all data from the kernel filesystem buffers to non-volatile storage, i.e., data which has been scheduled for writing via low-level I/O system calls. Higher-level I/O layers such as stdio may maintain separate buffers of their own.

As a function in C, the sync() call is typically declared as void sync(void) in <unistd.h>. The system call is also available via a command line utility also called sync, and similarly named functions in other languages such as Perl and Node.js (in the fs module).

The related system call fsync() commits just the buffered data relating to a specified file descriptor.[1] fdatasync() is also available to write out just the changes made to the data in the file, and not necessarily the file's related metadata.[2]

Some Unix systems run a kind of flush or update daemon, which calls the sync function on a regular basis. On some systems, the cron daemon does this, and on Linux it was handled by the pdflush daemon which was replaced by a new implementation and finally removed from the Linux kernel in 2012.[3] Buffers are also flushed when filesystems are unmounted or remounted read-only,[4] for example prior to system shutdown.

Some applications, such as LibreOffice, also call the sync function to save recovery information in an interval.

Database use

[edit]

In order to provide proper durability, databases need to use some form of sync in order to make sure the information written has made it to non-volatile storage rather than just being stored in a memory-based write cache that would be lost if power failed. PostgreSQL for example may use a variety of different sync calls, including fsync() and fdatasync(),[5] in order for commits to be durable.[6] Unfortunately, for any single client writing a series of records, a rotating hard drive can only commit once per rotation, which makes for at best a few hundred such commits per second.[7] Turning off the fsync requirement can therefore greatly improve commit performance, but at the expense of potentially introducing database corruption after a crash.

Databases also employ transaction log files (typically much smaller than the main data files) that have information about recent changes, such that changes can be reliably redone in case of crash; then the main data files can be synced less often.

Error reporting and checking

[edit]

To avoid any data loss return values of fsync() should be checked because when performing I/O operations that are buffered by the library or the kernel, errors may not be reported at the time of using the write() system call or the fflush() call, since the data may not be written to non-volatile storage but only be written to the memory page cache. Errors from writes are instead often reported during system calls to fsync(), msync() or close().[8] Prior to 2018, Linux's fsync() behavior under certain circumstances failed to report error status,[9][10] change behavior was proposed on 23 April 2018.[11]

Performance controversies

[edit]

Hard disks may default to using their own volatile write cache to buffer writes, which greatly improves performance while introducing a potential for lost writes.[12] Tools such as hdparm -F will instruct the HDD controller to flush the on-drive write cache buffer. The performance impact of turning caching off is so large that even the normally conservative FreeBSD community rejected disabling write caching by default in FreeBSD 4.3.[13]

In SCSI and in SATA with Native Command Queuing (but not in plain ATA, even with TCQ) the host can specify whether it wants to be notified of completion when the data hits the disk's platters or when it hits the disk's buffer (on-board cache). Assuming a correct hardware implementation, this feature allows the disk's on-board cache to be used while guaranteeing correct semantics for system calls like fsync.[14] This hardware feature is called Force Unit Access (FUA) and it allows consistency with less overhead than flushing the entire cache as done for ATA (or SATA non-NCQ) disks.[15] Although Linux enabled NCQ around 2007, it did not enable SATA/NCQ FUA until 2012, citing lack of support in the early drives.[16][17]

Firefox 3.0, released in 2008, introduced fsync system calls that were found to degrade its performance; the call was introduced in order to guarantee the integrity of the embedded SQLite database.[18] Linux Foundation chief technical officer Theodore Ts'o claims there is no need to "fear fsync", and that the real cause of Firefox 3 slowdown is the excessive use of fsync.[19] He also concedes however (quoting Mike Shaver) that

On some rather common Linux configurations, especially using the ext3 filesystem in the "data=ordered" mode, calling fsync doesn't just flush out the data for the file it's called on, but rather on all the buffered data for that filesystem.[20]

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Sync (Unix)

View on Grokipedia
from Grokipedia
In Unix-like operating systems, the sync command is a utility that forces the of from the kernel's in-memory buffers to persistent storage devices, ensuring that modified file , metadata such as superblocks and inodes, and delayed block I/O operations are written to disk. This process helps prevent or filesystem corruption in the event of a sudden power failure or system crash by flushing all pending writes. When invoked without arguments, sync performs a global flush across all mounted filesystems using the underlying sync(2) , which schedules but does not necessarily complete the writes before returning. The command's basic synopsis is sync [OPTION]... [FILE]..., where specifying one or more files limits the operation to those files or their containing filesystems, typically employing the fsync(2) system call to synchronize both data and metadata. Options such as --data (or -d) restrict synchronization to file data using fdatasync(2), omitting unnecessary metadata updates for efficiency, while --file-system (or -f) targets the entire filesystem containing the specified files via syncfs(2). Implemented as part of the GNU Coreutils package, sync adheres to POSIX standards for the underlying system call, which was first specified in Issue 4, Version 2 of the POSIX.1 standard and later moved to the base specification in Issue 5. Historically, sync has been a fundamental tool since the early development of Unix, providing administrators and users with a manual means to ensure before activities like unmounting filesystems or shutting down systems, though modern filesystems often employ automatic syncing mechanisms to reduce the need for frequent manual invocation. Its reliability depends on the host system's persistence guarantees, and while it returns only after scheduling writes, actual completion may vary by kernel implementation, such as in where it blocks until writes finish.

Overview

Definition and Purpose

In Unix-like operating systems, the sync mechanism serves as a critical interface for ensuring persistence by committing all modified information from the kernel's in-memory buffers to non-volatile storage devices, such as disks. This includes flushing superblocks, inodes, and file blocks that have been altered but not yet written to permanent storage. By doing so, sync overrides the kernel's default delayed write policies, which buffer changes in memory to optimize . The primary purpose of sync is to mitigate the risk of or corruption in scenarios like power failures, system crashes, or improper shutdowns, where unwritten buffered data could otherwise be lost. This need arises from the historical design of Unix kernels, which introduced a buffer cache in the 1970s to cache disk blocks in , thereby reducing physical I/O operations and improving overall system efficiency at the cost of potential data volatility until explicitly synchronized. Without sync, the kernel might retain modifications indefinitely in volatile RAM, relying on background daemons or periodic flushes that may not suffice during abrupt interruptions. Sync operates at two levels: a full-system synchronization that affects all mounted filesystems, ensuring comprehensive across the entire storage environment, and targeted limited to specific files or filesystems for more granular control. The interface, such as sync(), provides the foundational means to invoke these operations from user-space programs or the kernel itself.

Command Versus System Call

In Unix-like operating systems, the sync command serves as a user-facing utility program, typically located at /bin/sync in distributions, designed to flush all modified filesystem buffers across the entire to permanent storage. This command invokes the underlying sync() to ensure that data buffered in memory, including modified superblocks, inodes, and delayed block I/O operations, is written out to disk, thereby minimizing the risk of in the event of a crash or improper shutdown. As implemented in Coreutils package, the basic form of the sync command simply calls the sync() function once and exits, providing a straightforward, non-interactive mechanism for system administrators to trigger a global synchronization without specifying individual files or filesystems. In contrast, the sync() represents the kernel-level interface, standardized in .1-2001, which allows applications to directly request the flushing of filesystem buffers. Declared in <unistd.h> as void sync(void);, it schedules all pending modifications to filesystem metadata and cached file data for writing to the underlying filesystems, though POSIX permits the call to return before the writes complete. On specifically, the implementation ensures that the I/O operations finish before returning, providing stronger guarantees akin to a collective fsync() on all open files. Programmers invoke this syscall through libraries like libc, with the syscall number varying by architecture—for instance, 162 on x86_64 Linux—enabling precise integration into applications for targeted needs, such as after critical writes in database software or before unmounting filesystems. The primary distinction between the command and the lies in their scope and invocation context: the command offers a broad, system-wide operation that is simple to execute from the shell and blocks until the flush is completed on , making it suitable for administrative tasks like preparing for a , whereas the sync() allows developers to trigger a system-wide flush from within programs. For more targeted without affecting unrelated filesystems, developers use related calls like fsync(2), and the command supports file-specific options in modern implementations, but its core behavior remains tied to the kernel's flushing process.

History

Origins in Early Unix

The sync utility and system call were introduced in , released in May 1975 by , as a core component of the kernel's buffer management system to address delayed writes in the filesystem. This version marked the first widespread distribution of Unix outside , building on earlier research efforts at the laboratory since the late . The mechanism ensured that modified data in memory buffers was flushed to disk, preventing potential loss during system interruptions. Early documentation from the era, including setup guides and kernel commentaries, highlights its role in maintaining filesystem integrity from the outset. In its original implementation, sync operated through a straightforward kernel procedure called update, which was invoked by the sync command or automatically every 30 seconds by a background to proactively flush data. The update function iterated over mounted filesystems, writing out superblocks and inodes (lines 7217 and 7223 in the kernel source), followed by a call to bflush (line 7230), which scanned the buffer pool for blocks marked with the B_DELWRI indicating delayed writes. These dirty buffers, typically 512 bytes in size, were then written to disk using low-level primitives like bwrite, adjusting the available buffer list (av-list) via notavail to manage efficiently during the (line 5229). This simple loop-based approach reflected the minimalist design philosophy of early Unix, prioritizing reliability over complexity in the kernel's I/O subsystem. The primary rationale for sync stemmed from the hardware constraints of the time, particularly the PDP-11 minicomputers on which early Unix ran, featuring slow disk drives such as 1 MB fixed-head disks or 2.5 MB moving-head units with seek times in the tens of milliseconds. Without battery-backed caches or uninterruptible power supplies—common in later systems—power failures or crashes could corrupt the filesystem if buffered writes remained unflushed, as evidenced by at least one documented filesystem loss due to hardware failure in the early implementations. Delayed writes were employed to boost performance by deferring I/O and allowing immediate return to user programs, but sync provided an essential safeguard to force these writes, mitigating risks of data inconsistency on unreliable, non-volatile storage. Initial man pages from preceding versions like V5 (1974), which closely paralleled V6, explicitly recommended calling sync before halting the system to ensure integrity, underscoring its foundational importance in research Unix at Bell Labs during the 1970s.

Evolution in Modern Unix-like Systems

The standardization of the sync system call in POSIX.1-1988 marked a pivotal step in ensuring portability across systems, defining sync() as a required interface that flushes all modified buffers to permanent storage and returns void upon invocation, without specifying error conditions to simplify implementation across diverse hardware. This built on earlier Unix implementations but formalized it for broader adoption, influencing subsequent standards like POSIX.1-2001 and beyond, where the core behavior remained unchanged. In , enhancements to sync functionality emerged to address specific needs in multi-filesystem environments; the , introduced in kernel version 2.6.39 in 2011, allows of a single mounted filesystem identified by a , providing finer control over caching without affecting the entire system. Complementing this, the GNU coreutils implementation of the sync command gained options like --file-system in 2015, enabling targeted flushing of filesystems containing specified files rather than global . BSD variants, particularly , evolved sync mechanisms alongside filesystem advancements; enhancements for the (UFS) and Fast File System (FFS) included support for asynchronous mount options starting in 4.3BSD (1986), allowing delayed writes for performance while relying on explicit sync calls to ensure during critical operations. The sync utility itself became integrated into shutdown processes from 4.0BSD (1980) onward, automatically invoking flushes to prevent during system halts. A key milestone in the 1980s was the integration of sync with networked filesystems like NFS version 2 (introduced in ), where synchronous write semantics required careful handling of remote storage to avoid inconsistencies, as local sync operations would propagate changes over the network only upon explicit flushing. This adaptation highlighted the need for robust caching strategies in distributed Unix environments.

Usage

Command-Line Invocation

The sync command in Unix-like systems is invoked from the command line to flush filesystem buffers to disk, ensuring that modified data in memory is written to persistent storage. The basic syntax is sync [options] [file...], where omitting arguments synchronizes all buffered data across the system, while specifying one or more files limits the operation to those files and their containing filesystems. This behavior relies on underlying kernel system calls such as sync() for global flushing or fsync() for file-specific operations. In distributions using coreutils, the command supports a few targeted options to refine the process. The -d or --data option synchronizes only file and essential metadata using fdatasync(2), avoiding full metadata updates for efficiency. The -f or --file-system option explicitly synchronizes the entire filesystem containing the specified file via syncfs(2). Standard options like --help display usage information, and --version shows the program version; there is no built-in verbose mode, though the command can be paired with utilities like time to monitor execution duration. For example, running sync file.txt flushes buffers for file.txt by default, while sync -d file.txt targets only its blocks. In BSD variants such as , the sync command is simpler, with the synopsis limited to sync and no support for options or file arguments; it performs a global flush of all pending disk writes using the sync(2) . It is primarily invoked without parameters to ensure buffer completion before system operations like halting. Practical scenarios for the sync command include manual invocation prior to unmounting filesystems with umount to prevent and integration into scripts to guarantee writes after file copies. Historically, practices like the "triple sync" sequence (sync; sync; sync) were used on older systems to allow multiple passes for thorough flushing before shutdown, originating from early UNIX where sync was non-blocking and multiple invocations ensured completion. In modern contexts, sync is often unnecessary for routine s, as tools like reboot or halt handle flushing automatically, but it remains useful for targeted reliability in scripts or before critical maintenance.

System Call Parameters and Behavior

The sync() in systems is defined by the standard with the function signature void sync(void);, taking no parameters and returning no value upon success. It is declared in the <unistd.h> header file, allowing programs to invoke it directly for filesystem synchronization. Upon invocation, sync() schedules all modified in-core data—such as dirty buffers and filesystem metadata—for writing to permanent storage across all mounted filesystems, but it does not block the calling while waiting for the I/O operations to complete. This non-blocking behavior ensures that the returns promptly after initiating the writeback , though the actual disk writes may continue asynchronously in the background. In practice, this makes sync() suitable for periodic flushes in long-running applications without halting execution. For more targeted synchronization, related system calls include fsync(int fd), a POSIX function that flushes data and metadata for a specific open file descriptor to its underlying device, blocking until completion. Additionally, on Linux systems, syncfs(int fd) provides filesystem-specific synchronization, mirroring sync() but limited to the filesystem containing the file referenced by the descriptor fd. In C code, sync() is typically called as a standalone function within an application, often in scenarios requiring before critical operations. For example:

c

#include <unistd.h> int main() { // Perform file operations here... sync(); // Schedule all dirty data for writeback // Continue execution... return 0; }

#include <unistd.h> int main() { // Perform file operations here... sync(); // Schedule all dirty data for writeback // Continue execution... return 0; }

This snippet demonstrates embedding sync() to ensure buffered changes are queued for disk without further details on handling, as the call returns void on success and does not set errno.

Implementation

Kernel-Level Flushing Process

When the sync() is invoked in the , it initiates a multi-step to flush pending modifications from to stable storage. The kernel first wakes up flusher threads, which are kernel worker threads (kworker) responsible for handling writeback operations across backing devices. These threads identify and mark dirty inodes and buffers—representing modified filesystem metadata and blocks—for writeback using functions like writeback_inodes_sb(). This step ensures that all outstanding changes in the kernel's caches are queued for flushing. The flushing proceeds in phases to maintain filesystem . Metadata, including inodes and superblocks, is synchronized first through iterations over all mounted superblocks via sync_inodes_sb() and filesystem-specific sync_fs() operations, which may involve journaling mechanisms like those in to commit transactions atomically. Data blocks follow, with sync_bdevs() initiating writes to block devices in a non-blocking pass followed by a blocking one to await completion. Superblocks are updated in the final metadata synchronization to reflect the flushed state. The conceptual flow routes these operations from the (for file data) or buffer cache (for metadata) through the I/O scheduler, which merges and orders requests before submitting them to the disk. The sync() call provides strong durability guarantees by waiting for I/O completions on all relevant buffers and pages, ensuring data reaches stable storage before returning, akin to invoking fsync() on every file in the system. However, it relies on underlying filesystem drivers for specifics like journaling; for instance, ext4's sync_fs method flushes the journal to guarantee crash consistency. Special cases are handled to avoid unnecessary or erroneous operations. Requests on read-only mounted filesystems are ignored, as indicated by early returns in sync_filesystem() when sb_rdonly() is true. For memory-mapped files, sync() interacts with the by including their dirty pages in the writeback process, ensuring modifications made via are persisted alongside regular file I/O.

Filesystem-Specific Variations

On the filesystem, commonly used in distributions, the sync operation performs a full flush of dirty buffers to disk, including committing pending journal transactions to ensure filesystem consistency. This process leverages ext4's journaling mechanism, where metadata and data modifications are first logged in the journal before being applied to the main filesystem structures. Additionally, ext4 supports write barriers to enforce proper on-disk ordering during journal commits, which helps maintain even when using volatile disk caches, though at a potential performance cost. In , deployed on systems like and , sync integrates closely with the Adaptive Replacement Cache (ARC), ZFS's in-memory cache layer that holds both clean and dirty data blocks. When sync is invoked, it triggers the synchronization of the current transaction group (TXG), ensuring that modified data is persisted to stable storage as part of ZFS's design. The parameter zfs_dirty_data_sync controls the threshold of dirty data that prompts automatic TXG flushes during normal operation to prevent excessive memory usage by dirty buffers, distinguishing it from simpler buffer flushes in non-COW filesystems. The (UFS), prevalent in BSD variants, adopts a more straightforward approach to sync, primarily flushing data and metadata within its cylinder groups—logical divisions of the disk that localize related inodes, directories, and blocks to minimize seek times. Each cylinder group maintains its own summary information, which sync updates to reflect changes across the filesystem. For networked filesystems like NFS, sync on the client propagates pending writes to the remote server by issuing corresponding flush operations over the network protocol, ensuring data reaches the server's stable storage. This can result in hangs or timeouts if network connectivity is disrupted, as hard-mounted NFS volumes block until the server responds or a retry limit is reached. Client-side caching, including attribute and page caches, further complicates guarantees, as local buffers may not immediately reflect server-side changes, potentially leading to stale data visibility until occurs.

Applications

Role in Database Systems

In database systems running on operating systems, the broad filesystem flush performed by the sync(2) is typically avoided due to its high overhead and lack of , which can severely impact performance in write-intensive environments. Instead, databases employ targeted primitives such as fsync(2), fdatasync(2), or file descriptor flags like O_SYNC and O_DSYNC to ensure durability for critical components like transaction logs without flushing the entire filesystem cache. For example, uses the wal_sync_method configuration parameter to dictate how Write-Ahead Log (WAL) segments are forced to stable storage, with options including fsync (which synchronizes both and metadata), fdatasync (synchronizes and metadata only if required for correct retrieval), and open_sync (synchronous writes from the outset). This approach allows WAL commits to be acknowledged quickly while guaranteeing crash recovery, as WAL buffers are moved to the kernel cache and then synced via issue_xlog_fsync during commits. Specific implementations further optimize syncing to balance speed and integrity. In MySQL's engine, synchronization occurs primarily during fuzzy checkpoints, where dirty pages from the buffer pool are flushed to disk in small, incremental batches rather than all at once, reducing I/O contention and allowing user transactions to proceed uninterrupted. invokes fsync or equivalent on log files at commit points but batches data page writes to align with checkpoint intervals, configurable via parameters like innodb_flush_log_at_trx_commit. Similarly, leverages direct I/O to circumvent the filesystem buffer cache altogether, enabling the database to handle buffering internally and issue direct writes to storage devices; this is activated by setting FILESYSTEMIO_OPTIONS to DIRECTIO or SETALL, which minimizes double caching and associated sync overhead while maintaining control over flush timing through options. Bypassing full sync mechanisms introduces risks of during power failures or crashes, as unsynchronized kernel or hardware caches may lose recent writes, potentially invalidating committed transactions. To counter this, databases universally adopt (WAL), which durably records all changes to an append-only log before modifying the primary data structures, enabling atomic recovery without relying on filesystem-wide syncs; for instance, if a crash occurs mid-transaction, the WAL allows redoing committed operations and undoing incomplete ones. Disabling sync-like operations (e.g., via fsync=off) exacerbates these dangers, as evidenced by historical incidents where storage errors went undetected, leading to irrecoverable corruption in WAL-dependent systems. This preference for fine-grained control represents a historical from early Unix-era databases, which often relied on periodic, coarse-grained sync invocations to flush buffers en masse, to modern systems post-1980s that integrate WAL protocols for precise durability guarantees. Pioneered in research like IBM's System R project, which favored WAL over shadow paging for efficient recovery, and formalized in the 1992 ARIES algorithm supporting partial rollbacks and fine-granularity locking, this shift enables high-performance syncing tailored to transactional workloads.

Integration with Shutdown and Reboot

In Unix-like systems, the sync utility is integral to the shutdown process, ensuring that all pending filesystem writes are committed to disk before unmounting filesystems and halting the system. Traditional SysV systems invoke sync through shutdown scripts, such as the K* scripts in /etc/rc.d or /etc/init.d directories, which execute sync prior to unmounting to flush kernel buffers and maintain . In distributions using SysV or compatible systems, the shutdown -h command triggers these scripts, often resulting in multiple sync invocations—commonly twice in older implementations—to account for any delayed writes during the transition to . Modern init systems like , prevalent in contemporary distributions since the 2010s, automate this further through the systemd-shutdown service. This service syncs filesystems and block devices explicitly after unmounting non-essential filesystems and disabling swap, but before the final power-off or halt, preventing corruption from unclean shutdowns. Similarly, Upstart, an event-driven init system used in some releases during the early 2010s, integrates sync into its shutdown event handling by coordinating with underlying kernel and filesystem operations to flush buffers during service stoppage. For reboot operations, sync ensures completed writes before system restart. In FreeBSD, the utility flushes the filesystem cache to disk as a standard step, committing all modifications prior to termination and kernel invocation; this can be bypassed with the -n flag for unclean halts, though such usage risks and is reserved for recovery scenarios. Best practices emphasize manual sync execution before power-off in embedded or legacy Unix environments, where automated processes may lack robustness against sudden interruptions; administrators typically run sync followed by umount to safely quiesce filesystems. In contrast, modern distributions automate these steps via rc shutdown scripts or system integrations, reducing the need for manual intervention while upholding reliability. This automation evolved from 1970s-era Unix practices, where operators manually issued "sync; umount" commands before halting to mitigate buffer-related risks on early hardware.

Performance and Reliability

Timing and Blocking Characteristics

The sync utility in systems exhibits behavior where, per , it may return immediately after queuing all pending dirty buffers and filesystem metadata for writing to stable storage, with the kernel handling the actual I/O completion. However, in implementations, the underlying sync(2) blocks the calling process until the writes are completed, providing guarantees similar to fsync(2) across all files. The kernel's periodic writeback is governed by parameters such as dirty_expire_centisecs, which defaults to 3000 (30 seconds) and specifies the maximum age of dirty pages before they become eligible for background flushing, ensuring data does not linger indefinitely in memory. The effective duration of the flushing process following a sync invocation depends primarily on the volume of dirty data accumulated in the and the performance characteristics of the storage subsystem. On systems with substantial dirty data, such as large servers, the operation can take from seconds to several minutes; for example, flushing approximately 1 GB of data to a traditional HDD typically requires 10-30 seconds due to sequential write speeds of 100-200 MB/s, whereas on SSDs, the same amount may complete in under a second owing to much higher random and sequential write throughput. In setups, particularly or , these times are often amplified by the overhead of parity computations during writes, which can reduce effective throughput by 20-50% compared to single-disk configurations. To monitor the progress and impact of sync operations on system responsiveness, administrators can observe I/O activity with tools like iostat, which reports disk utilization and throughput in real-time, or inspect /proc/meminfo for fields such as "Dirty" (pages modified but not yet written) and "Writeback" (pages actively being written), both of which decrease as flushing completes. In very early kernels (pre-1.3.20), issuing multiple sync commands sequentially—often recommended as "sync; sync; sync"—helped ensure full propagation of queued writes, as the initial call did not wait for completion; this practice is a historical holdover and unnecessary in modern implementations or kernels since 1.3.20, where sync blocks until writes finish.

Controversies and Best Practices

The sync command and its underlying have faced criticism for their potential to block I/O queues and degrade system performance, particularly when invoked frequently or during high-load scenarios. In the , debates centered on the optimal frequency of sync calls, as excessive use could overwhelm slow mechanical disks, leading to noticeable slowdowns while insufficient calls risked during crashes. For instance, in database environments, the related fsync operation—often preferred for targeted flushing—exhibits variable performance, with latency ranging from 0.14 ms on high-end NVMe SSDs to over 18 ms on traditional HDDs (as measured in 2014 benchmarks), prompting ongoing discussions about trade-offs between durability and throughput. As an alternative to blunt sync usage, tools like ionice allow prioritization of I/O operations by assigning scheduling classes (e.g., idle or real-time), mitigating blocking effects without full system flushes. Best practices emphasize restraint in using sync, recommending it only for critical scenarios such as before unmounting filesystems or during manual shutdowns to ensure data integrity, rather than as a routine maintenance tool. For precision, fsync or fdatasync should be employed on specific files or descriptors instead of the broad sync, as the latter flushes all filesystem buffers and can unnecessarily impact unrelated workloads. In Linux environments, tuning kernel parameters like vm.dirty_ratio—which caps the percentage of system memory allowable for dirty pages before forced writes, defaulting to 20%—to a lower value (e.g., 10%) promotes proactive background flushing, thereby reducing the frequency of manual sync invocations and minimizing performance stalls. In very early Linux kernels (pre-1.3.20) and earlier Unix variants (e.g., Version 7 UNIX), multiple sync calls were sometimes required for reliability, as a single invocation did not guarantee completion, potentially leaving metadata like superblocks vulnerable during abrupt halts. Modern kernels, since version 1.3.20 and enhanced around 2.6 with write barriers, enforce strict ordering of disk writes to stable storage and integrate with journaling filesystems like to prevent corruption from volatile caches, eliminating the need for redundant sync sequences. In contemporary contexts as of 2025 with SSDs and NVMe storage, sync operations execute faster due to reduced latency—e.g., up to 7380 fsyncs per second on enterprise NVMe as of , with newer drives potentially higher—but debates persist in database applications, where aggressive caching can still amplify wear or introduce consistency risks if barriers are disabled for speed. Administrators are advised to verify hardware support for barriers and monitor I/O patterns to balance these concerns.

Error Handling

Return Values and Diagnostics

The sync() system call, as specified in the POSIX standard, is a void function that does not return a value and defines no error conditions, signifying that it unconditionally succeeds in queuing all pending filesystem updates for writing to stable storage. This design reflects the operation's fundamental role in ensuring data integrity without the possibility of partial execution at the interface level, though the actual completion of writes to disk may occur asynchronously after the call returns. In practice, implementations like Linux adhere to this by making sync(2) always successful, without setting errno, as it merely initiates the flushing process without blocking for verification. For applications invoking the directly, success is implied by the absence of a return value, and post-call checks of errno are unnecessary in standard cases since failures do not occur at the sync() level itself. However, underlying I/O operations triggered by sync()—such as those handled by filesystems—can rarely propagate severe errors like EIO ( error) if hardware issues prevent writes, though this would typically manifest through related calls like fsync() or kernel-level handling rather than sync() directly. POSIX compliance further guarantees that no partial flushes occur, meaning all modified buffers are scheduled atomically; errors are confined to catastrophic kernel events, such as panics, where the cannot proceed with normal operations. The sync command-line utility, which wraps invocations of the sync() (often multiple times for reliability), exits with status 0 upon successful completion and produces no standard output, aligning with Unix conventions for non-verbose tools. In cases of , such as invalid command-line options or invocation errors, it returns a general exit code of 1, though such scenarios are uncommon given the command's simplicity and lack of complex parameters. Diagnostics for sync operations primarily rely on system-level logging rather than command-specific output. Kernel messages detailing issues like failed writes during flushing are directed to syslog, where they can be reviewed for indicators of hardware faults or filesystem problems. For deeper inspection, the strace utility traces system calls made by the sync command or invoking processes, revealing details such as the sync(2) invocation and any subsequent signals or errors without altering the operation. This approach allows verification that flushes are initiated correctly, especially in scripted or automated contexts.

Troubleshooting Common Issues

One common issue with the sync command arises when executing it on networked filesystems like NFS, where it may hang indefinitely due to server unresponsiveness, network latency, or connectivity disruptions. To diagnose, examine kernel logs with dmesg for NFS-related errors such as "server not responding." As a resolution, mount the NFS filesystem with the soft option alongside a timeout value (e.g., timeo=600), which allows operations to fail after a specified period rather than blocking; alternatively, use syncfs(2) on a specific filesystem descriptor to limit the scope and potentially avoid global hangs, though it lacks built-in timeouts and may still report errors like EIO if I/O fails. Another frequent problem occurs when sync performs slowly or fails on nearly full disks, as the kernel attempts to flush dirty but encounters space exhaustion, leading to ENOSPC errors for syncfs(2). In such cases, monitor I/O activity with iotop to identify bottlenecks from pending writes. The recommended fix is to free up disk space by deleting unnecessary files or expanding the volume before retrying sync, ensuring writes can complete without exhaustion. For diagnostics across issues, [dmesg](/page/Dmesg) reveals kernel-level errors like I/O failures or filesystem warnings during sync operations. If a sync fails and leaves the filesystem in an inconsistent state, unmount the partition and run [fsck](/page/Fsck) to scan and repair metadata errors, such as orphaned inodes or block mismatches. Edge cases include loop devices, where sync flushes the virtual block device but may not immediately propagate changes to the underlying backing file unless the host filesystem is also synced, requiring multiple invocations of sync to ensure consistency. With noatime mounts, sync has minimal impact on access time updates since they are suppressed, avoiding unnecessary writes but potentially masking timestamp-related inconsistencies if applications rely on them. Following abrupt power loss, the kernel may trigger automatic [fsck](/page/Fsck) on boot for affected filesystems; if not, manually invoke it in to recover from potential journal inconsistencies or dirty blocks. In modern containerized environments like Docker (post-2020), sync within a operates on the container's filesystem view, but bind-mounted s may exhibit delayed propagation due to host-guest overhead; troubleshoot by verifying permissions and using synchronized file shares if development workflows involve frequent flushes.

References

  1. https://man7.org/linux/man-pages/man1/sync.1.html
  2. https://www.gnu.org/software/coreutils/manual/html_node/sync-invocation.html
  3. https://pubs.opengroup.org/onlinepubs/009695299/functions/sync.html
  4. https://man7.org/linux/man-pages/man2/sync.2.html
  5. https://docs.kernel.org/filesystems/buffer.html
  6. https://pubs.opengroup.org/onlinepubs/009695399/functions/sync.html
  7. https://linux.die.net/man/8/sync
  8. https://raw.githubusercontent.com/coreutils/coreutils/master/src/sync.c
  9. https://filippo.io/linux-syscall-table/
  10. https://cs3210.cc.gatech.edu/r/unix6.pdf
  11. https://minnie.tuhs.org/PUPS/Setup/v6_setup.html
  12. https://dsf.berkeley.edu/cs262/unix.pdf
  13. https://www.tuhs.org/Archive/Distributions/Research/Dennis_v5/v5man.pdf
  14. https://nvlpubs.nist.gov/nistpubs/Legacy/FIPS/fipspub151-1.pdf
  15. https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=8b2bf5295f353016d4f5e6a2317d55b6a8e7fd00
  16. https://man.freebsd.org/cgi/man.cgi?query=sync&sektion=8
  17. https://utcc.utoronto.ca/~cks/space/blog/unix/NFSWritesAndSync
  18. https://tldp.org/LDP/sag/html/sag.html
  19. https://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html
  20. https://elixir.bootlin.com/linux/latest/source/fs/sync.c#L73
  21. https://docs.kernel.org/core-api/workqueue.html
  22. https://elixir.bootlin.com/linux/latest/source/fs/sync.c#L123
  23. https://www.kernel.org/doc/html/latest/filesystems/ext4.html
  24. https://openzfs.github.io/openzfs-docs/man/master/7/zfs.7.html
  25. https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html
  26. https://docs.oracle.com/cd/E18752_01/html/817-5093/fsfilesysappx-23724.html
  27. https://access.redhat.com/solutions/28211
  28. https://www.postgresql.org/docs/current/wal-configuration.html
  29. https://dev.mysql.com/doc/refman/8.4/en/innodb-checkpoints.html
  30. https://oracle-base.com/articles/misc/direct-and-asynchronous-io
  31. https://www.architecture-weekly.com/p/the-write-ahead-log-a-foundation
  32. https://danluu.com/fsyncgate/
  33. https://cs.brown.edu/vldb99/IndustrialSpeakerSlides/vldb99_aries_slides.pdf
  34. https://web.stanford.edu/class/cs345d-01/rl/aries.pdf
  35. https://serverfault.com/questions/115069/is-execution-of-sync8-still-required-before-shutting-down-linux
  36. https://www.freedesktop.org/software/systemd/man/systemd-shutdown.html
  37. https://man.freebsd.org/cgi/man.cgi?query=reboot%288%29&sektion=8
  38. https://unix.stackexchange.com/questions/5260/is-there-truth-to-the-philosophy-that-you-should-sync-sync-sync-sync
  39. https://www.kernel.org/doc/html/latest/admin-guide/sysctl/vm.html
  40. https://www.baeldung.com/linux/proc-meminfo
  41. https://serverfault.com/questions/250707/why-does-mdadm-write-unusably-slow-when-mounted-synchronously
  42. https://unix.stackexchange.com/questions/48235/can-i-watch-the-progress-of-a-sync-operation
  43. https://www.percona.com/blog/fsync-performance-storage-devices/
  44. https://access.redhat.com/articles/16631
  45. https://lonesysadmin.net/2013/12/22/better-linux-disk-caching-performance-vm-dirty_ratio/
  46. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/storage_administration_guide/writebarrieronoff
  47. https://shapeshed.com/unix-exit-codes/
  48. https://man7.org/linux/man-pages/man1/strace.1.html
  49. https://man7.org/linux/man-pages/man2/syncfs.2.html
  50. https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/managing_file_systems/checking-and-repairing-a-file-system__managing-file-systems
  51. https://docs.docker.com/desktop/features/synchronized-file-sharing/
Add your contribution
Related Hubs
User Avatar
No comments yet.