Recent from talks
Nothing was collected or created yet.
Concurrency pattern
View on WikipediaA concurrency pattern is a software design pattern that supports concurrent processing.
Examples
[edit]Examples include:
See also
[edit]References
[edit]- ^ Douglas C. Schmidt, Michael Stal, Hans Rohnert, Frank Buschmann "Pattern-Oriented Software Architecture, Volume 2, Patterns for Concurrent and Networked Objects", Wiley, 2000
- ^ R. Greg Lavender, Douglas C. Schmidt (1995). "Active Object" (PDF). Archived from the original (PDF) on 2010-06-15. Retrieved 2010-06-17.
External links
[edit]- ScaleConf Presentation about concurrency patterns
- GopherCon Rethinking Classical Concurrency Patterns slides
- GoWiki: Learn Concurrency
Recordings about concurrency patterns from Software Engineering Radio:
Concurrency pattern
View on GrokipediaOverview
Definition
Concurrency patterns are reusable solutions to commonly occurring problems in concurrent programming, where multiple threads or processes execute simultaneously to coordinate access to shared resources and avoid issues such as race conditions and deadlocks. These patterns focus on efficient resource sharing, synchronization, and communication in multi-threaded environments, providing high-level abstractions that handle shared data access without delving into low-level implementation details.[1] Key characteristics of concurrency patterns include their emphasis on partitioning work among threads, managing inter-thread communication via mechanisms like queues, and ensuring thread-safe operations through synchronization primitives such as locks. Unlike general design patterns in sequential programming, which primarily address object structure and behavior in single-threaded contexts, concurrency patterns incorporate parallelism-specific elements to resolve issues arising from simultaneous execution. In relation to the Gang of Four (GoF) design patterns, which are oriented toward sequential object-oriented software, concurrency patterns extend these concepts by integrating concurrency controls like monitors and reactors to handle multi-threaded scenarios effectively. Concurrency patterns emerged in the 1990s amid the growing adoption of multi-threaded and networked systems, driven by demands for scalable applications in client-server architectures, and were formalized in seminal works such as Pattern-Oriented Software Architecture, Volume 2 published in 2000.Importance and Applications
Concurrency patterns play a crucial role in modern software engineering by enhancing scalability, responsiveness, and resource utilization in multi-core systems, allowing applications to effectively leverage parallel processing capabilities for improved performance.[5] These patterns enable developers to structure concurrent code in ways that minimize overhead and maximize throughput, particularly as hardware evolves toward greater parallelism.[6] By providing proven templates for handling shared resources and task coordination, they reduce the incidence of bugs arising from improper concurrency management, such as race conditions and deadlocks, thereby improving overall software reliability.[7] In practical applications, concurrency patterns are essential for web servers, where they facilitate the simultaneous handling of multiple client requests to maintain low latency and high availability under load.[5] In databases, they support transaction processing by ensuring consistent access to shared data across concurrent operations, preventing anomalies in high-throughput environments.[8] Real-time systems, such as embedded software in robotics and control applications, rely on these patterns to achieve predictable timing and coordination among parallel components.[7] Similarly, in distributed computing and cloud services, concurrency patterns enable efficient resource sharing and fault-tolerant operations across networked nodes.[9] The impact of concurrency patterns on software engineering is profound, as they empower the creation of high-performance applications that can achieve significant speedups on parallel hardware, guided by principles like Amdahl's Law, which quantifies the potential benefits of parallelization. Amdahl's Law illustrates that the maximum speedup for a program with a parallelizable fraction executed on processors is given by: [10] This formula underscores the necessity of concurrency patterns to maximize and minimize sequential bottlenecks, thereby realizing substantial performance gains in parallel sections.[5] Furthermore, these patterns address key challenges in concurrent systems, such as thread starvation—where certain tasks are indefinitely delayed—and the overhead from frequent context switching, which can degrade efficiency by consuming CPU cycles on thread management rather than computation. By encapsulating best practices for synchronization and resource allocation, concurrency patterns mitigate these issues, promoting more robust and efficient concurrent designs.[6] For instance, patterns like producer-consumer are applied in queue-based systems to balance workload distribution and prevent bottlenecks.[6]Fundamental Concepts
Concurrency versus Parallelism
Concurrency refers to the ability of a system to manage multiple tasks or computations that can progress independently or in an interleaved manner, without requiring simultaneous execution. This involves structuring programs as compositions of independently executing processes, allowing for the handling of multiple activities "at once" through techniques like time-sharing or asynchronous operations.[11] In contrast, parallelism involves the simultaneous execution of multiple computations, typically leveraging multiple processing units such as CPU cores to perform tasks concurrently in real time.[12] A classic example of concurrency without parallelism occurs on a single-core processor, where the operating system schedules multiple threads by rapidly switching between them, enabling multitasking such as a web server responding to several client requests sequentially but appearing simultaneous to users.[12] Parallelism, however, demands hardware support like multi-core CPUs; for instance, computing a vector dot product can be divided across multiple cores to execute parts of the calculation simultaneously, accelerating the overall result.[11] These distinctions highlight that concurrency focuses on program design for coordination, while parallelism emphasizes execution efficiency through resource utilization. The phrase "Concurrency is not parallelism" encapsulates this separation, originating from a 2012 talk by Rob Pike, emphasizing that concurrency addresses the structural challenges of dealing with multiple tasks, whereas parallelism is merely one possible outcome of executing those tasks on capable hardware.[11] This implies that concurrency patterns remain crucial for task coordination even in non-parallel environments, such as single-threaded applications with asynchronous I/O, to prevent issues like race conditions or inefficient resource use.[11] In terms of measurement, concurrency often prioritizes throughput, defined as the number of tasks completed per unit time, by enabling the system to handle a higher volume of work through interleaving or multiplexing.[11] Parallelism, on the other hand, typically aims to reduce latency, the time required to complete an individual task, by distributing workload across processors to shorten execution duration for compute-intensive operations.[12]Synchronization Primitives
Synchronization primitives are the foundational mechanisms in concurrent programming that enable threads or processes to coordinate access to shared resources, preventing issues such as race conditions and ensuring data consistency. These primitives provide low-level synchronization capabilities, often implemented through hardware instructions or operating system support, and serve as building blocks for higher-level concurrency patterns. By managing contention and signaling, they facilitate safe parallel execution in multi-threaded environments. A mutex, or mutual exclusion lock, is a synchronization primitive that guarantees only one thread can access a critical section of code or a shared resource at a time, thereby enforcing exclusive access to prevent concurrent modifications. Mutexes are typically implemented using atomic operations, such as test-and-set instructions, or through operating system kernel calls that block contending threads. For instance, in POSIX threads (pthreads), a mutex is acquired via the pthread_mutex_lock function, which spins or blocks until the lock is available.[13] Semaphores extend the concept of mutual exclusion by providing a counting mechanism to control access to a pool of resources, allowing multiple threads to proceed up to a specified limit. Invented by Edsger W. Dijkstra in 1965, a semaphore is an integer variable that supports two atomic operations: P (wait or decrement) and V (signal or increment), ensuring that the value never goes negative.[14] A binary semaphore, initialized to 1, behaves like a mutex for exclusive access, while a counting semaphore, initialized to a value greater than 1, signals availability for producer-consumer scenarios by tracking resource counts.[15] Condition variables complement mutexes by allowing threads to wait efficiently for specific conditions to become true, rather than busy-waiting, thus reducing CPU overhead in concurrent systems. Introduced in C. A. R. Hoare's 1974 work on monitors, a condition variable supports three key operations: wait(), which releases the associated mutex and blocks the thread until signaled; signal(), which wakes one waiting thread; and broadcast(), which wakes all waiting threads.[16] These operations are always used in conjunction with a mutex to protect the condition check and ensure thread safety, as exemplified in pthreads via pthread_cond_wait.[17] Atomic operations provide hardware-supported primitives for lock-free synchronization, enabling threads to perform read-modify-write sequences without interruption. A prominent example is the compare-and-swap (CAS) operation, which atomically checks if a memory location holds an expected value and, if so, replaces it with a new value; otherwise, it fails without modification. Formally, CAS can be expressed as: This primitive, supported in hardware on architectures like x86 via the CMPXCHG instruction, forms the basis for non-blocking algorithms by allowing optimistic updates with retry loops. While locking primitives like mutexes and semaphores offer simplicity and correctness guarantees, they introduce overhead from context switches and potential contention under high load, leading to scalability limits in multi-core systems. In contrast, non-blocking alternatives using atomic operations like CAS avoid blocking but may incur retry costs due to contention, resulting in trade-offs where lock-free approaches excel in low-contention or high-parallelism scenarios but can underperform in others due to increased complexity and ABA problems. Empirical studies show that lock-free queues can achieve higher throughput than locking versions under heavy contention on multi-processor systems, often comparable to or better than sophisticated lock-based designs.[18][19]Synchronization Patterns
Producer-Consumer Pattern
The producer-consumer pattern is a fundamental synchronization mechanism in concurrent programming that decouples the production of data from its consumption, allowing producers and consumers to operate at different speeds without direct coordination. In this pattern, one or more producer threads generate data items and place them into a shared buffer, while one or more consumer threads retrieve and process those items from the buffer. The buffer serves as an intermediary that smooths out discrepancies in production and consumption rates, preventing producers from overwhelming the system or consumers from idling unnecessarily. This approach originated in early work on process synchronization, where it was used to illustrate the need for coordination in shared-resource environments.[20] The core structure of the pattern revolves around a bounded buffer, typically implemented as a fixed-size circular queue to avoid unbounded memory growth and potential overflows. Producers append items to the buffer only when space is available, and consumers remove items only when the buffer is not empty. To manage access and availability, the pattern employs synchronization primitives such as a mutex for mutual exclusion on the buffer and semaphores to track the number of empty and full slots. The empty semaphore is initialized to the buffer's capacity, the full semaphore to zero, and the mutex to one, ensuring atomic operations during insertion and removal. This setup handles speed differences by blocking producers when the buffer is full and consumers when it is empty, thereby maintaining system stability.[21][22] The components interact through a well-defined protocol. For the producer, the sequence involves waiting on the empty semaphore to ensure space, acquiring the mutex to access the buffer, adding the item, releasing the mutex, and signaling the full semaphore to notify consumers. The consumer follows a symmetric process: waiting on the full semaphore for an available item, acquiring the mutex, removing the item, releasing the mutex, and signaling the empty semaphore. This orchestration prevents race conditions and ensures that buffer indices (e.g., input and output pointers in a circular array) are updated safely. Here is the standard pseudocode for the bounded buffer implementation using semaphores: Producer:do {
wait(empty);
wait(mutex);
/* add item to buffer */
buffer[in] = next_produced;
in = (in + 1) % BUFFER_SIZE;
signal(mutex);
signal(full);
} while (true);
do {
wait(empty);
wait(mutex);
/* add item to buffer */
buffer[in] = next_produced;
in = (in + 1) % BUFFER_SIZE;
signal(mutex);
signal(full);
} while (true);
do {
wait(full);
wait(mutex);
/* remove item from buffer */
next_consumed = buffer[out];
out = (out + 1) % BUFFER_SIZE;
signal(mutex);
signal(empty);
/* consume the item */
} while (true);
do {
wait(full);
wait(mutex);
/* remove item from buffer */
next_consumed = buffer[out];
out = (out + 1) % BUFFER_SIZE;
signal(mutex);
signal(empty);
/* consume the item */
} while (true);
empty is initialized to BUFFER_SIZE, full to 0, mutex to 1, and BUFFER_SIZE is the fixed buffer capacity.[21][20]
Variations of the pattern include unbounded queues, which use a dynamically growing linked list instead of a fixed array, eliminating the full semaphore and the risk of overflow but requiring careful memory management to avoid exhaustion. This simplifies implementation for cases where production rates are unpredictable but increases complexity in multi-producer scenarios due to potential contention on the tail pointer. Another variation appears in message-passing systems, where the buffer is replaced by channels that facilitate direct communication between producers and consumers without shared memory, as in Communicating Sequential Processes (CSP), enabling distributed coordination.[23][24]
The pattern offers key benefits, including the prevention of busy-waiting through blocking semaphores, which allow threads to suspend execution until conditions are met, conserving CPU resources compared to polling loops. Additionally, by buffering data and enabling concurrent operation, it improves overall throughput in systems where producers and consumers have mismatched processing speeds, allowing the pipeline to sustain higher data flow rates without bottlenecks.[21][22]
Reader-Writer Pattern
The reader-writer pattern addresses the challenge of concurrent access to shared resources where multiple readers can access the data simultaneously without interference, but writers require exclusive access to prevent data corruption, while also mitigating the risk of writer starvation in scenarios dominated by read operations.[25] This pattern optimizes throughput by allowing concurrent reads, which is particularly beneficial when reads vastly outnumber writes, as a simple mutual exclusion lock would serialize all access and underutilize the resource.[25] Two primary variants exist: reader-preference, which prioritizes readers by allowing new readers to join immediately if no writer is active, potentially leading to delayed writers but minimizing reader wait times; and writer-preference, which gives priority to waiting writers by blocking new readers once a writer is queued, ensuring writers proceed sooner but risking reader delays.[25] However, the reader-preference variant can lead to writer starvation if new readers continue to arrive while a writer is waiting, as new readers can acquire the lock before the waiting writer, whereas writer-preference may indefinitely postpone readers if writers arrive continuously.[25] Both variants rely on synchronization primitives like semaphores or locks to enforce these policies.[26] Implementation typically involves a shared lock for readers, an exclusive lock for writers, a counter to track active readers, and condition variables or semaphores to manage waiting threads. In the reader-preference approach using semaphores, amutex semaphore protects the reader count, and a wrt (write) semaphore ensures writer exclusion; readers increment a readcount and acquire wrt only if they are the first, while writers solely acquire wrt.[25] The writer-preference variant adds additional semaphores to block incoming readers when writers are pending.[25]
Here is pseudocode for the reader-preference variant:
Reader Process:
P(mutex)
readcount++
if (readcount == 1) P(wrt)
V(mutex)
-- perform read operation --
P(mutex)
readcount--
if (readcount == 0) V(wrt)
V(mutex)
P(mutex)
readcount++
if (readcount == 1) P(wrt)
V(mutex)
-- perform read operation --
P(mutex)
readcount--
if (readcount == 0) V(wrt)
V(mutex)
P(wrt)
-- perform write operation --
V(wrt)
P(wrt)
-- perform write operation --
V(wrt)
mutex preventing race conditions on readcount.[25]
The pattern finds applications in databases, where query operations (reads) far outnumber updates (writes), allowing multiple transactions to read data simultaneously while serializing modifications; similarly, in file systems, it supports concurrent reads of file contents alongside exclusive writes to maintain integrity.[27]
