Hubbry Logo
Data bufferData bufferMain
Open search
Data buffer
Community hub
Data buffer
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Data buffer
Data buffer
Data buffer
View on Wikipedia
from Wikipedia

In computer science, a data buffer (or just buffer) is a region of memory used to store data temporarily while it is being moved from one place to another. Typically, the data is stored in a buffer as it is retrieved from an input device (such as a microphone) or just before it is sent to an output device (such as speakers); however, a buffer may be used when data is moved between processes within a computer, comparable to buffers in telecommunication. Buffers can be implemented in a fixed memory location in hardware or by using a virtual data buffer in software that points at a location in the physical memory.

In all cases, the data stored in a data buffer is stored on a physical storage medium. The majority of buffers are implemented in software, which typically use RAM to store temporary data because of its much faster access time when compared with hard disk drives. Buffers are typically used when there is a difference between the rate at which data is received and the rate at which it can be processed, or in the case that these rates are variable, for example in a printer spooler or in online video streaming. In a distributed computing environment, data buffers are often implemented in the form of burst buffers, which provides distributed buffering services.

A buffer often adjusts timing by implementing a queue (or FIFO) algorithm in memory, simultaneously writing data into the queue at one rate and reading it at another rate.

Applications

[edit]

Buffers are often used in conjunction with I/O to hardware, such as disk drives, sending or receiving data to or from a network, or playing sound on a speaker. A line to a rollercoaster in an amusement park shares many similarities. People who ride the coaster come in at an unknown and often variable pace, but the roller coaster will be able to load people in bursts (as a coaster arrives and is loaded). The queue area acts as a buffer—a temporary space where those wishing to ride wait until the ride is available. Buffers are usually used in a FIFO (first in, first out) method, outputting data in the order it arrived.

Buffers can increase application performance by allowing synchronous operations such as file reads or writes to complete quickly instead of blocking while waiting for hardware interrupts to access a physical disk subsystem; instead, an operating system can immediately return a successful result from an API call, allowing an application to continue processing while the kernel completes the disk operation in the background. Further benefits can be achieved if the application is reading or writing small blocks of data that do not correspond to the block size of the disk subsystem, which allows a buffer to be used to aggregate many smaller read or write operations into block sizes that are more efficient for the disk subsystem, or in the case of a read, sometimes to completely avoid having to physically access a disk.

Telecommunication buffer

[edit]

A buffer routine or storage medium used in telecommunications compensates for a difference in rate of flow of data or time of occurrence of events when data is transferred from one device to another.

Buffers are used for many purposes, including:

  • Interconnecting two digital circuits operating at different rates.
  • Holding data for later use.
  • Allowing timing corrections to be made on a data stream.
  • Collecting binary data bits into groups that can then be operated on as a unit.
  • Delaying the transit time of a signal in order to allow other operations to occur.

Examples

[edit]

History

[edit]

An early mention of a print buffer is the "Outscriber" devised by image processing pioneer Russel A. Kirsch for the SEAC computer in 1952:[4]

One of the most important problems in the design of automatic digital computers is that of getting the calculated results out of the machine rapidly enough to avoid delaying the further progress of the calculations. In many of the problems to which a general-purpose computer is applied the amount of output data is relatively big — so big that serious inefficiency would result from forcing the computer to wait for these data to be typed on existing printing devices. This difficulty has been solved in the SEAC by providing magnetic recording devices as output units. These devices are able to receive information from the machine at rates up to 100 times as fast as an electric typewriter can be operated. Thus, better efficiency is achieved in recording the output data; transcription can be made later from the magnetic recording device to a printing device without tying up the main computer.

See also

[edit]

References

[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Data buffer

View on Grokipedia
from Grokipedia
A data buffer is a temporary of physical used to hold data during transfer between components in a computer system, compensating for differences in processing speeds or data transfer rates between devices such as hardware and the . This storage mechanism ensures smooth data flow by allowing faster components to proceed without waiting for slower ones, preventing bottlenecks in operations like reading from disks or transmitting over networks. In operating systems, data buffers are integral to I/O management, where they address mismatches in device transfer sizes and enable efficient handling of asynchronous operations. Common implementations include single buffering, which uses one buffer for sequential data staging; double buffering, employing two alternating buffers to overlap and I/O for improved throughput, as seen in rendering; and circular buffering, a queue-like structure of multiple buffers that cycles continuously for like audio or video. Buffers also play critical roles in networking, where they fragment large messages into packets for transmission and reassemble them at the destination, and in process synchronization, such as the producer-consumer problem, to coordinate data access between threads. Beyond I/O, data buffers appear in database management systems as part of buffer pools—collections of fixed-size frames that cache disk blocks to minimize physical reads. While buffers enhance , improper management can lead to issues like overflows, where excess data corrupts adjacent , though modern systems mitigate this through bounds checking and secure coding practices. Overall, data buffers form a foundational element in , enabling reliable and efficient data handling across software and hardware layers.

Fundamentals

Definition

A data buffer is a region of physical or virtual memory that serves as temporary storage for data during its transfer between two locations, devices, or processes, primarily to compensate for differences in data flow rates, event timing, or data handling capacities between the involved components. This mechanism allows the source and destination to operate at their optimal speeds without synchronization issues arising from mismatched rates or latency. Key attributes of data buffers include their size, which can be fixed (pre-allocated to a specific capacity) or variable (dynamically adjustable based on needs), and their storage medium, which is typically volatile such as RAM for high-speed operations but can be non-volatile like disk for longer-term holding in certain scenarios. Buffers also support various access models, ranging from single-producer/single-consumer patterns in simple producer-consumer setups to multi-producer/multi-consumer configurations in more complex concurrent environments. These attributes enable buffers to adapt to diverse system requirements while maintaining data integrity during transit. Unlike caches, which are optimized for repeated, frequent access to based on locality principles to reduce average access time, data buffers emphasize transient holding specifically for or streaming operations without inherent optimization for reuse. Similarly, while queues are data structures that enforce a particular ordering (typically first-in, first-out), data buffers do not inherently impose such ordering unless explicitly designed to do so, focusing instead on raw temporary storage capacity.

Purpose and Characteristics

Data buffers primarily serve to compensate for discrepancies in processing speeds between data producers and consumers, such as between a fast CPU and slower disk I/O operations, allowing the faster component to continue working without waiting. They also smooth out bursty data flows by temporarily holding variable-rate inputs, preventing disruptions in continuous processing pipelines. Additionally, buffers reduce blocking in concurrent systems by decoupling producer and consumer activities, enabling asynchronous operations that minimize idle time. Key characteristics of data buffers include their temporality, as data within them is typically overwritten or discarded once consumed, distinguishing them from persistent storage. Buffer sizes are determined based on factors like block transfer sizes from devices or requirements for hiding latency, often ranging from small units like 4 KB for file copies to larger allocations such as 4 MB for disk caches to optimize efficiency. In terms of performance impact, buffering decreases the frequency of context switches and I/O interruptions, thereby enhancing overall system responsiveness. The benefits of buffers encompass improved throughput, as seen in disk caching scenarios achieving rates up to 24.4 MB/sec compared to unbuffered access, and reduced latency in data pipelines through techniques like read-ahead buffering. They also provide in data streams by maintaining temporary copies that support recovery from transient failures without permanent loss. In the basic producer-consumer model, a deposits into the buffer while the retrieves it, coordinated by primitives such as semaphores to manage access and avoid conflicts.

Types

Linear Buffers

A linear buffer consists of a contiguous block of that is accessed sequentially from the beginning to the end, making it suitable for one-time transfers where elements are written and read in a straight-line order without looping back. This structure relies on a fixed-size allocation, typically implemented as an or similar primitive, to hold temporary during operations like reading from or writing to storage devices. The mechanics of a linear buffer involve head and tail pointers that advance linearly as is enqueued or dequeued; the tail pointer tracks the position for inserting new , while the head pointer indicates the location of the next item to be removed. When the buffer becomes full, further writes may result in unless the buffer is reset by moving pointers back to the start or reallocated to a larger contiguous region; similarly, upon emptying, the pointers reach the end and require reinitialization for reuse. Buffers like this generally facilitate rate matching between data producers and consumers, such as in device I/O where transfer speeds differ. Linear buffers find unique application in of files, where large datasets are loaded sequentially into memory for ordered processing without subsequent reuse, or in simple I/O that handle discrete, non-recurring transfers like reading configuration files. In these scenarios, the sequential nature ensures straightforward handling of in a single pass, avoiding the complexity of more . One key advantage of linear buffers is their simplicity in implementation, requiring only basic pointer arithmetic and contiguous allocation, which minimizes complexity and effort. They also impose low runtime overhead for short-lived operations, as there is no need for or additional logic to manage wrapping. Despite these benefits, linear buffers exhibit limitations in efficiency for continuous data streams, as reaching the end necessitates frequent resets or reallocations, potentially causing performance bottlenecks through repeated operations or temporary data halts. This makes them less ideal for scenarios demanding persistent, high-throughput data flow without interruptions.

Circular Buffers

A , also known as a ring buffer, is a fixed-size that uses a single as if it were connected end-to-end, enabling read and write pointers to wrap around to the beginning upon reaching the end. This FIFO-oriented design facilitates the continuous handling of data streams by overwriting the oldest entries once the buffer is full, without requiring data relocation or buffer resizing. The mechanics of a circular buffer rely on two pointers—one for the write position () and one for the read position (head)—managed through modulo arithmetic to compute effective indices within the fixed . For instance, a write operation places at buffer[(tail % size)] = [data](/page/Data), incrementing tail afterward, while reads use buffer[(head % size)] before advancing head. To distinguish a full buffer from an empty one (where both pointers coincide), a common approach reserves one slot unused, yielding an effective capacity of size - 1; the buffer is empty when head == tail and full when (tail + 1) % size == head. This structure offers constant-time O(1) operations for insertion and removal, eliminating the need to shift elements as in linear buffers, which enhances efficiency for like audio queues. Unlike linear buffers that cease operation upon filling and require reallocation, circular buffers support ongoing reuse through wrapping, optimizing memory in resource-constrained environments. Circular buffers emerged as an efficient queueing mechanism in early systems, with the concept documented in Donald Knuth's (Volume 1), and remain prevalent in embedded devices for handling asynchronous transfers.

Double Buffers

Double buffering, also known as ping-pong buffering, is a technique that employs two distinct buffer regions to facilitate seamless transfer in producer-consumer scenarios, where one buffer is filled with while the other is simultaneously consumed or processed. This approach allows the producer (e.g., a source like a disk or input ) to write to the inactive buffer without interrupting the consumer (e.g., a unit or ), ensuring continuous operation. The mechanics of double buffering involve alternating between the two buffers through a swap operation, typically implemented via pointer exchange or status flags that indicate which buffer is active for reading or writing. This swap occurs at designated synchronization points to prevent , often employing atomic operations or interrupt-driven queues to coordinate access between producers and consumers, particularly in environments where processing times for filling and consuming vary significantly. By decoupling these operations, double buffering maintains a steady data flow, avoiding stalls that would arise from waiting for one buffer to complete its cycle. A key advantage of double buffering is its ability to hide the latency of data preparation or transfer behind ongoing consumption, effectively doubling the throughput in pipelined systems by overlapping I/O and computation. This latency masking is especially beneficial in scenarios with mismatched speeds between data sources and sinks. It is commonly applied in graphics rendering, where front and back buffers alternate to display complete frames without flicker—rendering occurs in the back buffer while the front buffer is shown, followed by a swap. Similarly, in disk I/O operations, it enables efficient block transfers by allowing one buffer to receive from storage while the other is processed by the CPU.

Management and Implementation

Allocation Strategies

Allocation strategies for data buffers determine how memory is assigned to these temporary storage areas, balancing efficiency, predictability, and adaptability to varying workloads. Three primary approaches are employed: static, dynamic, and pool allocation. Static allocation assigns a fixed-size block of at , which remains constant throughout program execution. This method is particularly suited for embedded systems where memory constraints are tight and requirements are predictable, as it eliminates runtime overhead and ensures deterministic performance. In C, this can be implemented using fixed arrays declared globally or locally, while in C++, it involves stack-based variables or static members. However, static allocation lacks flexibility for handling variable data rates in buffers, potentially leading to wasted space if the fixed size exceeds actual needs. Dynamic allocation, in contrast, requests memory at runtime using functions like malloc and free in C or new and delete in C++, allowing buffers to resize based on immediate demands. This approach is ideal for applications with fluctuating data volumes, such as general-purpose computing tasks, but it introduces overhead from allocation calls and potential delays due to heap management. Trade-offs include higher memory footprint from metadata and the risk of exhaustion under heavy loads, though it provides greater adaptability than static methods. Pool allocation pre-allocates a collection of fixed-size blocks from which buffers can be quickly drawn and returned, minimizing repeated heap interactions and reducing fragmentation. This strategy reuses from dedicated pools tailored to specific buffer sizes, enhancing performance in high-frequency allocation scenarios like object caching. Key considerations in all strategies include managing to avoid excess usage and preventing fragmentation; for instance, buddy systems allocate power-of-two sized blocks to merge adjacent free spaces efficiently, thereby mitigating external fragmentation. Additionally, aligning buffers to hardware boundaries—such as 16-byte or 64-byte multiples—optimizes access speeds by enabling efficient SIMD instructions and DMA transfers. In operating system kernels, slab allocators extend pool concepts by maintaining caches of initialized objects, including buffer pools for network packets, to accelerate allocation and reduce initialization costs. Overall, static allocation offers predictability at the cost of inflexibility, while dynamic and pool methods provide but require careful management to prevent resource exhaustion.

Overflow and Error Handling

Buffer overflow occurs when a program attempts to write more data to a buffer than its allocated capacity, potentially overwriting adjacent locations and leading to or vulnerabilities. This condition arises from insufficient bounds checking during data input operations, such as in functions that copy strings or arrays without verifying lengths. A prominent type of buffer overflow is the stack-based buffer overflow, often exploited through techniques like stack smashing, where malicious code is injected to alter and execute arbitrary instructions. Buffer underflow, conversely, happens when a program reads from a buffer that lacks sufficient , typically because is consumed faster than it is produced, resulting in attempts to access uninitialized or invalid memory. This can cause program crashes, , or in some cases, issues if it exposes sensitive beyond the buffer's intended bounds. Common handling strategies for overflows include bounds checking to validate input sizes before writing, truncation of excess data to fit the buffer, blocking the operation until space is available, or dropping incoming data to prevent corruption. In software implementations, assertions can halt execution upon detecting an overflow, while exceptions in languages like C++ or Java provide a mechanism to signal and recover from the error gracefully. For underflows, similar checks ensure sufficient data exists before reading, often triggering waits or error returns. A notable real-world example of a buffer over-read vulnerability is the bug (CVE-2014-0160), disclosed in 2014, which affected the cryptography library and allowed attackers to read up to 64 kilobytes of server memory per request due to a missing length validation in the heartbeat extension. This vulnerability compromised private keys, passwords, and session cookies across numerous systems, highlighting the risks of unchecked buffer operations in widely used software. Mitigation techniques include stack canaries, which insert random sentinel values between the buffer and critical stack data like return addresses; any overflow corrupts the canary, detectable before function return. (ASLR) randomizes memory addresses to make exploitation harder by complicating return-to-libc or similar attacks. These defenses, often enabled by compilers like GCC, reduce vulnerability without fully eliminating the need for secure coding practices. In circular buffers, overflow handling typically follows a circular queue policy where, upon reaching capacity, new data overwrites the oldest entries, ensuring continuous operation without halting the producer. This approach prioritizes recent data retention, common in real-time systems like audio processing, but requires consumers to track valid data ranges to avoid reading stale information. Implementing overflow checks, such as bounds validation, incurs a overhead due to runtime verifications, though optimizations like compiler-assisted checks can mitigate this impact.

Applications

In Computing Systems

In operating systems, data buffers are essential for managing (I/O) operations in file systems, where they cache data in to bridge the gap between fast processors and slower storage devices. For example, employs a to store file contents temporarily in RAM, enabling subsequent reads and writes to be served directly from rather than accessing the disk each time, which significantly improves application . This caching mechanism aligns I/O operations with the file's , typically performing transfers at the page level to minimize overhead. Buffers also support (IPC) by providing temporary storage for exchange between processes. offer a unidirectional channel where the kernel buffers in a fixed-size queue, ensuring atomic writes up to the PIPE_BUF limit of 4096 bytes to prevent interleaving in concurrent scenarios. , in contrast, creates a directly accessible buffer region in the that multiple processes can map and use for high-speed without repeated kernel copies. Disk buffering specifically aggregates small, scattered writes into larger contiguous blocks before flushing to storage, which reduces the number of mechanical seeks on hard disk drives (HDDs) and enhances write efficiency. In CPU , instruction buffers—implemented as registers between stages—hold fetched instructions to facilitate overlapping execution across multiple pipeline phases, thereby increasing instruction throughput. In systems, buffers like the interact with swap space by allowing less frequently used pages to be swapped out to disk when physical RAM is under pressure, freeing for active processes while preserving . Double buffering further aids concurrency in multithreaded applications, where two buffers alternate roles—one for writing by a producer thread and one for reading by a consumer—reducing contention and enabling parallel operations without locks. Overall, buffering mitigates mechanical delays in HDDs, such as seek times averaging several milliseconds, by batching and prefetching in standard 4KB blocks that match common and page sizes. In shared environments, buffer overflows pose risks of if bounds are not enforced.

In Networking

In networking, data buffers play a critical role in managing the transmission of packets across interconnected devices, particularly in handling variability in arrival times and rates to prevent and ensure reliable delivery. Packet buffers in routers temporarily store incoming packets when output links are congested, allowing for orderly forwarding and mitigating immediate drops. For instance, in TCP implementations, receive windows utilize buffers to hold acknowledged data segments, enabling the receiver to control the flow from the sender based on available memory. Similarly, in (VoIP) systems compensate for packet delay variations by queuing arriving packets and releasing them at a steady rate, thus smoothing out network-induced jitter to maintain audio quality without perceptible disruptions. At the level, buffering occurs prominently in the OSI model's layer 2 () and layer 3 (network) to support congestion control mechanisms. Layer 2 switches and bridges use buffers to manage frame queuing during link-layer retransmissions, while layer 3 routers employ them for IP packet handling amid traffic bursts. Congestion control in these layers often involves (AQM) techniques to signal impending via packet drops or markings, preventing widespread network instability. Queueing disciplines further refine this process; for example, First-In-First-Out (FIFO) treats all packets equally in a single queue, suitable for simple environments, whereas priority queueing assigns higher precedence to latency-sensitive traffic, as implemented in routers to favor VoIP or signaling packets over bulk data. TCP's relies on buffers to implement flow control, where the receiver advertises its available buffer space in size announcements, limiting the sender's unacknowledged data to avoid overwhelming the endpoint. This mechanism dynamically adjusts transmission rates based on buffer occupancy, ensuring end-to-end reliability without explicit . However, excessive buffering in network devices has led to the problem, where large queues accumulate packets during congestion, inflating latency—sometimes to seconds—despite high throughput, a issue prominently addressed in networking communities starting around through AQM algorithms like PIE (Proportional Integral controller Enhanced). Deep packet inspection (DPI) processes, used in firewalls and intrusion detection systems, demand substantial buffer capacities to reassemble and analyze fragmented or out-of-order packet streams for signatures, enabling threat detection without dropping legitimate traffic. In contrast, networks prioritize low-latency applications by employing smaller, more efficient buffers with dynamic sizing at the (RLC) layer, often splitting responsibilities between RLC and (PDCP) layers to minimize queuing delays while supporting ultra-reliable low-latency communications (URLLC). Circular buffers are occasionally referenced in packet queue implementations for their efficiency in handling continuous streams without frequent reallocations.

In Multimedia Processing

In multimedia processing, data buffers play a crucial role in managing time-sensitive media streams, such as audio, video, and graphics, to ensure smooth playback and rendering without interruptions. Frame buffers in graphics processing units (GPUs) store data for rendered images, allowing the GPU to compose and update visual content efficiently before displaying it on screen. Similarly, audio buffers in sound cards hold samples of data, preventing glitches by compensating for variations in processing speed between the CPU and audio hardware. A key application is double buffering in graphics APIs like , where two buffers—one front (displayed) and one back (rendered to)—alternate to eliminate during updates. This technique synchronizes rendering with the display , producing tear-free visuals in real-time applications such as games and animations. In video streaming, adaptive buffering dynamically adjusts buffer sizes based on available bandwidth; for instance, employs algorithms that monitor network conditions to scale video quality and buffer depth, minimizing rebuffering events while maintaining continuous playback. In processing, buffers typically hold 512 samples at a 44.1 kHz sample rate, corresponding to approximately 11.6 milliseconds of audio, which balances low latency with CPU in digital audio workstations. If the buffer underruns—meaning it empties before new data arrives—audible artifacts like pops or clicks occur due to incomplete sample delivery to the . Modern advancements include AI-accelerated buffering in video codecs like , which optimizes real-time transcoding by predicting and pre-fetching data segments to reduce latency in scenarios. These buffers smooth rate variations between encoding, transmission, and decoding, enabling high-quality playback even under fluctuating conditions.

Historical Development

Origins in Early

The concept of data buffering in arose during the and 1950s as electronic computers transitioned from experimental machines to practical systems, addressing the significant speed disparities between rapid central processing units (CPUs) and slower electromechanical peripherals such as punch card readers and drives. These early buffers functioned as temporary storage areas to hold data from mechanical input devices, mitigating delays caused by their physical limitations—punch card readers, for instance, processed cards at rates of around 100 to 200 per minute, far slower than emerging CPU cycle times. By staging data in memory, buffering allowed CPUs to proceed with computations without idling, marking a foundational technique for efficient (I/O) management in pre-operating system era machines. A pivotal implementation occurred with the , the first commercial general-purpose electronic computer delivered in 1951, which incorporated dedicated tape buffers for data staging and overlapped I/O operations. The system featured two 60-word buffers—one for input and one for output—integrated with its UNISERVO tape drives, enabling asynchronous data transfer from while the CPU executed instructions. This design represented the earliest commercial example of buffered I/O, allowing the UNIVAC I to handle business and scientific workloads by decoupling tape read/write speeds (up to 7,200 characters per second) from the CPU's processing rate, thus reducing overall job turnaround times in environments reliant on offline data preparation. The UNIVAC I's buffering approach was essential for its role in high-profile applications, such as the 1952 U.S. presidential election prediction. Throughout the 1950s, batch processing systems further entrenched buffering practices, using emerging core memory as dedicated buffers to manage sequential job execution and data flow. These systems, common in installations like those employing or 650 computers, grouped programs and data into batches processed offline via punched cards or tape, with core memory—tiny magnetic rings invented around 1951—serving as high-speed buffers to temporarily hold input data and intermediate results. This buffered approach minimized CPU downtime during I/O waits, supporting the era's unit-record processing paradigms where entire decks of cards were read into core before computation began. Core memory's non-volatile nature and access times under 10 microseconds made it ideal for such buffering, enabling efficient handling of business data like or in resource-constrained environments. The terminology "buffer" itself was adapted from , where it described circuits introduced in the 1920s to match impedances between signal sources and loads, preventing reflections and signal degradation in early radio and systems. By the , this analogy extended to , portraying areas as "cushions" that isolated fast computational elements from slower storage mechanisms, a conceptual shift that underscored buffering's role in system stability.

Evolution in Modern Systems

In the and , advancements in operating systems integrated data buffers more deeply into kernel architectures to support efficient operations. Multics, developed starting in 1965, influenced subsequent systems by employing sophisticated buffering mechanisms for its hierarchical , enabling high-performance multitasking and data access. Unix, emerging in the early , adopted similar kernel buffers to manage file blocks and inodes, chaining them to optimize I/O throughput in environments. Paralleling these developments, the in the utilized packet buffers to mitigate and contention in early packet-switched networks, where preempting buffers was a key design consideration for reliability. The 1980s and 1990s saw data buffers evolve with the rise of graphical user interfaces and networked computing. Early GUI systems, such as the introduced in 1973, pioneered frame buffering for bit-mapped displays, laying groundwork for double buffering techniques to eliminate flicker during rendering, which became widespread in commercial GUIs like Windows by the late 1980s. In networking, the TCP/IP protocol suite, standardized in the 1980s, incorporated buffer management for flow control and congestion avoidance, addressing challenges like packet reassembly and queue delays in growing internetworks. Circular buffers, an efficient ring-based structure for continuous data streams, became widely adopted in systems during the 1980s for handling . From the onward, optimizations focused on reducing overhead and enhancing performance in storage and I/O. introduced zero-copy buffering with the splice() in 2006, allowing direct data transfer between kernel pipes without user-kernel copies, significantly improving throughput for file and network operations. Solid-state drives (SSDs), proliferating in the mid-, incorporated DRAM buffers to cache writes and support , distributing erase cycles evenly across flash cells to extend device lifespan. Post-2010 developments addressed scalability, security, and latency in distributed and consumer environments. Cloud storage systems like AWS S3 employed multipart upload buffering to handle large objects by dividing them into parts, enabling parallel transfers and fault-tolerant . Emerging AI techniques began applying for predictive buffer allocation, using models to anticipate I/O patterns and dynamically adjust sizes in workloads. In home networking, —excessive queuing delays in routers—drove innovations like from 2012, which drops packets based on delay thresholds to mitigate latency spikes without sacrificing throughput. The 2014 Heartbleed vulnerability, a buffer over-read in , heightened focus on secure buffer handling, prompting widespread audits and mitigations in cryptographic libraries to prevent memory leaks.

References

  1. https://faculty.eng.fau.edu/tsorgent/COP4610/COP4610M15Slides.pdf
  2. https://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/13_IOSystems.html
  3. https://courses.cs.duke.edu/spring03/cps216/lectures/17-buf.pdf
  4. https://dsp.stackexchange.com/questions/42735/what-is-the-difference-between-buffering-and-queuing
  5. https://csc-knu.github.io/sys-prog/books/Andrew%20S.%20Tanenbaum%20-%20Modern%20Operating%20Systems.pdf
  6. https://sigops.org/s/conferences/sosp/2013/papers/p423-zaharia.pdf
  7. https://www.brainkart.com/article/Linear-buffers_7736/
  8. https://www.geeksforgeeks.org/operating-systems/i-o-buffering-and-its-various-techniques/
  9. https://www.sciencedirect.com/topics/computer-science/circular-buffer
  10. https://www.geeksforgeeks.org/operating-systems/buffering-in-os/
  11. https://www.embedded.com/ring-buffer-basics/
  12. http://www.cs.emory.edu/~cheung/Courses/171/Syllabus/10-queue/13.html
  13. https://cs.brown.edu/courses/csci0300/2022/notes/l22.html
  14. https://www.geeksforgeeks.org/dsa/advantages-of-circular-queue-over-linear-queue/
  15. https://www.pcmag.com/encyclopedia/term/double-buffering
  16. https://homepage.cs.uiowa.edu/~dwjones/opsys/fall13/notes/12.shtml
  17. https://web.eecs.utk.edu/~bvanderz/teaching/guiSp11/notes/display-mgt/
  18. https://www.sciencedirect.com/topics/computer-science/dynamic-memory-allocation
  19. https://barrgroup.com/blog/how-allocate-dynamic-memory-safely
  20. https://www.geeksforgeeks.org/c/static-and-dynamic-memory-allocation-in-c/
  21. https://dspace.mit.edu/bitstream/handle/1721.1/30228/CS007.pdf?sequence=1&isAllowed=y
  22. https://erikdemaine.org/papers/Buddy_ActaInf/paper.pdf
  23. https://developer.ibm.com/articles/pa-dalign/
  24. https://people.eecs.berkeley.edu/~kubitron/courses/cs194-24-S14/hand-outs/bonwick_slab.pdf
  25. https://www.kernel.org/doc/gorman/html/understand/understand011.html
  26. https://owasp.org/www-community/vulnerabilities/Buffer_Overflow
  27. https://www.techtarget.com/searchsecurity/definition/buffer-overflow
  28. https://www.fortinet.com/resources/cyberglossary/buffer-overflow
  29. https://www.techtarget.com/searchsecurity/definition/buffer-underflow
  30. https://cwe.mitre.org/data/definitions/124.html
  31. https://www.portnox.com/cybersecurity-101/what-is-a-buffer-overflow/
  32. https://www.blackduck.com/blog/detect-prevent-and-mitigate-buffer-overflow-attacks.html
  33. https://www.heartbleed.com/
  34. https://www.cisa.gov/news-events/alerts/2014/04/08/openssl-heartbleed-vulnerability-cve-2014-0160
  35. https://www.sans.org/blog/stack-canaries-gingerly-sidestepping-the-cage
  36. https://engineering.purdue.edu/~vijay/papers/2005/bo-cacm.pdf
  37. https://www.baeldung.com/cs/circular-buffer
  38. https://embeddedartistry.com/blog/2017/05/17/creating-a-circular-buffer-in-c-and-c/
  39. https://www.sei.cmu.edu/blog/performance-of-compiler-assisted-memory-safety-checking/
  40. https://docs.kernel.org/filesystems/iomap/operations.html
  41. https://www.kernel.org/doc/ols/2006/ols2006v2-pages-73-78.pdf
  42. https://opensource.com/article/19/4/interprocess-communication-linux-channels
  43. https://docs.oracle.com/cd/E26502_01/html/E35299/svipc-posixipc.html
  44. https://www.usenix.org/event/usenix09/tech/full_papers/vandebogart/vandebogart.pdf
  45. https://cs.stanford.edu/people/eroberts/courses/soco/projects/risc/pipelining/index.html
  46. https://www.kernel.org/doc/gorman/html/understand/understand014.html
  47. https://www.embeddedrelated.com/showarticle/1360.php
  48. https://www.eng.utah.edu/~cs5460/slides/Lecture19.pdf
  49. https://www.rfc-editor.org/rfc/rfc2309.html
  50. https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/qos_conmgt/configuration/12-4t/qos-conmgt-12-4t-book.pdf
  51. https://datatracker.ietf.org/doc/html/draft-ietf-aqm-pie-01
  52. https://ieeexplore.ieee.org/document/8377190
  53. https://developer.samsung.com/galaxy-gamedev/resources/articles/gpu-framebuffer.html
  54. https://www.presonus.com/blogs/technical/digital-audio-latency-explained
  55. https://www.sweetwater.com/sweetcare/articles/which-buffer-size-setting-should-i-use-in-my-daw/
  56. https://dev.epicgames.com/community/learning/knowledge-base/xaRz/unreal-engine-best-practices-for-avoiding-underruns-in-custom-audio-code
  57. https://arxiv.org/pdf/2009.14165
  58. https://www.computerhistory.org/storageengine/tape-unit-developed-for-data-storage/
  59. http://www.bitsavers.org/pdf/univac/univac1/UNIVAC1_Programming_1959.pdf
  60. https://www.storagenewsletter.com/2018/06/18/history-1951-univac-i/
  61. https://shs.hal.science/halshs-01541602v2/document
  62. https://www.linfo.org/buffer.html
  63. https://resources.altium.com/p/mysterious-50-ohm-impedance-where-it-came-and-why-we-use-it
  64. https://dl.acm.org/doi/pdf/10.1145/989.990
  65. https://dl.acm.org/doi/pdf/10.1145/800213.806522
  66. https://dl.acm.org/doi/pdf/10.1145/1499949.1499981
  67. https://xeroxalto.computerhistory.org/_cd8_/altodocs/.fieldguide.press%212.pdf
  68. https://dl.acm.org/doi/pdf/10.1145/52325.52336
  69. https://lwn.net/Articles/178199/
  70. https://ieeexplore.ieee.org/document/10171179/
  71. https://docs.aws.amazon.com/code-library/latest/ug/s3_example_s3_Scenario_UsingLargeFiles_section.html
  72. https://arxiv.org/pdf/2404.10386
  73. https://queue.acm.org/detail.cfm?id=2209336
  74. https://cacm.acm.org/research/analysis-of-ssl-certificate-reissues-and-revocations-in-the-wake-of-heartbleed/
Add your contribution
Related Hubs
Contribute something
User Avatar
No comments yet.