Adaptive bitrate streaming
View on Wikipedia


Adaptive bitrate streaming is a technique used in streaming multimedia over computer networks.
While in the past most video or audio streaming technologies utilized streaming protocols such as RTP with RTSP, today's adaptive streaming technologies are based almost exclusively on HTTP,[1] and are designed to work efficiently over large distributed HTTP networks.
Adaptive bitrate streaming works by detecting a user's bandwidth and CPU capacity in real time, adjusting the quality of the media stream accordingly.[2] It requires the use of an encoder which encodes a single source media (video or audio) at multiple bit rates. The player client[3] switches between streaming the different encodings depending on available resources.[4] This results in providing very little buffering, faster start times and a good experience for both high-end and low-end connections.[5]
More specifically, adaptive bitrate streaming is a method of video streaming over HTTP where the source content is encoded at multiple bit rates. Each of the different bit rate streams are segmented into small multi-second parts.[6] The segment size can vary depending on the particular implementation, but they are typically between two and ten seconds.[4][6] First, the client downloads a manifest file that describes the available stream segments and their respective bit rates. During stream start-up, the client usually requests the segments from the lowest bit rate stream. If the client finds that the network throughput is greater than the bit rate of the downloaded segment, then it will request a higher bit rate segment. Later, if the client finds that the network throughput has deteriorated, it will request a lower bit rate segment. An adaptive bitrate (ABR) algorithm in the client performs the key function of deciding which bit rate segments to download, based on the current state of the network. Several types of ABR algorithms are in commercial use: throughput-based algorithms use the throughput achieved in recent prior downloads for decision-making (e.g., throughput rule in dash.js), buffer-based algorithms use only the client's current buffer level (e.g., BOLA[7] in dash.js), and hybrid algorithms combine both types of information (e.g., DYNAMIC[8] in dash.js).
Current uses
[edit]Post-production houses, content delivery networks and studios use adaptive bit rate technology in order to provide consumers with higher quality video using less manpower and fewer resources. The creation of multiple video outputs, particularly for adaptive bit rate streaming, adds great value to consumers.[9] If the technology is working properly, the end user or consumer's content should play back without interruption and potentially go unnoticed. Media companies have been actively using adaptive bit rate technology for many years now and it has essentially become standard practice for high-end streaming providers, permitting little buffering when streaming high-resolution feeds (begins with low-resolution and climbs).
Benefits of adaptive bitrate streaming
[edit]Traditional server-driven adaptive bitrate streaming provides consumers of streaming media with the best-possible experience, since the media server automatically adapts to any changes in each user's network and playback conditions.[10] The media and entertainment industry also benefit from adaptive bitrate streaming. As the video space grows, content delivery networks and video providers can provide customers with a superior viewing experience. Adaptive bitrate technology requires additional encoding, but simplifies the overall workflow and creates better results.
HTTP-based adaptive bitrate streaming technologies yield additional benefits over traditional server-driven adaptive bitrate streaming. First, since the streaming technology is built on top of HTTP, contrary to RTP-based adaptive streaming, the packets have no difficulties traversing firewalls and NAT devices. Second, since HTTP streaming is purely client-driven, all adaptation logic resides at the client. This reduces the requirement of persistent connections between server and client application. Furthermore, the server is not required to maintain session state information on each client, increasing scalability. Finally, existing HTTP delivery infrastructure, such as HTTP caches and servers, can be seamlessly adopted.[11][12][13][14]
A scalable CDN is used to deliver media streaming to an Internet audience. The CDN receives the stream from the source at its Origin server, then replicates it to many or all of its Edge cache servers. The end-user requests the stream and is redirected to the "closest" Edge server. This can be tested using libdash[15] and the Distributed DASH (D-DASH) dataset,[16] which has several mirrors across Europe, Asia and the US. The use of HTTP-based adaptive streaming allows the Edge server to run a simple HTTP server software, whose license cost is cheap or free, reducing software licensing cost, compared to costly media server licences (e.g., Adobe Flash Media Streaming Server). The CDN cost for HTTP streaming media is then similar to HTTP web caching CDN cost.
History
[edit]Adaptive bit rate over HTTP was created by the DVD Forum at the WG1 Special Streaming group in October 2002. The group was co-chaired by Toshiba and Phoenix Technologies, The expert group count with the collaboration of Microsoft, Apple Computer, DTS Inc., Warner Brothers, 20th Century Fox, Digital Deluxe, Disney, Macromedia and Akamai.[dubious – discuss][citation needed] The technology was originally called DVDoverIP and was an integral effort of the DVD ENAV book.[17] The concept came from storing MPEG-1 and MPEG-2 DVD TS Sectors into small 2 KB files, which will be served using an HTTP server to the player. The MPEG-1 segments provided the lower bandwidth stream, while the MPEG-2 segments provided a higher bit rate stream. The original XML schema provided a simple playlist of bit rates, languages and URL servers. The first working prototype was presented to the DVD Forum by Phoenix Technologies at the Harman Kardon Lab in Villingen, Germany.[citation needed]
Implementations
[edit]Adaptive bit rate streaming was introduced by Move Networks in 2006[citation needed] and is now being developed and utilized by Adobe Systems, Apple, Microsoft and Octoshape.[18] In October 2010, Move Networks was awarded a patent for their adaptive bit rate streaming (US patent number 7818444).[19]
Dynamic Adaptive Streaming over HTTP (DASH)
[edit]Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH, is the only adaptive bit-rate HTTP-based streaming solution that is an international standard[20] MPEG-DASH technology was developed under MPEG. Work on DASH started in 2010 and became a Draft International Standard in January 2011 and an International Standard in November 2011.[20][21][22] The MPEG-DASH standard was published as ISO/IEC 23009-1:2012 in April, 2012.
MPEG-DASH is a technology related to Adobe Systems HTTP Dynamic Streaming, Apple Inc. HTTP Live Streaming (HLS) and Microsoft Smooth Streaming.[23] DASH is based on Adaptive HTTP streaming (AHS) in 3GPP Release 9 and on HTTP Adaptive Streaming (HAS) in Open IPTV Forum Release 2.[24] As part of their collaboration with MPEG, 3GPP Release 10 has adopted DASH (with specific codecs and operating modes) for use over wireless networks.[24]
The goal of standardizing an adaptive streaming solution is to assure the market that the solution can work universally, unlike other solutions that are more specific to certain vendors, such as Apple’s HLS, Microsoft’s Smooth Streaming, or Adobe’s HDS.
Available implementations are the HTML5-based bitdash MPEG-DASH player[25] as well as the open source C++-based DASH client access library libdash of bitmovin GmbH,[15] the DASH tools of the Institute of Information Technology (ITEC) at Alpen-Adria University Klagenfurt,[3][26] the multimedia framework of the GPAC group at Telecom ParisTech,[27] and the dash.js[28] player of the DASH-IF.
Apple HTTP Live Streaming (HLS)
[edit]HTTP Live Streaming (HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. as part of QuickTime X and iOS. HLS supports both live and video on demand content. It works by breaking down media streams or files into short pieces (media segments), which are stored as MPEG-TS or fragmented MP4 files. This is typically done at multiple bitrates using a stream or file segmenter application, also known as a packager. One such segmenter implementation is provided by Apple.[29] Additional packagers are available, including free / open source offerings like Google's Shaka Packager [30] and various commercial tools as well - such as Unified Streaming.[31] The segmenter is also responsible for producing a set of playlist files in the M3U8 format which describe the media chunks. Each playlist is specific to a given bitrate, and contains the relative or absolute URLs to the chunks for that bitrate. The client is then responsible for requesting the appropriate playlist depending on available bandwidth.
HTTP Live Streaming is a standard feature in the iPhone 3.0 and newer versions.[32]
Apple has submitted its solution to the IETF for consideration as an Informational Request for Comments.[33] This was officially accepted as RFC 8216 A number of proprietary and open source solutions exist for both the server implementation (segmenter) and the client player.
HLS streams can be identified by the playlist URL format extension of m3u8 or MIME type of application/vnd.apple.mpegurl.[34] These adaptive streams can be made available in many different bitrates and the client device interacts with the server to obtain the best available bitrate which can reliably be delivered.
Playback of HLS is supported on many platforms, including Safari and native apps on macOS / iOS, Microsoft Edge on Windows 10, ExoPlayer on Android, and the Roku platform. Many Smart TVs also have native support for HLS. Playing HLS on other platforms like Chrome / Firefox is typically achieved via a browser / JavaScript player implementation. Many open source and commercial players are available, including hls.js, video.js http-streaming, BitMovin, JWPlayer, THEOplayer, etc.
Adobe HTTP Dynamic Streaming (HDS)
[edit]"HTTP Dynamic streaming is the process of efficiently delivering streaming video to users by dynamically switching among different streams of varying quality and size during playback. This provides users with the best possible viewing experience their bandwidth and local computer hardware (CPU) can support. Another major goal of dynamic streaming is to make this process smooth and seamless to users, so that if up-scaling or down-scaling the quality of the stream is necessary, it is a smooth and nearly unnoticeable switch without disrupting the continuous playback."[35]
The latest versions of Flash Player and Flash Media Server support adaptive bit-rate streaming over the traditional RTMP protocol, as well as HTTP, similar to the HTTP-based solutions from Apple and Microsoft,[36] HTTP dynamic streaming being supported in Flash Player 10.1 and later.[37] HTTP-based streaming has the advantage of not requiring any firewall ports to be opened outside of the normal ports used by web browsers. HTTP-based streaming also allows video fragments to be cached by browsers, proxies, and CDNs, drastically reducing the load on the source server.
Microsoft Smooth Streaming (MSS)
[edit]Smooth Streaming is an IIS Media Services extension that enables adaptive streaming of media to clients over HTTP.[38] The format specification is based on the ISO base media file format and standardized by Microsoft as the Protected Interoperable File Format.[39] Microsoft is actively involved with 3GPP, MPEG and DECE organizations' efforts to standardize adaptive bit-rate HTTP streaming. Microsoft provides Smooth Streaming Client software development kits for Silverlight and Windows Phone 7, as well as a Smooth Streaming Porting Kit that can be used for other client operating systems, such as Apple iOS, Android, and Linux.[40] IIS Media Services 4.0, released in November 2010, introduced a feature which enables Live Smooth Streaming H.264/AAC videos to be dynamically repackaged into the Apple HTTP Adaptive Streaming format and delivered to iOS devices without the need for re-encoding. Microsoft has successfully demonstrated delivery of both live and on-demand 1080p HD video with Smooth Streaming to Silverlight clients. In 2010, Microsoft also partnered with NVIDIA to demonstrate live streaming of 1080p stereoscopic 3D video to PCs equipped with NVIDIA 3D Vision technology.[41]
Common Media Application Format (CMAF)
[edit]CMAF is a presentation container format used for the delivery of both HLS and MPEG-DASH. Hence, it is intended to simplify delivery of HTTP-based streaming media. It was proposed in 2016 by Apple and Microsoft and officially published in 2018.[42]
QuavStreams Adaptive Streaming over HTTP
[edit]QuavStreams Adaptive Streaming is a multimedia streaming technology developed by Quavlive. The streaming server is an HTTP server that has multiple versions of each video, encoded at different bitrates and resolutions. The server delivers the encoded video/audio frames switching from one level to another, according to the current available bandwidth. The control is entirely server-based, so the client does not need special additional features. The streaming control employs feedback control theory.[43] Currently, QuavStreams supports H.264/MP3 codecs muxed into the FLV container and VP8/Vorbis codecs muxed into the WEBM container.
Uplynk
[edit]Uplynk delivers HD adaptive bitrate streaming to multiple platforms, including iOS, Android, Windows, Mac, Linux, and Roku, across various browser combinations, by encoding video in the cloud using a single non-proprietary adaptive streaming format. Rather than streaming and storing multiple formats for different platforms and devices, Uplynk stores and streams only one. The first studio to use this technology for delivery was Disney–ABC Television Group, using it for video encoding for web, mobile and tablet streaming apps on the ABC Player, ABC Family and Watch Disney apps, as well as the live Watch Disney Channel, Watch Disney Junior, and Watch Disney XD.[44][45]
Self-learning clients
[edit]In recent years, the benefits of self-learning algorithms in adaptive bitrate streaming have been investigated in academia. While most of the initial self-learning approaches are implemented at the server-side[46][47][48] (e.g., performing admission control using reinforcement learning or artificial neural networks), more recent research is focusing on the development of self-learning HTTP Adaptive Streaming clients. Multiple approaches have been presented in literature using the SARSA[49] or Q-learning[50] algorithm. In all of these approaches, the client state is modeled using, among others, information about the current perceived network throughput and buffer filling level. Based on this information, the self-learning client autonomously decides which quality level to select for the next video segment. The learning process is steered using feedback information, representing the Quality of Experience (QoE) (e.g., based on the quality level, the number of switches and the number of video freezes). Furthermore, it was shown that multi-agent Q-learning can be applied to improve QoE fairness among multiple adaptive streaming clients.[51]
Criticisms
[edit]HTTP-based adaptive bit rate technologies are significantly more operationally complex than traditional streaming technologies. Some of the documented considerations are things such as additional storage and encoding costs, and challenges with maintaining quality globally. There have also been some interesting dynamics found around the interactions between complex adaptive bit rate logic competing with complex TCP flow control logic.[11][52] [53] [54][55]
However, these criticisms have been outweighed in practice by the economics and scalability of HTTP delivery: whereas non-HTTP streaming solutions require massive deployment of specialized streaming server infrastructure, HTTP-based adaptive bit-rate streaming can leverage the same HTTP web servers used to deliver all other content over the Internet.[citation needed]
With no single clearly defined or open standard for the digital rights management used in the above methods, there is no 100% compatible way of delivering restricted or time-sensitive content to any device or player. This also proves to be a problem with digital rights management being employed by any streaming protocol.
The method of segmenting files into smaller files used by some implementations (as used by HTTP Live Streaming) could be deemed unnecessary due to the ability of HTTP clients to request byte ranges from a single video asset file that could have multiple video tracks at differing bit rates, with the manifest file only indicating track number and bit rate. However, this approach allows for serving of chunks by any simple HTTP server and therefore guarantees CDN compatibility. Implementations using byte ranges, such as Microsoft Smooth Streaming require a dedicated HTTP server, such as IIS, to respond to the requests for video asset chunks.
See also
[edit]- Multiple description coding
- Hierarchical modulation – alternative with reduced storage and authoring demands
References
[edit]- ^ Saamer Akhshabi; Ali C. Begen; Constantine Dovrolis (2011). An Experimental Evaluation of Rate-Adaptation Algorithms in Adaptive Streaming over HTTP. In Proceedings of the second annual ACM conference on Multimedia systems (MMSys '11). New York, NY, USA: ACM.
- ^ A. Bentaleb, B. Taani, A. Begen, C. Timmermer, and R. Zimmermann, "A Survey on Bitrate Adaptation Schemes for Streaming Media over HTTP", In IEEE Communications Surveys & (IEEE COMST), Volume 1 Issue 1, pp. 1-1, 2018.
- ^ a b DASH at ITEC, VLC Plugin, DASHEncoder and Dataset by C. Mueller, S. Lederer, C. Timmerer
- ^ a b "Proceedings Template – WORD" (PDF). Retrieved 16 December 2017.
- ^ Gannes, Liz (10 June 2009). "The Next Big Thing in Video: Adaptive Bitrate Streaming". Archived from the original on 19 June 2010. Retrieved 1 June 2010.
- ^ a b "mmsys2012-final36.pdf" (PDF). Retrieved 16 December 2017.
- ^ Spiteri, Kevin; Urgaonkar, Rahul; Sitaraman, Ramesh K. (2016). "BOLA: Near-optimal bitrate adaptation for online videos. IEEE INFOCOM, 2016, by Spiteri, Urgaonkar, and Sitaraman, IEEE INFOCOM, April 2016". arXiv:1601.06748. doi:10.1109/TNET.2020.2996964. S2CID 219792107.
{{cite journal}}: Cite journal requires|journal=(help) - ^ "From Theory to Practice: Improving Bitrate Adaptation in the DASH Reference Player, by Spiteri, Sitaraman and Sparacio, ACM Multimedia Systems Conference, June 2018" (PDF).
- ^ Marshall, Daniel (18 February 2010). "Show Report: Video Processing Critical to Digital Asset Management". Elemental Technologies. Archived from the original on 4 October 2011. Retrieved 15 October 2011.
- ^ Seufert, Michael; Egger, Sebastian; Slanina, Martin; Zinner, Thomas; Hoßfeld, Tobias; Tran-Gia, Phuoc (2015). "A Survey on Quality of Experience of HTTP Adaptive Streaming". IEEE Communications Surveys & Tutorials. 17 (1): 469–492. doi:10.1109/COMST.2014.2360940. S2CID 18220375.
- ^ a b Saamer Akhshabi; Ali C. Begen; Constantine Dovrolis. "An Experimental Evaluation of Rate-Adaptation Algorithms in Adaptive Streaming over HTTP" (PDF). Archived from the original (PDF) on 17 October 2011. Retrieved 15 October 2011.
{{cite journal}}: Cite journal requires|journal=(help) - ^ Anthony Vetro. "The MPEG-DASH Standard for Multimedia Streaming Over the Internet" (PDF). Archived from the original (PDF) on 4 March 2016. Retrieved 10 July 2015.
{{cite journal}}: Cite journal requires|journal=(help) - ^ Jan Ozer (28 April 2011). "What Is Adaptive Streaming?". Retrieved 10 July 2015.
{{cite journal}}: Cite journal requires|journal=(help) - ^ Jeroen Famaey; Steven Latré; Niels Bouten; Wim Van de Meerssche; Bart de Vleeschauwer; Werner Van Leekwijck; Filip De Turck (May 2013). "On the merits of SVC-based HTTP Adaptive Streaming": 419–426. Retrieved 10 July 2015.
{{cite journal}}: Cite journal requires|journal=(help) - ^ a b libdash: Open-source DASH client library by bitmovin
- ^ "Distributed DASH Datset | ITEC – Dynamic Adaptive Streaming over HTTP". Itec.uni-klu.ac.at. Retrieved 16 December 2017.
- ^ DVD Book Construction, DVD Forum, May 2005
- ^ Gannes, Liz (10 June 2009). "The Lowdown on Apple's HTTP Adaptive Bitrate Streaming". Archived from the original on 19 June 2010. Retrieved 24 June 2010.
- ^ "Move Gets Streaming Patent; Are Adobe & Apple Hosed? – Online Video News". Gigaom.com. 15 September 2010. Archived from the original on 22 October 2011. Retrieved 15 October 2011.
- ^ a b "MPEG ratifies its draft standard for DASH". MPEG. 2 December 2011. Archived from the original on 20 August 2012. Retrieved 26 August 2012.
- ^ Timmerer, Christian (26 April 2012). "HTTP streaming of MPEG media – blog entry". Multimediacommunication.blogspot.com. Retrieved 16 December 2017.
- ^ "ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH)". Iso.org. Retrieved 16 December 2017.
- ^ Updates on DASH – blog entry
- ^ a b ETSI 3GPP 3GPP TS 26.247; Transparent end-to-end packet-switched streaming service (PSS); Progressive Download and Dynamic Adaptive Streaming over HTTP (3GP-DASH)
- ^ "bitdash HTML5 MPEG-DASH player". Dash-player.com. 22 January 2016. Archived from the original on 10 July 2016. Retrieved 16 December 2017.
- ^ "A VLC media player plugin enabling dynamic adaptive streaming over HTTP" (PDF). Retrieved 16 December 2017.
- ^ "GPAC Telecom ParisTech". Archived from the original on 24 February 2012. Retrieved 28 March 2013.
- ^ "dash.js". Github.com. Retrieved 16 December 2017.
- ^ Mac Developer Library, Apple, retrieved 2 June 2014
- ^ Shaka Packager Github Repository, Google, retrieved 3 January 2023
- ^ Unified Streaming, Unified Streaming, retrieved 3 January 2023
- ^ Prince McLean (9 July 2009). "Apple launches HTTP Live Streaming standard in iPhone 3.0". AppleInsider. Archived from the original on 13 May 2019. Retrieved 15 October 2011.
- ^ R. Pantos, HTTP Live Streaming, IETF, retrieved 11 October 2011
- ^ RFC 8216. sec. 4. doi:10.17487/RFC8216.
- ^ Hassoun, David. "Dynamic streaming in Flash Media Server 3.5 – Part 1: Overview of the new capabilities". Adobe Developer Connection. Adobe Systems. Archived from the original on 30 March 2014.
- ^ "HTTP Dynamic Streaming". Adobe Systems. Retrieved 13 October 2010.
- ^ "FAQ HTTP Dynamic Streaming". Adobe Systems. Retrieved 12 January 2015.
- ^ "Smooth Streaming". IIS.net. Archived from the original on 15 June 2010. Retrieved 24 June 2010.
- ^ Chris Knowlton (8 September 2009), Protected Interoperable File Format, Microsoft, retrieved 15 October 2011
- ^ "Microsoft End-to-End Platform Powers Next-Generation Silverlight and IIS Media Experiences Across Multiple Screens". Microsoft. 8 April 2010. Retrieved 30 July 2011.
- ^ "First Day of IBC". Microsoft. Archived from the original on 2 February 2011. Retrieved 22 January 2011.
- ^ Traci Ruether (23 January 2019). "What Is CMAF?". Retrieved 13 January 2022.
- ^ Luca De Cicco; Saverio Mascolo; Vittorio Palmisano. "Feedback Control for Adaptive Live Video Streaming" (PDF). MMSYS2011. Retrieved 9 September 2012.
- ^ Dean Takahashi (16 January 2013). "Uplynk creates a cheap and efficient way for Disney to stream videos". VentureBeat. Retrieved 16 December 2017.
- ^ Dreier, Troy (16 January 2013). "UpLynk Emerges from Stealth Mode; DisneyABC Is First Customer – Streaming Media Magazine". Streamingmedia.com. Retrieved 16 December 2017.
- ^ Y. Fei; V. W. S. Wong; V. C. M. Leung (2006). "Efficient QoS provisioning for adaptive multimedia in mobile communication networks by reinforcement learning". Mobile Networks and Applications. 11 (1): 101–110. CiteSeerX 10.1.1.70.1430. doi:10.1007/s11036-005-4464-2. S2CID 13022779.
- ^ V. Charvillat; R. Grigoras (2007). "Reinforcement learning for dynamic multimedia adaptation". Journal of Network and Computer Applications. 30 (3): 1034–1058. doi:10.1016/j.jnca.2005.12.010.
- ^ D. W. McClary; V. R. Syrotiuk; V. Lecuire (2008). "Adaptive audio streaming in mobile ad hoc networks using neural networks". Ad Hoc Networks. 6 (4): 524–538. doi:10.1016/j.adhoc.2007.04.005.
- ^ V. Menkovski; A. Liotta (2013). "Intelligent control for adaptive video streaming". IEEE International Conference on Consumer Electronics (ICCE). Washington, DC. pp. 127–128. doi:10.1109/ICCE.2013.6486825.
- ^ M. Claeys; S. Latré; J. Famaey; F. De Turck (2014). "Design and evaluation of a self-learning HTTP adaptive video streaming client". IEEE Communications Letters. 18 (4): 716–719. Bibcode:2014IComL..18..716C. doi:10.1109/lcomm.2014.020414.132649. hdl:1854/LU-5733061. S2CID 26955239.
- ^ S. Petrangeli; M. Claeys; S. Latré; J. Famaey; F. De Turck (2014). "A multi-agent Q-Learning-based framework for achieving fairness in HTTP Adaptive Streaming". IEEE Network Operations and Management Symposium (NOMS). Krakow. pp. 1–9. doi:10.1109/NOMS.2014.6838245.
- ^ Pete Mastin (28 January 2011). "Is adaptive bit rate the yellow brick road, or fool's gold for HD streaming?". Archived from the original on 7 September 2011. Retrieved 15 October 2011.
- ^ Luca De Cicco; Saverio Mascolo. "An Experimental Investigation of the Akamai Adaptive Video Streaming" (PDF). Retrieved 29 November 2011.
{{cite journal}}: Cite journal requires|journal=(help) - ^ "Adaptive streaming: a comparison". Archived from the original on 19 April 2014. Retrieved 17 April 2014.
{{cite journal}}: Cite journal requires|journal=(help) - ^ Chris Knowlton (28 January 2010). "Adaptive Streaming Comparison".
{{cite journal}}: Cite journal requires|journal=(help)
Further reading
[edit]- The Next Big Thing in Video: Adaptive Bitrate Streaming Archived 19 June 2010 at the Wayback Machine
Adaptive bitrate streaming
View on GrokipediaFundamentals
Definition
Adaptive bitrate streaming is a video delivery technique that encodes the same content into multiple bitrate variants, allowing the client device to dynamically select and download short segments of the stream based on real-time network conditions, thereby optimizing playback quality and minimizing buffering.[6][7] In contrast to fixed-bitrate streaming, which transmits video at a constant quality level and risks interruptions like stalling or rebuffering when bandwidth varies, adaptive bitrate streaming divides the content into brief, independent segments—typically 2 to 10 seconds in duration—enabling seamless switches between quality levels during playback without requiring a full file download.[7][8][9] A key element of this approach is the bitrate ladder, which consists of a predefined set of quality variants, such as 360p resolution at around 500 kbps for lower-bandwidth scenarios up to 1080p at approximately 5 Mbps for higher-capacity connections, all delivered via standard HTTP protocols.[10][11] Common implementations utilize standards like MPEG-DASH or HTTP Live Streaming (HLS) to facilitate this adaptive process.[12][1]Core Principles
Adaptive bitrate streaming operates on the principle of segment-based delivery, where video content is divided into small, self-contained chunks or segments, typically lasting 2 to 10 seconds each. These segments are encoded and made available in multiple bitrate variants, ranging from low to high quality, allowing clients to select and switch between them seamlessly without interrupting playback. This approach ensures that playback can adapt to varying network conditions by fetching the next segment at an appropriate quality level, minimizing visual artifacts from abrupt quality changes.[13] Central to this mechanism is the client's bandwidth estimation and adaptation logic, which continuously monitors network throughput—often calculated as the harmonic mean of recent download speeds—and buffer occupancy levels to inform bitrate selection. For instance, if the estimated throughput exceeds the bitrate of the current variant while the buffer remains sufficiently filled (e.g., above a target threshold like 60 seconds), the client switches to a higher-quality variant for the subsequent segment; conversely, it downswitches if buffer levels drop or throughput declines to prevent rebuffering. This client-driven decision-making process prioritizes smooth playback by balancing quality maximization with stall avoidance, using algorithms such as proportional-integral-derivative (PID) controllers to stabilize adaptations over time.[14][13] The reliance on HTTP for delivery further underpins these principles through its stateless nature, where each segment request is an independent GET operation without requiring persistent server connections or session state maintenance. This design facilitates scalability by enabling the use of standard HTTP infrastructure and content delivery networks (CDNs), which cache segments at edge locations to handle massive concurrent requests efficiently and reduce latency without server-side overhead. By avoiding stateful protocols, adaptive bitrate streaming achieves high reliability across diverse networks, including those behind firewalls or NATs.[15][13]Technical Components
Content Preparation
Content preparation for adaptive bitrate streaming involves server-side encoding and segmentation processes to generate multiple versions of the media suitable for dynamic quality adjustment during playback. This preparation ensures that the content can be delivered efficiently over varying network conditions by creating a set of renditions that balance visual quality with bandwidth usage.[16] Multi-bitrate encoding is the foundational step, where the source video is transcoded into several variants using efficient codecs such as H.264/AVC, H.265/HEVC, or AV1. These codecs compress the video while preserving quality, with H.264 being the most widely supported, H.265 offering better compression efficiency, and AV1 providing royalty-free high-performance encoding for modern applications. The output forms a bitrate ladder, typically comprising 3 to 8 variants at different resolutions and bitrates—for example, ranging from 360p at 500 Kbps to 4K at 25 Mbps—to optimize the trade-off between file size and perceptual quality without excessive storage demands.[17][16] Following encoding, the variants are segmented into short, fixed-duration fragments, commonly in MPEG-2 Transport Stream (TS) format for compatibility with protocols like HLS or fragmented MP4 (fMP4) for broader standards support such as DASH. Tools like FFmpeg facilitate this chopping process through commands that divide the video into chunks, while Bento4's mp4dash utility specifically handles fMP4 fragmentation and multi-variant packaging for adaptive presentations. Segment durations are typically 2 to 4 seconds to enable smooth bitrate switching with minimal latency, though longer durations up to 10 seconds may be used for efficiency in stable networks.[18][19] Throughout preparation, careful attention is given to audio synchronization and subtitle integration across all variants to maintain seamless playback. Audio tracks must align precisely with video using common time anchors and identical timescales (e.g., matching frame rates and sample rates to avoid drift), often achieved via synchronized encoding pipelines that enforce segment boundary alignment. Subtitles or captions are embedded or provided as separate tracks compatible with multi-language support, ensuring they remain timed correctly regardless of the selected bitrate variant. These prepared segments are subsequently referenced in manifest files to guide client-side selection.[20][9]Manifest Files and Segmentation
Manifest files, also known as playlists, serve as metadata descriptors in adaptive bitrate streaming, organizing the delivery of media content by listing available variants and their corresponding segments. These files enable clients to select appropriate quality levels based on network conditions without requiring direct server interaction for adaptation decisions. In MPEG-DASH, the manifest is an XML-based Media Presentation Description (MPD) file that structures the media presentation into hierarchical elements, including Periods for temporal divisions, Adaptation Sets grouping related media components like video and audio, and Representations detailing specific variants within each set.[12] Each Representation specifies attributes such as bandwidth (e.g., in bits per second), codecs, and for video, resolution (e.g., 1920x1080), while Segment information provides URLs or templates for retrieving media files, along with durations typically ranging from 2 to 10 seconds.[21] In HTTP Live Streaming (HLS), the manifest uses a text-based M3U8 playlist format, consisting of a Master Playlist that enumerates variant streams via tags like EXT-X-STREAM-INF, which include bandwidth, average bandwidth, codecs, and optional resolution for video variants.[22] Individual Media Playlists linked from the Master Playlist list segment URLs (e.g., to .ts files) and their durations, often standardized at around 6 seconds per segment, allowing seamless switching between variants during playback.[23] Both MPD and M3U8 formats ensure that all variant information—encompassing multiple bitrates, resolutions, and associated segment locations—is centralized, facilitating efficient content discovery and adaptation.[22] Manifests differ in their update mechanisms depending on the streaming type: static manifests are used for video-on-demand (VOD) content, where the file remains unchanged after initial generation, listing all segments in advance for complete playback without further server polling.[24] In contrast, dynamic manifests support live streaming by periodically updating to append new segments as they become available, with MPDs employing attributes like availabilityStartTime and minimumUpdatePeriod to guide client refreshes, ensuring real-time incorporation of ongoing content.[21] For HLS, live playlists omit the EXT-X-ENDLIST tag and use EXT-X-MEDIA-SEQUENCE to track segment progression, requiring clients to reload the playlist at intervals based on the target duration to detect updates.[22] Segmentation in adaptive bitrate streaming relies on standardized container formats to encapsulate media data, with the ISO Base Media File Format (ISOBMFF) serving as the primary container for compatibility across protocols like MPEG-DASH. The Common Media Application Format (CMAF, ISO/IEC 23000-19) leverages ISOBMFF for consistent packaging across HLS and DASH, with recent amendments as of 2024 introducing a new Structural CMAF Brand Profile for improved structural support.[25] ISOBMFF structures segments as self-contained units, typically comprising a Movie Fragment Box (moof) for metadata and one or more Media Data Boxes (mdat) for the encoded samples, enabling random access and efficient partial downloads without dependency on full-file parsing.[26] This format ensures interoperability among devices and players by adhering to defined brands (e.g., via File Type Box or Segment Type Box), supporting features like initialization segments for decoder setup and fragmented media segments for progressive playback.[26] In HLS, segments often use MPEG-2 Transport Stream (TS) containers, but ISOBMFF-based fragmented MP4 is increasingly adopted for broader compatibility and lower latency.[22]Client-Side Adaptation
Client-side adaptation refers to the mechanisms implemented in playback devices to dynamically select and switch between different bitrate variants of a video stream based on real-time network conditions and device capabilities. This process occurs at the client end, where the media player analyzes factors such as available bandwidth, buffer occupancy, and decoding resources to fetch the next video segment at an optimal quality level. By making these decisions per segment, typically lasting 2-10 seconds, the player aims to balance high video quality with uninterrupted playback, avoiding both excessive buffering and quality degradation. Adaptation algorithms form the core of this logic, categorizing into buffer-based, rate-based, and hybrid approaches. Buffer-based algorithms, exemplified by the Buffer Occupancy-based Lyapunov Algorithm (BOLA), decide the bitrate by comparing the current buffer level to a target occupancy, selecting the highest quality that maintains the buffer above a minimum threshold to prevent rebuffering while maximizing average bitrate. Rate-based algorithms estimate available throughput from the download times of recent segments and choose the highest bitrate variant below this estimate, providing responsive adaptation to short-term network fluctuations but potentially leading to oscillations in volatile conditions. Hybrid models integrate both, using throughput estimates for immediate decisions and buffer status for stability, as in algorithms that weight recent bandwidth measurements against buffer health to dampen unnecessary switches.[27] Switching heuristics govern the timing and manner of bitrate changes to ensure perceptual smoothness. Abrupt transitions occur immediately at segment boundaries, which can cause noticeable quality jumps if not aligned properly, whereas smooth transitions employ gradual heuristics, such as limiting switch frequency or prioritizing variants with similar quality levels.[28] To avoid visible artifacts like blurring or blocking during switches, clients handle quality ramps—pre-encoded segments with progressive quality buildup or decay at switch points—by selecting alignment points that minimize perceptual disruption.[28] Error handling in client-side adaptation focuses on robustness against network impairments, such as packet loss, which increases segment download times and triggers fallback to lower bitrates via the active algorithm. For instance, if throughput drops below the current bitrate due to loss-induced delays, the player immediately selects a safer variant to refill the buffer, often incorporating retry logic for failed fetches before declaring a stall. These features are embedded in popular open-source player frameworks; Shaka Player employs a throughput-based estimator with configurable rules for switch thresholds and error recovery, while Video.js integrates adaptive logic through its HTTP Streaming module, allowing custom heuristics for bitrate fallback and bandwidth probing on errors.[29][30] Such mechanisms help reduce playback interruptions in imperfect networks.Benefits
Network Efficiency
Adaptive bitrate streaming optimizes bandwidth usage by dynamically adjusting the video quality to match available network throughput, preventing the over-delivery of high-bitrate content during periods of congestion. This mechanism contrasts with fixed-bitrate streaming, where a constant data rate often leads to buffering or wasted resources when the network cannot sustain it. Studies indicate that such dynamic adjustments can reduce overall data consumption compared to fixed-bitrate approaches, as adaptive clients select lower-bitrate segments only when necessary, conserving bandwidth without fully compromising playback. For instance, quality-aware adaptive bitrate schemes have demonstrated up to 43% bandwidth savings while maintaining target quality levels.[3] The integration of adaptive bitrate streaming with content delivery networks (CDNs) enhances scalability by leveraging HTTP-based segmentation and caching. Short video segments at multiple bitrates are cached at edge servers, allowing clients to fetch the most appropriate version based on real-time conditions, which distributes load across the network and reduces origin server strain. This caching strategy lowers latency by up to 18% and decreases encoding demands by 25% in live streaming scenarios, enabling CDNs to handle millions of concurrent users efficiently.[31] On mobile networks, adaptive bitrate streaming mitigates the challenges of variable connectivity by selecting lower bitrates during bandwidth fluctuations, thereby extending battery life through reduced data transmission and fewer rebuffering events. Energy-efficient variants of adaptive algorithms can cut mobile device energy consumption by up to 12%, as lower data volumes decrease radio activity and processing overhead. Additionally, these savings translate to lower data costs for users, particularly in metered plans where high-definition streaming might otherwise exceed monthly allowances.[3]Viewer Experience
Adaptive bitrate streaming significantly enhances the viewer experience by preemptively adjusting video quality to network conditions, thereby reducing buffering and playback stalls. By monitoring available bandwidth and buffer levels in real-time, ABR algorithms switch to lower bitrates before the playback buffer depletes, preventing interruptions that disrupt viewing continuity. This proactive adaptation maintains smooth playback, with studies showing that effective ABR implementations can minimize rebuffering far better than traditional streaming methods under variable conditions.[32][33] In terms of quality maximization, ABR enables viewers to enjoy higher bitrates and sharper video resolution during periods of stable or high-bandwidth connections, optimizing perceptual quality without manual intervention. The technology supports seamless transitions between bitrate variants, which are typically imperceptible to the human eye due to short segment durations (often 2-10 seconds) and careful encoding to minimize visual artifacts at switch points. For instance, buffer-based ABR schemes have demonstrated average video rates up to 1950 kbps while stabilizing quality fluctuations, leading to higher overall satisfaction scores in quality-of-experience (QoE) assessments.[32][33] Furthermore, ABR promotes accessibility by accommodating a wide range of devices, from low-end mobile phones with limited processing power to high-resolution 4K televisions. Adaptation algorithms consider device-specific factors such as screen size, decoding capabilities, and battery life, ensuring playable quality across heterogeneous environments without requiring custom implementations per device. This broad compatibility extends ABR's utility in both live streaming and video-on-demand scenarios, where consistent performance across platforms is essential for inclusive viewing.[32]History
Early Developments
Prior to the 2000s, precursors to adaptive bitrate streaming appeared in the form of variable bitrate (VBR) encoding techniques used in digital video standards like MPEG-2, which was foundational for DVDs released in 1996 and digital broadcast television.[34] VBR allowed encoders to allocate more bits to complex scenes with high motion or detail while using fewer for simpler ones, optimizing storage and quality on fixed media like DVDs or in broadcast transmissions.[34] However, these methods were static, applied during encoding without real-time adjustments during playback or delivery, limiting their applicability to dynamic network environments.[34] The 2000s marked a breakthrough with the introduction of true adaptive streaming systems, pioneered by Move Networks in 2007 for IPTV applications.[35] Move's proprietary system utilized HTTP to deliver video in small chunks, enabling the player to monitor download speeds and dynamically select segments encoded at different bitrates based on available bandwidth and device capabilities.[35] This innovation addressed the limitations of earlier progressive download and fixed-bitrate streaming, influencing the shift toward web-based video delivery by allowing seamless quality adjustments without proprietary protocols.[35] These early developments were driven by the inherent challenges of bandwidth variability in nascent broadband networks during the early 2000s, where actual speeds often fell short of advertised rates—such as 45% of headline speeds for 8 Mbit/s connections—and fluctuated due to factors like network congestion, peak usage, and line quality.[36] Such inconsistencies caused buffering and delivery failures in video streaming, as early broadband download speeds hovered around 600 kbit/s, making consistent playback of even low-resolution content unreliable without real-time adaptation.[36] This variability underscored the need for systems that could adjust bitrate on the fly, paving the way for broader standardization efforts in subsequent years.[36]Key Milestones and Standardization
In 2008, Microsoft introduced Smooth Streaming as an HTTP-based adaptive streaming protocol integrated with Internet Information Services (IIS) 7.0, enabling dynamic bitrate adjustment for Silverlight playback.[37] This marked an early proprietary effort to address variable network conditions in video delivery. The following year, in May 2009, Apple released HTTP Live Streaming (HLS), the first version of its protocol designed for iOS devices, which segmented video into small chunks for adaptive quality switching over HTTP.[38] By 2012, the industry shifted toward open standards with the publication of MPEG-DASH (Dynamic Adaptive Streaming over HTTP) as ISO/IEC 23009-1 in April, developed collaboratively by MPEG and 3GPP to promote interoperability across devices and networks using HTTP.[39] This standardization unified fragmented proprietary approaches, allowing servers and clients to describe and deliver media segments in a vendor-agnostic manner.[12] Following 2015, adaptive streaming saw accelerated growth through enhanced compatibility standards and platform integrations. In 2016, Apple and Microsoft proposed the Common Media Application Format (CMAF) to MPEG, aiming to enable a single fragmented MP4-based container for both HLS and DASH, reducing encoding and storage overhead.[40] CMAF was formalized as ISO/IEC 23000-19 in 2018, facilitating cross-protocol adoption.[41] Major platforms like Netflix and YouTube widely implemented these technologies during this period, with Netflix optimizing its per-title encodes for adaptive delivery by 2015[42] and YouTube leveraging DASH for scalable video distribution since 2013,[43] driving global streaming efficiency.[44]Standards and Protocols
MPEG-DASH
MPEG-DASH, formally known as Dynamic Adaptive Streaming over HTTP and standardized as ISO/IEC 23009-1, is an international open standard developed by the Moving Picture Experts Group (MPEG) for delivering multimedia content adaptively over HTTP.[45][12] It relies on Media Presentation Description (MPD) files, which are XML-based manifests that describe the structure, timing, and availability of segmented media resources, including multiple bitrate variants for adaptation to network conditions.[12] The standard supports both live streaming and video-on-demand (VOD) scenarios, enabling efficient delivery of content encoded with various codecs ranging from Advanced Video Coding (AVC/H.264) to Versatile Video Coding (VVC/H.266), as well as audio formats like AAC.[46][12] A core architectural element of MPEG-DASH is its period-based timeline model within the MPD, where the overall media presentation is divided into discrete periods that define synchronized playback intervals and facilitate dynamic updates for live content.[47] This structure supports multi-period content, allowing for complex presentations with multiple synchronized video, audio, and subtitle tracks, akin to advanced disc-based media features, while maintaining flexibility for content packaging and delivery.[47] Furthermore, the standard enables server-side ad insertion through period boundaries, where ads can be seamlessly stitched into the stream on the server, reducing client-side complexity and improving personalization without interrupting playback.[47] MPEG-DASH has seen broad adoption in the streaming industry, with major platforms like YouTube utilizing it as the primary protocol for HTML5-based video delivery, supporting codecs such as H.264 and VP9 to reach billions of users.[48] Ongoing developments as of 2025 include DASH-IF guidelines for MPD Patch to handle varying segment durations (February 2025) and a Watermarking API for encoder integration (December 2024), alongside MPEG's consideration of new technologies for enhanced DASH functionality.[49][50][51] Its integration with web technologies, particularly through the Media Source Extensions (MSE) API in modern browsers like Chrome, Firefox, and Safari, allows for plugin-free adaptive playback, promoting widespread interoperability across devices and ecosystems.[47] Unlike Apple's proprietary HTTP Live Streaming (HLS), MPEG-DASH's vendor-neutral design fosters greater cross-platform compatibility and innovation in streaming services.[48]HTTP Live Streaming (HLS)
HTTP Live Streaming (HLS) is an HTTP-based adaptive bitrate streaming protocol developed by Apple Inc. and first released in 2009 to enable reliable delivery of live and on-demand audio and video content over standard web servers. The protocol segments media files into short chunks, typically 2 to 10 seconds in duration, and uses extended M3U (M3U8) playlist files—identified by the MIME typeapplication/vnd.apple.mpegurl—to index and sequence these segments for playback. Media segments are commonly packaged in MPEG-2 Transport Stream (TS) format with the MIME type video/mp2t, though support for other containers has been added over time. To facilitate adaptive bitrate streaming, HLS employs master playlists that reference multiple variant streams encoded at varying bitrates (e.g., from 145 kbit/s to 20,000 kbit/s) and resolutions, enabling clients to dynamically select the optimal quality based on available bandwidth and device capabilities while minimizing buffering and stalls.[52][22]
Since its inception, HLS has undergone significant evolution to address emerging needs in streaming technology. In 2016, Apple extended the protocol to support fragmented MP4 (fMP4) as a container format alongside TS, paving the way for compatibility with the Common Media Application Format (CMAF) and improving efficiency for cross-protocol interoperability. Low-latency HLS was introduced in 2019, incorporating features like partial segments via the #EXT-X-PART tag and preload hints with #EXT-X-PRELOAD-HINT to reduce end-to-end latency to approximately 2-5 seconds without sacrificing scalability, making it suitable for interactive live events. Enhancements through the 2020s have focused on security through advanced FairPlay Streaming DRM integration, including sample-level AES encryption (SAMPLE-AES-CTR), and optimized support for 4K and HDR video, with authoring specifications recommending specific bitrates, frame rates (up to 60 fps for SDR, 30 fps for HDR), and codecs like HEVC for high-quality delivery on capable devices. As of June 2025, further updates include new video projection specifiers in REQ-VIDEO-LAYOUT (e.g., PROJ-RECT, PROJ-EQUI), INSTREAM-ID support for all media types in EXT-X-MEDIA (requiring EXT-X-VERSION 13+), updated CHANNEL attributes for spatial audio (e.g., 3OA for third-order ambisonics), new EXT-X-MEDIA characteristics like "public.machine-generated", a DATERANGE schema ("com.apple.hls.preload") for resource preloading, and skip button controls for interstitials, alongside support for custom media selection and signaling for APMP and AIV content.[53][54][52][55][56]
HLS maintains dominance within the Apple ecosystem, offering native playback support in iOS, macOS, tvOS, watchOS, and the Safari browser without requiring plugins or extensions, which ensures optimal performance and integration on Apple hardware. This device-centric design has driven its widespread adoption by over-the-top (OTT) services seeking broad compatibility, such as Netflix, which employs HLS for streaming on Apple platforms to deliver adaptive, high-quality video across varying network conditions and ensure consistent viewer experiences.[1][57]