Hubbry Logo
logo
FFmpeg
Community hub

FFmpeg

logo
0 subscribers
Read side by side
from Wikipedia
FFmpeg
Original authorsFabrice Bellard
Bobby Bingham (libavfilter)[1]
DeveloperFFmpeg team
Initial releaseDecember 20, 2000; 24 years ago (2000-12-20)[2]
Stable release
8.0[3] Edit this on Wikidata / 22 August 2025
Repositorygit.ffmpeg.org/ffmpeg.git
Written inC and Assembly[4]
Operating systemVarious, including Windows, macOS, and Linux (executable programs are only available from third parties, as the project only distributes source code)[5][6]
Platformx86, ARM, PowerPC, MIPS, RISC-V, DEC Alpha, Blackfin, AVR32, SH-4, and SPARC; may be compiled for other desktop computers
TypeMultimedia framework
LicenseLGPL-2.1-or-later, GPL-2.0-or-later
Unredistributable if compiled with any software with a license incompatible with the GPL[7]
Websiteffmpeg.org

FFmpeg is a free and open-source software project consisting of a suite of libraries and programs for handling video, audio, and other multimedia files and streams. At its core is the command-line ffmpeg tool itself, designed for processing video and audio files. It is widely used for format transcoding, basic editing (trimming and concatenation), video scaling, video post-production effects, and standards compliance (SMPTE, ITU).

FFmpeg also includes other tools: ffplay, a simple media player, and ffprobe, a command-line tool to display media information. Among included libraries are libavcodec, an audio/video codec library used by many commercial and free software products, libavformat (Lavf),[8] an audio/video container mux and demux library, and libavfilter, a library for enhancing and editing filters through a GStreamer-like filtergraph.[9]

FFmpeg is part of the workflow of many other software projects, and its libraries are a core part of software media players such as VLC, and has been included in core processing for YouTube and Bilibili.[10] Encoders and decoders for many audio and video file formats are included, making it highly useful for the transcoding of common and uncommon media files.

FFmpeg is published under the LGPL-2.1-or-later or GPL-2.0-or-later, depending on which options are enabled.[11]

History

[edit]

The project was started by Fabrice Bellard[11] (using the pseudonym "Gérard Lantau") in 2000, and was led by Michael Niedermayer from 2004 until 2015.[12] Some FFmpeg developers were also part of the MPlayer project.

The "FF" in FFmpeg stands for "fast forward"[13] The logo represents a zigzag scan pattern that shows how MPEG video codecs handle entropy encoding.[14]

On March 13, 2011, a group of FFmpeg developers decided to fork the project under the name Libav.[15][16][17] The group decided to fork the project due to a disagreement with the leadership of FFmpeg.[clarification needed][18][19][20] Libav was declared abandoned in 2020.[21]

On January 10, 2014, two Google employees announced that over 1000 bugs had been fixed in FFmpeg during the previous two years by means of fuzz testing.[22]

In January 2018, the ffserver command-line program – a long-time component of FFmpeg – was removed.[23] The developers had previously deprecated the program citing high maintenance efforts due to its use of internal application programming interfaces.[24]

The project publishes a new release every three months on average. While release versions are available from the website for download, FFmpeg developers recommend that users compile the software from source using the latest build from their source code, using the Git version control system.[25]

Codec history

[edit]

Two video coding formats with corresponding codecs and one container format have been created within the FFmpeg project so far. The two video codecs are the lossless FFV1, and the lossless and lossy Snow codec. Development of Snow has stalled, while its bit-stream format has not been finalized yet, making it experimental since 2011. The multimedia container format called NUT is no longer being actively developed, but still maintained.[26]

In summer 2010, FFmpeg developers Fiona Glaser, Ronald Bultje, and David Conrad, announced the ffvp8 decoder. Through testing, they determined that ffvp8 was faster than Google's own libvpx decoder.[27][28] Starting with version 0.6, FFmpeg also supported WebM and VP8.[29]

In October 2013, a native VP9[30] decoder and OpenHEVC, an open source High Efficiency Video Coding (HEVC) decoder, were added to FFmpeg.[31] In 2016 the native AAC encoder was considered stable, removing support for the two external AAC encoders from VisualOn and FAAC. FFmpeg 3.0 (nicknamed "Einstein") retained build support for the Fraunhofer FDK AAC encoder.[32] Since version 3.4 "Cantor" FFmpeg supported the FITS image format.[33] Since November 2018 in version 4.1 "al-Khwarizmi" AV1 can be muxed in MP4 and Matroska, including WebM.[34][35]

Components

[edit]

Command-line tools

[edit]
  • ffmpeg is a command-line tool that converts audio or video formats. It can also capture and encode in real-time from various hardware and software sources[36] such as a TV capture card.
  • ffplay is a simple media player utilizing SDL and the FFmpeg libraries.
  • ffprobe is a command-line tool to display media information (text, CSV, XML, JSON), see also MediaInfo.

Libraries

[edit]
  • libswresample is a library containing audio resampling routines.
  • libavcodec is a library containing all of the native FFmpeg audio/video encoders and decoders. Most codecs were developed from scratch to ensure best performance and high code reusability.
  • libavformat (Lavf)[8] is a library containing demuxers and muxers for audio/video container formats.
  • libavutil is a helper library containing routines common to different parts of FFmpeg. This library includes hash functions, ciphers, LZO decompressor and Base64 encoder/decoder.
  • libswscale is a library containing video image scaling and colorspace/pixelformat conversion routines.
  • libavfilter is the substitute for vhook which allows the video/audio to be modified or examined (for debugging) between the decoder and the encoder. Filters have been ported from many projects including MPlayer and avisynth.
  • libavdevice is a library containing audio/video io through internal and external devices.

Supported hardware

[edit]

CPUs

[edit]

FFmpeg encompasses software implementations of video and audio compressing and decompressing algorithms. These can be compiled and run on many different instruction sets, including x86 (IA-32 and x86-64), PPC (PowerPC), ARM, DEC Alpha, SPARC, and MIPS.[37]

Special purpose hardware

[edit]

There are a variety of application-specific integrated circuits (ASICs) for audio/video compression and decompression. These ASICs can partially or completely offload the computation from the host CPU. Instead of a complete implementation of an algorithm, only the API is required to use such an ASIC.[38]

Firm ASIC purpose supported by FFmpeg Details
AMD UVD decoding Yes via VDPAU API and VAAPI
VCE encoding Yes via VAAPI, considered experimental[39]
Amlogic Amlogic Video Engine decoding ?
BlackMagic DeckLink encoding/decoding Yes real-time ingest and playout
Broadcom Crystal HD decoding Yes
Qualcomm Hexagon encoding/decoding Yes hwaccel[40]
Intel Intel Clear Video decoding Yes (libmfx, VAAPI)
Intel Quick Sync Video encoding/decoding Yes (libmfx, VAAPI)
Nvidia PureVideo / NVDEC decoding Yes via the VDPAU API as of FFmpeg v1.2 (deprecated)
via CUVID API as of FFmpeg v3.1[41]
NVENC encoding Yes as of FFmpeg v2.6

The following APIs are also supported: DirectX Video Acceleration (DXVA2, Windows), Direct3D 11 (D3D11VA, Windows), Media Foundation (Windows), Vulkan (VKVA), VideoToolbox (iOS, iPadOS, macOS), RockChip MPP, OpenCL, OpenMAX, MMAL (Raspberry Pi), MediaCodec (Android OS), V4L2 (Linux). Depending on the environment, these APIs may lead to specific ASICs, to GPGPU routines, or to SIMD CPU code.[38]

Supported codecs and formats

[edit]

Image formats

[edit]

FFmpeg supports many common and some uncommon image formats.

The PGMYUV image format is a homebrew variant of the binary (P5) PGM Netpbm format. FFmpeg also supports 16-bit depths of the PGM and PPM formats, and the binary (P7) PAM format with or without alpha channel, depth 8 bit or 16 bit for pix_fmts monob, gray, gray16be, rgb24, rgb48be, ya8, rgba, rgb64be.

Supported formats

[edit]

In addition to FFV1 and Snow formats, which were created and developed from within FFmpeg, the project also supports the following formats:

Group Format type Format name
ISO/IEC/ITU-T Video MPEG-1 Part 2, H.261 (Px64),[42] H.262/MPEG-2 Part 2, H.263,[42] MPEG-4 Part 2, H.264/MPEG-4 AVC, HEVC/H.265[31] (MPEG-H Part 2), MPEG-4 VCB (a.k.a. VP8), Motion JPEG, IEC DV video and CD+G
Audio MP1, MP2, MP3, AAC, HE-AAC, MPEG-4 ALS, G.711 μ-law, G.711 A-law, G.721 (a.k.a. G.726 32k), G.722, G.722.2 (a.k.a. AMR-WB), G.723 (a.k.a. G.726 24k and 40k), G.723.1, G.726, G.729, G.729D, IEC DV audio and Direct Stream Transfer
Subtitle MPEG-4 Timed Text (a.k.a. 3GPP Timed Text)
Image JPEG, Lossless JPEG, JPEG-LS, JPEG 2000, JPEG XL,[43] PNG, CCITT G3 and CCITT G4
Alliance for Open Media Video AV1[44]
Image AVIF[45]
EIA Subtitle EIA-608
CEA Subtitle CEA-708
SMPTE Video SMPTE 314M (a.k.a. DVCAM and DVCPRO), SMPTE 370M (a.k.a. DVCPRO HD), VC-1 (a.k.a. WMV3), VC-2 (a.k.a. Dirac Pro), VC-3 (a.k.a. AVID DNxHD)
Audio SMPTE 302M
Image DPX
ATSC/ETSI/DVB Audio Full Rate (GSM 06.10), AC-3 (Dolby Digital), Enhanced AC-3 (Dolby Digital Plus) and DTS Coherent Acoustics (a.k.a. DTS or DCA)
Subtitle DVB Subtitling (ETSI 300 743)
DVD Forum/Dolby Audio MLP / Dolby TrueHD
Subtitle DVD-Video subtitles
Xperi/DTS, Inc/QDesign Audio DTS Coherent Acoustics (a.k.a. DTS or DCA), DTS Extended Surround (a.k.a. DTS-ES), DTS 96/24, DTS-HD High Resolution Audio, DTS Express (a.k.a. DTS-HD LBR), DTS-HD Master Audio, QDesign Music Codec 1 and 2
Blu-ray Disc Association Subtitle PGS (Presentation Graphics Stream)
3GPP Audio AMR-NB, AMR-WB (a.k.a. G.722.2)
3GPP2 Audio QCELP-8 (a.k.a. SmartRate or IS-96C), QCELP-13 (a.k.a. PureVoice or IS-733) and Enhanced Variable Rate Codec (EVRC. a.k.a. IS-127)
World Wide Web Consortium Video Animated GIF[46]
Subtitle WebVTT
Image GIF, and SVG (via librsvg)
IETF Video FFV1
Audio iLBC (via libilbc), Opus and Comfort noise
International Voice Association Audio DSS-SP
SAC Video AVS video, AVS2 video[47] (via libdavs2), and AVS3 video (via libuavs3d)
Microsoft Video Microsoft RLE, Microsoft Video 1, Cinepak, Microsoft MPEG-4 v1, v2 and v3, Windows Media Video (WMV1, WMV2, WMV3/VC-1), WMV Screen and Mimic codec
Audio Windows Media Audio (WMA1, WMA2, WMA Pro and WMA Lossless), XMA (XMA1 and XMA2),[48] MSN Siren, MS-GSM and MS-ADPCM
Subtitle SAMI
Image Windows Bitmap, WMV Image (WMV9 Image and WMV9 Image v2), DirectDraw Surface, and MSP[49]
Interactive Multimedia Association Audio IMA ADPCM
Intel / Digital Video Interactive Video RTV 2.1 (Indeo 2), Indeo 3, 4 and 5,[42] and Intel H.263
Audio DVI4 (a.k.a. IMA DVI ADPCM), Intel Music Coder, and Indeo Audio Coder
RealNetworks Video RealVideo Fractal Codec (a.k.a. Iterated Systems ClearVideo), 1, 2, 3 and 4
Audio RealAudio v1 – v10, and RealAudio Lossless[50]
Subtitle RealText
Apple / Spruce Technologies Video Cinepak (Apple Compact Video), ProRes, Sorenson 3 Codec, QuickTime Animation (Apple Animation), QuickTime Graphics (Apple Graphics), Apple Video, Apple Intermediate Codec and Pixlet[51]
Audio ALAC
Image QuickDraw PICT
Subtitle Spruce subtitle (STL)
Adobe Flash Player (SWF) Video Screen video, Screen video 2, Sorenson Spark and VP6
Audio Adobe SWF ADPCM and Nellymoser Asao
Adobe / Aldus Image TIFF, PSD,[51] and DNG
Xiph.Org Video Theora
Audio Speex,[52] Vorbis, Opus and FLAC
Subtitle Ogg Writ
Sony Audio Adaptive Transform Acoustic Coding (ATRAC1, ATRAC3, ATRAC3Plus,[53] and ATRAC9[47])[42] and PSX ADPCM
NTT Audio TwinVQ
Google / On2 / GIPS Video Duck TrueMotion 1, Duck TrueMotion 2, Duck TrueMotion 2.0 Real Time, VP3, VP4,[54] VP5,[42] VP6,[42] VP7, VP8,[55] VP9[30] and animated WebP
Audio DK ADPCM Audio 3/4, On2 AVC and iLBC (via libilbc)
Image WebP[56]
Epic Games / RAD Game Tools Video Smacker video and Bink video
Audio Bink audio
CRI Middleware Audio ADX ADPCM, and HCA
Nintendo / NERD Video Mobiclip video
Audio GCADPCM (a.k.a. ADPCM THP), FastAudio, and ADPCM IMA MOFLEX
Synaptics / DSP Group Audio Truespeech
Electronic Arts / Criterion Games / Black Box Games / Westwood Studios Video RenderWare TXD,[57] Madcow, CMV, TGV, TGQ, TQI, Midivid VQ (MVDV), MidiVid 3.0 (MV30), Midivid Archival (MVHA), and Vector Quantized Animation (VQA)
Audio Electronic Arts ADPCM variants
Netpbm Image PBM, PGM, PPM, PNM, PAM, PFM and PHM
MIT/X Consortium/The Open Group Image XBM,[50] XPM and xwd
HPE / SGI / Silicon Graphics Video Silicon Graphics RLE 8-bit video,[46] Silicon Graphics MVC1/2[46]
Image Silicon Graphics Image
Oracle/Sun Microsystems Image Sun Raster
IBM Video IBM UltiMotion
Avid Technology / Truevision Video Avid 1:1x, Avid Meridien,[50] Avid DNxHD, Avid DNx444,[53] and DNxHR
Image Targa[46]
Autodesk / Alias Video Autodesk Animator Studio Codec and FLIC
Image Alias PIX
Activision Blizzard / Activision / Infocom Audio ADPCM Zork
Konami / Hudson Soft Video HVQM4 Video
Audio Konami MTAF, and ADPCM IMA HVQM4
Grass Valley / Canopus Video HQ, HQA, HQX and Lossless
Vizrt / NewTek Video SpeedHQ
Image Vizrt Binary Image[45]
Academy Software Foundation / ILM Image OpenEXR[50]
Mozilla Corporation Video APNG[56]
Matrox Video Matrox Uncompressed SD (M101) / HD (M102)
AMD/ATI Video ATI VCR1/VCR2
Asus Video ASUS V1/V2 codec
Commodore Video CDXL codec
Kodak Image Photo CD
Blackmagic Design / Cintel Image Cintel RAW
Houghton Mifflin Harcourt / The Learning Company / ZSoft Corporation Image PCX
Australian National University Image X-Face[46]
Bluetooth Special Interest Group Audio SBC, and mSBC
Qualcomm / CSR Audio QCELP, aptX, and aptX HD
Open Mobile Alliance / WAP Forum Image Wireless Bitmap

Muxers

[edit]

Output formats (container formats and other ways of creating output streams) in FFmpeg are called "muxers". FFmpeg supports, among others, the following:

Pixel formats

[edit]
Type Color Packed Planar Palette
Without alpha With alpha Without alpha With alpha Chroma-interleaved With alpha
Monochrome Binary (1-bit monochrome) monoblack, monowhite
Grayscale 8/9/10/12/14/16bpp 16/32bpp
RGB RGB 1:2:1 (4-bit color) 4bpp
RGB 3:3:2 (8-bit color) 8bpp
RGB 5:5:5 (High color) 16bpp
RGB 5:6:5 (High color) 16bpp
RGB/BGR 24/30[p 1]/48bpp 32[p 2]/64bpp 8bit->32bpp
GBR[p 3] 8/9/10/12/14/16bpc 8/10/12/16bpc
RGB Float RGB 32bpc 16/32bpc
GBR 32bpc 32bpc
YUV YVU 4:1:0 (9bpp (YVU9))[p 4]
YUV 4:1:0 9bpp
YUV 4:1:1 8bpc (UYYVYY) 8bpc (8bpc (NV11))
YVU 4:2:0 (8bpc (YV12))[p 4] 8 (NV21)
YUV 4:2:0 8[p 5]/9/10/12/14/16bpc 8/9/10/16bpc 8 (NV12)/10 (P010)/12 (P012)/16bpc (P016)
YVU 4:2:2 (8bpc (YV16))[p 4] (8bpc (NV61))
YUV 4:2:2 8 (YUYV[p 6] and UYVY)/10 (Y210)/12bpc (Y212)[p 7] 8[p 8]/9/10/12/14/16bpc 8/9/10/12/16bpc 8 (NV16)/10 (NV20 and P210)/16bpc (P216)
YUV 4:4:0 8/10/12bpc
YVU 4:4:4 (8bpc (YV24))[p 4] 8bpc (NV42)
YUV 4:4:4 8 (VUYX)/10[p 9]/12bpc[p 10] 8[p 11] / 16bpc (AYUV64)[p 12] 8[p 13]/9/10/12/14/16bpc 8/9/10/12/16bpc 8 (NV24)/10 (P410)/ 16bpc (P416)
XYZ XYZ 4:4:4[p 14] 12bpc
Bayer BGGR/RGGB/GBRG/GRBG 8/16bpp
  1. ^ 10-bit color components with 2-bit padding (X2RGB10)
  2. ^ RGBx (rgb0) and xBGR (0bgr) are also supported
  3. ^ used in YUV-centric codecs such like H.264
  4. ^ a b c d YVU9, YV12, YV16, and YV24 are supported as rawvideo codec in FFmpeg.
  5. ^ I420 a.k.a. YUV420P
  6. ^ aka YUY2 in Windows
  7. ^ UYVY 10bpc without a padding is supported as bitpacked codec in FFmpeg. UYVY 10bpc with 2-bits padding is supported as v210 codec in FFmpeg. 16bpc (Y216) is supported as targa_y216 codec in FFmpeg.
  8. ^ I422 a.k.a. YUV422P
  9. ^ XV30 a.k.a. XVYU2101010
  10. ^ XV36
  11. ^ VUYA a.k.a. AYUV
  12. ^ 10bpc (Y410), 12bpc (Y412), and Y416 (16bpc) are not supported.
  13. ^ I444 a.k.a. YUV444P
  14. ^ used in JPEG2000

FFmpeg does not support IMC1-IMC4, AI44, CYMK, RGBE, Log RGB and other formats. It also does not yet support ARGB 1:5:5:5, 2:10:10:10, or other BMP bitfield formats that are not commonly used.

Supported protocols

[edit]

Open standards

[edit]

Supported filters

[edit]

FFmpeg supports, among others, the following filters.[68]

Audio

[edit]

Video

[edit]
  • Transformations
    • Cropping (crop, cropdetect)
    • Fading (fade)
    • Scaling (scale)
    • Padding (pad)
    • Rotation (rotate)
    • Transposition (transpose)
    • Others:
      • Lens correction (lenscorrection)
      • OpenCV filtering (ocv)
      • Perspective correction (perspective)
  • Temporal editing
    • Framerate (fps, framerate)
    • Looping (loop)
    • Trimming (trim)
  • Deinterlacing (bwdif, idet, kerndeint, nnedi, yadif, w3fdif)
  • Inverse Telecine
  • Filtering
  • Denoising (atadenoise, bitplanenoise, dctdnoiz, owdenoise, removegrain)
  • Logo removal (delogo, removelogo)
  • Subtitles (ASS, subtitles)
  • Alpha channel editing (alphaextract, alphamerge)
  • Keying (chromakey, colorkey, lumakey)
  • Frame detection
    • Black frame detection (blackdetect, blackframe)
    • Thumbnail selection (thumbnail)
  • Frame Blending (blend, tblend, overlay)
  • Video stabilization (vidstabdetect, vidstabtransform)
  • Color and Level adjustments
    • Balance and levels (colorbalance, colorlevels)
    • Channel mixing (colorchannelmixer)
    • Color space (colorspace)
    • Parametric adjustments (curves, eq)
  • Histograms and visualization
  • Drawing
  • OCR
  • Quality measures
  • Lookup Tables
    • lut, lutrgb, lutyuv, lut2, lut3d, haldclut

Supported test patterns

[edit]

Supported LUT formats

[edit]

Supported media and interfaces

[edit]

FFmpeg supports the following devices via external libraries.[70]

Media

[edit]

Physical interfaces

[edit]
  • IEEE 1394 (a.k.a. FireWire; via libdc1394 and libraw1394; input only)
  • IEC 61883 (via libiec61883; input only)
  • DeckLink
  • Brooktree video capture chip (via bktr driver; input only)

Audio IO

[edit]

Video IO

[edit]

Screen capture and output

[edit]

Others

[edit]

Applications

[edit]
[edit]

FFmpeg contains more than 100 codecs,[71] most of which use compression techniques of one kind or another. Many such compression techniques may be subject to legal claims relating to software patents.[72] Such claims may be enforceable in countries like the United States which have implemented software patents, but are considered unenforceable or void in member countries of the European Union, for example.[73][original research] Patents for many older codecs, including AC3 and all MPEG-1 and MPEG-2 codecs, have expired.[citation needed]

FFmpeg is licensed under the LGPL license, but if a particular build of FFmpeg is linked against any GPL libraries (notably x264), then the entire binary is licensed under the GPL.

Projects using FFmpeg

[edit]

FFmpeg is used by software such as Blender, Cinelerra-GG Infinity, HandBrake, Kodi, MPC-HC, Plex, Shotcut, VirtualDub2 (a VirtualDub fork),[74] VLC media player, xine and YouTube.[75][76] It handles video and audio playback in Google Chrome[76] and the Linux version of Firefox.[77] GUI front-ends for FFmpeg have been developed, including Multimedia Xpert[78] and XMedia Recode.

FFmpeg is used by ffdshow, FFmpegInterop, the GStreamer FFmpeg plug-in, LAV Filters and OpenMAX IL to expand the encoding and decoding capabilities of their respective multimedia platforms.

As part of NASA's Mars 2020 mission, FFmpeg is used by the Perseverance rover on Mars for image and video compression before footage is sent to Earth.[79]

Embedded applications

[edit]

FFmpeg is also being used in embedded applications, where it can be used with custom hardware to simplify version and dependency management and also to provide operating system abstraction across multiple different OS and processor manufacturers. For example[80]

See also

[edit]

References

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
FFmpeg is a free and open-source multimedia framework that provides libraries and command-line tools for decoding, encoding, transcoding, muxing, demuxing, streaming, filtering, and playing a wide array of audio, video, and related multimedia formats, from legacy standards to the latest developments.[1] It encompasses core libraries such as libavcodec for handling audio and video codecs, libavformat for multimedia container formats, libswscale for image scaling and pixel format conversion, and libavutil for utility functions, enabling both developers to build applications and end users to perform media processing tasks.[1] The project originated in 2000, launched by French programmer Fabrice Bellard, who led its early development and established it as a comprehensive solution for multimedia manipulation under an open-source model.[2] Since then, FFmpeg has evolved through community contributions, emphasizing technical excellence, minimal external dependencies by favoring its own implementations, and rigorous testing via the FATE infrastructure to ensure reliability across diverse scenarios.[1] FFmpeg's flagship command-line tool, ffmpeg, functions as a universal media converter capable of ingesting inputs from files, live devices, or streams, applying complex filters for effects like resizing or overlaying, and outputting to numerous formats. Common examples include converting WebM to MP4 using the simple command ffmpeg -i input.webm output.mp4 and creating a video from a PNG image sequence using ffmpeg -framerate 24 -i image_%03d.png -c:v libx264 -pix_fmt yuv420p output.mp4, which has made it indispensable for tasks ranging from format conversion to real-time streaming and media synthesis.[3][4] Additional utilities include ffplay, a simple media player for testing playback, and ffprobe, for inspecting media properties.[3] The framework's high portability allows it to compile and operate seamlessly on platforms including Linux, macOS, Windows, BSDs, and Solaris, with ongoing updates to address security vulnerabilities and incorporate new codecs.[1]

Overview

Definition and Purpose

FFmpeg is a leading open-source multimedia framework designed for handling the recording, conversion, and streaming of audio and video content. It serves as a comprehensive collection of libraries and tools that enable developers and users to process multimedia data efficiently across various applications, from simple file conversions to complex streaming setups. As a versatile solution, FFmpeg supports a broad range of operations essential for multimedia manipulation, making it a foundational component in software for video editing, broadcasting, and content delivery.[1] At its core, FFmpeg provides capabilities for demuxing (separating streams from containers), decoding (converting compressed media into raw data), encoding (compressing raw data into specified formats), muxing (combining streams into containers), transcoding (converting between formats), filtering (applying effects or transformations), and streaming (transmitting media over networks). These functions allow it to handle inputs from diverse sources, such as files, live devices, or network streams, and output them in numerous formats while supporting real-time processing. The framework's design emphasizes flexibility, enabling tasks like format conversion without quality loss or the addition of subtitles and metadata during processing.[1][5] FFmpeg is renowned for its cross-platform compatibility, operating seamlessly on Windows, macOS, and Linux systems, which broadens its accessibility for users and integrators worldwide. Its primary interface is a powerful command-line tool that facilitates scripting and automation, allowing complex workflows to be executed via simple text commands or integrated into larger programs. This command-line approach, combined with its lightweight footprint, makes FFmpeg ideal for both standalone use and embedding in applications like media servers or mobile apps.[6][7] A typical FFmpeg processing pipeline follows a sequential flow: input demuxing and decoding, optional filtering for modifications, encoding to the target codec, and final muxing for output delivery. This modular pipeline ensures efficient handling of multimedia tasks, from basic conversions to advanced real-time streaming scenarios.[5]

Licensing and Development Model

The project's core libraries, such as libavcodec and libavformat, are primarily released under the GNU Lesser General Public License (LGPL) version 2.1 or later, which permits integration into both open-source and proprietary applications provided that the source code of the libraries is made available and dynamic linking is used where applicable.[8] However, certain optional components, including GPL-licensed encoders such as libx264 and non-free components such as libfdk_aac, as well as advanced filters, are only available under the GNU General Public License (GPL) version 2 or later when enabled, requiring derivative works to also be licensed under GPL.[9] The command-line tools, including the flagship ffmpeg executable, are licensed under the GPL version 2 or later to leverage these full-featured components.[8] The development of FFmpeg follows an open-source, community-driven model hosted on a public Git repository at git.ffmpeg.org/ffmpeg.git, where changes are tracked and merged through a patch-based submission process.[10] Contributions are coordinated via the ffmpeg-devel mailing list for technical discussions and code reviews, supplemented by real-time collaboration on IRC channels #ffmpeg and #ffmpeg-devel on the Libera.Chat network.[11] To submit patches, developers must adhere to strict coding rules outlined in the project's documentation, ensuring compatibility with existing APIs and performance standards.[10] Governance of FFmpeg is managed by the FFmpeg Multimedia Community, a collective of active contributors who form the General Assembly; active status is granted to those who have authored more than 20 patches in the preceding 36 months or serve as maintainers for specific modules.[12] Module maintainers oversee targeted areas like codecs or demuxers, reviewing and committing contributions while enforcing project policies.[10] All contributions must be licensed under the LGPL 2.1 or GPL 2 (or any later version).[10] Handling of third-party code is governed by policies that prioritize license compatibility; external libraries are either relicensed to match FFmpeg's terms, distributed separately, or disabled by default to avoid conflicts with non-free or incompatible licenses.[8] In one notable instance, a historical fork known as Libav led to temporary divergence, but FFmpeg has since integrated many of its advancements, fostering renewed collaboration among developers.[13]

History

Origins and Early Development

FFmpeg was founded in late 2000 by French programmer Fabrice Bellard under the pseudonym Gérard Lantau, initiated as an independent project and soon integrated into the MPlayer project as its multimedia engine.[14][15] The initial motivation stemmed from the need for a complete, cross-platform solution for handling multimedia formats, particularly amid ongoing GPL licensing disputes that limited the use of proprietary codecs in open-source software.[16] Bellard aimed to create a versatile framework for decoding, encoding, and processing audio and video, licensed under the LGPL to facilitate integration into various applications while adhering to open-source principles.[14] The project was initiated in late 2000, with the first public release on December 20, 2000. Active development began in the second half of 2001.[14][17] A key early milestone was the integration of libavcodec, a core library providing support for multiple audio and video codecs, which laid the foundation for FFmpeg's extensive format compatibility.[15] Initially a solo effort by Bellard, the project quickly attracted contributions from developers involved in MPlayer, reflecting overlapping communities in the open-source multimedia space.[16] Bellard led FFmpeg until 2003, after which he stepped away to pursue other initiatives, such as QEMU.[15] Michael Niedermayer assumed the role of maintainer in 2003, ushering in a transition to a collaborative, community-driven model that encouraged broader participation and rapid iteration. Niedermayer led the project until 2015, after which it continued under a collaborative team model.[16] This shift was evident in the growing number of contributors and the project's increasing stability, culminating in the first official release, version 0.5, in March 2009.[15] By 2010, FFmpeg had established itself as a robust multimedia toolkit, though internal tensions would later lead to the 2011 fork creating Libav, a pivotal event that eventually prompted reconciliation and unified development.[16]

Major Releases and Milestones

FFmpeg releases are assigned codenames honoring notable scientists and mathematicians.[18] Certain branches, typically the .1 minor releases of odd major versions such as 5.1 "Riemann" and 7.1 "Péter", are designated as long-term support (LTS) versions maintained for at least three years.[19] FFmpeg's major version numbers align with increments in its core library versions, providing clarity on compatibility for developers integrating libraries like libavformat. Specifically, FFmpeg 4.x corresponds to libavformat 58.x, FFmpeg 5.x to 59.x, FFmpeg 6.x to 60.x, and FFmpeg 7.x to 61.x.[18] In 2015, FFmpeg reconciled with the Libav fork by merging key improvements from Libav's master branch into FFmpeg 2.8 "Feynman," released on September 9, which incorporated changes up to Libav master as of June 10, 2015, and Libav 11 as of June 11, 2015, thereby unifying development efforts and reducing fragmentation in the multimedia ecosystem.[20] This integration enhanced compatibility and feature parity without fully resolving underlying governance differences. FFmpeg 4.0 "Wu," released on April 20, 2018, marked a significant milestone with initial support for the AV1 video codec, including a decoder and low-latency encoding options, alongside major hardware acceleration enhancements such as AMD's Advanced Media Framework (AMF) for GPU encoding on Radeon hardware and improved VA-API integration for Intel and AMD platforms. These additions broadened FFmpeg's applicability in professional video workflows, enabling efficient transcoding on consumer-grade hardware. The release also dropped support for outdated platforms like Windows XP and the deprecated ffserver tool. Subsequent versions built on this foundation, with FFmpeg 5.0 "Lorentz," released on January 17, 2022, introducing an AV1 low-overhead bitstream format muxer and slice-based threading in the swscale library for faster scaling operations, while expanding support for external AV1 encoders like SVT-AV1. FFmpeg 6.0 "Von Neumann," released on February 28, 2023, advanced filter capabilities, including improvements to neural network-based tools such as the sr (super-resolution) filter using convolutional neural networks for AI-driven upscaling and the arnndn filter for recurrent neural network audio noise reduction, alongside multi-threading for the ffmpeg command-line interface and RISC-V optimizations. By 2024 and 2025, FFmpeg emphasized emerging codecs and AI integration. FFmpeg 7.0 "Dijkstra," released on April 5, 2024, added a native Versatile Video Coding (VVC, or H.266) decoder supporting a substantial subset of features, optimized for multi-threading and on par with reference implementations in performance.[21] FFmpeg 8.0 "Huffman," released on August 22, 2025, further integrated machine learning with the Whisper filter for AI-based speech transcription, enhanced Vulkan compute shaders for AV1 encoding and VVC VA-API decoding, and refined ML upscaling via the sr filter for real-time 1080p-to-4K workflows. These developments responded to patent challenges around proprietary codecs like HEVC and VVC by prioritizing open standards such as AV1 while providing optional support for patented formats, with users responsible for licensing compliance to avoid infringement risks.[8]

Core Components

Libraries

FFmpeg's modular architecture is built around a set of core libraries that enable multimedia processing tasks such as decoding, encoding, filtering, and format handling. These libraries are designed to be reusable by external applications and form the foundation for FFmpeg's command-line tools.[1] The libavcodec library provides a generic framework for encoding and decoding audio, video, and subtitle streams, incorporating numerous decoders, encoders, and bitstream filters to support a wide range of codecs.[22] It serves as the central component for compression and decompression operations in the multimedia pipeline.[23] libavformat handles the multiplexing and demultiplexing of audio, video, and subtitle streams into various container formats, while also supporting multiple input and output protocols for streaming and file I/O.[24] This library abstracts the complexities of media container structures and network protocols, enabling seamless data flow between sources and sinks.[23] libavutil offers a collection of utility functions essential for portable multimedia programming, including safe string handling, random number generation, data structures like dictionaries and lists, mathematics routines, and error management tools.[25] It acts as a foundational layer, providing common functionalities that prevent code duplication across other FFmpeg libraries. Among the other key libraries, libswscale performs optimized scaling, colorspace conversion, and pixel format transformations for video frames.[26] Configuration of scaling is achieved through sws_flags, which allows selection of various algorithms and modifiers as detailed in the Filters and Effects section on geometric operations.[27] libswresample specializes in audio resampling, rematrixing, and sample format conversions to ensure compatibility across different audio configurations.[28] libavfilter implements a flexible framework for audio and video filtering, supporting a variety of sources, filters, and sinks to build complex processing chains.[29] libavdevice provides a generic framework for grabbing from and rendering to multimedia input/output devices, such as Video4Linux2 and ALSA.[30] These libraries exhibit interdependencies that create a cohesive processing pipeline: libavutil supplies utilities to all others; libavformat demuxes input streams into packets, which libavcodec decodes into raw frames; these frames may then pass through libavfilter for effects, libswscale for video adjustments, or libswresample for audio modifications before libavcodec re-encodes them and libavformat muxes the output.[23] This layered design allows for efficient, modular media manipulation, with higher-level libraries relying on lower ones for core operations. The command-line tools in FFmpeg are constructed on top of these libraries to provide user-friendly interfaces for common tasks.[1]

Command-Line Tools

FFmpeg provides several command-line tools that enable users to perform multimedia processing tasks directly from the terminal, leveraging the project's core libraries for demuxing, decoding, encoding, and muxing. These tools are designed for efficiency and flexibility, allowing tasks such as format conversion, media playback, and stream analysis without requiring graphical interfaces or custom programming.[31] The primary tool, ffmpeg, serves as a versatile multimedia converter and processor. It reads input from files, devices, or streams, applies optional processing like filtering or transcoding, and outputs to various formats. The basic syntax follows the form ffmpeg [global options] [{input options} -i input] ... [output options] output, where -i specifies the input file or URL. For instance, to convert an AVI file to MP4 with re-encoding, the command is ffmpeg -i input.avi output.mp4, which automatically selects appropriate codecs for the output container. Similarly, converting WebM to MP4 is straightforward using ffmpeg -i input.webm output.mp4. This conversion remains quick, accessible, and with no major difficulties, as demonstrated by FFmpeg and confirmed by various tools and guides.[32] To avoid re-encoding and simply remux streams, users can employ ffmpeg -i input.avi -c copy output.mkv, preserving quality while changing the container. Another useful remuxing operation is incorporating external audio and/or subtitles into an MP4 container without re-encoding the video or audio streams. This is particularly useful for adding soft subtitles (embedded as separate streams that can be toggled on or off in compatible players), including support for Chinese characters. For proper display of Chinese characters in soft subtitles, the subtitle file (e.g., SRT) must be encoded in UTF-8 to avoid garbled text. Additionally, specifying language metadata enhances player compatibility. If the input video file already contains both video and audio streams and only subtitles need to be added:
ffmpeg -i input.mp4 -i chinese.srt -c copy -c:s mov_text -metadata:s:s:0 language=zho output.mp4
This embeds the subtitles as a soft stream without re-encoding the video or audio, making the process fast and preserving original quality. The -c:s mov_text converts the SRT subtitles to the MP4-compatible mov_text subtitle format. The language is set to 'zho' for Chinese. For cases with additional audio and subtitles:
ffmpeg -i video.mp4 -i audio.aac -i subtitles.srt -c:v copy -c:a copy -c:s mov_text output.mp4
Replace video.mp4, audio.aac, subtitles.srt, input.mp4, chinese.srt, and output.mp4 with the actual filenames. The video codec (e.g., H.264/AVC) and audio codec (e.g., AAC) must be compatible with the MP4 container for stream copying to succeed. For explicit stream mapping (useful if inputs have multiple streams), add mapping options such as -map 0:v -map 1:a -map 2:s before the output filename.[5] Common options include -c or -codec for selecting specific audio/video codecs (e.g., -c:v libx264 for H.264 encoding), -b for bitrate control (e.g., -b:v 2M for 2 Mbps video bitrate), and input/output specifiers like -ss for seeking to a timestamp or -t for duration limiting. These options facilitate scripting for batch processing, such as converting multiple files via loops in shell scripts. The tool relies on FFmpeg's libraries like libavformat for handling formats and libavcodec for codec operations.[3] In addition to the -ss option for seeking to a specific start position, FFmpeg includes the -sseof position (input) option, which seeks relative to the end of file—negative values specify seconds earlier in the file, and 0 is at EOF. This is particularly useful for trimming from the end without probing duration first (e.g., ffmpeg -sseof -2 -i input.mp4 -c copy output.mp4 to cut off the last 2 seconds with stream copy). The -sseof option first appeared in the FFmpeg 3.x series, with community references and usage becoming prominent around 2016–2018, and it was documented as a stable feature by the time of FFmpeg 4.0 (released April 2018). A standard command is: ffmpeg -framerate 24 -i image_%03d.png -c:v libx264 -pix_fmt yuv420p output.mp4 Replace image_%03d.png with the actual file pattern (e.g., frame_%04d.png for four-digit zero-padded numbering). Adjust -framerate (e.g., to 30) to set the desired playback speed. Add -vf scale=1920:1080 for resizing to 1920x1080 resolution. Use -crf 18 for higher quality (lower CRF values provide better quality at larger file sizes; the libx264 default is 23). This method remains standard for generating videos from image sequences.[33] Converting a video clip, such as an MP4, to a high-quality looping animated GIF is another common use case. A widely recommended command employs palette optimization for improved colors, frame rate reduction and scaling for manageable file size, Lanczos resampling for sharpness, and infinite looping: ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse" -loop 0 output.gif Key options include:
  • fps=10: Reduces frame rate to balance smoothness and file size (adjust to 12-15 for smoother animation if file size allows).
  • scale=480:-1:flags=lanczos: Resizes to 480 pixels width while preserving aspect ratio, using high-quality Lanczos resampling.
  • split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse: Splits the stream, generates a custom palette from one copy, and applies it to the other for optimized 256-color representation suitable for GIF.
  • -loop 0: Enables infinite looping, essential for animated GIFs.
For improved quality in gradients and reduced banding, dithering can be added: ffmpeg -i input.mp4 -vf "fps=10,scale=480:-1:flags=lanczos,split[s0][s1];[s0]palettegen=max_colors=256:stats_mode=full[p];[s1][p]paletteuse=dither=floyd_steinberg" -loop 0 output.gif Here, palettegen=max_colors=256:stats_mode=full analyzes all frames for a more accurate palette, and dither=floyd_steinberg applies error-diffusion dithering. To create a seamless loop from a specific segment of the video, use -ss (start time) and -t (duration) to trim the input, e.g., ffmpeg -ss 00:00:05 -t 00:00:03 .... This approach is a standard method for generating animated GIFs from video sources using FFmpeg's filters.[34][35] A practical application of ffmpeg involves concatenating multiple MP4 video files without re-encoding to prevent audio glitches or pops, achieved by leveraging MPEG-TS's design for seamless broadcasting concatenation. Inputs are first converted to MPEG-TS using stream copy, such as ffmpeg -i input.mp4 -c copy -bsf:v h264_mp4toannexb output.ts for H.264 video adjustment, allowing the TS files to be joined via the concat demuxer or protocol without re-encoding. The result is then remuxed to MP4 with ffmpeg -i concatenated.ts -c copy -bsf:a aac_adtstoasc final.mp4 to correct AAC audio headers.[36] For Matroska (MKV) files with identical stream parameters (codec, resolution, frame rate, pixel format for video; similar for audio), direct concatenation without re-encoding is possible using the concat demuxer. Create a text file (e.g., list.txt) listing the files:
file 'input1.mkv'
file 'input2.mkv'
file 'input3.mkv'
Then run the command:
ffmpeg -f concat -safe 0 -i list.txt -c copy output.mkv
-f concat specifies the concat demuxer, -safe 0 allows absolute paths or non-standard filenames if needed, and -c copy copies streams without re-encoding. This method is efficient when inputs match exactly. If streams differ slightly, re-encoding may be required (e.g., using -c:v libx264).[36] Similar lossless concatenation can be applied to MP3 audio files, provided they have identical encoding parameters (e.g., sample rate, bitrate, channels). As of 2026, the best and most recommended way to merge MP3 files losslessly in the Linux terminal is using FFmpeg with the concat demuxer method. This avoids re-encoding and preserves original quality. Create a text file (e.g., mylist.txt) listing the files:
file 'file1.mp3'
file 'file2.mp3'
This can be generated automatically for all MP3 files in the current directory using:
printf "file '%s'\n" *.mp3 > mylist.txt
Then run:
ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.mp3
The -safe 0 option permits absolute paths or filenames with special characters. This method is reliable for MP3 files due to their straightforward stream structure. FFmpeg is preferred over alternatives such as mp3wrap (an older tool that may lose metadata) or mp3cat (a lightweight Go-based tool in maintenance mode since 2024), as FFmpeg remains actively maintained and more versatile.[36][37] While the concat demuxer enables lossless merging of compatible MP3 files using stream copy, re-encoding may be desired for bitrate adjustment, quality modification, or applying audio filters. When no -c copy is specified, FFmpeg defaults to re-encoding audio for conversions to the same format, using the libmp3lame encoder for MP3 output. A basic command to re-encode an MP3 file is:
ffmpeg -i input.mp3 output.mp3
This decodes the input and re-encodes using default libmp3lame settings, typically approximately 128 kbit/s CBR. For explicit control and to ensure re-encoding with specified parameters:
ffmpeg -i input.mp3 -c:a libmp3lame -b:a 192k output.mp3
The -c:a libmp3lame specifies the LAME MP3 encoder, and -b:a 192k sets the bitrate (adjustable; use -q:a 0-9 for VBR quality control, where lower values indicate higher quality). Applying an audio filter forces re-encoding, as filters require decoded frames. For example:
ffmpeg -i input.mp3 -af volume=1.5 output.mp3
This applies the filter and re-encodes the audio.[5][38] ffplay functions as a lightweight media player for quick playback and inspection of audio/video files or streams. Built on FFmpeg libraries and SDL for rendering, it supports basic playback controls, including keyboard shortcuts for seeking forward: the right arrow key seeks forward 10 seconds, the up arrow key seeks forward 1 minute, and the Page Up key seeks forward 10 minutes. ffplay does not support continuous fast-forward playback (e.g., 2x speed) and instead relies on these discrete time jumps via the keys. It is ideal for testing media compatibility without external players. The standard invocation is ffplay [options] input_file, which plays the input using default settings. Key options include -autoexit to quit upon reaching the end, -loop 0 for infinite looping, -vf for simple video filters (e.g., -vf scale=640:480 to resize), and -af for audio adjustments. For example, ffplay -autoexit input.mp4 plays a file and exits automatically, useful for verifying stream integrity. It displays stream information like resolution and framerate during playback, aiding debugging.[39] ffplay can also play RTSP streams. For reliable playback over unreliable networks, specify TCP transport instead of the default UDP: ffplay -rtsp_transport tcp rtsp://your.rtsp.url:554/stream. This ensures data delivery without packet loss.[39] On Ubuntu systems using the GNOME desktop environment, users can create a desktop launcher for a specific RTSP stream with ffplay by creating a .desktop file on the desktop (e.g., ~/Desktop/rtsp-stream.desktop) with the following content (replacing the URL as appropriate):
[Desktop Entry]
Type=Application
Terminal=false
Icon=multimedia-video-player
Name=RTSP Stream
Exec=ffplay -rtsp_transport tcp rtsp://your.rtsp.url:554/stream
Comment=Launch RTSP stream with ffplay
After saving the file, right-click it on the desktop and select "Allow Launching" (or open Properties > Permissions and check "Allow executing file as program"). Double-clicking the icon launches ffplay with the specified stream. In recent Ubuntu versions (e.g., 24.04 and later), desktop icons are supported natively; if icons do not appear, ensure desktop configuration is enabled or use GNOME extensions as needed. ffprobe acts as a multimedia stream analyzer, extracting detailed properties and metadata from files or streams in human-readable or structured formats. It probes inputs without decoding the entire content, making it efficient for large files. Usage is ffprobe [options] input_file, with common flags like -show_format to display container details (e.g., duration, bitrate), -show_streams to list stream specifics (e.g., codec, resolution, sample rate), and -show_entries for targeted output (e.g., -show_entries format=duration:[size](/page/Size) for file duration and size). An example command ffprobe -v quiet -print_format json -show_format -show_streams input.mkv outputs JSON-formatted data suitable for scripting or integration. This tool is particularly valuable for inspecting metadata like tags or chapters before processing with ffmpeg.[40] To display only audio stream information for an MP4 file (or other containers, as ffprobe handles formats generically), use: ffprobe -show_streams -select_streams a input.mp4. The -select_streams a option filters for audio streams (a = audio), while -show_streams provides details such as codec, sample rate, channels, and bit rate. For cleaner output suppressing banners and errors, use: ffprobe -v error -hide_banner -show_streams -select_streams a input.mp4. FFmpeg previously included ffserver, a streaming server for HTTP-based media delivery from files or live inputs, configurable via a dedicated .conf file for feeds and streams. However, it was deprecated due to maintenance challenges and fully removed in January 2018, with users directed to external tools or older release branches (e.g., 3.4) for legacy needs.[41]

Supported Media Handling

Codecs

FFmpeg's libavcodec library provides extensive support for both audio and video codecs, encompassing decoders and encoders for a variety of compression standards. This enables users to handle media transcoding, streaming, and playback across diverse applications.[42] Audio codec support in FFmpeg includes native implementations for key lossy formats such as AAC (specifically AAC-LC), which serves as the default encoder for efficient, high-fidelity compression suitable for multimedia containers and is recommended for live streaming as it is required for reliable ingest on major platforms such as Twitch. MP3 encoding is facilitated through the external libmp3lame library, which serves as the default MP3 encoder when re-encoding audio to MP3 format without stream copying (for example, ffmpeg -i input.mp3 output.mp3 re-encodes using libmp3lame with default parameters, typically around 128 kbps CBR). Offering broad compatibility despite its older design. For detailed command-line usage and options to control bitrate or quality (such as -b:a for CBR or -q:a for VBR), refer to the Command-Line Tools section. Opus, integrated via libopus, excels in low-latency applications like VoIP and provides superior quality at low bitrates compared to AAC or MP3, though it is not supported for ingest on these platforms. Vorbis, using libvorbis, delivers open-source lossy compression with strong performance for music and general audio. These codecs are complemented by lossless options like FLAC, which preserves exact audio fidelity without generational loss, making it ideal for archival purposes. Hardware acceleration for audio codecs is available through APIs such as those in VA-API or NVENC when supported by the underlying hardware.[42][43][44][5] FFmpeg also supports the synthesis of MIDI files to audio formats such as WAV using the optional libfluidsynth library, which requires compilation with the --enable-libfluidsynth flag. This enables the fluidsynth decoder within libavcodec to render MIDI events into audio using a default soundfont (typically a system default or one compiled in). A basic conversion uses the command:
ffmpeg -i input.mid output.wav
FFmpeg does not provide a direct command-line option to specify a custom soundfont for its built-in fluidsynth decoder. For custom soundfonts and finer control over synthesis parameters (such as gain or sample rate), the standalone FluidSynth tool is recommended instead. An example workflow outputs raw audio with FluidSynth:
fluidsynth -ni -F output.raw -r 44100 -g 0.8 /path/to/soundfont.sf2 input.mid
followed by conversion to WAV via FFmpeg:
ffmpeg -f s16le -ar 44100 -ac 2 -i output.raw output.wav
Alternatively, FluidSynth can use audio drivers for direct output in supported setups.[42][45] Video codec capabilities cover major standards, with H.264/AVC decoding and encoding widely supported for its balance of quality and efficiency in broadcast and web video. H.265/HEVC, offering approximately 50% better compression than H.264 at similar quality levels, is handled via dedicated decoders and encoders. VP9 provides royalty-free encoding and decoding for web-optimized video, while AV1, the successor to VP9, achieves even higher efficiency with up to 30% bitrate savings over HEVC, both integrated for modern streaming needs. FFmpeg includes support for H.266/VVC since version 7.0 (2024), with a native decoder providing full support as of version 7.1, and encoding available through libvvenc wrappers, positioning it for ultra-high-definition applications. Lossless video codecs like FFV1 ensure bit-exact reproduction, contrasting with the lossy nature of H.264, HEVC, VP9, and AV1, which discard data to reduce file sizes. Patent-free alternatives such as VP9 and AV1 avoid licensing royalties, unlike H.264 and HEVC, which involve patent pools.[42][46][47][48] FFmpeg differentiates between decoders, which unpack compressed streams, and encoders, which compress raw media, with many codecs supporting both but some limited to one direction—for example, certain proprietary decoders lack encoding counterparts. Third-party libraries significantly extend functionality; libx264 delivers tunable H.264 encoding with advanced rate control and preset options for optimized performance, while libx265 provides similar enhancements for HEVC, both compilable into FFmpeg for superior results over basic native encoders. These integrations, maintained by projects like VideoLAN, allow developers to leverage community-driven improvements without altering FFmpeg's core.[42][46]

Formats and Containers

FFmpeg's libavformat library provides extensive support for container formats, which package multiple media streams such as audio, video, and subtitles into a single file or stream.[49] Common container formats include MP4 (also known as QuickTime or MOV), Matroska (MKV), Audio Video Interleave (AVI), and WebM, each designed to handle synchronized playback of diverse media elements while supporting metadata, chapters, and attachments.[49] Additionally, FFmpeg accommodates segmented container formats like HTTP Live Streaming (HLS) and Dynamic Adaptive Streaming over HTTP (DASH), enabling the creation of playlist-based files for adaptive bitrate delivery in live or on-demand scenarios.[49] Muxers and demuxers form the core of FFmpeg's format handling, with muxers responsible for combining encoded audio, video, and subtitle streams into a cohesive container, ensuring proper synchronization and encapsulation according to the format's specifications. For example, FFmpeg's Matroska muxer strictly supports only audio, video, and subtitle streams and does not support data streams such as timed_id3 metadata (common in HLS/TS inputs), resulting in the error "Only audio, video, and subtitles are supported for Matroska" when attempting to mux such a stream into a Matroska (.mkv) container.[50] Similarly, MP4 containers support subtitles in the mov_text format (compatible with QuickTime/MOV), and FFmpeg can mux SRT subtitles into MP4 via conversion with -c:s mov_text while using stream copy for video and audio streams (-c:v copy and -c:a copy) without re-encoding, preserving their quality.[5] Demuxers perform the reverse operation, parsing the container to extract individual streams for decoding or further processing, thereby facilitating tasks like transcoding or stream analysis.[24] This bidirectional capability allows FFmpeg to seamlessly convert between formats without re-encoding the underlying media, preserving quality and efficiency.[5] For static media, FFmpeg supports a variety of image formats through dedicated muxers and demuxers, including Portable Network Graphics (PNG) for lossless compression, Joint Photographic Experts Group (JPEG) for lossy photographic images, and Tagged Image File Format (TIFF) for high-quality, multi-page archiving.[46] These formats are particularly useful for handling single-frame extractions or sequences in applications like thumbnail generation or image processing pipelines.[49] FFmpeg manages pixel formats essential for video representation and interoperability, prominently supporting YUV variants (such as YUV420p and YUV444p) for efficient color storage in broadcast and compression workflows, alongside RGB variants (like RGB24 and RGBA) for graphics and display applications.[51] The libswscale library enables conversions between these formats, adjusting color spaces and bit depths to match target containers or hardware requirements while minimizing information loss.[52] These containers often embed various codecs to compress the streams they hold.[49]

Protocols

FFmpeg provides extensive support for input and output protocols, enabling the handling of media streams across local systems, networks, and the internet. These protocols facilitate tasks such as live streaming, file transfer, and adaptive playback, integrating seamlessly with FFmpeg's demuxing and muxing capabilities. The framework's protocol layer abstracts access to resources, allowing uniform treatment of diverse sources like network endpoints and local paths.[53] Among open standards, FFmpeg implements HTTP for retrieving and serving media over the web, supporting features like partial content requests (via byte-range headers) and custom user agents to mimic browsers or clients. This protocol is essential for downloading or uploading streams, with options to set timeouts and connection reuse for efficiency. RTP (Real-time Transport Protocol) enables the transport of audio and video packets over IP networks, often paired with RTCP for quality feedback, making it suitable for low-latency live streaming applications. FFmpeg can generate RTP payloads from various codecs and handle multicast or unicast delivery. RTSP (Real-time Streaming Protocol) allows control of streaming sessions, including play, pause, and setup commands; FFmpeg acts as an RTSP client to pull streams from servers, for example via ffplay for playback, and supports the -rtsp_transport tcp option to force TCP transport for improved reliability on unstable or lossy networks. With additional configuration, FFmpeg can act as a basic server for pushing content. UDP (User Datagram Protocol) supports lightweight, connectionless transmission for real-time media, ideal for broadcast scenarios where speed trumps reliability, with configurable buffer sizes to manage packet loss.[53] For de facto standards in adaptive streaming, FFmpeg natively handles HLS (HTTP Live Streaming), which breaks media into segmented TS files delivered via HTTP manifests (.m3u8), enabling bitrate adaptation based on network conditions. This protocol supports live and VOD playback, with encryption options like AES-128 for protected content. Similarly, DASH (Dynamic Adaptive Streaming over HTTP) is supported through HTTP-based delivery of MPD manifests and segmented media, allowing dynamic switching between quality levels; FFmpeg can generate DASH-compliant outputs for broad compatibility with web players.[53] To achieve low-latency handling of live inputs such as HLS streams (including those from platforms like YouTube), FFmpeg supports configuration options that minimize input buffering, probing, and analysis time. Flags such as -fflags nobuffer (to disable input buffering), -flags low_delay (to enable low-delay mode), -probesize 32, and -analyzeduration 0 reduce startup delays and facilitate quicker stream processing. For encoding with minimal added latency, the -tune zerolatency option optimizes encoders like libx264 for real-time applications. Multiple FFmpeg instances can be run in parallel to process several streams simultaneously. An example command for low-latency ingestion of an HLS stream and output to RTSP is:
ffmpeg -fflags nobuffer -flags low_delay -probesize 32 -analyzeduration 0 -i "https://example.com/playlist.m3u8" -c copy -f rtsp rtsp://localhost:8554/stream
This approach supports real-time or near-real-time applications requiring reduced latency in live stream handling.[53][5][54] Local and inter-process access is managed through file-based protocols. The file protocol treats local filesystems and devices (e.g., /dev/video0) as streamable inputs or outputs, supporting seekable access and atomic writes for temporary files. Pipes enable seamless communication between processes by reading from standard input (stdin) or writing to standard output (stdout), commonly used in scripting pipelines like ffmpeg -i input.mp4 -f null - for analysis without file I/O.[53] Security is integrated via HTTPS, an extension of the HTTP protocol that encrypts traffic using TLS/SSL, requiring FFmpeg to be compiled with libraries such as OpenSSL or GnuTLS for certificate validation and secure connections. Authentication mechanisms include HTTP Basic and Digest schemes, specified via URL credentials (e.g., http://user:pass@host) or headers, allowing access to protected servers without exposing tokens in plain text. These features ensure compliant handling of authenticated streams in enterprise or restricted environments.[53]

Hardware Acceleration

CPU Architectures

FFmpeg provides extensive support for various CPU architectures through hand-written assembly optimizations that leverage SIMD instruction sets, enabling significant performance improvements in multimedia processing tasks such as codec decoding. These optimizations are architecture-specific and are enabled during compilation based on the target platform's capabilities, allowing FFmpeg to run efficiently on diverse hardware from desktops to embedded systems.[55] For x86 and AMD64 architectures, FFmpeg utilizes a range of SIMD extensions including MMX, SSE, SSE2, SSSE3, SSE4, AVX, AVX2, and AVX-512 to accelerate operations like motion compensation and transform computations in video codecs. Recent developments have introduced hand-tuned AVX-512 assembly code, yielding performance boosts of up to 94 times for certain workloads on compatible Intel and AMD processors, such as those in the Zen 4 and Zen 5 families.[56] These extensions are particularly effective for parallelizing pixel-level operations in decoding formats like H.264 and HEVC.[55] On ARM architectures, FFmpeg supports NEON SIMD instructions for both 32-bit ARMv7 and 64-bit AArch64 (ARMv8), which are widely used in mobile and embedded devices for efficient vector processing. NEON optimizations enhance throughput in codec decoding by handling multiple data elements simultaneously, with specific assembly paths tailored for cores like Cortex-A72 and A76.[57] AArch64 further benefits from advanced extensions like dotprod and i8mm, integrated into FFmpeg's build system for improved matrix multiplications in video processing.[58] Support for RISC-V architectures includes the RISC-V Vector (RVV) extension for SIMD operations, with optimizations merged for many digital signal processing (DSP) components as of FFmpeg 8.0 in 2025. These enhancements target vectorized workloads in codecs and filters, improving performance on RISC-V hardware such as SiFive processors and other embedded systems.[6] Support for other architectures includes MIPS with its SIMD Architecture (MSA) extensions, targeting processors like the MIPS 74Kc for optimized multimedia handling in embedded applications, and PowerPC with AltiVec (VMX) for vector operations on older systems like G4 and G5 processors.[59][60] These optimizations, while less extensively developed than x86 or ARM, provide essential acceleration for niche platforms.[61] Architecture-specific optimizations are controlled via FFmpeg's configure script during compilation; for instance, SIMD support can be enabled with --enable-asm (default on supported platforms), while individual extensions like SSE, AVX, or NEON can be disabled using flags such as --disable-sse, --disable-avx512, or --disable-neon if needed for compatibility or testing.[62] Cross-compilation flags, such as --arch=arm or --cpu=cortex-a53 for ARM, further tailor the build to specific CPU models, ensuring runtime detection and selection of the appropriate optimized code paths.

Specialized Hardware

FFmpeg integrates with specialized hardware to accelerate multimedia processing, leveraging GPUs, ASICs, and FPGAs for encoding, decoding, and filtering tasks beyond general CPU capabilities. This support enables offloading compute-intensive operations to dedicated silicon, improving performance in scenarios like real-time transcoding and high-resolution video handling. The framework's hardware paths are designed to be modular, allowing seamless fallback to software processing when hardware is unavailable or unsupported.[63] For GPU acceleration, FFmpeg provides robust support for NVIDIA hardware through NVENC for encoding and NVDEC (formerly CUVID) for decoding, both utilizing CUDA for integration with NVIDIA GPUs. This enables hardware-accelerated handling of codecs such as H.264, HEVC, AV1, and VP9, with NVENC offering low-latency encoding suitable for live streaming. FFmpeg does not have a dedicated CUDA/GPU-accelerated concatenation filter (e.g., no "concat_cuda" exists). The standard "concat" filter is CPU-based and does not support direct CUDA frame processing. CUDA filters exist (e.g., scale_cuda, overlay_cuda), but none provide concatenation functionality.[63] To concatenate multiple H.264 or H.265 videos using GPU acceleration with NVENC, use the concat demuxer: create a text file (e.g., mylist.txt) with entries like
file 'video1.mp4'
file 'video2.mp4'
file 'video3.mp4'
then run FFmpeg with
ffmpeg -hwaccel cuda -hwaccel_output_format cuda -f concat -safe 0 -i mylist.txt -c:v h264_nvenc output.mp4
(for H.265 use -c:v hevc_nvenc). The -hwaccel cuda -hwaccel_output_format cuda options ensure decoded frames remain in GPU memory for efficient processing. If the input videos are compatible (identical codec, resolution, pixel format, frame rate, etc.), use stream copy with -c copy for the fastest concatenation without re-encoding:
ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.mp4
This method involves minimal GPU usage. When inputs are incompatible or require additional processing (e.g., scaling, filtering), re-encoding with NVENC is required; keeping frames on the GPU via -hwaccel_output_format cuda maximizes efficiency and performance.[63][64] AMD GPUs are supported via the Advanced Media Framework (AMF), which facilitates accelerated encoding and decoding of H.264, HEVC, and AV1 on compatible Radeon and Instinct hardware, emphasizing cross-platform compatibility including Linux via Vulkan. Intel Quick Sync Video (QSV) integration allows for efficient encoding and decoding on Intel integrated GPUs, supporting multiple codecs through the oneVPL library (formerly Intel Media SDK), and is particularly effective for consumer-grade hardware in tasks like 4K video processing.[63][65][46][66] Platform-specific APIs extend this acceleration to ASICs and FPGAs. On Linux, VAAPI (Video Acceleration API) provides a unified interface for hardware decoding and encoding on Intel, AMD, and NVIDIA GPUs, utilizing libva to access underlying silicon like Intel's Quick Sync or AMD's UVD/VCE, with support for codecs including H.264, HEVC, VP9, and AV1. For macOS, VideoToolbox framework integration enables hardware-accelerated decoding and encoding using Apple's unified GPU architecture, covering H.264, HEVC, ProRes, and VP9, optimized for Metal-based rendering. On Windows, DirectX Video Acceleration (DXVA2) supports decoding of H.264, VC-1, and MPEG-2 via DirectX, interfacing with GPUs from various vendors for efficient surface handling and reduced CPU load. These APIs abstract hardware specifics, allowing FFmpeg to target diverse ASICs without vendor lock-in.[63][67][63][63] FFmpeg also incorporates compute APIs for broader hardware tasks. OpenCL support enables parallel processing in filters and effects, requiring an OpenCL 1.2 driver from GPU vendors, and is used for operations like deinterlacing and scaling on compatible devices. Vulkan integration provides low-level access for video decoding (H.264, HEVC, VP9, AV1) and emerging encoding capabilities, promoting portability across GPUs from AMD, Intel, and NVIDIA through a single API, with recent additions including Vulkan-based FFv1 codec handling.[63][63][6] Configuration for specialized hardware involves build-time options to enable specific backends and runtime flags for selection. During compilation, flags such as --enable-nvenc, --enable-amf, --enable-vaapi, --enable-videotoolbox, and --enable-opencl activate the respective libraries, requiring dependencies like CUDA SDK for NVIDIA or libva for VAAPI. At runtime, options like -hwaccel cuda or -hwaccel vaapi direct FFmpeg to use hardware paths, with automatic detection of available devices and fallback to CPU if needed. This dual-layer approach ensures flexibility across environments.[63][46][63]

Filters and Effects

Audio Filters

FFmpeg provides a wide array of audio filters through its libavfilter library, enabling users to manipulate audio streams for tasks such as adjustment, enhancement, and effects application during processing or transcoding.[68] These filters operate on raw audio frames and can be applied to inputs from files, live captures, or streams, supporting formats like PCM and various sample rates.[68] The library's design allows for efficient, graph-based chaining of filters, making it suitable for both offline batch processing and real-time scenarios.[69] Basic audio filters handle fundamental manipulations like volume control, sample rate conversion, and channel mixing. The volume filter adjusts the amplitude of audio samples, accepting a gain parameter in decibels (dB) or a linear multiplier (e.g., 0.5 for half volume), which helps normalize loudness or attenuate signals to prevent clipping.[70] For resampling, the aresample filter changes the sample rate while optionally preserving audio fidelity through high-quality interpolation algorithms from libswresample, such as sinc-based methods; it supports options like osr for output sample rate (e.g., 44100 Hz) and precision for filter quality levels up to 33 bits.[71] Channel mixing is achieved with filters like amix, which combines multiple input audio streams into one by weighted summation (controllable via weights option, e.g., "1 0.5" for primary and secondary inputs), and pan, which remaps and mixes channels with precise gain per channel (e.g., pan=mono|c0=0.5*FL+0.5*FR to downmix stereo to mono).[72][73] Advanced filters offer sophisticated processing for audio enhancement. Equalization is implemented via the equalizer filter, a parametric EQ that boosts or cuts specific frequency bands using Infinite Impulse Response (IIR) designs; key options include frequency (center freq in Hz), width_type (e.g., "hertz" or "q-factor"), and gain in dB (e.g., equalizer=f=1000:w=100:g=5 to boost 1 kHz by 5 dB).[74] Noise reduction filters, such as afftdn (FFT-domain denoising), apply spectral subtraction to suppress stationary noise by estimating and subtracting noise profiles from the frequency domain, with parameters like noise_reduction in dB (range 0.01-97, default 12 dB) controlling aggressiveness; alternatives include anlmdn for non-local means denoising, which averages similar temporal-spectral blocks to reduce broadband noise while preserving transients.[75] For echo effects and simulation (often used in post-processing to model or mitigate reflections), the aecho filter adds delayed and attenuated copies of the input, mimicking acoustic echoes; it uses options like in_gain (input scaling), out_gain (output scaling), delays (e.g., "500|1000" for 0.5 and 1 seconds in ms), and decays (attenuation factors) to create realistic reverb or test cancellation scenarios.[76] Audio filters are chained using libavfilter's filter graph syntax, which describes directed connections between filters in a string format applied via command-line options like -af or programmatically through the API. A simple chain might look like volume=0.8,equalizer=f=3000:g=3,aresample=48000, processing input sequentially from left to right; complex graphs use labels and splits, e.g., [in]split=2[a][b]; [a]volume=1.2[b]amix=inputs=2[out], allowing parallel paths and recombination.[77] For Finite Impulse Response (FIR) filtering, the afir filter applies arbitrary linear-phase filters defined by external coefficients (e.g., generated via afirloudnorm or external tools), supporting gain compensation and dry/wet mixing (e.g., afir=gain=-10:dry=0.5:wet=1:irfile=fir_coeffs.txt) for custom equalization or room correction.[78] This graph-based approach ensures efficient buffering and format negotiation between filters.[69] FFmpeg's audio filters support real-time processing for live streams, such as microphone inputs or network broadcasts, by integrating with low-latency capture devices and protocols like RTMP. Filters like aresample and volume are optimized for minimal delay, and the -re flag simulates real-time input rates during testing; however, computationally intensive filters (e.g., afftdn) may introduce latency, necessitating hardware acceleration or simplified chains for broadcast applications.[54] In live pipelines, filters can synchronize with video via shared timestamps, ensuring lip-sync in multimedia streams.[5]

Video Filters

FFmpeg provides a comprehensive set of video filters through its libavfilter library, enabling users to apply transformations, effects, and analytical operations to video streams during processing. These filters are invoked via the -vf option in the command line or through API integrations, allowing for complex filter graphs that chain multiple operations. Video filters operate on pixel data, supporting various color spaces and formats, and are essential for tasks ranging from basic resizing to sophisticated post-production workflows.[68]

Geometric Filters

Geometric filters in FFmpeg handle spatial manipulations of video frames, such as resizing, trimming, orientation adjustments, and compositing. The scale filter resizes input video to specified dimensions while preserving aspect ratios or applying algorithms like bicubic interpolation for quality preservation; for example, scale=[1920:1080](/page/1920×1080) upsamples to full HD, with options like flags=lanczos for sharper results. The flags parameter in the scale filter (corresponding to the global -sws_flags option in the ffmpeg command-line tool) allows selection of a scaling algorithm (only one primary algorithm should be selected) and combination with additional modifiers using +. The default algorithm is bicubic. Scaling algorithms:
  • fast_bilinear (fast bilinear)
  • bilinear
  • bicubic (default)
  • experimental
  • neighbor (nearest neighbor)
  • area (averaging area)
  • bicublin (bicubic for luma, bilinear for chroma)
  • gauss (Gaussian)
  • sinc
  • lanczos (Lanczos, default width/alpha=3 adjustable via param0)
  • spline (natural bicubic spline)
Additional combinable flags:
  • accurate_rnd (accurate rounding)
  • full_chroma_int (full chroma interpolation)
  • full_chroma_inp (full chroma input)
  • bitexact (bitexact output)
  • print_info (enable printing/debug logging)
  • unstable (allow experimental code for testing)
Examples include flags=lanczos+accurate_rnd.[27][79][80] The crop filter extracts a rectangular region from the frame, defined by width, height, and offsets, useful for removing borders or focusing on regions of interest, as in crop=iw:ih-100:0:0 to trim the bottom 100 pixels while maintaining input width and height (iw, ih).[81] Rotation is achieved via the rotate filter, which applies arbitrary angles in radians or degrees with optional bilinear interpolation, such as rotate=PI/2 for a 90-degree clockwise turn, often combined with scale to adjust for altered dimensions.[82] The overlay filter composites one video or image stream onto another at specified coordinates, supporting transparency via alpha channels and dynamic positioning with expressions like overlay=10:main_w-overlay_w-10, enabling picture-in-picture effects or watermarking.[83]

Effects Filters

FFmpeg's effects filters modify visual attributes like sharpness, smoothness, and color balance, facilitating artistic or corrective adjustments. The unsharp filter enhances edge details by applying separable convolution kernels separately to luma and chroma channels; parameters include luma_msize_x for matrix size and luma_amount for strength, as in unsharp=luma_msize_x=5:luma_amount=1.0 to subtly sharpen without artifacts.[84] For blurring, the boxblur filter uses a rectangular averaging kernel with configurable radius and power, such as boxblur=10:2 for a moderate Gaussian-like effect, while gblur offers Gaussian blurring with sigma control for smoother results.[85] Color correction is supported by the lut3d filter, which applies 3D lookup tables (LUTs) in formats like .cube for mapping input colors to output values, commonly used in grading workflows like lut3d=file=correction.cube.[86] The curves filter enables piecewise parametric adjustments to tonal ranges via RGB or individual channel presets, such as curves=r='0/0 0.5/0.58 1/1' to lift shadows in the red channel, providing precise control akin to professional editing software.[87]

Test Patterns

Test pattern filters generate synthetic video sources for calibration, debugging, and quality assessment without requiring input media. The smptebars source produces standard SMPTE color bars, including white, yellow, cyan, green, magenta, red, and blue bars with a PLUGE (Picture Line-Up Generation Equipment) pattern at the bottom, configurable for resolution and duration via options like smptebars=size=1920x1080:rate=30, aiding in color space verification.[88] The testsrc filter creates a dynamic pattern featuring a color cycle, scrolling gradient, and overlaid timestamp, with parameters for frame rate and size, such as testsrc=size=640x480:rate=25:duration=10, useful for testing decoder performance. For noise simulation, the noise filter adds patterns like snow by applying additive or multiplicative grain, with noise=all_seed=12345:all_strength=10:type=s generating static-like snow across all components to mimic broadcast interference.[89][90]

Advanced Filters

Advanced video filters in FFmpeg address temporal and content-specific processing, including artifact removal, rate adjustments, and text integration. Deinterlacing is handled by filters like yadif, which performs linear blending or temporal interpolation on interlaced fields to produce progressive frames, with modes such as yadif=1 for single-rate deinterlacing and parity=tff for top-field-first content, effectively doubling vertical resolution without combing.[91] Frame rate conversion uses the fps filter for simple dropping or duplication, like fps=25 to output at 25 fps, or minterpolate for motion-compensated interpolation via optical flow, as in minterpolate=fps=60:mi_mode=mci to smoothly upscale from 30 to 60 fps while minimizing judder.[92][93] Subtitle burning embeds text overlays permanently using the subtitles filter, which renders ASS/SSA or SRT files via libass onto the video. To support Chinese characters and other non-ASCII text properly, subtitle files must be encoded in UTF-8. When burning Chinese subtitles, specify a font supporting Chinese glyphs using the force_style option to prevent garbled text, such as Fontname=Microsoft YaHei or Fontname=Noto Sans CJK. If rendering issues occur due to font availability, the fontsdir option can specify a directory containing the required fonts, or additional force_style adjustments can be applied. For example, burning SRT subtitles with customized styling:
ffmpeg -i input.mp4 -vf "subtitles=chinese.srt:force_style='Fontname=Microsoft YaHei,Fontsize=24,PrimaryColour=&H00FFFFFF&'" output.mp4
ASS format is recommended for complex styling and can be burned using the ass filter:
ffmpeg -i input.mp4 -vf ass=chinese.ass output.mp4
or via the subtitles filter with an .ass file. Subtitles are baked into the pixel data for distribution.[94][95][96] Creating high-quality animated GIFs from video inputs, such as MP4 files, relies on advanced filter chains that optimize frame rate, size, and color palette. The palettegen filter generates a custom palette from the video stream, typically limited to 256 colors for GIF compatibility, with options like max_colors=256 and stats_mode=full to analyze full frame statistics for improved color accuracy. The paletteuse filter applies the palette to the video, with dithering via dither=floyd_steinberg to reduce banding in gradients and subtle color transitions. These are integrated with fps for frame rate reduction (e.g., fps=10 to balance smoothness and file size), scale for resizing (e.g., scale=480:-1:flags=lanczos to maintain aspect ratio with sharp Lanczos resampling), and split to efficiently generate and apply the palette in a single pass. The output uses -loop 0 for infinite looping. For an example command demonstrating this process in MP4 to GIF conversion, see the Command-Line Tools section.[34][35]

Concatenation

The concat filter concatenates multiple video and/or audio streams sequentially within a filter graph. It requires that the input streams to the filter have compatible parameters (e.g., matching resolution, pixel format, frame rate for video; sample format, rate, channels for audio) and is typically used via -filter_complex with options like n (number of segments), v (video streams per segment), and a (audio streams per segment). For example, concat=n=2:v=1:a=1 joins two inputs with one video and one audio stream each.[97] The concat filter can be used to concatenate files even when they have incompatible parameters, such as different codecs or resolutions. This requires decoding the inputs, applying transformations if needed (e.g., scaling for different resolutions), concatenating the processed streams, and re-encoding the output to a common codec. In contrast, direct concatenation without re-encoding using the concat demuxer and -c copy is not possible if stream parameters (including codecs) differ, as inputs must have identical parameters.[36] For example, to concatenate two MP4 files that may have different codecs:
ffmpeg -i file1.mp4 -i file2.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[v][a]" -map "[v]" -map "[a]" -c:v libx264 -c:a aac output.mp4
Adjust n= for more files. If resolutions differ, insert scaling filters before concatenation, for example: [0:v]scale=1920:1080[v0]; [1:v]scale=1920:1080[v1]; [v0][0:a][v1][1:a]concat=n=2:v=1:a=1[v][a]. For better quality control, include bitrate options like -b:v 5M.[36] FFmpeg does not provide a dedicated GPU-accelerated concatenation filter such as "concat_cuda". The standard concat filter is CPU-based and lacks native support for CUDA or other hardware acceleration frameworks. Although several CUDA-accelerated filters exist (e.g., scale_cuda for resizing, overlay_cuda for compositing, bwdif_cuda for deinterlacing), none offer concatenation capabilities.[68] To concatenate multiple H.264 or H.265 videos with GPU acceleration via NVENC, use the concat demuxer rather than the filter: create a text file listing inputs (e.g., file 'input1.mp4'\nfile 'input2.mp4'), then process with -f concat -safe 0 -i list.txt. Enable hardware acceleration using -hwaccel cuda -hwaccel_output_format cuda to decode frames into GPU memory, and encode with -c:v h264_nvenc (for H.264) or -c:v hevc_nvenc (for H.265). If inputs are compatible in codec, resolution, frame rate, and other parameters, use -c copy for fast, no-reencode concatenation with minimal GPU involvement. For incompatible inputs requiring processing or format alignment, re-encoding with NVENC is necessary, benefiting from retained GPU memory for efficiency.[36]

Recent Additions

As of FFmpeg 7.0 (released April 2024) and 8.0 (released November 2025), new filters have expanded capabilities in audio and video processing. Audio additions include adrc for dynamic range compression to normalize audio levels and showcwt for visualizing continuous wavelet transforms to analyze time-frequency content. Video enhancements feature tiltandshift for simulating tilt-shift lens effects to create miniature perspectives, quirc for detecting and decoding QR codes in frames, and a dnn backend enabling machine learning-based filters for tasks like super-resolution or style transfer. These updates, detailed in the official changelog, support advanced workflows including AI integration.[21][98][99]

Input and Output Interfaces

Media Sources

FFmpeg supports a wide range of file-based media inputs, enabling access to local disks and network shares through its demuxers and the standard file protocol. Users can specify input files directly via the -i option, such as ffmpeg -i input.mp4, where the tool reads from local storage or mounted network locations without requiring special configuration for basic access. This capability extends to various container formats, including MP4, AVI, MKV, and WAV, allowing seamless demuxing of audio, video, and subtitle streams from stored media.[49][3] For stream sources, FFmpeg handles live feeds and broadcast inputs primarily through supported protocols integrated into its input system. It can ingest real-time streams via protocols like HTTP Live Streaming (HLS), Real-Time Messaging Protocol (RTMP), and UDP multicast for broadcast TV signals, treating them as uniform inputs for processing. For instance, a command like ffmpeg -i http://example.com/stream.m3u8 captures segmented live content, while DVB inputs for over-the-air TV are accessible via device interfaces like V4L2 when hardware support is configured. This protocol-based access ensures compatibility with dynamic sources like web broadcasts or IP-based television feeds.[53][54][3] To minimize added latency when ingesting live streams for real-time or low-latency applications, FFmpeg supports specific input flags such as -fflags nobuffer, -flags low_delay, -probesize 32, and -analyzeduration 0. These reduce input buffering, enable low-delay mode in applicable codecs, and minimize initial probing and analysis time for faster stream startup. For example, these can be applied as ffmpeg -fflags nobuffer -flags low_delay -probesize 32 -analyzeduration 0 -i http://example.com/stream.m3u8 ....[5] Capture functionality in FFmpeg allows direct input from multimedia devices such as microphones and webcams using platform-specific APIs. On Linux systems, the ALSA input device captures audio from microphones, as in ffmpeg -f alsa -i hw:0, supporting mono, stereo, or multichannel recording depending on the hardware. For video, the Video4Linux2 (V4L2) device enables webcam capture, e.g., ffmpeg -f v4l2 -i /dev/video0, providing live video streams for encoding or streaming. On Windows, DirectShow serves as the API for both audio and video captures from similar devices. On macOS and iOS, AVFoundation provides capture capabilities, e.g., ffmpeg -f avfoundation -i "0:0" for the default video and audio devices, ensuring cross-platform accessibility to real-time sources.[100][101][102][103][104] Metadata handling in FFmpeg involves extraction during demuxing, where global properties like tags, chapters, and subtitles are parsed from input sources. Demuxers retrieve embedded tags such as title, artist, album, and encoder information, which can be inspected using ffprobe or preserved in outputs. Chapters are extracted as timed segments with metadata, supporting navigation in formats like Matroska, while subtitles appear as separate streams that can be isolated, e.g., via ffmpeg -i input.mkv -map 0:s:0 subtitles.srt. This process ensures comprehensive access to ancillary data without altering the core media streams.[3][40][105][49]

Physical and Virtual Interfaces

FFmpeg supports a range of physical and virtual interfaces for capturing and outputting media through hardware devices and software abstractions, primarily via the libavdevice library. These interfaces enable interaction with system-level audio and video hardware on various platforms, allowing FFmpeg to function as a versatile tool for live input and display without relying on file-based or network sources.[100] For physical audio input and output on Linux systems, FFmpeg utilizes the Advanced Linux Sound Architecture (ALSA), which provides direct access to sound cards and PCM devices. Users specify an ALSA device by name, such as "hw:0" for the default hardware interface, enabling low-latency capture or playback of audio streams. Similarly, PulseAudio integration allows FFmpeg to connect to networked or virtual audio sinks and sources, with options like "default" or specific server names for seamless integration into desktop environments. As of 2025, PipeWire can also be used for audio capture and playback on Linux via ALSA or PulseAudio compatibility layers.[100][101][106] Physical video capture is handled through the Video4Linux2 (V4L2) API on Linux, supporting devices like webcams and capture cards by exposing them as /dev/video nodes. USB 3.0 HDMI to UVC capture cards, in particular, enable low-latency video input from HDMI sources such as TV tuners or antennas, with typical latencies of 2–8 seconds depending on hardware and configuration; these integrate as standard V4L2 video devices. FFmpeg can query device capabilities, set formats, and capture frames directly, with parameters such as pixel format and resolution configurable via command-line options. For output to professional interfaces like HDMI or SDI, FFmpeg relies on vendor-specific drivers, such as those for Blackmagic DeckLink cards, which use the dedicated 'decklink' device format and support embedding audio in the signal for broadcast workflows.[100][102] Virtual interfaces extend FFmpeg's capabilities to software-emulated devices. On X11-based systems, the x11grab input device captures screen content by specifying display coordinates and framerate, facilitating desktop recording without hardware intervention. For Wayland compositors, screen capture is possible via kmsgrab for framebuffer access (often requiring elevated permissions) or emerging PipeWire integrations, with desktop portals for permission handling in recent builds as of 2025. Virtual audio cables, often implemented through PulseAudio modules or JACK, allow FFmpeg to route audio between applications as if using physical hardware, using null sinks or loopback devices for internal processing.[100][107] Display output includes the Linux framebuffer (fbdev) device, which renders video directly to console framebuffers for headless or embedded setups, bypassing graphical servers. Platform-specific outputs are used across systems, such as xv for X11 or audiotoolbox for macOS, with audio synchronization options available.[100][107]

Applications and Integrations

Third-Party Software

FFmpeg's libraries are widely integrated into third-party media players, enabling support for a broad range of formats and codecs without requiring users to interact directly with the command-line tool. VLC media player, developed by VideoLAN, incorporates FFmpeg's libavcodec and libavformat libraries for decoding and demuxing audio and video streams, allowing it to handle diverse file types seamlessly. Similarly, Media Player Classic - Home Cinema (MPC-HC) replaced its internal filters with LAV Filters in version 1.7.0, which are based on FFmpeg, providing enhanced stability and performance for DirectShow-based playback on Windows.[108] mpv, a command-line media player, builds statically with FFmpeg to support extensive video and audio codec compatibility, emphasizing minimalism and high-quality rendering.[109] Media servers such as Plex and Jellyfin also rely on FFmpeg for transcoding and playback. Plex employs a customized fork of FFmpeg as its transcoder to convert media on-the-fly for compatible streaming to client devices.[110] Jellyfin integrates FFmpeg directly for handling various input formats and hardware-accelerated transcoding in self-hosted environments.[111] In video editing software, FFmpeg serves as a foundational component for format handling and encoding tasks. HandBrake, an open-source video transcoder, relies on FFmpeg under the hood to open and process input files from nearly any source format, including those beyond native disc support like DVD and Blu-ray.[112] Adobe Premiere Pro integrates FFmpeg capabilities through third-party plugins such as Voukoder, which enables exporting to FFmpeg-supported encoders like x264 and x265 directly within the application, bypassing Adobe's native encoders for more flexible workflows.[113] DaVinci Resolve, Blackmagic Design's professional editing suite, supports FFmpeg for preparing and converting media files to ensure compatibility, particularly on Linux where users employ FFmpeg commands to transcode inputs into Resolve-accepted formats like DNxHR.[114] Streaming applications leverage FFmpeg for real-time encoding and output handling. OBS Studio, a popular open-source tool for live streaming and recording, bundles FFmpeg libraries for muxing and supports additional FFmpeg encoders via plugins, facilitating high-performance streams to platforms like Twitch and YouTube.[115] YouTube upload tools, including command-line utilities like yt-dlp, incorporate FFmpeg for post-processing tasks such as format conversion and metadata embedding to meet YouTube's strict upload requirements.[116] For web and mobile environments, FFmpeg enables client-side multimedia processing through ports and kits tailored for these platforms. Browsers support FFmpeg via WebAssembly implementations like ffmpeg.wasm, a pure JavaScript/WebAssembly port that allows video recording, conversion, and streaming directly in the browser without server dependencies.[117] On mobile devices, FFmpeg is integrated into Android and iOS apps using frameworks like FFmpegKit, which provides pre-built libraries for tasks such as video editing and transcoding in applications developed with React Native or native code.[118]

Embedded and Commercial Uses

FFmpeg's versatility and open-source nature make it suitable for deployment in resource-constrained embedded environments, where customized builds enable multimedia processing on devices with limited computational power. In router firmware such as OpenWRT, FFmpeg is integrated as a core package for tasks like streaming and recording video from IP cameras, allowing routers to function as lightweight media servers or recorders without dedicated hardware.[119][120] On Raspberry Pi single-board computers, FFmpeg supports diverse projects including live video streaming to web interfaces and real-time processing of camera feeds, often compiled with hardware acceleration to optimize performance on ARM architectures.[121][122] These embedded applications leverage FFmpeg's modular design, which permits the exclusion of unnecessary codecs and protocols to reduce binary size and memory footprint. In commercial products, FFmpeg powers set-top boxes and surveillance systems by handling video encoding, decoding, and streaming efficiently. For instance, media players like E2iPlayer on Android-based set-top boxes automatically install FFmpeg libraries to support playback and conversion of diverse formats during operation.[123] In video surveillance, FFmpeg is widely used to capture and store RTSP streams from IP cameras in network video recorders (NVRs), enabling continuous recording with minimal re-encoding to preserve quality and lower CPU usage.[124] Cloud-based services, such as AWS Elemental MediaLive and MediaPackage, integrate FFmpeg for RTMP input handling and transcoding workflows, facilitating scalable live video ingestion and distribution for broadcasters.[125][126] To suit low-resource embedded systems, developers create stripped-down FFmpeg builds by disabling optional components like unused demuxers, muxers, and external libraries during compilation, resulting in leaner binaries that fit within tight storage and runtime constraints.[127][128] This approach is particularly effective for ARM-based devices, where cross-compilation tools generate optimized versions tailored to specific hardware, such as those running on Ampere processors for efficient video processing.[129] Case studies highlight FFmpeg's role in specialized commercial integrations. In automotive infotainment systems, FFmpeg's ffprobe utility is employed in media managers to scan and metadata-extract from files, supporting constrained environments for in-vehicle entertainment without full library overhead.[130] For medical imaging, tools like the open-source AMIDE software use FFmpeg to generate MPEG-1 movies from rendered image series, aiding visualization in nuclear medicine applications, while DICOM processing pipelines convert medical scans to MP4 for broader compatibility and analysis.[131][132] These deployments underscore FFmpeg's adaptability in proprietary systems requiring reliable, high-performance multimedia handling. FFmpeg encounters significant legal considerations due to the patented nature of many multimedia codecs it supports, such as H.264/AVC managed by the MPEG-LA patent pool (now Via Licensing Alliance). To mitigate potential infringement, the project avoids including patented encoding implementations directly in its core codebase, instead recommending external libraries like the GPL-licensed libx264 for H.264 encoding. Users and distributors must independently secure patent licenses from relevant pools, as FFmpeg does not provide such coverage, and implementing or distributing conformant codecs may trigger royalty obligations depending on usage scale and commercial intent.[8] The licensing structure of FFmpeg, primarily under the GNU Lesser General Public License (LGPL) version 2.1 or later, with some components like certain filters under the GNU General Public License (GPL), imposes specific implications for integration into proprietary software. Dynamic linking to LGPL portions allows proprietary applications to remain closed-source without requiring disclosure of their code, provided the FFmpeg library can be replaced by users. However, static linking restricts this flexibility; it necessitates providing object files of the proprietary application to enable relinking with modified FFmpeg versions, or it may trigger full GPL obligations if GPL elements are involved, potentially requiring the entire application to be open-sourced.[8][133] Distribution of FFmpeg binaries must adhere to LGPL requirements for source code availability, ensuring that recipients can obtain the complete corresponding source—either bundled in a tarball or zip file, hosted alongside the binary, or via a written offer valid for at least three years. Binary redistribution without these measures violates the license, and inclusion of non-free or GPL-enabled components (e.g., via compilation flags like --enable-nonfree or --enable-gpl) further limits permissible uses in proprietary contexts.[8] Historically, FFmpeg has navigated tensions with patent pools like MPEG-LA without facing direct lawsuits, though community interactions highlight ongoing risks. In 2010, MPEG-LA issued clarifications asserting that distributing H.264 decoding software like FFmpeg beyond 100,000 units for non-personal use requires a license fee, sparking debates on open-source patent exposure. The FFmpeg community countered by underscoring the project's source-only distribution model and shifting compliance responsibility to end-users and vendors, which defused immediate threats and reinforced user diligence in patent matters. In December 2025, FFmpeg developers filed a DMCA takedown request against Rockchip's Linux MPP repository on GitHub for violating the LGPL license by copying FFmpeg code without attribution and relicensing it under the Apache License.[134][135][136]

Forensic video applications

FFmpeg is employed in forensic video examinations for tasks such as metadata inspection using ffprobe, format conversion and stream copying to preserve original data integrity, visually lossless transcoding (e.g., H.264 with quantization parameter set to 0), mathematically lossless compression using the FFv1 codec, and evidence validation via frame-level MD5 hashing. These capabilities support processing video evidence while maintaining its integrity and potential admissibility in legal proceedings.[137] FFmpeg is cross-platform, running natively on desktop operating systems including Windows, macOS, and Linux, in server environments through command-line interfaces, and on mobile platforms via ports and libraries such as FFmpegKit (supporting Android and iOS) and applications such as FFmpeg Media Encoder on Android.[138] No dedicated graphical user interface application specifically branded for forensic or evidence video compression exists across mobile, desktop, and server platforms. FFmpeg serves as the closest match due to its open-source nature, versatility, and adoption in forensic workflows. Specialized forensic video tools such as Amped FIVE and MotionDSP Forensic primarily focus on enhancement, restoration, and analysis (e.g., super-resolution, deblurring, stabilization) but are generally limited to desktop environments (primarily Windows) and lack broad support for mobile and server platforms.[139][140]

References

User Avatar
No comments yet.