Hubbry Logo
Text-to-video modelText-to-video modelMain
Open search
Text-to-video model
Community hub
Text-to-video model
logo
8 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Text-to-video model
Text-to-video model
Text-to-video model
View on Wikipedia
from Wikipedia

A compilation video generated using OpenAI's Sora 2 text-to-video model

A text-to-video model is a form of generative artificial intelligence that uses a natural language description as input to produce a video relevant to the input text.[1] Advancements during the 2020s in the generation of high-quality, text-conditioned videos have largely been driven by the development of video diffusion models.[2]

Models

[edit]

There are different models, including open source models. Chinese-language input[3] CogVideo is the earliest text-to-video model "of 9.4 billion parameters" to be developed, with its demo version of open source codes first presented on GitHub in 2022.[4] That year, Meta Platforms released a partial text-to-video model called "Make-A-Video",[5][6][7] and Google's Brain (later Google DeepMind) introduced Imagen Video, a text-to-video model with 3D U-Net.[8][6][9][10][11]

In February 2023, Runway released Gen-1 and Gen-2, among the first commercially available text-to-video and video-to-video models accessible to the public through a web interface. Gen-1, initially released as a video-to-video model, allowed users to transform existing video footage using text or image prompts.[12] Gen-2, introduced in March 2023 and made publicly available in June 2023, added text-to-video capabilities, enabling users to generate videos from text prompts alone.[13]

In March 2023, a research paper titled "VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation" was published, presenting a novel approach to video generation.[14] The VideoFusion model decomposes the diffusion process into two components: base noise and residual noise, which are shared across frames to ensure temporal coherence. By utilizing a pre-trained image diffusion model as a base generator, the model efficiently generated high-quality and coherent videos. Fine-tuning the pre-trained model on video data addressed the domain gap between image and video data, enhancing the model's ability to produce realistic and consistent video sequences.[15] In the same month, Adobe introduced Firefly AI as part of its features.[16]

In January 2024, Google announced development of a text-to-video model named Lumiere which is anticipated to integrate advanced video editing capabilities.[17] Matthias Niessner and Lourdes Agapito at AI company Synthesia work on developing 3D neural rendering techniques that can synthesise realistic video by using 2D and 3D neural representations of shape, appearances, and motion for controllable video synthesis of avatars.[18] In June 2024, Luma Labs launched its Dream Machine video tool.[19][20] That same month,[21] Kuaishou extended its Kling AI text-to-video model to international users. In July 2024, TikTok owner ByteDance released Jimeng AI in China, through its subsidiary, Faceu Technology.[22] By September 2024, the Chinese AI company MiniMax debuted its video-01 model, joining other established AI model companies like Zhipu AI, Baichuan, and Moonshot AI, which contribute to China's involvement in AI technology.[23] In December 2024 Lightricks launched LTX Video as an open source model.[24]

Alternative approaches to text-to-video models include[25] Google's Phenaki, Hour One, Colossyan,[3] Runway's Gen-3 Alpha,[26][27] and OpenAI's Sora,[28][29] Several additional text-to-video models, such as Plug-and-Play, Text2LIVE, and TuneAVideo, have emerged.[30] FLUX.1 developer Black Forest Labs has announced its text-to-video model SOTA.[31] Google was preparing to launch a video generation tool named Veo for YouTube Shorts in 2025.[32] In May 2025, Google launched the Veo 3 iteration of the model. It was noted for its impressive audio generation capabilities, which were a previous limitation for text-to-video models.[33] In July 2025 Lightricks released an update to LTX Video capable of generating clips reaching 60 seconds,[34][35] and in October 2025 it released LTX-2, with audio capabilities built in.[36]

Architecture and training

[edit]

There are several architectures that have been used to create text-to-video models. Similar to text-to-image models, these models can be trained using Recurrent Neural Networks (RNNs) such as long short-term memory (LSTM) networks, which has been used for Pixel Transformation Models and Stochastic Video Generation Models, which aid in consistency and realism respectively.[37] An alternative for these include transformer models. Generative adversarial networks (GANs), Variational autoencoders (VAEs), — which can aid in the prediction of human motion[38] — and diffusion models have also been used to develop the image generation aspects of the model.[39]

Text-video datasets used to train models include, but are not limited to, WebVid-10M, HDVILA-100M, CCV, ActivityNet, and Panda-70M.[40][41] These datasets contain millions of original videos of interest, generated videos, captioned-videos, and textual information that help train models for accuracy. Text-video datasets used to train models include, but are not limited to PromptSource, DiffusionDB, and VidProM.[40][41] These datasets provide the range of text inputs needed to teach models how to interpret a variety of textual prompts.

The video generation process involves synchronizing the text inputs with video frames, ensuring alignment and consistency throughout the sequence. This predictive process is subject to decline in quality as the length of the video increases due to resource limitations.[41] The Will Smith Eating Spaghetti test is a benchmark for models.[42]

Limitations

[edit]

Despite the rapid evolution of text-to-video models in their performance, a primary limitation is that they are very computationally heavy which limits its capacity to provide high quality and lengthy outputs.[43][44] Additionally, these models require a large amount of specific training data to be able to generate high quality and coherent outputs, which brings about the issue of accessibility.[44][43]

Moreover, models may misinterpret textual prompts, resulting in video outputs that deviate from the intended meaning. This can occur due to limitations in capturing semantic context embedded in text, which affects the model's ability to align generated video with the user's intended message.[44][41] Various models, including Make-A-Video, Imagen Video, Phenaki, CogVideo, GODIVA, and NUWA, are currently being tested and refined to enhance their alignment capabilities and overall performance in text-to-video generation.[44]

Another issue with the outputs is that text or fine details in AI-generated videos often appear garbled, a problem that stable diffusion models also struggle with. Examples include distorted hands and unreadable text.

Ethics

[edit]

The deployment of text-to-video models raises ethical considerations related to content generation. These models have the potential to create inappropriate or unauthorized content, including explicit material, graphic violence, misinformation, and likenesses of real individuals without consent.[40] Ensuring that AI-generated content complies with established standards for safe and ethical usage is essential, as content generated by these models may not always be easily identified as harmful or misleading. The ability of AI to recognize and filter out NSFW or copyrighted content remains an ongoing challenge, with implications for both creators and audiences.[40]

Impacts and applications

[edit]

Text-to-video models offer a broad range of applications that may benefit various fields, from educational and promotional to creative industries. These models can streamline content creation for training videos, movie previews, gaming assets, and visualizations, making it easier to generate content.[45]

During the Russo-Ukrainian war, fake videos made with Artificial Intelligence were created as part of a propaganda war against Ukraine and shared in social media. These included depictions of children in the Ukrainian Armed Forces, fake ads targeting children encouraging them to denounce critics of the Ukrainian government, or fictitious statements by Ukrainian President Volodymyr Zelenskyy about the country's surrender, among others.[46][47][48][49][50][51]

Movies

[edit]

Kaur vs Kore is the first Indian feature film made using generative AI which features dual role for the AI character of Sunny Leone, set to release in 2026.[52][53][54]

Chiranjeevi Hanuman – The Eternal is an Indian movie made entirely using Generative AI created by Vijay Subramaniam which is set for theatrical release in 2026. The movie was widely criticised by the Film makers in the Bollywood industry for entirely relying on AI and use of AI was seen as an existential threat to their career.[55][56][57]

Series

[edit]

Mahabharat: Ek Dharmayudh is an Indian mythological OTT series released on October 2025 and streamed on JioHotstar. It is recognized as the first series created entirely using artificial intelligence to generate visuals and character animations and consists of 100 episodes.[58][59][60]

Comparison of models

[edit]
Model/Product Company Year released Status Key features Capabilities Pricing Video length Supported languages
Synthesia Synthesia 2019 Released AI avatars, multilingual support for 60+ languages, customization options[61] Specialized in realistic AI avatars for corporate training and marketing[61] Subscription-based, starting around $30/month Varies based on subscription 60+
Vexub Vexub 2023 Released Text-to-video from prompt, focus on TikTok and YouTube storytelling formats for social media[62] Generates AI videos (1–15 mins) from text prompts; includes editing and voice features[62] Subscription-based, with various plans Up to ~15 minutes 70+
InVideo AI InVideo 2021 Released AI-powered video creation, large stock library, AI talking avatars[61] Tailored for social media content with platform-specific templates[61] Free plan available, Paid plans starting at $16/month Varies depending on content type Multiple (not specified)
Fliki Fliki AI 2022 Released Text-to-video with AI avatars and voices, extensive language and voice support[61] Supports 65+ AI avatars and 2,000+ voices in 70 languages[61] Free plan available, Paid plans starting at $30/month Varies based on subscription 70+
Runway Gen-2 Runway AI 2023 Released Multimodal video generation from text, images, or videos[63] High-quality visuals, various modes like stylization and storyboard[63] Free trial, Paid plans (details not specified) Up to 16 seconds Multiple (not specified)
Pika Labs Pika Labs 2024 Beta Dynamic video generation, camera and motion customization[64] User-friendly, focused on natural dynamic generation[64] Currently free during beta Flexible, supports longer videos with frame continuation Multiple (not specified)
Runway Gen-3 Alpha Runway AI 2024 Alpha Enhanced visual fidelity, photorealistic humans, fine-grained temporal control[65] Ultra-realistic video generation with precise key-framing and industry-level customization[65] Free trial available, custom pricing for enterprises Up to 10 seconds per clip, extendable Multiple (not specified)
Google Veo Google 2024 Released Google Gemini prompting, voice acting, sound effects, background music. Cinema style realistic videos.[66] Can generate very realistic and detailed character models/scenes/clips, with accommodating and matching voice acting, ambient sounds, and background music. Ability to extend clips with continuity.[67] Varies ($250 Google Pro/Ultra AI subscription, and additional AI credit Top-Ups) Eight seconds for individual clips (however clips can be continued/extended as separate clips) 50+
OpenAI Sora OpenAI 2024 Alpha Deep language understanding, high-quality cinematic visuals, multi-shot videos[68] Capable of creating detailed, dynamic, and emotionally expressive videos; still under development with safety measures[68] Pricing not yet disclosed Expected to generate longer videos; duration specifics TBD Multiple (not specified)
Runway Gen-4 Runway 2025 Released Consistent characters across scenes,[69] world consistency,[70] camera control, physics simulation Generates 5-10 second clips with consistent characters, objects, and environments across multiple shots[71] Credit-based subscription, part of paid plans 5-10 seconds Multiple (not specified)

See also

[edit]

References

[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia

Text-to-video model

View on Grokipedia
from Grokipedia
A text-to-video model is a system that synthesizes video sequences from textual descriptions, typically by conditioning spatiotemporal processes on text embeddings derived from large language models to iteratively denoise latent video representations into coherent frames with motion. These models build on architectures originally developed for static , extending them to capture temporal dependencies through mechanisms like 3D convolutions, transformer-based , or flow-matching to model dynamics across frames. Early approaches relied on autoregressive or GAN-based methods, but models have dominated since 2022 due to superior sample quality and scalability, as evidenced by benchmarks showing reduced perceptual artifacts in generated clips. Key advancements include OpenAI's Sora series, with Sora 2 released in 2025 featuring improved physical accuracy, native audio integration, and enhanced controllability via transformer architecture to generate high-definition videos with complex scene compositions and simulated physics, offering free access as of 2026 with limitations on resolution and duration in some access modes. Google's Lumiere, introduced in early 2024, uses space-time on latent patches to produce diverse, realistic motion in shorter clips, outperforming prior models in motion coherence per human evaluations. Stability AI's Stable Video Diffusion, also from 2023-2024 iterations, enables fine-tuning for customized outputs via open-source latent adapted for video, facilitating applications in and effects prototyping. These models have achieved notable in rendering objects, lighting, and basic interactions, with quantitative metrics like FVD scores dropping below 200 on datasets such as UCF-101, indicating improved alignment with real video distributions. Despite progress, persistent limitations include failures in long-term , violation of physical laws in scenarios (e.g., impossible trajectories or mass conservation errors), and computational demands exceeding hundreds of GPU-hours per clip, stemming from on web-scraped datasets that prioritize statistical correlations over causal mechanisms. Controversies arise from risks of misuse in fabricating deceptive content, prompting calls for watermarking and regulatory scrutiny, alongside debates over in corpora dominated by unlicensed media. Empirical evaluations reveal systemic biases toward over-representation of common motifs, yielding less reliable outputs for underrepresented cultural or physical contexts.

Definition and Historical Development

Core Concept and Foundational Principles

Text-to-video models are systems designed to synthesize dynamic video sequences from textual prompts, producing frames that maintain spatial fidelity within each image and temporal coherence across the sequence to depict plausible motion and events. These models condition the generation process on text embeddings derived from pre-trained language encoders, such as CLIP or T5, to align output semantics with descriptive inputs like "a jumping over a fence in ." The core objective is to approximate the p(v∣t)p(\mathbf{v} | \mathbf{t}), where v\mathbf{v} represents the video and t\mathbf{t} the text prompt, enabling controllable synthesis of novel content not present in training data. Unlike static image generation, video models must explicitly capture inter-frame dependencies to avoid artifacts like flickering or implausible dynamics, which arise from the high-dimensional nature of video data—typically involving thousands of pixels per frame over dozens of frames. At their foundation, contemporary text-to-video models predominantly leverage diffusion processes, a probabilistic framework inspired by , where a forward diffusion gradually corrupts video latents with isotropic over TT timesteps until reaching a tractable noise distribution, and a reverse denoising process iteratively reconstructs structured data conditioned on text. This reverse process parameterizes a that learns to predict noise or denoised samples, formalized as training to minimize a variational lower bound on the data likelihood, often simplified to denoising score matching for scalability. Empirical success stems from diffusion's ability to model complex multimodal distributions without adversarial training instabilities, as demonstrated in early video adaptations achieving coherent short clips of 2-10 seconds at resolutions up to 256x256 pixels. Causal modeling of motion relies on data-driven learning of spatio-temporal correlations, though outputs can deviate from physical realism if training datasets underrepresent edge cases like rare interactions or long-range dependencies. To mitigate the exponential compute costs of pixel-space —arising from video's volumetric data footprint (e.g., H×W×T×CH \times W \times T \times C dimensions)—foundational implementations compress videos into lower-dimensional latent representations via spatiotemporal autoencoders, such as variational autoencoders (VAEs) or vector-quantized variants, before applying . This latent paradigm, first scaled for images in , preserves perceptual quality while reducing parameters and steps, enabling training on datasets with billions of frame-text pairs sourced from web videos. Architecturally, models extend 2D backbones with 3D convolutions or temporal mechanisms in transformer-based diffusion transformers (DiTs) to propagate across time, ensuring consistent object trajectories and scene flows; for instance, bidirectional causal masking in some designs allows global context while simulating forward generation. Cross- layers fuse text conditionals into the denoising network at multiple scales, with classifier-free guidance amplifying adherence to prompts by interpolating between conditional and unconditional predictions during sampling, boosting semantic fidelity at the cost of diversity. These principles prioritize empirical scalability over exhaustive physical , relying on vast, diverse training corpora to implicitly encode causal structures like or occlusion, though evaluations reveal persistent gaps in handling complex interactions or extended durations without fine-tuning or cascaded refinement stages. Source surveys, such as those aggregating peer-reviewed works up to mid-2024, underscore diffusion's dominance due to its stable training dynamics and superior sample quality over GAN-based predecessors, which suffered mode collapse in temporal domains.

Early Research and Precursors (Pre-2022)

Early efforts in text-to-video generation prior to 2022 primarily relied on generative adversarial networks (GANs) and variational autoencoders (VAEs) to produce short, low-resolution video clips conditioned on textual descriptions, often limited to simple scenes due to computational constraints and dataset scarcity. These approaches decomposed video synthesis into static scene layout (e.g., background and objects) and dynamic motion elements, using text embeddings to guide generation. Datasets such as the Video Description Corpus (MSVD) provided paired text-video data, but lacked the scale and diversity needed for complex outputs, resulting in generations typically under 10 seconds long and resolutions below 64x64 pixels. A foundational work, "Video Generation From Text" (2017), introduced a hybrid VAE-GAN model that automatically curated a text-video corpus from online sources and separated static "gist" features for layout from dynamic filters conditioned on text, enabling plausible but rudimentary videos like "a man playing guitar." Building on this, the 2017 ACM paper "Generating Videos from Captions" employed encoder-decoder architectures with LSTM for temporal modeling, focusing on caption-driven synthesis but struggling with motion realism. GAN variants advanced the field: the 2019 IJCAI paper "Conditional GAN with Discriminative Filter Generation for Text-to-Video Synthesis" used adaptive filters in the discriminator to improve text alignment and temporal coherence, outperforming baselines on MSVD in human evaluations of . Similarly, IRC-GAN (2019) integrated recurrent convolutions to refine adversarial , reducing mode in motion . Later pre-2022 developments included TiVGAN (2020), a step-wise evolutionary GAN that first generated images from text before extending to video frames, achieving better frame consistency on datasets like Pororo. GODIVA (2021) shifted toward transformer-based autoregressive modeling for open-domain videos, generating up to 16-frame clips at higher fidelity but still prone to artifacts in . These models highlighted persistent challenges: poor temporal consistency (e.g., flickering objects), limited beyond domains, and high training instability from GANs, paving the way for -based paradigms post-2021. Evaluation metrics, such as adapted Scores or human judgments, underscored qualitative improvements but quantitative gaps in realism compared to later diffusion models.

Breakthrough Era (2022–2023)

In late 2022, the field of text-to-video generation experienced rapid advancements driven by diffusion-based architectures, which extended successful text-to-image techniques like to incorporate temporal dynamics. These models leveraged large datasets of captioned videos to learn spatiotemporal representations, enabling the synthesis of coherent motion from static textual prompts, though outputs remained constrained to short clips of 2–10 seconds at resolutions up to 256x256 or 512x512 pixels. On September 29, 2022, announced Make-A-Video, a pipeline that inflates text-conditioned image features into video latents using a spatiotemporal upsampler and decoder trained on millions of video-text pairs. The model generated whimsical, low-fidelity clips emphasizing creative but often artifact-prone motion, such as animated scenes of animals or objects, without public release due to ethical risks like . Google Research followed in October 2022 with Phenaki, introduced via a preprint on October 5, which pioneered variable-length generation by employing a bidirectional masked transformer (MaskGIT) to autoregressively predict discrete video tokens conditioned on evolving text sequences. Capable of producing clips up to 2 minutes long at 128x128 resolution, Phenaki demonstrated narrative continuity across scenes—e.g., a prompt sequence describing a character riding a bicycle through changing environments—but suffered from compounding errors in longer outputs and required extensive computational resources for training on diverse, open-domain video data. Concurrently, unveiled Imagen Video on October 6, 2022, a cascaded diffusion system building on the Imagen , comprising a base low-resolution video generator followed by spatial and temporal super-resolution stages to yield high-definition results up to 1280x768 at 24 frames per second. It prioritized fidelity in physics simulation and human motion over length, generating 2–4 second clips with superior semantic alignment to prompts compared to predecessors, yet like others, it was withheld from public access to mitigate misuse potential. By 2023, refinements emerged, including Meta's Video on November 16, which applied efficient sampling to Emu image embeddings for faster, higher-quality 5-second clips at , reducing training costs through from larger teacher models. These efforts highlighted diffusion's efficacy for causal video modeling but underscored persistent challenges: temporal inconsistency, high latency (often minutes per clip on GPU clusters), and data biases amplifying stereotypes in outputs, as empirically observed in evaluations against human-rated coherence metrics.

Commercial Acceleration (2024–Present)

In 2024, text-to-video models transitioned from research prototypes to commercially viable products, with major firms releasing accessible platforms that enabled widespread user experimentation and integration into creative workflows. OpenAI's Sora, initially previewed in February, launched a faster variant called Sora Turbo on December 9, 2024, allowing limited public access through Plus subscriptions and emphasizing safeguards against misuse. Concurrently, introduced Gen-3 Alpha on June 17, 2024, a model trained on videos and images to support text-to-video, image-to-video, and text-to-image generation, powering tools used by millions for professional-grade outputs up to 10 seconds at 1280x768 resolution. Luma AI's Dream Machine followed on June 12, 2024, generating high-quality clips from text or images in minutes, with subsequent updates like version 1.5 in August enhancing motion coherence and realism. announced Veo in May 2024, integrating it into Vertex AI for enterprise video generation from text or images, focusing on cost reduction and production efficiency. Kuaishou's Kling AI emerged as a competitor, offering text-to-video capabilities with hyper-realistic dynamics, initially limited but expanding to global access via web interfaces. This proliferation spurred competitive advancements, including longer clip durations, improved physics simulation, and multimodal inputs, driven by training on vast datasets. By mid-2024, models like Gen-3 Alpha and Dream Machine supported extensions beyond initial generations, enabling users to create coherent sequences through iterative prompting, though computational costs remained high—often requiring paid credits for high-fidelity renders. Commercial platforms introduced tiered pricing, such as Runway's subscription model for unlimited generations, contrasting earlier research-only demos and accelerating adoption in , , and . Into 2025, acceleration intensified with iterative releases emphasizing speed, audio synchronization, and mobile accessibility. unveiled Sora 2 on September 30, 2025, incorporating audio generation for dialogue and effects alongside visuals, launched via an app that amassed over 1 million downloads in under five days—surpassing ChatGPT's initial uptake—and enabling remixing of user-generated clips. released Kling AI 2.5 Turbo on September 26, 2025, upgrading text-to-video quality with faster inference and enhanced detail in motion and lighting. Luma expanded Dream Machine with an app in November 2024 and Ray 2 in January 2025, prioritizing boundary-pushing video synthesis for 25 million registered users by late 2024. advanced Veo to version 3 in 2025, accessible through the Gemini app requiring Google AI Pro or Ultra subscriptions and generating 8-second videos with sound from text prompts or uploaded images, integrating it with tools like Flow for cinematic scene creation and optimizing for in . These updates reflected a market shift toward integrated ecosystems, where models not only generated videos but also supported editing, upscaling, and provenance tracking to address authenticity concerns. The era marked a surge in venture investment and enterprise adoption, with platforms reporting exponential user growth amid benchmarks showing superior temporal consistency over 2023 predecessors—e.g., Veo 3's lip-sync accuracy and Sora 2's multimodal fidelity. However, challenges persisted, including high inference costs (often $0.01–$0.10 per second of video) and ethical debates over deepfakes, prompting features like watermarks in Sora and Veo outputs. Competition from Chinese firms like highlighted global disparities in data access and regulation, accelerating open-source alternatives while proprietary leaders maintained edges in scale and refinement. By October 2025, text-to-video tools had democratized short-form content creation, with applications in and , though full-length video coherence remained an ongoing frontier.

Technical Architecture and Training

Core Architectures (Diffusion Models, Transformers, and Hybrids)

constitute the primary paradigm for text-to-video generation, extending the denoising process from images to spatiotemporal data by iteratively refining into coherent video sequences conditioned on textual descriptions. These models typically encode videos into latent representations via autoencoders to reduce computational demands, then apply a reverse that predicts noise removal across frames while preserving temporal consistency through mechanisms like 3D convolutions or layers. Early implementations, such as VideoLDM, leverage latent models (LDMs) to synthesize high-resolution videos by factorizing the denoising into spatial and temporal components, enabling efficient training on large datasets of captioned videos. This approach mitigates the quadratic growth in parameters inherent to full , achieving resolutions up to 256x256 at 49 frames with reduced VRAM usage compared to pixel-space . Transformer architectures have increasingly supplanted convolutional s in diffusion-based video models, offering superior scalability through self- mechanisms that process sequences of spacetime patches—discrete tokens derived from compressed video latents arranged along spatial and temporal dimensions. The Diffusion Transformer (DiT), originally proposed for image generation, replaces blocks with layers comprising multi-head and feed-forward networks, facilitating longer context modeling and parallel computation essential for video's extended sequences. In text-to-video applications, DiTs condition generation via cross- to text embeddings from large language models, as seen in models like CogVideoX, which integrates a specialized to enhance motion dynamics and textual fidelity during diffusion steps. OpenAI's Sora exemplifies this shift, employing a DiT operating on spacetime latent patches to simulate physical world dynamics, supporting videos up to 60 seconds at resolution through hierarchical patch encoding that unifies image and video processing. Hybrid architectures combine diffusion's probabilistic sampling with transformer's sequential reasoning, often merging latent diffusion backbones with autoregressive or parallel transformer components to address limitations in long-range coherence and efficiency. For instance, Vchitect-2.0 introduces a parallel design within a framework, partitioning video across spatial and temporal axes to scale generation for high-resolution, long-duration outputs while maintaining causal masking for autoregressive-like dependencies. Other hybrids, such as Hydra- models, integrate state-space models with DiTs in a , leveraging the former's linear complexity for temporal to produce extended videos beyond training lengths, as demonstrated in evaluations yielding improved FID scores on benchmarks like UCF-101. These fusions exploit 's robustness to mode collapse alongside 's expressivity, though they introduce trade-offs in training stability requiring techniques like flow matching for accelerated convergence.

Data Requirements and Training Paradigms

Text-to-video models necessitate expansive s of video clips annotated with textual descriptions to capture correlations between and spatiotemporal content. Prominent examples include WebVid-10M, which contains 10.7 million video-text pairs encompassing roughly 52,000 hours of footage scraped from stock video platforms, enabling large-scale pre-training for conditional generation. Another key resource is InternVid, a video-centric with millions of clips paired with captions, designed to foster transferable representations across multimodal tasks. These corpora prioritize diversity in actions, environments, and durations—typically short clips of 10–30 seconds—to train models on realistic dynamics, though sourcing high-fidelity annotations remains resource-intensive due to manual or automated captioning limitations. Data quality demands extend beyond scale to temporal consistency and resolution variety, as low-quality inputs propagate artifacts in generated outputs. Datasets like VidGen-1M aggregate 1 million clips with detailed, human-verified captions to address gaps in consistency, often filtering for resolutions above and frame rates exceeding 24 fps. Kinetics variants, such as Kinetics-700 with over 650,000 YouTube-sourced videos across 700 action classes, supplement these by providing labeled motion primitives, though they require additional text pairing for direct text-to-video use. Overall, training corpora aggregate billions of frames, with proprietary efforts reportedly scaling to hundreds of thousands of hours, underscoring the empirical necessity of volume for emergent capabilities like physics simulation in outputs. Training paradigms predominantly leverage diffusion processes conditioned on text embeddings from models like CLIP or T5, extending 2D image to 3D spatiotemporal domains. Latent models compress videos via spatiotemporal variational autoencoders into lower-dimensional representations, applying addition and denoising iteratively to reduce overhead—often by factors of 8–16 compared to pixel-space . Common approaches factorize modeling into spatial (via blocks) and temporal (via or ) components, as in VideoLDM, trained end-to-end on text-video pairs with objectives minimizing reconstruction error under classifier-free guidance for prompt adherence. Joint pre-training on images and videos initializes parameters from text-to-image systems, exploiting abundant static data to bootstrap video-specific temporal layers, followed by video-only fine-tuning on datasets like WebVid. This paradigm, evident in models like Sora, incorporates world-modeling objectives to enforce physical realism, with training spanning thousands of GPU-hours on clusters exceeding 10,000 H100 equivalents. Hierarchical strategies, such as patch-based , further optimize for high resolutions by progressively refining coarse-to-fine latents, mitigating the quadratic scaling of in long sequences. Such methods empirically outperform autoregressive alternatives in coherence but demand careful hyperparameter tuning to avoid mode collapse in underrepresented dynamics.

Inference and Generation Processes

In text-to-video diffusion models, inference begins with encoding the input text prompt using a pre-trained text encoder, such as CLIP or T5, to produce conditioning embeddings that guide the generation process. These embeddings are injected into a denoising network, typically a augmented with temporal layers or 3D convolutions, which operates in a compressed to reduce computational overhead. The process initializes a sequence of noisy latent representations for the video frames—often starting from pure —and iteratively refines them over multiple timesteps, predicting and subtracting noise at each step to reconstruct coherent spatiotemporal content. The core denoising loop employs classifier-free guidance, where the model samples from both conditioned and unconditioned distributions to amplify adherence to the prompt, enhancing semantic alignment while mitigating mode collapse. Effective prompting strategies, as documented by model developers, emphasize specifying the subject, actions, visual style, camera movements, and duration to improve output coherence and fidelity. For temporal consistency across frames, architectures incorporate mechanisms like temporal blocks or flow-based priors that propagate motion information, preventing artifacts such as flickering or inconsistent object trajectories; for instance, models like VideoLDM insert lightweight temporal convolution layers into the to model inter-frame dependencies without full 3D parameterization. Sampling schedulers, such as DDIM or PLMS, accelerate this reverse diffusion by skipping intermediate steps, typically reducing from 1000 to 20-50 iterations while preserving quality. Upon completing denoising, the refined latent video is decoded frame-by-frame via a variational autoencoder (VAE) to pixel space, often followed by super-resolution or upsampling modules to achieve higher resolutions like 576x1024. In models emphasizing efficiency, such as those using consistency distillation, inference bypasses iterative denoising entirely by directly mapping noise to clean latents in one or few steps, cutting generation time from minutes to seconds on consumer hardware. Proprietary systems like OpenAI's Sora extend this pipeline to longer durations (up to 60 seconds) by scaling diffusion over spacetime patches, though exact details remain undisclosed, relying on massive parallel computation for photorealistic outputs. Commercial models such as Runway ML (Gen-3), Kling AI, and Luma Dream Machine exhibit approximate generation times for short clips (e.g., 5-10 seconds) of 30 seconds to a few minutes for Runway, 1-5 minutes for Luma, and 5-30 minutes or longer for Kling due to queues, with video analysis for reference-guided generation integrated into the overall process; these times vary by video length, resolution, subscription tier, and system load. These processes demand significant GPU resources, with optimizations like latent-space operations enabling feasible deployment on clusters of A100 or H100 equivalents.

Computational Demands and Optimization Techniques

Text-to-video models, predominantly based on processes extended to spatiotemporal , impose substantial computational demands during both and phases due to the high dimensionality of video sequences, which encompass spatial frames and temporal dynamics. such models typically requires clusters of thousands of high-end GPUs; for instance, proprietary systems like OpenAI's Sora have been estimated to utilize between 4,200 and 10,500 H100 GPUs for approximately one month to achieve production-scale capabilities. Open-source alternatives, such as Open-Sora 2.0, demonstrate that commercial-level performance can be attained with optimized pipelines costing around $200,000 in compute resources, leveraging progressive multi-stage from low-resolution (e.g., 256×256 pixels) to higher resolutions while minimizing overall GPU-hours through data-efficient curation and architectural efficiencies. These demands stem from the need to process vast datasets of video-text pairs, often exceeding billions of frames, to learn coherent motion and semantics, resulting in floating-point operations (FLOPs) orders of magnitude higher than text-to-image counterparts—potentially in the range of 10^24 to 10^25 FLOPs for frontier models, though exact figures for closed systems remain undisclosed. Inference for text-to-video generation further amplifies resource intensity, as it involves iterative denoising over extended latent sequences to produce temporally consistent outputs, often limited on consumer hardware. For example, generating short clips (e.g., 4 seconds at 240p resolution) with open implementations like Open-Sora on a single 3090 GPU consumes significant VRAM and requires about one minute per clip, constraining output length and quality due to memory bottlenecks. Lighter open-source models can, however, be run locally on consumer PCs equipped with GPUs having at least 8-12 GB VRAM to achieve decent inference speed and quality, particularly for shorter clips or lower resolutions. Production deployments, such as those for Sora 2, support up to 1080p resolution and 20-second durations but necessitate specialized accelerators like H100 clusters for real-time or batch , with rendering times scaling quadratically with video length and resolution. These constraints arise causally from the autoregressive or parallel sampling of frame sequences in models, where maintaining physical realism demands high-fidelity latent representations that exceed the 24-48 GB VRAM typical of high-end consumer GPUs. Optimization techniques have emerged to mitigate these demands, focusing on architectural innovations, efficiencies, and accelerations while preserving generative fidelity. Latent architectures compress videos into lower-dimensional spaces prior to processing, reducing spatial and temporal compute by factors of 10-100 compared to pixel-space methods, as implemented in two-stage pipelines that first generate coarse latents and refine them progressively. Diffusion Transformers (DiT) hybridize attention mechanisms with steps for scalable video modeling, enabling efficient handling of long sequences via causal masking and rotary positional encodings, as seen in Open-Sora's design which achieves high-quality outputs with reduced parameter counts through expert mixtures and flow-matching alternatives to traditional denoising. optimizations include adaptive sampling schedules that align step counts with perceptual quality, cutting generation time by up to 50% without quality loss, alongside hardware-specific accelerations like TensorRT for transformer-based models, which fuse operations and quantize weights to 8-bit precision for 2-4x speedups on GPUs. Additional strategies encompass to smaller student models, zero-shot conditioning to avoid full retraining, and tokenization efficiencies like VidTok, which chunks videos into compact representations to lower footprints during both phases. These techniques collectively enable broader , though they often trade marginal fidelity for practicality in resource-constrained settings.

Key Models and Comparative Analysis

Pioneering and Open-Source Models

One of the earliest open-source text-to-video models was Alibaba's ModelScope Text-to-Video Synthesis, a multi-stage with 1.7 billion parameters capable of generating videos from English text descriptions using a UNet3D architecture. Released in late 2022, it marked a foundational step in accessible diffusion-based video generation by providing pre-trained weights and for , though outputs were limited to short clips with moderate fidelity due to training on constrained datasets. In 2022, THUDM's CogVideo emerged as another pioneering effort, employing architectures to produce coherent video sequences from textual prompts, with initial versions generating 4-second clips at 240x426 resolution. Its open-source release facilitated rapid experimentation, influencing subsequent models by demonstrating scalable autoregressive generation, albeit with challenges in temporal consistency and computational efficiency. AnimateDiff, introduced in early 2023, advanced open-source capabilities by integrating lightweight motion modules into existing text-to-image models, enabling animation without full retraining. This plug-and-play approach generated 16-24 frame videos at 512x512 resolution, prioritizing motion smoothness over novel content creation, and spurred community extensions like custom adapters for longer sequences. Stability AI's Stable Video Diffusion, released on November 21, 2023, represented a significant milestone as the first open foundation model extending Stable Diffusion to video, supporting text-to-video and image-to-video synthesis for 14-25 frames at 576x1024 resolution. Trained on millions of video-text pairs, it achieved higher realism through latent diffusion techniques but required substantial GPU resources for inference, such as high-end consumer GPUs (e.g., RTX 4090s, potentially multiple for optimal performance) or cloud instances, enabling generation of decent videos locally without relying on proprietary APIs, with open weights available on Hugging Face for fine-tuning. Subsequent developments include Genmo's Mochi 1, noted for high-quality smooth motion, accurate prompt adherence, and uncensored outputs; Tencent's HunyuanVideo, providing accurate adherence to prompts alongside multi-language text-to-video support; the Wan 2.2 (14B version), enabling text-to-video and image-to-video generation at high resolutions; SkyReels for cinematic realism; and models such as Lightricks' LTX-Video, recognized for strong performance, and THUDM's CogVideoX, versatile across various workflows. These models, available on Hugging Face, can be run locally via interfaces like ComfyUI or SwarmUI and support integration via the Diffusers library for building AI video generation tools. These advancements build on earlier efforts, offering specialized strengths in motion, accessibility, and efficiency while maintaining open-source availability for community adaptation, though they continue to face challenges in long-form generation and resource demands.

Proprietary Leaders (Sora, , Kling, etc.)

OpenAI's Sora, first previewed on February 15, 2024, represents a flagship text-to-video model capable of generating high-definition videos up to 20 seconds in length from textual prompts, emphasizing visual quality and prompt adherence through advanced diffusion transformer architectures, though it may produce errors such as object deformation or unnatural movement in complex physics, multi-character interactions, or extreme actions. Full public access via sora.com launched on December 9, 2024, supporting videos up to resolution and 20 seconds initially, with integration into for Plus and Pro subscribers. An upgraded Sora 2, released September 30, 2025, introduced synchronized audio generation including dialogue and ambient sounds, improved physics simulation and understanding, and enhanced consistency such as reduced flickering, alongside a dedicated app for remixing and user appearances in clips. As of February 2026, the Sora 2 Pro supports 9:16 vertical format and maintains consistent character appearance via "character cameo" feature. It shifted API access from Microsoft Azure to the OpenAI API, with per-second pricing substantially lower than prior structures to support scalability. Initially available in the and , rollout excludes regions including the EU and Australia. Access remains gated behind paid tiers, with daily usage quotas based on subscription levels to manage computational demands; generation prohibits depictions of real persons, especially in image-to-video modes, and restricts violent or sensitive content per OpenAI policies. Runway ML's Gen-3 Alpha, unveiled June 17, 2024, powers proprietary text-to-video, image-to-video, and text-to-image tools through joint training on video and image datasets, enabling coherent motion and stylistic control, with high-quality motion and creative control up to 10 seconds base duration, extendable by generating sequential clips and merging them. A Turbo variant followed in August 2024, offering sevenfold speed increases at half the cost while maintaining output fidelity for clips up to several seconds. Users access these via Runway's platform with credit-based subscriptions starting at standard tiers providing limited monthly generation, such as 62 seconds of Gen-3 video; Runway offers a free plan with limited credits for short videos. The model excels in integrating text overlays and novel scene dynamics but requires precise prompting for optimal results. As of late 2024/early 2025, features and availability change rapidly; users should check official sites for current status. Kuaishou's Kling AI, debuting June 10, 2024, employs a diffusion-based with 3D spatio-temporal joint attention to produce fluid, high-fidelity videos from text or prompts, supporting up to two minutes at resolution in select plans and known for high-resolution, realistic motion in text-to-video and image-to-video generation. Subsequent iterations include Kling 1.6 in December 2024 for enhanced generation stability and Kling 2.5 Turbo in September 2025, which improves reference fidelity in elements like color, lighting, and texture while accelerating inference. Available through Kuaishou's platform with credit systems, Kling provides free daily credits for text-to-video generation and prioritizes realistic motion modeling but faces regional access restrictions outside . As of 2025, among Runway, Kling, Pika, and Luma Dream Machine, Kling AI is widely regarded as the best for generating realistic, cinematic slow-motion videos, including fashion content like model walks. It excels in natural motion, physics, and smooth slow-motion effects. Runway Gen-3 is a strong second for artistic cinematic styles and prompt adherence, suitable for fashion videos. Luma Dream Machine and Pika Labs are capable but generally rank lower in realism and motion quality for cinematic slow-motion use. All tools prohibit explicit NSFW content, but tasteful lingerie fashion videos may be possible with careful prompting (subject to platform filters and updates). As of late 2024/early 2025, features and availability change rapidly; users should check official sites for current status. Other notable proprietary entrants include Google's Veo 3.1, which excels in realism and photorealism, with native 9:16 vertical support for TikTok/Reels, text-to-video generation, and improved character consistency using reference images ("Ingredients to Video" feature) for persistent characters, expressions, and objects across scenes; limited free access via waitlist in Google Labs/VideoFX, not broadly free. while Runway Gen-4 provides faster generation speeds (e.g., 10-second videos in 30 seconds via Turbo), supports 9:16 aspect ratio, text-to-video generation, and strong character consistency using reference images for characters, locations, and objects across scenes, alongside enhanced professional control tools such as reference-driven consistency and advanced prompting for motion and scenes. Veo 3.1 is integrated with Gemini for consumer access, generating videos with synchronized sound from text prompts or uploaded images, requiring Google AI Pro or Ultra subscriptions for credit-based generation, and integrated into YouTube Shorts to enable AI generation of short videos from text or image prompts directly within the platform; Luma AI's Dream Machine, powered by the Ray3 model, supports text-to-video and image-to-video with features like keyframe control, video extension, looping, character consistency, and modification via natural language prompts, generating coherent multi-shot videos up to approximately 10 seconds emphasizing natural motion and well-suited for creative storytelling and fantastical scenes, offering a free tier with limited generations per month; MiniMax's Hailuo AI, which generates videos from text and image prompts with enhanced motion smoothness and style consistency, and Pika Labs, founded in April 2023 by former Stanford AI PhD students Demi Guo and Chenlin Meng, offers user-friendly text-to-video and image-to-video generation up to 12 seconds or more good for stylized clips including cute or imaginative content, with a free plan featuring daily credits and watermarked videos, whose models like Pika 2.1 focus on text-to-video generation with API access, facilitating rapid iteration for creative workflows, both operating under subscription models with backends as of 2025. As of 2025, these tools demonstrate capabilities in generating whimsical scenes like a cute alien animal, such as a fluffy, big-eyed alien creature in a colorful environment, effectively with detailed prompts (e.g., "a cute fluffy alien creature with big sparkling eyes exploring a glowing forest"). As of late 2025, the leading AI video generators capable of creating historical evolution videos from text prompts include Kling AI, Runway Gen-3, Luma Dream Machine, and Google Veo. These tools excel in temporal consistency, realism, and longer video durations (up to 1-2 minutes or more with extensions), making them suitable for depicting sequential historical or evolutionary processes. Kling AI is frequently praised for high-quality motion and complex scene handling. OpenAI Sora remains limited in access but is highly anticipated for advanced capabilities in 2026. No single tool is definitively "best" for 2026, as the field evolves rapidly. For videos exceeding native durations, a common workaround involves generating multiple short clips sequentially and combining them in editing software such as CapCut or Adobe Premiere. For very long durations spanning minutes, hybrid tools like Synthesia, HeyGen, Pictory, or InVideo utilize virtual presenters or stock footage to produce extended text-to-video content without strict length limits. In 2026, leading models for high resolution include Google Veo 3.1 (native 4K with upscaling to 8K via post-processing in tools like Gemini), Luma Ray3 (Hi-Fi 4K HDR), and LTX-2 (native 4K at 50fps). Native 8K generation remains uncommon, typically relying on upscaling; other prominent options like OpenAI Sora 2 and Runway Gen-4.5 generally output at 4K or lower resolutions. In 2025 evaluations, top proprietary AI video generators included Google Veo 3, recognized as best overall for cinematic quality, audio integration, and usability; Kling for visual realism and motion; OpenAI Sora for prompt adherence, storyboarding, and community features; Runway for advanced editing tools; and Hailuo MiniMax for excellent prompt adherence and speed. Rankings varied by source, with Veo 3 and Sora frequently highlighted as leaders. These leaders maintain closed architectures to protect training data and IP, contrasting open-source alternatives, though their outputs often require post-processing for production use due to inconsistencies in long-form coherence. As of late 2024/early 2025, features and availability change rapidly; users should check official sites for current status. As of February 2026, top free AI video generators (with free tiers or fully free access) include Sora 2 (OpenAI) as the best overall free option with high-quality text-to-video and audio sync, requiring no subscription but with limited resolution and duration in some modes; Runway, offering a strong free tier for creative generation, image-to-video, and editing on a credits-based system; Kling AI, providing free daily credits for photorealistic human motion and lip-sync; Pika, with free monthly credits suited for creative, social media-focused videos; and Luma Dream Machine, allowing free draft videos though limited and often watermarked. Many impose restrictions such as limited credits, watermarks, or short clip lengths, with the "best" option varying by use case, such as realism versus creativity.

Performance Metrics and Benchmarks

Text-to-video models are evaluated using a combination of automatic metrics assessing visual fidelity, temporal dynamics, and semantic alignment, alongside human preference studies to capture subjective quality. Key automatic metrics include Fréchet Video Distance (FVD), which quantifies distributional differences between generated and reference videos by incorporating temporal structure, often yielding lower scores (indicating better performance) for advanced models like those achieving FVD values below 200 on standard datasets such as UCF-101. measures per-frame realism, with state-of-the-art open-source models reporting FID scores around 10-20 on benchmarks like MSRVTT. CLIPScore evaluates text-video alignment by computing between text embeddings and video frame features, where scores exceeding 0.3 typically indicate strong prompt adherence. Comprehensive benchmarks dissect performance across granular dimensions to address limitations in holistic metrics like FVD, which can overlook specific failures such as flickering or inconsistency. EvalCrafter, introduced in 2023 and updated through 2024 evaluations, assesses models on 700 diverse prompts using 17 metrics spanning visual quality (e.g., aesthetic and sharpness via LAION-Aesthetics), content quality (e.g., object presence via DINO), motion quality (e.g., warping error and amplitude classification), and text-video alignment (e.g., CLIP and scores), with overall rankings derived from weighted human preferences aligning objective scores to user favorability. VBench, with its 2025 iteration VBench-2.0, employs a hierarchical suite of 16+ dimensions including subject consistency, temporal flickering (measured via frame-to-frame variance), motion smoothness (optical flow-based), and spatial relationships, normalizing scores between approximately 0.3 and 0.8 across open and closed models; human annotations confirm alignment with automatic evaluations, revealing persistent gaps in long-sequence consistency. T2V-CompBench, presented at CVPR 2025, focuses on compositional abilities with multi-level metrics (MLLM-based, detection-based, tracking-based) to probe complex scene interactions, highlighting deficiencies in attribute binding and temporal ordering. Proprietary models often outperform open-source counterparts in practical benchmarks emphasizing real-world deployability, such as maximum video length, resolution, and generation efficiency, though direct quantitative comparisons are constrained by limited access and datasets. OpenAI's Sora supports resolution videos up to 60 seconds at 24 FPS, enabling complex multi-shot narratives with high , as demonstrated in February 2024 previews, surpassing earlier limits of 5-10 seconds in models like Gen-3. Kling achieves 720p- outputs of 5-10 seconds at 24-30 FPS with render times of 121-574 seconds, excelling in motion realism per user tests. Gen-3 targets for 4-8 seconds at 24 FPS with ~45-second , prioritizing cinematic versatility. These capabilities reflect scaling laws where increased parameters and correlate with improved , yet benchmarks like Video-Bench reveal discrepancies between automatic scores and human-aligned preferences, with MLLM evaluators (e.g., GPT-4V) exposing over-optimism in metrics for dynamic scenes. Academic evaluations lag commercial releases, as models like Sora evade full until open APIs emerge, underscoring the need for standardized, accessible protocols to mitigate evaluation biases toward accessible open-source systems.

Evolution of Capabilities Across Iterations

Early text-to-video models, emerging around , relied on extensions of image diffusion techniques and produced clips typically limited to 2-5 seconds in duration, with resolutions under 256x256 pixels, frequent motion artifacts, and poor temporal coherence, such as unnatural object deformations or inconsistent backgrounds. These limitations stemmed from challenges in modeling spatiotemporal dependencies, often addressed via cascaded architectures separating spatial and temporal generation. By 2023-early 2024, iterations like Runway's Gen-2 introduced hybrid diffusion-transformer architectures, extending clip lengths to 4-16 seconds and supporting inputs beyond text, such as images for stylized extensions, while improving adherence to prompts through better factorization for motion. Runway's Gen-3 Alpha, released June 2024, advanced this via large-scale multimodal training on proprietary infrastructure, enabling video-to-video conditioning, higher stylistic control, and sequences up to 10 seconds at 720p with enhanced world simulation for plausible physics and multi-entity interactions. Similarly, Kling AI's initial 2024 release supported up to 10-second clips with basic motion brushes for localized edits, evolving by mid-2025 to Kling 2.0/2.5, which added cinematic lighting, slow-motion fidelity, and durations exceeding 2 minutes through upgraded and diffusion priors. OpenAI's Sora, announced February 2024, represented a pivotal by scaling transformer-based spatiotemporal patches to generate up to 60-second videos at , achieving superior , causal motion (e.g., realistic bouncing or ), and multi-shot consistency via a unified video tokenizer trained on vast internet-scale data. Sora 2, launched September 2025, further refined these with explicit physics simulation layers, reducing hallucinations in dynamic scenes and adding precise controllability for elements like camera paths, while maintaining or extending length capabilities. Across models, iterative gains correlated with compute scaling—often 10-100x increases per version—and dataset curation emphasizing high-quality video frames, yielding measurable uplifts in benchmarks like VBench for motion smoothness (from ~0.6 to 0.9 normalized scores) and human preference evaluations. By 2026, these tools and others are expected to offer longer videos, better coherence, and more advanced features due to rapid progress in AI video synthesis.
Model IterationRelease DateKey Capability AdvancesMax DurationResolution
Runway Gen-2Early 2024Image-conditioned generation, improved prompt fidelity4-16s
Gen-3 AlphaJune 2024Multimodal (text/image/video) inputs, enhanced temporal modeling10s++
Sora (v1)Feb 2024Spatiotemporal transformers, complex scene causality60s
Sora 2Sep 2025Physics-aware simulation, advanced controls60s+
Kling 1.xMid-2024Motion brushes, basic 3D awareness10s
Kling 2.0/2.52025Cinematic aesthetics, extended sequencing2min+
These evolutions reflect a shift from frame-by-frame to holistic video understanding, though persistent gaps remain in long-form coherence and rare-event , as evidenced by failure modes in benchmarks like Dynabench where later models still score below 0.8 on edge-case dynamics.

Applications and Broader Impacts

Creative and Commercial Deployments

Text-to-video models enable filmmakers and artists to prototype scenes, generate , and experiment with cinematic styles efficiently. OpenAI's Sora, released in February 2024 and updated to Sora 2 in September 2025, supports video generation up to one minute in length, allowing creators to produce photorealistic, animated, or surreal content from textual descriptions; collaborations with artists such as Minne Atairu have demonstrated its use in artistic video explorations adhering closely to prompts. Runway ML's tools, including Gen-3 and Gen-4, facilitate scene editing and background replacement in film production, with applications in for independent shorts and feature films. In and , these models accelerate content creation for commercials and campaigns. provides AI-driven generation for professional ads, enabling teams to produce customized videos without traditional shooting constraints, and grants users full commercial rights to outputs. , developed by , has been used to fabricate CGI product advertisements from text prompts, as in cases where users generated full promotional videos simulating high-value production at minimal cost. These models also support the creation of specialized content, such as Spanish-language horror videos set in 2026. Creators begin by writing the story in Spanish, dividing it into scenes with detailed visual descriptions, for example, "Un pasillo oscuro en 2026 con luces parpadeantes y sombras que se mueven." Clips are then generated scene-by-scene using text-to-video tools supporting Spanish prompts, including Kling AI for videos up to two minutes, Runway Gen-3 or Gen-4 for high-fidelity cinematic results, and Luma Dream Machine for realistic or stylized outputs. Prompts might specify, "En 2026, una figura siniestra emerge de la niebla en una ciudad abandonada, estilo terror psicológico, iluminación oscura, cámara lenta." Narration in Spanish with eerie tones is added via AI voice tools like ElevenLabs or PlayHT. Final assembly involves editing clips, incorporating royalty-free horror music from sources like Epidemic Sound or YouTube Audio Library, and applying effects in software such as CapCut, DaVinci Resolve, or Adobe Premiere, before exporting the combined video. For longer videos, multiple clips are generated and stitched together. As of February 2026, leading AI tools for generating animated cartoons from text prompts, scripts, or stories include Invideo AI, which produces cartoon videos featuring AI-generated scripts, multilingual voiceovers, subtitles, and text-based editing capabilities; Vyond, focused on animated character videos that convert prompts or scripts into scenes with character movements, voiceovers, and timeline editing; and Krikey AI, enabling text-to-3D animations with talking avatars, lip-synced voiceovers, and customizable 3D characters and videos. Complementary options such as Luma Dream Machine provide character consistency in animations, while OpenAI's Sora delivers high-quality story-to-video generation, including animated styles through targeted prompts. E-commerce platforms leverage for personalized product videos, automating script-to-visual workflows to enhance conversion rates through dynamic demonstrations. In broadcasting, Hour One's NVIDIA-accelerated platform converts text into videos featuring virtual humans for and training content, streamlining production for outlets requiring rapid, scalable output. To produce videos exceeding typical short clip durations, such as beyond 6 seconds, practitioners generate multiple sequential segments from extended or continued text prompts and combine them using video editing tools like CapCut or Adobe Premiere, with models like Kling AI supporting up to two minutes natively and Runway Gen-3 enabling 10-second clips extendable through this method. These deployments highlight efficiency gains, though outputs often require human post-editing for narrative coherence and brand alignment. In early 2026, popular AI video generation tools for monetized content creators on platforms like YouTube and TikTok include OpenAI's Sora for strong narrative and integration, Google Veo 3 for high-fidelity and physics-aware motion, Runway Gen-4/4.5 for advanced controls and professional quality, Kling AI for realistic humans and lip-sync, Luma Dream Machine for fast cinematic output, and Pika for creative effects. These tools enable efficient creation of high-quality text-to-video or image-to-video content for shorts, long-form videos, and marketing to drive views and revenue.

Economic Productivity Gains and Job Market Dynamics

Text-to-video models streamline by automating the generation of footage from textual prompts, reducing the time and labor traditionally required for scripting, storyboarding, and initial rendering. Tools such as Runway ML and OpenAI's Sora enable creators to produce promotional videos or ads in minutes rather than days, facilitating rapid iteration and cost savings in content workflows. In and , generative AI applications, including text-to-video, are projected to lower production costs by 10% across media sectors and up to 30% in and , allowing smaller teams to scale output without proportional increases in personnel or equipment. AI-assisted video scripting alone shortens phases by approximately 53%, boosting overall efficiency in and commercial deployments. These productivity enhancements, however, coincide with job market shifts, particularly in (VFX), , and roles vulnerable to . A January 2024 report from The Animation Guild, based on surveys of industry professionals, estimated that generative AI could disrupt around 204,000 U.S. jobs over three years, with one-third of respondents anticipating displacement for 3D modelers, sound editors, and broadcast video technicians due to automated generation of assets and edits. Freelance markets provide empirical evidence of early effects, where occupations highly exposed to generative AI—such as and tied to video adjuncts—saw a 2% decline in contracts and 5% earnings reduction by mid-2025. Despite displacement risks in routine tasks, text-to-video fosters new roles in AI oversight, such as and output refinement, while expanding demand for high-level creative direction as cheaper production enables more content volume. Broader generative AI integration, encompassing video tools, is forecasted to add 1.5 percentage points to annual labor growth, potentially offsetting losses through increased economic activity in and spend. Empirical patterns from prior waves suggest net job creation in adjacent fields, though transition costs—evident in entry-level roles—underscore the need for reskilling amid uneven across firm sizes.

Societal and Cultural Transformations

Text-to-video models have lowered barriers to , enabling non-experts to generate coherent, high-fidelity clips from textual prompts, thereby expanding access to visual beyond professional studios. This shift has accelerated content creation in domains like short-form , educational tutorials, and independent filmmaking, with tools such as OpenAI's Sora facilitating outputs that mimic without requiring cameras, actors, or editing software. By mid-2025, Sora's public app garnered over 1 million downloads in its launch week, reflecting rapid societal uptake for personal experimentation and viral content generation. Similarly, models like Runway Gen-3 and Kling AI have supported transitions from static images to dynamic sequences, compressing traditional production timelines from weeks to minutes. Culturally, these technologies foster emergent aesthetics emphasizing spectacle, , and rapid iteration, paralleling the novelty-driven appeal of early cinema where audiences embraced experimental visuals over narrative depth. This has manifested in novel art forms, such as AI-generated music videos and abstract animations shared on platforms like , where creators leverage text-to-video for hyper-personalized narratives unbound by physical constraints. However, the abundance of risks eroding distinctions between authentic and fabricated content, prompting cultural reevaluations of visual evidence in and historical documentation. experts have highlighted how lifelike outputs from Sora exacerbate challenges in discerning truth, potentially undermining public discourse. On a societal level, text-to-video amplifies inequalities in cultural production while promising broader participation; affluent users or those with prompt-engineering skills gain disproportionate influence, whereas marginalized creators may face amplified competition from automated outputs. Empirical assessments indicate risks to creative labor markets, with generative video automating and pre-visualization tasks historically performed by artists, as evidenced by concerns over Sora's encroachment on workflows. Yet, this also catalyzes hybrid practices where AI augments human intent, potentially enriching global through accessible tools for underrepresented voices in regions with limited resources. Brookings analyses underscore that while surges, unmitigated adoption could contract employment in and VFX by prioritizing efficiency over artisanal craft.

Democratization of Media Production

Text-to-video models enable individuals and small teams to produce complex video content from simple textual prompts, bypassing traditional requirements for cameras, lighting, actors, and crews. This shift reduces production costs dramatically; for instance, generating a short promotional video that once required thousands of dollars in equipment and labor can now be achieved on consumer hardware for under $100 in compute fees, depending on model access. Such empowers independent creators, marketers, and small businesses to compete with larger studios, fostering a proliferation of user-generated media on platforms like and . Adoption data underscores this trend: as of 2025, 85% of content creators have experimented with AI video tools, with 52% integrating them regularly into workflows, while 50% of small businesses report using AI-generated videos for tasks like product demos, which boost conversion rates by up to 40%. Tools like Runway ML, with its Gen-3 model released in 2024, provide intuitive interfaces for rapid iteration, allowing solo creators to output cinematic clips in minutes rather than days, thus leveling the playing field against resource-intensive traditional pipelines. Open-source alternatives, accessible via limited free tiers such as the Hugging Face Inference API for models like Zeroscope and ModelScope (with rate limiting) or Replicate's initial free credits for variants like Stable Video Diffusion, further amplify this by enabling customization without proprietary subscriptions; as of February 2026, several AI text-to-video generators offer free plans with no watermark, including Pixelbin (no signup required, limited to 3 videos per month using models like Google Veo and Kling), FlexClip (watermark-free with a 1-minute limit), and Canva (basic AI video creation without watermark), though constrained by credits, video length, features, and restrictions on violent content such as dynamic animal battles (e.g., crocodile vs. snake vs. tiger); advanced realistic generations remain possible with high-end models but are limited in quality and length on free tiers, with no fully unlimited free options due to compute demands. Proprietary models like OpenAI's Sora offer higher fidelity for polished outputs accessible via APIs. The market reflects surging demand, with the text-to-video AI sector valued at $250 million in and projected to reach $2.48 billion by 2032 at a 33.2% CAGR, driven largely by non-professional users seeking efficient . This extends to and non-profits, where low-barrier tools facilitate custom animations and explainers without hiring specialists, though empirical limitations in consistency and originality persist, requiring human oversight for professional viability. Overall, these models causalize a causal chain from idea to output, prioritizing speed and over artisanal craft, which has expanded media diversity but also intensified content saturation online.

Technical Challenges and Empirical Limitations

Fidelity and Consistency Shortcomings

Text-to-video models, predominantly based on processes, frequently exhibit shortcomings in , manifesting as degraded visual quality such as blurring, artifacts, and insufficient detail retention in generated frames. These issues arise from the inherent challenges in scaling image techniques to sequential frames, where prediction struggles to maintain sharp edges and textures under temporal constraints. For instance, models trained on limited high-resolution video datasets often produce outputs with over-smoothing effects, reducing perceptual realism compared to real footage. Temporal consistency represents a core limitation, with generated videos showing flickering objects, discontinuous motions, and erratic changes in entity appearances across frames when relying solely on text prompts. This stems from the autoregressive or frame-by-frame denoising in models, which lacks robust mechanisms for enforcing inter-frame coherence without auxiliary conditioning like or reference images. Empirical evaluations reveal that even advanced architectures fail to preserve logical flow in actions, such as stable trajectories for moving subjects, leading to unnatural or morphing. Spatial inconsistencies compound these problems, where elements like backgrounds or character poses deform unpredictably within individual frames or sequences, undermining narrative continuity. Diffusion-based approaches exacerbate this due to probabilistic sampling, which introduces variability that current training paradigms—often optimized for static image metrics—do not fully mitigate for dynamic scenes. Benchmarks indicate that without specialized plug-in methods for motion disentanglement or spatiotemporal augmentation, outputs diverge significantly from prompt-specified compositions, particularly in complex interactions involving multiple entities. These and consistency deficits persist across model scales, as larger parameter counts improve single-frame quality but demand disproportionate compute for video-length coherence, highlighting a gap between image and video generation paradigms. Real-world testing underscores that human evaluators rate such videos lower on alignment and realism metrics, with temporal artifacts reducing usability in applications requiring precise .

Scalability and Resource Constraints

Training text-to-video models necessitates immense computational resources, primarily due to the high-dimensional nature of video data, which encompasses spatial and temporal dimensions across numerous frames. Proprietary models like OpenAI's Sora require access to specialized data centers with thousands of high-end GPUs, with training costs for comparable open-source alternatives such as Open-Sora 2.0 amounting to around $200,000—still 5-10 times lower than estimates for leading closed systems. This disparity arises from the need to process petabytes of video sets, performing trillions of floating-point operations to learn coherent motion and scene dynamics, often leveraging architectures optimized for scaling but demanding proportional increases in hardware. Inference scalability remains constrained by per-generation compute intensity, where producing a single short video clip can require GPU hours equivalent to those for hundreds of text or generations. Text-to-video tasks, involving frame-by-frame consistency via techniques like sliding windows on short-clip training data, amplify this burden, leading to generation times of minutes to hours even on optimized servers. Services from providers like and enforce strict quotas and queues to manage demand, as unrestricted access would overwhelm available infrastructure; for example, early Sora deployments limited outputs to prevent server overload. Energy consumption poses a critical bottleneck, with dominating 80-90% of total AI compute in data centers and text-to-video emerging as particularly power-hungry due to its multimodal complexity. Projections suggest that scaling text-to-video generation at could drive annual energy use to levels comparable to India's national consumption, far exceeding text-based models. The associated is estimated to be orders of magnitude higher than for static synthesis, prompting scrutiny of in deployments reliant on fossil-fuel-powered grids. Hardware availability further limits , as consumer-grade setups lack the VRAM (often 80+ GB per GPU) for viable , confining advanced usage to cloud providers with escalating costs—potentially $10-100 per minute of output depending on resolution and . Architectural efforts toward , such as distilled models or quantization, offer partial mitigation but against , underscoring a fundamental tension between capability scaling laws and practical resource realism.

Evaluation Metrics and Real-World Testing Gaps

Common automatic metrics for text-to-video models include Fréchet Video Distance (FVD), which measures distributional similarity between generated and real videos; CLIP Score, assessing text-video alignment via ; and Inception Score (IS), evaluating visual diversity and appeal. These metrics enable scalable comparisons but often prioritize frame-level or short-sequence properties over holistic video attributes. Limitations of these automatic metrics stem from their inadequate capture of temporal dynamics, semantic reasoning, and human-perceived quality, rendering them unreliable proxies for overall performance. For instance, FVD and CLIP Score underperform in assessing motion or factual consistency, prompting reliance on evaluations despite their subjectivity and cost. Protocols like Text-to-Video Human Evaluation (T2VHE) address this by standardizing annotator training and dynamic modules, achieving higher reproducibility while reducing costs by nearly 50%. Emerging benchmarks introduce targeted metrics, such as DEVIL's dynamics scores for range, , and , which correlate over 90% with ratings by emphasizing multi-granularity temporal assessment. Similarly, T2VScore combines text-video alignment and expert-mixture evaluation on datasets like TVGE with 2,543 -judged samples. EvalCrafter extends this across video (via aesthetics and technicality), alignment (e.g., Detection-Score for objects), motion (e.g., Flow-Score), and temporal consistency (e.g., Warping Error), using 700 real-user prompts. Real-world testing reveals gaps in models' adherence to physics, world knowledge, and diverse scenarios, as benchmarks like PhyWorldBench demonstrate failures in 1,050 prompts across fundamental motion, interactions, and anti-physics cases, with state-of-the-art models exhibiting violations of and rigid-body dynamics. T2VWorldBench, spanning 1,200 prompts in categories like and culture, shows advanced models producing semantically inconsistent outputs lacking factual accuracy, underscoring deficiencies in commonsense integration. Evaluations remain constrained to short clips and curated prompts, limiting insights into long-form generation, user-varied inputs, and deployment-scale robustness.

Controversies, Risks, and Policy Debates

Intellectual Property Disputes and Training Data Sourcing

Text-to-video models, such as those developed by and , rely on expansive datasets comprising billions of video clips sourced primarily from public internet repositories like , often without explicit licensing from copyright holders. This practice has sparked disputes, centering on whether the ingestion and analysis of copyrighted videos for constitutes unauthorized under copyright law. Proponents of the models argue that processes transform data into non-expressive parameters, akin to human learning, and qualify as ; however, critics contend that mass copying undermines creators' exclusive rights to and works, depriving them of potential licensing in an emerging AI data market valued at billions. A prominent case involves ML, where a leaked internal from July 2024 revealed plans to systematically download, tag, and train on thousands of videos, including copyrighted content, without permission. The document outlined categorization by attributes like camera motion and scene type, highlighting deliberate sourcing strategies that bypassed 's prohibiting such scraping for commercial AI development. has not faced a direct over this leak as of October 2025, but it echoes broader class-action suits against video AI firms; for instance, artists and creators filed claims against , Stability AI, and in 2021, alleging unauthorized use of visual works in training datasets that extend to video generation. OpenAI's Sora model has similarly drawn scrutiny, with reports indicating training on unlicensed internet videos contributing to outputs that replicate protected elements, prompting policy shifts. In September 2025, OpenAI announced an opt-out mechanism for Sora 2, allowing copyright holders to block generation of their characters unless explicitly permitted, reversing an initial opt-in approach amid backlash from studios and the Motion Picture Association. This followed accusations that Sora's training data ingestion violated copyrights, paralleling over 25 pending U.S. suits against AI firms for similar practices across modalities. The U.S. Copyright Office's May 2025 report on generative AI training emphasized that while models do not retain literal copies, the initial data copying phase implicates reproduction rights, recommending legislative clarity on opt-out systems and licensing to balance innovation with owner protections. These disputes underscore sourcing challenges: datasets like those derived from web crawls often include pirated or licensed footage inadvertently, amplifying infringement risks, while alternatives remain scarce due to high costs. Some firms, such as , have pursued licensed deals—paying $1.5 billion for training data access—suggesting viable paths forward, though most text-to-video developers continue relying on defenses amid unresolved litigation. Courts have issued mixed rulings; a February 2025 decision rejected where training deprived licensing markets, signaling potential liability for video AI if outputs compete with originals. As of October 2025, no text-to-video-specific precedent has settled the core training question, leaving models exposed to claims that could reshape data acquisition norms.

Potential for Misuse (Deepfakes, Propaganda)

Text-to-video models, such as OpenAI's Sora and variants of Stable Video Diffusion, enable the generation of highly realistic videos from textual prompts, including depictions of specific individuals performing fabricated actions or delivering false statements, thereby lowering barriers to production compared to traditional techniques. These capabilities exploit diffusion-based architectures to synthesize coherent motion and facial expressions, often indistinguishable from authentic footage without forensic analysis. Following Sora's public release as an app in September 2025, users rapidly generated unauthorized featuring celebrities' likenesses, including actors like , leading to widespread backlash over privacy violations and non-consensual portrayals. The app achieved 1 million downloads within its first week, amplifying the scale of such misuse, with reports of videos depicting deceased figures in fabricated scenarios raising additional ethical concerns about . In response, imposed restrictions on likeness usage and deepfake outputs, influenced by pressure from , though enforcement relies on user opt-ins and prompt monitoring, which experts note as imperfect safeguards. For , text-to-video models heighten risks of by enabling scalable fabrication of political events or speeches, potentially eroding trust in visual media during elections or conflicts. In the global elections, AI-generated videos contributed to viral , though most instances involved low-fidelity "AI slop" or memes rather than sophisticated capable of swaying outcomes, as evidenced by post-election analyses showing no decisive electoral impact from such content. Despite this, projections for 2025 onward warn of escalating threats, given models' improving fidelity and accessibility, with peer-reviewed studies highlighting vulnerabilities in detection systems against diffusion-generated forgeries. Empirical limitations in real-world testing underscore that while current deepfake detection achieves up to 96% accuracy in controlled settings, generalization to novel text-to-video outputs remains inconsistent.

Bias Amplification from Training Datasets

Text-to-video models are trained on expansive datasets of video clips annotated with textual descriptions, frequently derived from web-scraped content that mirrors imbalances in media representation, such as disproportionate depictions of males in executive roles or Western-centric cultural narratives. These datasets propagate empirical correlations from real-world sources, including underrepresentation of non-Western ethnicities or females in STEM professions, which models internalize during . In diffusion-based architectures prevalent in text-to-video generation, such as those underlying models like Sora, bias amplification arises mechanistically: the iterative denoising process optimizes for high-likelihood trajectories in latent space, thereby exaggerating dataset imbalances as the model prioritizes frequently observed patterns over rarer, equally valid ones. This results in generated videos that intensify stereotypes; for instance, prompts for "a leader addressing a team" yield outputs where male figures dominate at rates exceeding their already skewed prevalence in training videos. Studies confirm this effect scales with model depth and dataset size, where deeper networks amplify variance in biased directions due to compounded error reinforcement in generative sampling. Empirical audits of Sora, conducted via systematic prompting with gender-neutral and stereotypical cues, reveal persistent associations—e.g., tasks linked to s in over 80% of outputs despite neutral inputs—directly attributable to reflections of societal media patterns rather than algorithmic . Analogous amplification appears in racial portrayals, where generative outputs for neutral occupation prompts overrepresent lighter-skinned individuals in high-status roles, surpassing base rates in source videos by leveraging correlated visual cues like attire or settings. Such dynamics stem from causal dependencies in : prevalent co-occurrences (e.g., "CEO" with male attire in videos) become overfitted priors, sidelining underrepresented variants absent sufficient counterexamples. While proprietary datasets obscure full quantification, open analyses indicate amplification ratios can exceed 1.5-2x relative to input distributions, as measured in controlled generation experiments. Mitigation efforts, including targeted fine-tuning on debiased subsets or , show partial efficacy but falter against entrenched latent encodings from initial training. This underscores a core limitation: without curated, balanced data reflecting causal diversity in real-world variance, models risk entrenching amplified distortions that misrepresent empirical realities.

Regulatory Approaches: Innovation vs. Precautionary Principles

The in AI regulation posits that potential harms from technologies like text-to-video models—such as amplified misuse or —should prompt preemptive restrictions until safety is demonstrably assured, prioritizing risk aversion over unproven benefits. This approach, rooted in environmental and health precedents, has been critiqued for historically delaying innovations without commensurate evidence of reduced harms, as seen in stalled advancements in where regulatory burdens exceeded empirical justifications for caution. In the context of text-to-video generation, proponents argue it necessitates upfront compliance testing to mitigate societal risks, though empirical data on AI-specific harms remains sparse relative to modeled scenarios. The European Union's AI Act, effective from August 1, 2024, exemplifies a precautionary framework applied to generative models including text-to-video systems, classifying general-purpose AI (GPAI) like OpenAI's Sora under transparency mandates rather than outright high-risk bans. Providers must disclose training data summaries, watermark outputs for detectability, and conduct risk assessments for systemic threats, with fines up to 7% of global turnover for non-compliance; text-to-video tools face added scrutiny for copyrighted material ingestion, aligning with EU copyright directives. This regime aims to preempt deepfake proliferation—evidenced by incidents like AI-generated videos influencing public discourse—but critics, including U.S.-based policy analysts, contend it imposes asymmetric burdens on European innovators, potentially ceding global leadership to less-regulated jurisdictions. In contrast, permissionless innovation advocates favor minimal barriers, allowing text-to-video deployment with post-hoc remedies for verifiable harms, arguing that adaptive better fosters empirical learning and economic gains—U.S. GDP projections from AI advancement estimate trillions in value by 2030 if unhindered. The lacks comprehensive federal AI statutes as of October 2025, relying instead on targeted measures like the TAKE IT DOWN Act (signed May 22, 2025), which criminalizes non-consensual without broadly encumbering model development. State-level responses, such as California's 2019 election ad disclosure laws and over a dozen 2024 enactments restricting political synthetics, emphasize misuse over foundational tech constraints, reflecting a view that overregulation risks echoing past tech suppressions without proportional safety dividends. Policy debates highlight tensions: precautionary models may amplify biases in regulatory bodies toward risk exaggeration, as academic and media sources often overstate AI existential threats absent causal evidence, while innovation proponents cite historical precedents where light-touch policies accelerated diffusion and self-correction, such as yielding net societal benefits despite initial fears. For text-to-video, empirical gaps persist— detections improved 40% via watermarking standards in 2024 trials, suggesting targeted tools suffice over blanket precaution—yet calls for harmonized global approaches intensify, with U.S. frameworks potentially influencing via market dominance.

References

  1. https://arxiv.org/abs/2510.04999
  2. https://arxiv.org/abs/2311.06329
  3. https://openreview.net/pdf?id=sgDFqNTdaN
  4. https://openai.com/index/sora/
  5. https://openai.com/index/sora-2/
  6. https://openreview.net/pdf/2b1f7482437bc21a773608619b12a496ae166188.pdf
  7. https://arxiv.org/abs/2402.14780
  8. https://proceedings.neurips.cc/paper_files/paper/2024/file/c6483c8a68083af3383f91ee0dc6db95-Paper-Conference.pdf
  9. https://arxiv.org/abs/2405.03150
  10. https://lilianweng.github.io/posts/2024-04-12-diffusion-video/
  11. https://towardsdatascience.com/the-evolution-of-text-to-video-models-1577878043bd
  12. https://arxiv.org/abs/1710.00421
  13. https://www.ijcai.org/proceedings/2019/0276.pdf
  14. https://www.ijcai.org/Proceedings/2019/0307.pdf
  15. https://arxiv.org/abs/2009.02018
  16. https://huggingface.co/blog/text-to-video
  17. https://ai.meta.com/blog/generative-ai-text-to-video/
  18. https://arxiv.org/abs/2210.02399
  19. https://imagen.research.google/video/
  20. https://www.theverge.com/2022/10/6/23390607/ai-text-to-video-google-imagen-phenaki-new-research
  21. https://ai.meta.com/blog/emu-text-to-video-generation-image-editing-research/
  22. https://openai.com/index/sora-is-here/
  23. https://runwayml.com/research/introducing-gen-3-alpha
  24. https://venturebeat.com/ai/luma-ai-debuts-dream-machine-for-realistic-video-generation-heating-up-ai-media-race
  25. https://cloud.google.com/blog/products/ai-machine-learning/introducing-veo-and-imagen-3-on-vertex-ai
  26. https://klingai.com/global/
  27. https://runwayml.com/
  28. https://www.bbc.com/news/articles/crkjgrvg6z4o
  29. https://ir.kuaishou.com/news-releases/news-release-details/kling-ai-launches-25-turbo-video-model-industry-leading
  30. https://finance.yahoo.com/news/people-show-world-see-launch-140000209.html
  31. https://gemini.google.com/veo
  32. https://skywork.ai/blog/sora-2-ai-video-tools-platform-impact-2025/
  33. https://www.datacamp.com/blog/top-video-generation-models
  34. https://www.synthesia.io/post/best-ai-video-generators
  35. https://research.nvidia.com/labs/toronto-ai/VideoLDM/
  36. https://arxiv.org/abs/2408.06072
  37. https://openai.com/index/video-generation-models-as-world-simulators/
  38. https://arxiv.org/abs/2501.08453
  39. https://arxiv.org/abs/2204.03458
  40. https://snap-research.github.io/hpdm/
  41. https://huggingface.co/ali-vilab/text-to-video-ms-1.7b
  42. https://wandb.ai/byyoung3/ml-news/reports/Experimenting-with-Stable-Video-Diffusion---Vmlldzo3NDA0MDUy
  43. https://help.runwayml.com/hc/en-us/articles/47313737321107-Text-to-Video-Prompting-Guide
  44. https://deepmind.google/models/veo/prompt-guide/
  45. https://huggingface.co/docs/diffusers/en/using-diffusers/svd
  46. https://openaccess.thecvf.com/content/WACV2025/papers/Zhang_FastVideoEdit_Leveraging_Consistency_Models_for_Efficient_Text-to-Video_Editing_WACV_2025_paper.pdf
  47. https://www.reddit.com/r/MediaSynthesis/comments/11vzx7a/modelscopetexttovideosynthesis_model_in_open/
  48. https://www.fabianmosele.com/ai-timeline
  49. https://github.com/guoyww/AnimateDiff
  50. https://stable-diffusion-art.com/animatediff/
  51. https://stability.ai/news/stable-video-diffusion-open-ai-video-model
  52. https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1
  53. https://stable-diffusion-art.com/stable-video-diffusion-img2vid/
  54. https://huggingface.co/genmo/mochi-1-preview
  55. https://huggingface.co/Tencent-Hunyuan/HunyuanVideo
  56. https://huggingface.co/THUDM/CogVideoX-5b
  57. https://github.com/huggingface/diffusers
  58. https://github.com/showlab/Awesome-Video-Diffusion
  59. https://www.cnbc.com/2025/09/30/openai-sora-ai-video-generator-us-canada.html
  60. https://openai.com/policies/creating-sora-videos-in-line-with-our-policies/
  61. https://the-decoder.com/runways-new-gen-3-alpha-turbo-video-model-is-7x-faster-at-half-the-cost/
  62. https://www.reddit.com/r/singularity/comments/1dsy8ze/your_move_sora_runwaymls_gen_3_video_model/
  63. https://www.maginative.com/article/kuaishou-unveils-kling-a-text-to-video-model-to-challenge-openais-sora/
  64. https://skywork.ai/blog/kling-ai-text-to-video-explained/
  65. https://ir.kuaishou.com/news-releases/news-release-details/kuaishou-kling-ai-integrates-deepseek-lowering-entry-barrier-ai
  66. https://deepmind.google/models/veo/
  67. https://kie.ai/runway-gen4-turbo
  68. https://blog.youtube/news-and-events/generative-ai-creation-tools-made-on-youtube-2025/
  69. https://hailuoai.video/
  70. https://ulazai.com/ai-video-models-guide-2025/
  71. https://sacra.com/c/pika/
  72. https://www.tomsguide.com/features/5-best-ai-video-generators-tested-and-compared
  73. https://iacis.org/iis/2025/1_iis_2025_377-393.pdf
  74. https://openaccess.thecvf.com/content/CVPR2024/papers/Liu_EvalCrafter_Benchmarking_and_Evaluating_Large_Video_Generation_Models_CVPR_2024_paper.pdf
  75. https://www.pixazo.ai/blog/ai-video-generation-models-comparison-t2v
  76. https://runwayml.com/research
  77. https://app.klingai.com/global/release-notes/2025-09-19
  78. https://openai.com/index/sora
  79. https://openai.com/sora/
  80. https://www.bitechnology.com/what-is-runway-ml-how-does-it-work/
  81. https://runwayml.com/ai-for-ads
  82. https://help.runwayml.com/hc/en-us/articles/18927776141715-Usage-rights
  83. https://www.aibase.tech/news/features/create-a-product-commercial-video-with-kling-ai/
  84. https://kling.ai
  85. https://runwayml.com
  86. https://lumalabs.ai
  87. https://elevenlabs.io
  88. https://play.ht
  89. https://invideo.io/make/ai-cartoon-video-generator/
  90. https://www.vyond.com/product/vyond-go/
  91. https://www.krikey.ai/
  92. https://quantilus.com/article/how-ecommerce-brands-are-using-ai-for-product-videos-a-case-study-on-revolutionizing-content-strategy/
  93. https://www.nvidia.com/en-us/customer-stories/virtual-human-meets-virtual-studio-ai-powers-text-to-video-for-broadcast/
  94. https://www.brookings.edu/articles/how-openais-sora-hurts-the-creative-industries/
  95. https://manus.im/blog/best-ai-video-generator
  96. https://superagi.com/from-text-to-video-how-ai-tools-like-runway-ml-and-openai-sora-are-transforming-motion-design-in-2025/
  97. https://reelmind.ai/blog/the-evolution-of-ai-video-openai-sora-s-impact-on-the-creator-economy
  98. https://www.morganstanley.com/insights/articles/ai-in-media-entertainment-benefits-and-risks
  99. https://www.zebracat.ai/post/ai-video-creation-statistics
  100. https://animationguild.org/wp-content/uploads/2024/01/Future-Unscripted-The-Impact-of-Generative-Artificial-Intelligence-on-Entertainment-Industry-Jobs-pages-1.pdf
  101. https://www.latimes.com/entertainment-arts/business/story/2024-01-30/ai-artificial-intelligence-impact-report-entertainment-industry
  102. https://www.brookings.edu/articles/is-generative-ai-a-job-killer-evidence-from-the-freelance-market/
  103. https://www.argil.ai/blog/video-production-jobs-in-2024-how-ai-is-transforming-content-creation-careers
  104. https://www.gspublishing.com/content/research/en/reports/2023/03/27/d64e052b-0f6e-45d7-967b-d7be35fabd16.html
  105. https://educationalvoice.co.uk/ai-animation-and-job-displacement/
  106. https://www.latimes.com/business/story/2025-10-26/sora-the-bizarre-mind-bending-ai-slop-machine
  107. https://mmg-1.com/the-transformative-impact-of-generative-ai-on-computer-generated-imagery-in-filmmaking/
  108. https://theconversation.com/spectacle-weirdness-and-novelty-what-early-cinema-tells-us-about-the-appeal-of-ai-slop-265071
  109. https://www.theguardian.com/us-news/2025/oct/04/openai-sora-violence-racism
  110. https://arxiv.org/pdf/2406.11859
  111. https://www.entrepreneur.com/growing-a-business/ai-video-has-changed-marketing-forever-heres-how-to-adapt/497898
  112. https://www.swiftask.ai/blog/text-to-video-ai
  113. https://www.onemoreshot.ai/blog/ai-video-creation-trends-2025
  114. https://skywork.ai/blog/ai-video/sora-2-vs-runway-gen-3-honest-comparison-for-creators/
  115. https://www.pixelbin.io/ai-tools/video-generator
  116. https://www.flexclip.com/create/artificial-intelligence-video.html
  117. https://www.canva.com/features/ai-video-generator/
  118. https://noamkroll.com/what-sora-ai-video-means-for-indie-filmmakers/
  119. https://www.credenceresearch.com/report/text-to-video-ai-market
  120. https://www.wevideo.com/blog/ai-in-action
  121. https://sustineriattorneys.com/2024/10/14/the-breakthrough-of-text-to-video-ai-in-modern-content-creation/
  122. https://www.labellerr.com/blog/evaluating-and-finetuning-text-to-video-model/
  123. https://artgor.medium.com/paper-review-star-spatial-temporal-augmentation-with-text-to-video-models-for-real-world-video-ff0ddcc6352f
  124. https://www.researchgate.net/figure/Limitations-of-text-to-video-diffusion-models-for-generating-consistent-videos-Top_fig4_385977708
  125. https://arxiv.org/html/2502.17863v1
  126. https://dl.acm.org/doi/10.1145/3702250.3702298
  127. https://arxiv.org/html/2403.02332v3
  128. https://labelbox.com/guides/a-comprehensive-approach-to-evaluating-text-to-video-models/
  129. https://arxiv.org/html/2503.09642v1
  130. https://www.scribbledata.io/blog/exploring-openais-sora-and-text-to-video-models-a-complete-guide/
  131. https://gradientflow.com/sora-turbo-real-world-constraints/
  132. https://workplaceinsight.net/text-to-video-genai-will-help-drive-energy-consumption-of-openai-to-same-level-as-india/
  133. https://www.technologyreview.com/2025/05/20/1116327/ai-energy-usage-climate-footprint-big-tech/
  134. https://futurism.com/future-society/ai-power-usage-text-to-video-generator
  135. https://www.leylinepro.ai/blog/how-to-train-video-generation-ai
  136. https://arxiv.org/abs/2401.07781
  137. https://arxiv.org/abs/2406.08845
  138. https://arxiv.org/abs/2407.01094
  139. https://arxiv.org/abs/2507.13428
  140. https://arxiv.org/abs/2507.18107
  141. https://venturebeat.com/ai/runway-faces-backlash-after-report-of-copying-ai-video-training-data-from-youtube
  142. https://www.marketingaiinstitute.com/blog/runway-youtube
  143. https://www.copyright.gov/ai/Copyright-and-Artificial-Intelligence-Part-3-Generative-AI-Training-Report-Pre-Publication-Version.pdf
  144. https://www.arl.org/blog/training-generative-ai-models-on-copyrighted-works-is-fair-use/
  145. https://www.debevoise.com/insights/publications/2025/02/an-early-win-for-copyright-owners-in-ai-cases-as
  146. https://blog.trialline.net/generative-ai-lawsuits-timeline/
  147. https://www.techdogs.com/tech-news/td-newsdesk/stability-midjourney-and-runway-make-their-stand-in-the-ai-copyright-lawsuit
  148. https://deadline.com/2025/10/mpa-sora-2-openai-copyright-infringement-1236571849/
  149. https://www.wsj.com/tech/ai/openais-new-sora-video-generator-to-require-copyright-holders-to-opt-out-071d8b2a
  150. https://www.mccarter.com/insights/court-sets-new-limits-on-use-of-copyrighted-materials-to-train-ai-models/
  151. https://www.wired.com/story/ai-copyright-case-tracker/
  152. https://aimagazine.com/news/sora-the-app-igniting-mass-downloads-copyright-concerns
  153. https://time.com/7327031/openai-sora-deepfakes-privacy/
  154. https://link.springer.com/chapter/10.1007/978-3-032-00635-6_9
  155. https://pmc.ncbi.nlm.nih.gov/articles/PMC12305536/
  156. https://www.cnbc.com/2025/10/20/open-ai-sora-bryan-cranston-sag-aftra.html
  157. https://www.nbcnews.com/tech/tech-news/are-deepfakes-dead-people-rewriting-rcna235982
  158. https://www.businessinsider.com/sora-video-openai-fetish-content-my-face-problem-2025-10
  159. https://www.npr.org/2025/10/10/nx-s1-5567162/sora-ai-openai-deepfake
  160. https://www.npr.org/2024/12/21/nx-s1-5220301/deepfakes-memes-artificial-intelligence-elections
  161. https://www.weforum.org/stories/2025/01/deepfakes-different-threat-than-expected/
  162. https://www.sciencedirect.com/science/article/pii/S2444569X25001271
  163. https://arxiv.org/html/2410.04372v1
  164. https://www.sciencedirect.com/science/article/pii/S2212473X25000355
  165. https://drexel.edu/news/archive/2024/April/machine-learning-generative-ai-video-detection
  166. https://news.mit.edu/2024/study-large-language-models-datasets-lack-transparency-0830
  167. https://myscale.com/blog/biases-impacting-text-2-video-model-accuracy/
  168. https://arxiv.org/abs/2501.01987
  169. https://arxiv.org/abs/2505.17560
  170. https://www.researchgate.net/publication/391050839_BIAS_AMPLIFICATION_IN_DIFFUSION_MODELS_DETECTING_AND_CORRECTING_ETHICAL_DRIFT_AT_THE_LATENT_LEVEL
  171. https://www.bloomberg.com/graphics/2023-generative-ai-bias/
  172. https://openaccess.thecvf.com/content/CVPR2024/papers/Chen_Would_Deep_Generative_Models_Amplify_Bias_in_Future_Models_CVPR_2024_paper.pdf
  173. https://pmc.ncbi.nlm.nih.gov/articles/PMC12103484/
  174. https://www.researchgate.net/publication/391241260_BIAS_AMPLIFICATION_IN_DIFFUSION_MODELS_ETHICAL_RISKS_AND_MITIGATION_APPROACHES
  175. https://itif.org/publications/2019/02/04/ten-ways-precautionary-principle-undermines-progress-artificial-intelligence/
  176. https://www.aei.org/research-products/report/the-precautionary-principle-safety-regulation-and-ai-this-time-it-really-is-different/
  177. https://dash.harvard.edu/server/api/core/bitstreams/942665e9-90e9-42e7-a2f1-34afc8f3b4ce/content
  178. https://artificialintelligenceact.eu/high-level-summary/
  179. https://www.europarl.europa.eu/topics/en/article/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence
  180. https://www.wilmerhale.com/en/insights/blogs/wilmerhale-privacy-and-cybersecurity-law/20241002-navigating-generative-ai-under-the-european-unions-artificial-intelligence-act
  181. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
  182. https://www.dajv.de/ai-act/the-us-innovates-the-eu-regulates-contrasting-approaches-to-ai-regulation-across-the-atlantic/
  183. https://www.congress.gov/crs-product/R48555
  184. https://www.mintz.com/insights-center/viewpoints/54731/2025-05-22-president-trump-signs-ai-deepfake-act-law-and-house
  185. https://www.ncsl.org/technology-and-communication/deceptive-audio-or-visual-media-deepfakes-2024-legislation
  186. https://www.brennancenter.org/our-work/research-reports/regulating-ai-deepfakes-and-synthetic-media-political-arena
  187. https://research.lancaster-university.uk/files/423283414/LancasterWP2025_003.pdf
  188. https://www.cato.org/blog/what-might-good-ai-policy-look-four-principles-light-touch-approach-artificial-intelligence
  189. https://www.freestatefoundation.org/wp-content/uploads/2025/06/Hindsight-Should-Not-Hasten-AI-Regulation-060525.pdf
Add your contribution
Related Hubs
User Avatar
No comments yet.