A research project from Google Research, announced in January 2024, presenting a diffusion model that generates video from text. Researchers from the Weizmann Institute, Tel Aviv University, and the Technion also contributed. Whereas conventional video generation models follow a “create keyframes first, then fill in and smooth the gaps” pipeline, Lumiere generates the entire temporal duration of a video in a single pass. This design aims to deliver both large-scale motion and temporal consistency at the same time. Beyond text-to-video, the paper also demonstrates applications such as image-to-video, stylized generation with a reference image, and video inpainting. Note that Lumiere is a research result released as a paper and a project page ─ it is not offered as an app or API that general users can access.
Key Features
- Single-pass generation with a Space-Time U-Net: The model uses a U-Net that performs down- and up-sampling in both the spatial and temporal dimensions, generating all frames of a video in one pass. Unlike conventional methods that split keyframe generation from temporal super-resolution, temporal consistency is less likely to break down along the way
- 5-second, 80-frame, 16 fps output: The paper describes generating 80 frames at 16 fps, that is, five seconds of video. The researchers report that competing methods of the time tended to produce roughly four-second clips with limited motion, while their approach preserved larger motion while maintaining consistency
- Built on a pre-trained text-to-image model: Rather than training a video model from scratch, it extends an existing text-to-image diffusion model by leveraging its pre-trained weights
- Image-to-video generation: Starting from a still image, it can produce video that adds motion while preserving the original content
- Stylized generation and video stylization: It can generate video that reflects the touch and texture of a reference image, and it can also apply style transfer to an existing video consistently across frames
- Cinemagraphs and inpainting: The same framework extends to cinemagraphs, which animate only part of a still image, and to video inpainting, which rewrites the contents of a masked region
Pricing
| Plan | Price | Details |
|---|---|---|
| Not generally available | ─ | Only the research paper and project page are public. No app, API, or pre-trained weights have been announced for general release |
Pricing information is current as of August 2026. Please check the official project page for the latest status.
Pros & Cons
✅ Pros
- Because the whole video is generated at once, subjects and backgrounds are less likely to drift as time progresses
- It demonstrated a direction that combines large motion with consistency, influencing later research on video generation
- Text-to-video, image animation, style transfer, and inpainting are all handled within the same framework
- The project page publishes many generated examples, so you can judge the method’s capabilities with your own eyes
⚠️ Cons
- It is a research project and is not offered as a service that general users can try
- No official implementation code or pre-trained weights have been released, which makes reproducing it on your own difficult
- Output length is around five seconds, so it is not suited to long-form video production as is
- It was announced in January 2024, so it belongs to a different generation than the commercial video generation services that followed
Comparison with Similar Services
Placing Lumiere next to the models and services used as comparisons in the paper makes its position clearer.
| Item | Lumiere | Runway | Pika | Stable Video Diffusion |
|---|---|---|---|---|
| Provider | Google Research | Runway | Pika Labs | Stability AI |
| Positioning | Research project | Commercial service | Commercial service | Publicly released model |
| General access | Not available | Available | Available | Available (model released) |
| Generation approach | Whole duration at once | Keyframes plus temporal interpolation is the norm | Keyframes plus temporal interpolation is the norm | Video extension of an image model |
| Main use | Research and validation of methods | Video production, advertising | Short-form video for social media | Experiments and integration in your own environment |
The video generation features Google offers to general users are delivered through a separate product line, not through Lumiere. It is more accurate to treat Lumiere as a research result that presented the underlying technical ideas.
Who Is It For
- Engineers and researchers who want to understand the internal structure and design philosophy of video generation models
- Anyone who wants to see the difference between keyframe-based and single-pass generation with concrete examples
- Writers covering AI who want to follow how video generation technology has evolved
- Developers gathering material for method selection when adding video generation to their own products
Summary
Lumiere is a Google Research model that sought to combine large motion with temporal consistency by generating the entire duration of a video at once. A defining trait is that it handles not only text-to-video but also image animation, style transfer, and inpainting within a single framework. That said, it is not a generally available service, and neither code nor weights have been published, so it is not a tool for anyone who wants to make a video right now. It is best regarded as a valuable reference for those who want to understand how video generation works and to keep up with the direction of the technology.