v0.39.0
📦 diffusersView on GitHub →
✨ 10 features🔧 9 symbols
Summary
This release introduces nine new state-of-the-art pipelines including Cosmos 3, Ideogram 4, Krea 2, and DiffusionGemma, significantly expanding multimodal and video generation capabilities.
✨ New Features
- Introduced Cosmos 3 pipeline, NVIDIA's unified world foundation model (WFM) built on a Mixture-of-Transformers (MoT) architecture, supporting video-to-video and action-conditioned generation, and a sound encoder.
- Introduced Ideogram 4 pipeline, a flow-matching text-to-image model featuring structured prompt upsampling and LoRA loading support.
- Introduced Krea 2 (K2) pipeline, a flow-matching text-to-image model using a single-stream MMDiT with grouped-query attention and supporting base/TDM checkpoints and a LoRA DreamBooth trainer.
- Introduced DreamLite pipeline for text-to-image and image-editing, featuring a custom 2D U-Net (`DreamLiteUNetModel`) and a distilled `DreamLiteMobilePipeline` for low-latency generation.
- Introduced PRX Pixel pipeline, a pixel-space text-to-image model that denoises raw RGB images directly (no VAE needed) using a ~7B `PRXTransformer2DModel`.
- Introduced Motif-Video pipeline, a 2B parameter diffusion transformer for text-to-video and image-to-video generation featuring Shared Cross-Attention and rectified flow matching.
- Introduced AnyFlow framework, the first any-step video diffusion framework built on flow maps, supporting bidirectional and FAR causal pipelines for various video tasks.
- Introduced JoyAI-Image pipeline, a unified multimodal foundation model supporting image understanding, text-to-image generation, and instruction-guided image editing, including spatial editing capabilities.
- Introduced DiffusionGemma pipeline, a block-diffusion encoder-decoder language model utilizing a causal encoder and bidirectional decoder with a new `BlockRefinementScheduler`.
- Introduced Anima pipeline, a 2 billion parameter text-to-image model focused on anime concepts, characters, and styles.