2022-12-02 11:42:09 +00:00
# What is latent blending?
2022-11-21 23:20:07 +00:00
2022-12-09 11:52:50 +00:00
Latent blending allows you to generate smooth video transitions between two prompts. It is based on [stable diffusion 2.1 ](https://stability.ai/blog/stablediffusion2-1-release7-dec-2022 ) and remixes the latent reprensetation using spherical linear interpolations. This results in imperceptible transitions, where one image slowly turns into another one.
2022-11-21 23:20:07 +00:00
2023-01-09 08:00:24 +00:00
# Example 1: Simple transition
2023-01-09 07:58:03 +00:00
2023-01-09 08:00:24 +00:00
To run a simple transition between two prompts, run `example1_standard.py`
2022-11-21 23:20:07 +00:00
2023-01-09 08:00:24 +00:00
# Example 2: Inpainting transition
To run a transition between two prompts where you want some part of the image to remain static, run `example2_inpaint.py`
2022-11-21 23:20:07 +00:00
2022-12-02 11:42:09 +00:00
# Example 3: concatenated transition
2023-01-09 08:00:24 +00:00
To run multiple transition between K prompts, resulting in a stitched video, run `example3_multitrans.py`
2022-11-21 23:20:07 +00:00
2022-12-02 11:42:09 +00:00
# Relevant parameters
2022-11-21 23:20:07 +00:00
2022-12-02 11:42:09 +00:00
# Installation
#### Packages
```commandline
2022-12-02 12:08:17 +00:00
pip install -r requirements.txt
2022-12-02 11:42:09 +00:00
```
2022-12-02 12:08:17 +00:00
#### Download Models from Huggingface
2022-12-09 11:52:50 +00:00
[Download the Stable Diffusion v2-1_768 Model ](https://huggingface.co/stabilityai/stable-diffusion-2-1 )
2022-12-02 11:42:09 +00:00
[Download the Stable Diffusion 2.0 Inpainting Model (optional) ](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting )
2022-12-09 11:52:50 +00:00
#### (Optional) Install [Xformers](https://github.com/facebookresearch/xformers)
2022-12-02 12:43:35 +00:00
With xformers, stable diffusion 2 will run a bit faster. The recommended way of installation is via the supplied binaries (Linux).
2022-12-02 11:42:09 +00:00
```commandline
2022-12-02 12:08:17 +00:00
conda install xformers -c xformers/label/dev
2022-12-02 11:42:09 +00:00
```
2022-12-02 12:08:17 +00:00
Alternatively, you can build it from source:
2022-12-02 11:42:09 +00:00
```commandline
2022-12-02 12:08:17 +00:00
# (Optional) Makes the build much faster
pip install ninja
# Set TORCH_CUDA_ARCH_LIST if running and building on different GPU types
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
# (this can take dozens of minutes)
2022-12-02 11:42:09 +00:00
```
# How does it work
2022-11-21 23:20:07 +00:00
2022-12-02 11:42:09 +00:00
what makes a transition a good transition?
* absence of movement
* every frame looks like a credible photo
