latentblending/README.md

# What is latent blending?

Latent blending allows you to generate smooth video transitions between two prompts. It is based on [stable diffusion 2.0](https://stability.ai/blog/stable-diffusion-v2-release) and remixes the latent reprensetation using spherical linear interpolations. This results in imperceptible transitions, where one image slowly turns into another one. 

# Example 1: simple transition
(mp4), code

# Example 2: inpainting transition
(mp4), code

# Example 3: concatenated transition
(mp4), code

# Relevant parameters


# Installation
#### Packages
```commandline
pip install -r requirements.txt
```
#### Download Models from Huggingface
[Download the Stable Diffusion 2.0 Standard Model](https://huggingface.co/stabilityai/stable-diffusion-2)

[Download the Stable Diffusion 2.0 Inpainting Model (optional)](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)

#### Install [Xformers](https://github.com/facebookresearch/xformers)
With xformers, stable diffusion 2 will run a bit faster. The recommended way of installation is via the supplied binaries (Linux).

```commandline
conda install xformers -c xformers/label/dev
```

Alternatively, you can build it from source:
```commandline
# (Optional) Makes the build much faster
pip install ninja
# Set TORCH_CUDA_ARCH_LIST if running and building on different GPU types
pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers
# (this can take dozens of minutes)
```

# How does it work
![](animation.gif)

what makes a transition a good transition?
* absence of movement
* every frame looks like a credible photo
Update README.md 2022-12-02 11:42:09 +00:00			`# What is latent blending?`
Update README.md 2022-11-21 23:20:07 +00:00
Update README.md 2022-12-02 11:51:40 +00:00			`Latent blending allows you to generate smooth video transitions between two prompts. It is based on [stable diffusion 2.0](https://stability.ai/blog/stable-diffusion-v2-release) and remixes the latent reprensetation using spherical linear interpolations. This results in imperceptible transitions, where one image slowly turns into another one.`
Update README.md 2022-11-21 23:20:07 +00:00
Update README.md 2022-12-02 11:42:09 +00:00			`# Example 1: simple transition`
Update README.md 2022-11-21 23:20:07 +00:00			`(mp4), code`

Update README.md 2022-12-02 11:42:09 +00:00			`# Example 2: inpainting transition`
Update README.md 2022-11-21 23:20:07 +00:00			`(mp4), code`

Update README.md 2022-12-02 11:42:09 +00:00			`# Example 3: concatenated transition`
Update README.md 2022-11-21 23:20:07 +00:00			`(mp4), code`

Update README.md 2022-12-02 11:42:09 +00:00			`# Relevant parameters`

Update README.md 2022-11-21 23:20:07 +00:00
Update README.md 2022-12-02 11:42:09 +00:00			`# Installation`
			`#### Packages`
			```commandline
Update README.md 2022-12-02 12:08:17 +00:00			`pip install -r requirements.txt`
Update README.md 2022-12-02 11:42:09 +00:00			```
Update README.md 2022-12-02 12:08:17 +00:00			`#### Download Models from Huggingface`
Update README.md 2022-12-02 11:42:09 +00:00			`[Download the Stable Diffusion 2.0 Standard Model](https://huggingface.co/stabilityai/stable-diffusion-2)`

			`[Download the Stable Diffusion 2.0 Inpainting Model (optional)](https://huggingface.co/stabilityai/stable-diffusion-2-inpainting)`

Update README.md 2022-12-02 12:08:17 +00:00			`#### Install [Xformers](https://github.com/facebookresearch/xformers)`
Update README.md 2022-12-02 12:43:35 +00:00			`With xformers, stable diffusion 2 will run a bit faster. The recommended way of installation is via the supplied binaries (Linux).`
Update README.md 2022-12-02 11:42:09 +00:00
			```commandline
Update README.md 2022-12-02 12:08:17 +00:00			`conda install xformers -c xformers/label/dev`
Update README.md 2022-12-02 11:42:09 +00:00			```

Update README.md 2022-12-02 12:08:17 +00:00			`Alternatively, you can build it from source:`
Update README.md 2022-12-02 11:42:09 +00:00			```commandline
Update README.md 2022-12-02 12:08:17 +00:00			`# (Optional) Makes the build much faster`
			`pip install ninja`
			`# Set TORCH_CUDA_ARCH_LIST if running and building on different GPU types`
			`pip install -v -U git+https://github.com/facebookresearch/xformers.git@main#egg=xformers`
			`# (this can take dozens of minutes)`
Update README.md 2022-12-02 11:42:09 +00:00			```

			`# How does it work`
			`![](animation.gif)`
Update README.md 2022-11-21 23:20:07 +00:00
Update README.md 2022-12-02 11:42:09 +00:00			`what makes a transition a good transition?`
			`* absence of movement`
			`* every frame looks like a credible photo`