[kolors] Add PAG support
# What does this PR do?
Adds PAG support to Kolors. and some other fixes.
Fixes # (issue)
## Before submitting
- [X] Did you read the [contributor guideline](https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md)?
- [X] Did you read our [philosophy doc](https://github.com/huggingface/diffusers/blob/main/PHILOSOPHY.md) (important for complex PRs)?
- [X] Was this discussed/approved via a GitHub issue or the [forum](https://discuss.huggingface.co/c/discussion-related-to-httpsgithubcomhuggingfacediffusers/63)? Please add a link to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
[documentation guidelines](https://github.com/huggingface/diffusers/tree/main/docs), and
[here are tips on formatting docstrings](https://github.com/huggingface/diffusers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?
## Example
```sh
pip install git+https://github.com/modelslab/diffusers_plus_plus
```
```py
import torch
from diffusers import AutoPipelineForText2Image, DPMSolverMultistepScheduler
from transformers import CLIPVisionModelWithProjection
pipe = AutoPipelineForText2Image.from_pretrained(
"Kwai-Kolors/Kolors-diffusers",
torch_dtype=torch.float16,
variant="fp16",
enable_pag=True,
pag_applied_layers=["down.block_2.attentions_1", "up.block_0.attentions_1"],
).to("cuda")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)
image = pipe(
prompt="",
guidance_scale=1.5,
num_inference_steps=20,
pag_scale=.4,
height=1024,
width=768,
).images[0]
```
## Generations (comparisons across PAG scales)
| w/o PAG Kolors | with PAG (scale .2) | with PAG (scale .4) | with PAG (scale .8) |
| -------------------- | -------------------------- | --------------------------- | -------------------------- |
|  |  |  |  |
_credits for the original code @asomoza_
合并状态:已合并 合并于 2024-08-05 关闭于 2024-08-05 1 条评论