ITADN

[kolors] Add PAG support

#25Pull RequestAbhinavJangra29 创建于 2024-08-03已合并
A
AbhinavJangra29commented
# What does this PR do? Adds PAG support to Kolors. and some other fixes. Fixes # (issue) ## Before submitting - [X] Did you read the [contributor guideline](https://github.com/huggingface/diffusers/blob/main/CONTRIBUTING.md)? - [X] Did you read our [philosophy doc](https://github.com/huggingface/diffusers/blob/main/PHILOSOPHY.md) (important for complex PRs)? - [X] Was this discussed/approved via a GitHub issue or the [forum](https://discuss.huggingface.co/c/discussion-related-to-httpsgithubcomhuggingfacediffusers/63)? Please add a link to it if that's the case. - [X] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/diffusers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/diffusers/tree/main/docs#writing-source-documentation). - [X] Did you write any new necessary tests? ## Example ```sh pip install git+https://github.com/modelslab/diffusers_plus_plus ``` ```py import torch from diffusers import AutoPipelineForText2Image, DPMSolverMultistepScheduler from transformers import CLIPVisionModelWithProjection pipe = AutoPipelineForText2Image.from_pretrained( "Kwai-Kolors/Kolors-diffusers", torch_dtype=torch.float16, variant="fp16", enable_pag=True, pag_applied_layers=["down.block_2.attentions_1", "up.block_0.attentions_1"], ).to("cuda") pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True) image = pipe( prompt="", guidance_scale=1.5, num_inference_steps=20, pag_scale=.4, height=1024, width=768, ).images[0] ``` ## Generations (comparisons across PAG scales) | w/o PAG Kolors | with PAG (scale .2) | with PAG (scale .4) | with PAG (scale .8) | | -------------------- | -------------------------- | --------------------------- | -------------------------- | | ![no_pag](https://github.com/user-attachments/assets/588ead88-2d0a-4515-99ca-62364deca5f3) | ![pag](https://github.com/user-attachments/assets/25eeee70-ee80-42fa-a651-33a1293510a3) | ![pag_4](https://github.com/user-attachments/assets/b7096fb1-7a2d-4e75-9f16-cc61406e276f) | ![pag_8](https://github.com/user-attachments/assets/a5b7586d-e7e3-489c-bf8f-462c9ef18a72) | _credits for the original code @asomoza_
合并状态:已合并 合并于 2024-08-05 关闭于 2024-08-05 1 条评论