---
title: Qwen-Image
metatags:
    description: "Deploy Qwen-Image with SGLang - community contribution guide for Qwen's image generation model."
---

import { DiffusionModelTags } from '/src/snippets/diffusion/model-tags.jsx';
import { QwenImageDeployment } from '/src/snippets/diffusion/qwen-image-deployment.jsx';

<DiffusionModelTags tags={["image", "text-to-image", "text rendering", "NVFP4"]} />

## 1. Model Introduction

[Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) is a text-to-image diffusion model developed by the Qwen team.

For more details, please refer to the [official Qwen-Image HuggingFace page](https://huggingface.co/Qwen/Qwen-Image), the [Blog](https://qwenlm.github.io/blog/qwen-image/), and the [Tech Report](https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Image/Qwen_Image.pdf).

## 2. SGLang-diffusion Installation

SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements.

Please refer to the [official SGLang-diffusion installation guide](../../../docs/sglang-diffusion/installation) for installation instructions.

## 3. Model Deployment

This section provides deployment configurations optimized for different hardware platforms and use cases.

### 3.1 Basic Configuration

Qwen-Image is a text-to-image model. The recommended launch configurations vary by hardware. SGLang supports serving Qwen-Image on NVIDIA B200, B300, H200, H100, AMD MI300X, MI325X, MI355X GPUs and Ascend A2, A3 NPUs.

**Interactive Command Generator**: Use the configuration selector below to automatically generate the appropriate deployment command for your hardware platform.

<QwenImageDeployment />

For the validated ModelOpt NVFP4 checkpoint on Blackwell, load the published
Qwen-Image-2512 NVFP4 repo directly:

```bash Command
sglang serve \
  --model-path lmsys/qwen-image-2512-modelopt-nvfp4-sglang \
  --ulysses-degree=1 \
  --ring-degree=1
```

For high-resolution B200 generations, the FlashInfer CUTLASS FP4 GEMM backend
can be faster than the default TensorRT-LLM FP4 GEMM backend:

```bash Command
SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cutlass \
sglang generate \
  --model-path lmsys/qwen-image-2512-modelopt-nvfp4-sglang \
  --width 2048 --height 2048 \
  --prompt "A tiny astronaut reading a book under a glass greenhouse" \
  --save-output
```

### 3.2 Configuration Tips

Currently supported optimizations are listed [here](/docs/sglang-diffusion/compatibility_matrix).

- `--vae-path`: Path to a custom VAE model or HuggingFace model ID (e.g., fal/FLUX.2-Tiny-AutoEncoder). If not specified, the VAE will be loaded from the main model path.
- `--num-gpus`: Number of GPUs to use
- `--tp-size`: Tensor parallelism size (only for the encoder; should not be larger than 1 if text encoder offload is enabled, as layer-wise offload plus prefetch is faster)
- `--sp-degree`: Sequence parallelism size (typically should match the number of GPUs)
- `--ulysses-degree`: The degree of DeepSpeed-Ulysses-style SP in USP
- `--ring-degree`: The degree of ring attention-style SP in USP

**AMD ROCm Notes**: Requires SGLang >= v0.5.8.

## 4. API Usage

For complete API documentation, please refer to the [official API usage guide](../../../docs/sglang-diffusion/api/openai_api).

### 4.1 Generate an Image

```python Example
import base64
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")

response = client.images.generate(
    model="Qwen/Qwen-Image",
    prompt="A logo With Bold Large text: SGL Diffusion",
    n=1,
    response_format="b64_json",
)

# Save the generated image
image_bytes = base64.b64decode(response.data[0].b64_json)
with open("output.png", "wb") as f:
    f.write(image_bytes)
```

### 4.2 Advanced Usage

#### 4.2.1 Cache-DiT Acceleration

SGLang integrates [Cache-DiT](https://github.com/vipshop/cache-dit), a caching acceleration engine for Diffusion Transformers (DiT), to achieve up to 7.4x inference speedup with minimal quality loss. You can set `SGLANG_CACHE_DIT_ENABLED=True` to enable it. For more details, please refer to the SGLang Cache-DiT [documentation](../../../docs/sglang-diffusion/cache_dit).

**Basic Usage**

```bash Command
SGLANG_CACHE_DIT_ENABLED=true sglang serve --model-path Qwen/Qwen-Image
```

**Advanced Usage**

- DBCache Parameters: DBCache controls block-level caching behavior:

<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
  <colgroup>
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
  </colgroup>
  <thead>
    <tr style={{borderBottom: "2px solid #d55816"}}>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Fn</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_FN`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of first blocks to always compute</td>
    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Bn</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_BN`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Number of last blocks to always compute</td>
    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>W</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_WARMUP`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>4</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Warmup steps before caching starts</td>
    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>R</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_RDT`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>0.24</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Residual difference threshold</td>
    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>MC</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_MC`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>3</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Maximum continuous cached steps</td>
    </tr>
  </tbody>
</table>
- TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion:

<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}>
  <colgroup>
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
    <col style={{width: "25.0%"}} />
  </colgroup>
  <thead>
    <tr style={{borderBottom: "2px solid #d55816"}}>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Parameter</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Env Variable</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.02)"}}>Default</th>
      <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700, whiteSpace: "nowrap", backgroundColor: "rgba(255,255,255,0.05)"}}>Description</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Enable</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TAYLORSEER`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>false</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Enable TaylorSeer calibrator</td>
    </tr>
    <tr>
      <td style={{padding: "9px 12px", fontWeight: 500, backgroundColor: "rgba(255,255,255,0.02)"}}>Order</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>`SGLANG_CACHE_DIT_TS_ORDER`</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.02)"}}>1</td>
      <td style={{padding: "9px 12px", backgroundColor: "rgba(255,255,255,0.05)"}}>Taylor expansion order (1 or 2)</td>
    </tr>
  </tbody>
</table>

  Combined Configuration Example:

```bash Command
SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
sglang serve --model-path Qwen/Qwen-Image
```

#### 4.2.2 CPU Offload

- `--dit-cpu-offload`: Use CPU offload for DiT inference. Enable if run out of memory.
- `--text-encoder-cpu-offload`: Use CPU offload for text encoder inference.
- `--vae-cpu-offload`: Use CPU offload for VAE.
- `--pin-cpu-memory`: Pin memory for CPU offload. Only added as a temp workaround if it throws "CUDA error: invalid argument".

## 5. Benchmark

Test Environment:

- Hardware: AMD Instinct MI300X GPU (1x)
- Model: Qwen/Qwen-Image
- Docker Image: lmsysorg/sglang:v0.5.8-rocm700-mi30x
- sglang diffusion version: 0.5.8

### 5.1 Speedup Benchmark

#### 5.1.1 Generate an image

<Tabs>
  <Tab title="AMD MI300X">
    **Server Command**:

    ```shell Command
    sglang serve --model-path Qwen/Qwen-Image \
        --ulysses-degree=1 --ring-degree=1 --port 30000
    ```

    **Benchmark Command**:

    ```shell Command
    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
    ```

    **Result**:

    ```text Output
    ================= Serving Benchmark Result =================
    Task:                                    text-to-image
    Model:                                   Qwen/Qwen-Image
    Dataset:                                 vbench
    --------------------------------------------------
    Benchmark duration (s):                  29.04
    Request rate:                            inf
    Max request concurrency:                 1
    Successful requests:                     1/1
    --------------------------------------------------
    Request throughput (req/s):              0.03
    Latency Mean (s):                        29.0378
    Latency Median (s):                      29.0378
    Latency P99 (s):                         29.0378
    --------------------------------------------------
    Peak Memory Max (MB):                    48018.83
    Peak Memory Mean (MB):                   48018.83
    Peak Memory Median (MB):                 48018.83
    ============================================================
    ```
  </Tab>
  <Tab title="Ascend A3">
    **Server Command**:

    ```shell Command
    #One A3 card has 2 npu chips
    sglang serve --tp-size 2 --sp-degree 1 --model-path Qwen/Qwen-Image --num-gpus 2
    ```

    **Benchmark Command**:

    ```shell Command
    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 1 --max-concurrency 1
    ```

    **Result**:

    ```text Output
    ================= Serving Benchmark Result =================
    Task:                                         text-to-image
    Model:                                        Qwen/Qwen-Image
    Dataset:                                      vbench
    --------------------------------------------------
    Benchmark duration (s):                       36.26
    Request rate:                                 inf
    Max request concurrency:                      1
    Successful requests:                          1/1
    Completed outputs:                            1
    Outputs per prompt:                           1
    --------------------------------------------------
    Request throughput (req/s):                   0.03
    Output throughput (outputs/s):                0.03
    Latency Mean (s):                             36.26
    Latency Median (s):                           36.26
    Latency P90 (s):                              36.26
    Latency P95 (s):                              36.26
    Latency P99 (s):                              36.26
    --------------------------------------------------
    Peak Memory Max (MB):                         36984.00
    Peak Memory Mean (MB):                        36984.00
    Peak Memory Median (MB):                      36984.00
    ------------------------------------------------------------
    ```
  </Tab>
</Tabs>

#### 5.1.2 Generate images with high concurrency

<Tabs>
  <Tab title="AMD MI300X">
    **Benchmark Command**:

    ```shell Command
    python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
        --backend sglang-image --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20 --port 30000
    ```

    **Result**:

    ```text Output
    ================= Serving Benchmark Result =================
    Task:                                    text-to-image
    Model:                                   Qwen/Qwen-Image
    Dataset:                                 vbench
    --------------------------------------------------
    Benchmark duration (s):                  300.79
    Request rate:                            inf
    Max request concurrency:                 20
    Successful requests:                     14/20
    --------------------------------------------------
    Request throughput (req/s):              0.05
    Latency Mean (s):                        154.5368
    Latency Median (s):                      154.8363
    Latency P99 (s):                         285.4603
    --------------------------------------------------
    Peak Memory Max (MB):                    48030.31
    Peak Memory Mean (MB):                   48030.30
    Peak Memory Median (MB):                 48030.29
    ============================================================
    ```
  </Tab>
  <Tab title="Ascend A3">
    **Benchmark Command**:

    ```shell Command
    python -m sglang.multimodal_gen.benchmarks.bench_serving --dataset vbench --task text-to-image --num-prompts 20 --max-concurrency 20
    ```

    **Result**:

    ```text Output
    ================= Serving Benchmark Result =================
    Task:                                         text-to-image
    Model:                                        Qwen/Qwen-Image
    Dataset:                                      vbench
    --------------------------------------------------
    Benchmark duration (s):                       300.81
    Request rate:                                 inf
    Max request concurrency:                      20
    Successful requests:                          8/20
    Completed outputs:                            8
    Outputs per prompt:                           1
    --------------------------------------------------
    Request throughput (req/s):                   0.03
    Output throughput (outputs/s):                0.03
    Latency Mean (s):                             166.61
    Latency Median (s):                           167.02
    Latency P90 (s):                              270.80
    Latency P95 (s):                              283.48
    Latency P99 (s):                              293.64
    --------------------------------------------------
    Peak Memory Max (MB):                         36984.00
    Peak Memory Mean (MB):                        36984.00
    Peak Memory Median (MB):                      36984.00
    ------------------------------------------------------------
    ```
  </Tab>
</Tabs>
