---
title: "DeepSeek-V3.2"
metatags:
  description: "Deploy DeepSeek-V3.2 model with SGLang on Ascend NPUs, including multi-node PD disaggregation mode."
---

## Introduction

DeepSeek-V3.2 is a Mixture-of-Experts (MoE) large language model developed by DeepSeek, featuring 685B total parameters
with 37B active parameters. It employs Multi-head Latent Attention (MLA) and DeepSeekMoE architecture, with built-in
multi-token prediction (MTP) for speculative decoding. DeepSeek-V3.2 introduces the DeepSeek Sparse Attention (DSA)
mechanism, a fine-grained sparse attention mechanism powered by a lightning indexer, enabling significant efficiency
improvements in long-context scenarios.

This document demonstrates the deployment of DeepSeek-V3.2 on Ascend NPUs using SGLang, including multi-node PD
disaggregation mode, feature configuration, and performance optimization.

This document is validated and written based on **SGLang v0.5.16**. The current model (DeepSeek-V3.2) is fully supported in
this version. To use the latest features (e.g., PD disaggregation, speculative decoding, DSA context parallel), it is
recommended to use v0.5.16 or a later version.

## Supported features

| Feature                       | Example usage                                                                                 |
|-------------------------------|-----------------------------------------------------------------------------------------------|
| Tensor Parallelism            | `--tp-size 32`                                                                                |
| Data Parallelism              | `--dp-size 8`                                                                                 |
| Expert Parallelism            | `--ep-size 32 \`<br/>`--moe-a2a-backend deepep \`<br/>`--deepep-mode low_latency`                             |
| Context Parallelism           | `--enable-nsa-prefill-context-parallel \`<br/>`--nsa-prefill-cp-mode in-seq-split \`<br/>`--attn-cp-size 32`  |
| PD Disaggregation             | `--disaggregation-mode prefill \`<br/>`--disaggregation-transfer-backend ascend`                      |
| Quantization                  | `--quantization modelslim`                                                                    |
| Speculative Decoding          | `--speculative-algorithm NEXTN \`<br/>`--speculative-num-steps 3 \`<br/>`--speculative-eagle-topk 1 \`<br/>`--speculative-num-draft-tokens 4` |
| NPU Graph                     | enabled by default; disable with `--disable-cuda-graph`;<br/>control range via `--cuda-graph-bs` or `--cuda-graph-max-bs-decode`; e.g., `--cuda-graph-bs 1 2 3 4 5 6 7 8` |
| Overlap Schedule              | `export SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1`                                                  |
| DP LM Head                    | `--enable-dp-lm-head`                                                                         |

<Note>
The values in the **Example usage** column are for illustration only. Adjust them according to your hardware, deployment
mode, and workload. For parameter details, see
[Feature descriptions](/docs/hardware-platforms/ascend-npus/optimization/parameter_tuning#feature-descriptions); for
recommended configurations for each deployment scenario, see [Best practices](#best-practices).
</Note>

For feature compatibility and conflict information between features,
see [Feature Compatibility](/docs/hardware-platforms/ascend-npus/optimization/parameter_tuning#feature-compatibility).

## Prerequisites

### Environment

Before following this tutorial, complete the environment setup in the documents below:

- [Ascend NPU Quickstart](/docs/hardware-platforms/ascend-npus/getting-started/quick_start) — the fastest way to get started.
  It walks you through launching the official container image, starting the SGLang server, and sending a test request.
  Recommended if you are new to SGLang on Ascend.
- [SGLang Installation with NPU Support](/docs/hardware-platforms/ascend-npus/getting-started/installation) — the full installation guide.
  It covers the component version mapping (CANN, TorchNPU, Triton, kernels, etc.), building from source or from a
  Dockerfile, and recommended system settings (CPU power scheme, NUMA, swap). Use it when you need to install or customize
  the environment instead of using the official image.

### Model weights

Before downloading model weights, check the model size to reserve enough disk space.
For multi-node deployment, download the weights to a shared directory accessible to all nodes.

- [DeepSeek-V3.2-W8A8](https://www.modelscope.cn/models/sgl-npu/DeepSeek-V3.2-W8A8) (Quantized version, 694.47GB)

<Info>
The W8A8 variant (694.47GB) can be deployed on 16 × 64GB of device memory (`--tp-size 16`), which corresponds to one full
A3 node (8 cards, 16 dies) or two A2 nodes.
</Info>

This is the minimum recommended configuration. For optimized configurations,
see [Best practices](#best-practices), which may require additional cards or nodes.

For the hardware specifications (memory per die, dies per card, and the difference between A2 and A3),
see [Ascend NPU Reference — Hardware](/docs/hardware-platforms/ascend-npus/reference/glossary#hardware).

## Installation

<Warning>
Ensure sufficient disk space before pulling images. The Docker image requires at least **30GB** of free space.
</Warning>

The dependencies required for the NPU runtime environment have been integrated into a Docker image and uploaded to the
online platform. You can directly pull it.

Both **stable releases** and **daily builds** are available. The following command is based on the stable release tag.
For details, see [Docker image versions](/docs/hardware-platforms/ascend-npus/faq#8-docker-image-versions-stable-release-vs-daily-build).

<Tabs>
  <Tab title="Atlas 800I A3">

```bash Command
docker pull quay.io/ascend/sglang:cann9.0.0-a3-v0.5.16

docker run -itd --shm-size=16g --name ${NAME} \
--privileged=true --net=host \
-v /var/queue_schedule:/var/queue_schedule \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /usr/local/sbin:/usr/local/sbin \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware \
--device=/dev/davinci0:/dev/davinci0  \
--device=/dev/davinci1:/dev/davinci1  \
--device=/dev/davinci2:/dev/davinci2  \
--device=/dev/davinci3:/dev/davinci3  \
--device=/dev/davinci4:/dev/davinci4  \
--device=/dev/davinci5:/dev/davinci5  \
--device=/dev/davinci6:/dev/davinci6  \
--device=/dev/davinci7:/dev/davinci7  \
--device=/dev/davinci8:/dev/davinci8  \
--device=/dev/davinci9:/dev/davinci9  \
--device=/dev/davinci10:/dev/davinci10  \
--device=/dev/davinci11:/dev/davinci11  \
--device=/dev/davinci12:/dev/davinci12  \
--device=/dev/davinci13:/dev/davinci13  \
--device=/dev/davinci14:/dev/davinci14  \
--device=/dev/davinci15:/dev/davinci15  \
--device=/dev/davinci_manager:/dev/davinci_manager \
--device=/dev/hisi_hdc:/dev/hisi_hdc \
--entrypoint=bash \
quay.io/ascend/sglang:cann9.0.0-a3-v0.5.16
```

  </Tab>
  <Tab title="Atlas 800I A2">

```bash Command
docker pull quay.io/ascend/sglang:cann9.0.0-910b-v0.5.16

docker run -itd --shm-size=16g --name ${NAME} \
--privileged=true --net=host \
-v /var/queue_schedule:/var/queue_schedule \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /usr/local/sbin:/usr/local/sbin \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware \
--device=/dev/davinci0:/dev/davinci0  \
--device=/dev/davinci1:/dev/davinci1  \
--device=/dev/davinci2:/dev/davinci2  \
--device=/dev/davinci3:/dev/davinci3  \
--device=/dev/davinci4:/dev/davinci4  \
--device=/dev/davinci5:/dev/davinci5  \
--device=/dev/davinci6:/dev/davinci6  \
--device=/dev/davinci7:/dev/davinci7  \
--device=/dev/davinci_manager:/dev/davinci_manager \
--device=/dev/hisi_hdc:/dev/hisi_hdc \
--entrypoint=bash \
quay.io/ascend/sglang:cann9.0.0-910b-v0.5.16
```

  </Tab>
</Tabs>

<Tip>
- If the model weights have already been downloaded to a shared directory, use `-v` to mount the model path into the
  container, for example: `-v /path/to/models:/models`.
- Replace `${NAME}` with your own container name or remove `--name` to use default name.
</Tip>

## Online service deployment

### Multi-node PD disaggregation deployment

PD disaggregation splits the prefill and decode stages onto separate nodes, reducing interference and improving
throughput for high-concurrency scenarios. This scenario is already covered in the best practice. For the complete, optimized
deployment commands and benchmark data, see
[DeepSeek-V3.2 Best Practice — PD Disaggregation On A3](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/deepseek_v3_2#pd-disaggregation).

## Functional verification

After the service is started, you can invoke the model by sending a prompt:

```shell
# ============================================================
# Before running, update the following variables:
#   HOST: the server host address (e.g., localhost)
#   PORT: the server port number (e.g., 6689)
# ============================================================

curl http://${HOST}:${PORT}/generate \
    -H "Content-Type: application/json" \
    -d '{
        "text": "What is the capital of France?",
        "sampling_params": {
            "max_new_tokens": 64,
            "temperature": 0
        }
    }'
```

Expected result: an HTTP 200 response with the generated text containing "Paris".

Once the server prints `The server is fired up and ready to roll!` in the logs, it is ready to accept requests. For more
testing examples (Health Check, Generate, Chat Completions, and port usage guidance),
see [Testing the Service](/docs/hardware-platforms/ascend-npus/getting-started/installation#testing-the-service).

## Accuracy evaluation

For accuracy evaluation methods and datasets, see [Accuracy Evaluation on Ascend NPU](/docs/hardware-platforms/ascend-npus/evaluation/accuracy_evaluation).

## Performance

For performance data and benchmark commands, see [Performance Testing on Ascend NPU](/docs/hardware-platforms/ascend-npus/evaluation/performance_testing).

## Best practices

### Best practice configuration reference

For complete optimal configurations with deployment scripts and benchmark commands, see the
[DeepSeek-V3.2 Best Practice](/docs/hardware-platforms/ascend-npus/model-deployment/best-practices/deepseek_v3_2) page.

## Performance tuning

For the full list of supported features, see [Supported features](#supported-features). For detailed optimization
guidance, see [Optimization on Ascend NPU](/docs/hardware-platforms/ascend-npus/optimization/parameter_tuning).

## FAQ

For common environment, installation, and general parameter issues, please refer to the [Ascend NPU FAQ](/docs/hardware-platforms/ascend-npus/faq).
