---
title: "Environment Variables"
metatags:
    description: "Reference commonly used Ascend NPU environment variables for configuring SGLang runtime behavior."
---
SGLang supports various environment variables related to Ascend NPU that can be used to configure its runtime behavior.
This document provides a list of commonly used environment variables and aims to stay updated over time.

## Directly Used in SGLang

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>SGLANG_NPU_USE_MLAPO</code></td>
      <td>Adopts the <code>MLAPO</code> fusion operator in attention <br/> preprocessing stage of the MLA model.</td>
      <td><code>false</code></td>
    </tr>
    <tr>
      <td><code>SGLANG_USE_FIA_NZ</code></td>
      <td>Reshapes KV Cache for FIA NZ format.<br/> <code>SGLANG_USE_FIA_NZ</code> must be enabled with <code>SGLANG_NPU_USE_MLAPO</code></td>
      <td><code>false</code></td>
    </tr>
    <tr>
      <td><code>SGLANG_NPU_USE_MULTI_STREAM</code></td>
      <td>Enable dual-stream computation of shared experts <br/> and routing experts in DeepSeek models.<br/> Enable dual-stream computation in DeepSeek DSA Indexer.</td>
      <td><code>false</code></td>
    </tr>
    <tr>
      <td><code>SGLANG_NPU_DISABLE_ACL_FORMAT_WEIGHT</code></td>
      <td>Disable cast model weight tensor to a specific NPU <br/> ACL format.</td>
      <td><code>false</code></td>
    </tr>
    <tr>
      <td><code>SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK</code></td>
      <td>The maximum number of dispatched tokens on each rank.</td>
      <td><code>128</code></td>
    </tr>
  </tbody>
</table>

## Used in DeepEP Ascend

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>DEEPEP_NORMAL_LONG_SEQ_PER_ROUND_TOKENS</code></td>
      <td>Enable long-sequence token pipelining in dispatch stage. Indicates <br/> the number of tokens transmitted per round on each rank.</td>
      <td><code>8192</code></td>
    </tr>
    <tr>
      <td><code>DEEPEP_NORMAL_LONG_SEQ_ROUND</code></td>
      <td>Enable long-sequence token pipelining in dispatch stage. Indicates <br/> the number of rounds transmitted on each rank.</td>
      <td><code>1</code></td>
    </tr>
    <tr>
      <td><code>DEEPEP_NORMAL_COMBINE_ENABLE_LONG_SEQ</code></td>
      <td>Enable long-sequence token pipelining in combine stage. <br/> The value <code>0</code> means disabled.</td>
      <td><code>0</code></td>
    </tr>
    <tr>
      <td><code>MOE_ENABLE_TOPK_NEG_ONE</code></td>
      <td>Needs to be enabled when the expert ID to be processed by <br/> DEEPEP contains -1.</td>
      <td><code>0</code></td>
    </tr>
    <tr>
      <td><code>DEEP_NORMAL_MODE_USE_INT8_QUANT</code></td>
      <td>
        <strong>Deprecated — will be removed in a future release.</strong>
        <br/>
        When set to <code>1</code>, quantizes intermediate activations to INT8 in
        <br/>the DeepEP dispatch operator during normal mode, reducing communication
        <br/>volume for W8A8-quantized MoE models. This variable will become a no-op;
        <br/>the quantization behavior will be inferred automatically.
      </td>
      <td><code>0</code></td>
    </tr>
    <tr>
      <td><code>DEEPEP_HCCL_BUFFSIZE</code></td>
      <td>Configures the HCCL buffer size (in MB) for the process groups used by <br/> DeepEP MoE All-to-All communication (the default group and <code>moe</code>-named <br/> groups), allowing the MoE communication buffer to be tuned independently from <br/> non-MoE groups. If unset, falls back to <code>HCCL_BUFFSIZE</code>, then to <code>200</code>. <br/> A larger buffer reduces All-to-All latency at the cost of more HBM usage.</td>
      <td><code>200</code></td>
    </tr>
  </tbody>
</table>

## Others

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td><code>TASK_QUEUE_ENABLE</code></td>
      <td>Used to control the optimization level of the dispatch queue<br/> about the task_queue operator. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/730/comref/Envvariables/docs/zh/environment_variable_reference/TASK_QUEUE_ENABLE.md">Detail</a></td>
      <td><code>1</code></td>
    </tr>
    <tr>
      <td><code>INF_NAN_MODE_ENABLE</code></td>
      <td>Controls whether the chip uses saturation mode or INF_NAN mode. <a href="https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/envref/envref_07_0056.html">Detail</a></td>
      <td><code>1</code></td>
    </tr>
    <tr>
      <td><code>STREAMS_PER_DEVICE</code></td>
      <td>Configures the maximum number of streams for the stream pool. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/720/comref/Envvariables/Envir_041.html">Detail</a></td>
      <td><code>32</code></td>
    </tr>
    <tr>
      <td><code>PYTORCH_NPU_ALLOC_CONF</code></td>
      <td>Controls the behavior of the cache allocator. <br/>This variable changes memory usage and may cause performance fluctuations. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/700/comref/Envvariables/Envir_012.html">Detail</a></td>
      <td></td>
    </tr>
    <tr>
      <td><code>ASCEND_MF_STORE_URL</code></td>
      <td>The address of config store in MemFabric during PD separation, <br/>which is generally set to the IP address of the P primary node<br/> with an arbitrary port number.</td>
      <td></td>
    </tr>
    <tr>
      <td><code>ASCEND_LAUNCH_BLOCKING</code></td>
      <td>Controls whether synchronous mode is enabled during operator execution. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/710/comref/Envvariables/Envir_006.html">Detail</a></td>
      <td><code>0</code></td>
    </tr>
    <tr>
      <td><code>HCCL_OP_EXPANSION_MODE</code></td>
      <td>Configures the expansion position for communication algorithm scheduling. <a href="https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/envref/envref_07_0094.html">Detail</a></td>
      <td></td>
    </tr>
    <tr>
      <td><code>HCCL_BUFFSIZE</code></td>
      <td>Controls the size of the buffer area for shared data between two NPUs. <br/>The unit is MB, and the value must be greater than or equal to 1. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/60RC3/ptmoddevg/trainingmigrguide/performance_tuning_0047.html">Detail</a></td>
      <td><code>200</code></td>
    </tr>
    <tr>
      <td><code>HCCL_SOCKET_IFNAME</code></td>
      <td>Configures the name of the network card used by the Host <br/>during HCCL initialization. <a href="https://www.hiascend.com/document/detail/zh/canncommercial/81RC1/apiref/envvar/envref_07_0075.html">Detail</a></td>
      <td></td>
    </tr>
    <tr>
      <td><code>GLOO_SOCKET_IFNAME</code></td>
      <td>Configures the network interface name for GLOO communication.</td>
      <td></td>
    </tr>
  </tbody>
</table>
