Installation (Ascend NPU)#
Note: Ascend NPU support is maintained on the
ascend-v1.0.5 branch
rather than main. All code, configurations, and examples referenced in this document
refer to that branch — make sure to check it out as described below.
Prerequisites#
Hardware Requirements#
The following hardware configuration has been extensively tested:
NPU: 16x NPU per node
CPU: 64 cores per node
Memory: 1TB per node
Network: RoCE 3.2 Tbps
Storage:
1TB local storage for single-node experiments
10TB shared storage (NAS) for distributed experiments
Software Requirements#
Component |
Version |
|---|---|
Operating System |
Ubuntu, EulerOS or any system meeting the requirements below |
Ascend HDK |
25.5.1 |
CANN |
9.0.1 |
Git LFS |
Required for downloading models, datasets, and AReaL code. See installation guide |
Docker |
27.2.0 |
AReaL Image (A2) |
|
AReaL Image (A3) |
|
Note: This tutorial does not cover the installation of CANN, or shared storage mounting, as these depend on your specific node configuration and system version. Please complete these installations independently. You can check out more details from the vLLM-Ascend community at this page.
Runtime Environment#
We recommend using Docker with our provided image for NPU. The A2 image targets Atlas A2
and the A3 image targets Atlas A3; both are built from the same recipe (see
Dockerfile.a2
/
Dockerfile.a3
on the ascend-v1.0.5 branch) and ship the following pre-built stack. The versions
below match the published v1.0.5 image:
Component |
Version |
|---|---|
Base image (A2) |
|
Base image (A3) |
|
Runtime OS |
Ubuntu 22.04.5 LTS |
Python |
3.11.15 |
CANN |
9.0.1 |
PyTorch |
2.10.0 |
torch_npu |
2.10.0.post2 |
vLLM |
0.23.0 (patched v0.23.0 source) |
vLLM-Ascend |
|
Transformers |
5.5.4 |
Megatron-Core |
core_v0.16.1 |
MindSpeed |
core_r0.16.0 at |
Megatron-Bridge |
|
All other AReaL Python dependencies are pre-installed from pyproject.npu.toml.
Megatron-LM, MindSpeed, and Megatron-Bridge sources live under /areal-workspace and
are made importable via PYTHONPATH.
Create Container#
WORK_DIR=<your_workspace>
CONTAINER_WORK_DIR=<your_container_workspace>
# Use A2/A3 image depending on your hardware type
# IMAGE=ghcr.io/hwvanici/areal_npu:v1.0.5-a2
IMAGE=ghcr.io/hwvanici/areal_npu:v1.0.5-a3
CONTAINER_NAME=areal_npu
cd ${WORK_DIR}
docker pull ${IMAGE}
docker run -itd --cap-add=SYS_PTRACE --net=host \
--device=/dev/davinci0 \
--device=/dev/davinci1 \
--device=/dev/davinci2 \
--device=/dev/davinci3 \
--device=/dev/davinci4 \
--device=/dev/davinci5 \
--device=/dev/davinci6 \
--device=/dev/davinci7 \
--device=/dev/davinci8 \
--device=/dev/davinci9 \
--device=/dev/davinci10 \
--device=/dev/davinci11 \
--device=/dev/davinci12 \
--device=/dev/davinci13 \
--device=/dev/davinci14 \
--device=/dev/davinci15 \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
--shm-size=1200g \
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /var/log/npu/:/usr/slog \
-v ${WORK_DIR}:${CONTAINER_WORK_DIR} \
--privileged=true \
--name ${CONTAINER_NAME} \
${IMAGE} \
/bin/bash
For multi-node training: Ensure a shared storage path is mounted on every node (and mounted to the container if you are using Docker). This path will be used to save checkpoints and logs.
Custom Environment Installation#
The image includes a built-in copy of the AReaL source code under /AReaL, but it may
be out of date. We recommend removing it and installing AReaL from the latest source.
This only replaces the AReaL package itself — all dependencies (including MindSpeed and
Megatron-Bridge under /areal-workspace) stay untouched.
rm -rf /AReaL
git clone https://github.com/areal-project/AReaL
cd AReaL
# Checkout to ascend branch
git checkout ascend-v1.0.5
# Install AReaL. All Python dependencies (from pyproject.npu.toml) are already
# installed in the image, so only the AReaL package itself needs to be installed.
uv pip install --no-deps -e . --system
(Optional) Launch Ray Cluster for Distributed Training#
On the first node, start the Ray Head:
ray start --head
On all other nodes, start the Ray Worker:
# Replace with the actual IP address of the first node
RAY_HEAD_IP=xxx.xxx.xxx.xxx
ray start --address $RAY_HEAD_IP
You should see the Ray resource status displayed when running ray status.
Properly set the n_nodes argument in AReaL’s training command, then AReaL’s training
script will automatically detect the resources and allocate workers to the cluster.
Next Steps#
Check the quickstart section to get familiar with launching AReaL jobs.
On NPU, we recommend starting from the vision-language model (VLM) GRPO examples in
examples/vlm_npu/
on the ascend-v1.0.5 branch, which train on the Geometry3K dataset. For example, to
train Qwen2.5-VL-3B on a single node (16 NPUs, vLLM rollout + Megatron training):
# Some models are optimized by vllm-ascend, but the optimized variants
# may be unsuitable for RLHF training. Disable them before launching.
export USE_OPTIMIZED_MODEL=0
python examples/vlm/geometry3k_grpo.py \
--config examples/vlm_npu/qwen2_5_vl_3b_geometry3k_grpo.yaml
See the
examples/vlm_npu/README.md
on the ascend-v1.0.5 branch for the full list of configurations (Qwen2.5-VL, Qwen3-VL,
Qwen3.5, Qwen3.6 dense and MoE), dataset preparation, and multi-node training with Ray.
If you want to run multi-node training, make sure your Ray cluster is started as
described above before launching the job.
Note: On Ascend NPU, rollout is supported through the vllm engine (via the
vLLM-Ascend plugin); SGLang is not available. Both the fsdp and megatron (through
MindSpeed) training engines are supported.