Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
1858821
Add Action
tianshuo78520a Oct 13, 2025
5357263
update
tianshuo78520a Oct 14, 2025
062331b
update
tianshuo78520a Oct 14, 2025
ba83b92
update
tianshuo78520a Oct 14, 2025
7a42c65
update
tianshuo78520a Oct 14, 2025
8b9ef34
update
tianshuo78520a Oct 14, 2025
9d152dc
update
tianshuo78520a Oct 14, 2025
f73acc6
update
tianshuo78520a Oct 14, 2025
49dffae
update
tianshuo78520a Oct 14, 2025
5a54ef5
update
tianshuo78520a Oct 14, 2025
4104415
update
tianshuo78520a Oct 14, 2025
b0dbf75
update
tianshuo78520a Oct 14, 2025
866600a
update
tianshuo78520a Oct 14, 2025
653cf91
update
tianshuo78520a Oct 14, 2025
ec77703
update
tianshuo78520a Oct 14, 2025
c5d24b4
update
tianshuo78520a Oct 14, 2025
e55234a
add gcu
tianshuo78520a Oct 14, 2025
8183828
update gcu
tianshuo78520a Oct 14, 2025
ad6af77
update gcu
tianshuo78520a Oct 14, 2025
4a934bf
update gcu
tianshuo78520a Oct 14, 2025
cd65823
update gcu
tianshuo78520a Oct 14, 2025
ab54fa1
test
tianshuo78520a Oct 15, 2025
ee13d88
Update _Codestyle-Check.yml
tianshuo78520a Oct 15, 2025
817a744
update
tianshuo78520a Oct 15, 2025
7370475
Merge branch 'tianshuo-test1' of https://github.com/PaddlePaddle/Padd…
tianshuo78520a Oct 15, 2025
0aa2bc9
Update
tianshuo78520a Oct 16, 2025
86eee35
Update
tianshuo78520a Oct 16, 2025
95927e9
Update
tianshuo78520a Oct 16, 2025
b903099
Update
tianshuo78520a Oct 16, 2025
46f8f57
Update
tianshuo78520a Oct 16, 2025
382a2a2
Update
tianshuo78520a Oct 16, 2025
19a3034
Update
tianshuo78520a Oct 16, 2025
d39db81
Update
tianshuo78520a Oct 16, 2025
23c3e16
Update
tianshuo78520a Oct 16, 2025
7b72e09
Update
tianshuo78520a Oct 16, 2025
2e1450e
Update
tianshuo78520a Oct 16, 2025
6dedb00
Update
tianshuo78520a Oct 16, 2025
3c6e018
install zstd
tianshuo78520a Oct 16, 2025
dc54bc6
update
tianshuo78520a Oct 16, 2025
b894d32
update
tianshuo78520a Oct 16, 2025
3d9b31f
update
tianshuo78520a Oct 16, 2025
1d585b5
test
tianshuo78520a Oct 17, 2025
645a81e
test
tianshuo78520a Oct 17, 2025
247c920
test
tianshuo78520a Oct 17, 2025
d1f3fb7
test
tianshuo78520a Oct 17, 2025
5f000f1
update
tianshuo78520a Oct 17, 2025
1d899c2
update
tianshuo78520a Oct 17, 2025
3c88941
update
tianshuo78520a Oct 17, 2025
41f351d
test
tianshuo78520a Oct 17, 2025
6273458
test
tianshuo78520a Oct 17, 2025
b17b68e
test
tianshuo78520a Oct 17, 2025
0bf31b3
test
tianshuo78520a Oct 17, 2025
a743d57
test
tianshuo78520a Oct 17, 2025
49400e6
Update
tianshuo78520a Oct 17, 2025
327b906
Update
tianshuo78520a Oct 20, 2025
90306ac
Update
tianshuo78520a Oct 20, 2025
6606296
Update
tianshuo78520a Oct 20, 2025
abf833d
Update
tianshuo78520a Oct 20, 2025
ce6bc9b
update
tianshuo78520a Oct 20, 2025
bc0c9f0
update
tianshuo78520a Oct 20, 2025
eee363e
Update
tianshuo78520a Oct 20, 2025
bcf8a46
Test
tianshuo78520a Oct 21, 2025
3ecfb6a
Add Check bypass
tianshuo78520a Oct 21, 2025
1492043
Add Check bypass
tianshuo78520a Oct 21, 2025
d065a87
Add Check bypass
tianshuo78520a Oct 21, 2025
a88c2a8
update check-bypass
tianshuo78520a Oct 21, 2025
93baa97
update check-bypass
tianshuo78520a Oct 21, 2025
6246b61
update check-bypass
tianshuo78520a Oct 21, 2025
d408741
update check-bypass
tianshuo78520a Oct 21, 2025
840deba
update check-bypass
tianshuo78520a Oct 21, 2025
0bae39c
update check-bypass
tianshuo78520a Oct 21, 2025
7150b26
update check-bypass
tianshuo78520a Oct 21, 2025
4977aa0
update
tianshuo78520a Oct 21, 2025
49e9249
update
tianshuo78520a Oct 21, 2025
49aa398
Update
tianshuo78520a Oct 22, 2025
7c9d42d
Update
tianshuo78520a Oct 22, 2025
4655ee5
Update
tianshuo78520a Oct 22, 2025
ac62e4a
Update
tianshuo78520a Oct 22, 2025
b48e57b
Update
tianshuo78520a Oct 22, 2025
5507309
Update
tianshuo78520a Oct 22, 2025
97f67e6
Update
tianshuo78520a Oct 22, 2025
a160b05
Update
tianshuo78520a Oct 22, 2025
c32c882
Update
tianshuo78520a Oct 22, 2025
fd7ef6f
Update
tianshuo78520a Oct 22, 2025
50d53b7
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddlecusto…
tianshuo78520a Oct 23, 2025
3dcc6f4
update
tianshuo78520a Oct 23, 2025
f39dd1c
Update
tianshuo78520a Oct 23, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/actions/check-bypass/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: "Check bypass"
description: "A custom action to encapsulate PFCCLab/ci-bypass"
inputs:
github-token:
description: "GitHub token"
required: true
workflow-name:
description: "Workflow name"
required: true
outputs:
can-skip:
description: "Whether the workflow can be skipped."
value: ${{ steps.check-bypass.outputs.can-skip }}

runs:
using: "composite"
steps:
- id: check-bypass
name: Check Bypass
env:
CI_TEAM_MEMBERS: '["tianshuo78520a", "yongqiangma", "Yuqiang Ge"]'
uses: PFCCLab/ci-bypass@v2
with:
github-token: ${{ inputs.github-token }}
non-pull-request-event-strategy: 'never-skipped'
type: 'composite'
composite-rule: |
{
"any": [
{
"type": "labeled",
"label": ["skip-ci: ${{ inputs.workflow-name }}", "skip-ci: all"],
"username": ${{ env.CI_TEAM_MEMBERS }}
},
{
"type": "commented",
"comment-pattern": [".*/skip-ci ${{ inputs.workflow-name }}.*", ".*/skip-ci all.*"],
"username": ${{ env.CI_TEAM_MEMBERS }}
}
]
}
63 changes: 63 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: CI

on:
pull_request:
types: [opened, synchronize]
branches: [develop, release/**, fleety_*]

permissions: read-all

concurrency:
group: ${{ github.event.pull_request.number }}-${{ github.workflow }}
cancel-in-progress: true

env:
PR_ID: ${{ github.event.pull_request.number }}
COMMIT_ID: ${{ github.event.pull_request.head.sha }}

jobs:
clone:
name: Clone-linux
uses: ./.github/workflows/_Clone-linux.yml
with:
workflow-name: 'CI'

Codestyle-Check:
name: Codestyle-Check
uses: ./.github/workflows/_Codestyle-Check.yml
needs: [clone]

gcu:
name: gcu
uses: ./.github/workflows/_GCU.yml
needs: [Codestyle-Check]

hpu:
name: hpu
uses: ./.github/workflows/_HPU.yml
needs: [Codestyle-Check]

mlu:
name: mlu
uses: ./.github/workflows/_MLU.yml
needs: [Codestyle-Check]

ixuca:
name: ixuca
uses: ./.github/workflows/_IXUCA.yml
needs: [Codestyle-Check]

sdaa:
name: sdaa
uses: ./.github/workflows/_SDAA.yml
needs: [Codestyle-Check]

npu-910b-arm:
name: npu-910B-ARM
uses: ./.github/workflows/_910B-ARM.yml
needs: [Codestyle-Check]

npu-910b-x86:
name: npu-910B-X86
uses: ./.github/workflows/_910B-X86.yml
needs: [Codestyle-Check]
145 changes: 145 additions & 0 deletions .github/workflows/_910B-ARM.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
name: PR-CI-910B-ARM

on:
workflow_call:
inputs:
workflow-name:
type: string
required: false
clone_dir:
type: string
required: false
default: 'PaddlecustomDevice'
is_pr:
type: string
required: false
default: 'true'

jobs:
check-bypass:
name: Check bypass
if: ${{ inputs.can-skip != 'true' }}
uses: ./.github/workflows/check-bypass.yml
with:
workflow-name: '910b-arm'
secrets:
github-token: ${{ secrets.GITHUB_TOKEN }}

PR-CI-910B-ARM:
name: PR-CI-910B-ARM
needs: check-bypass
if: ${{ needs.check-bypass.outputs.can-skip != 'true' }}
runs-on:
group: 910B-ARM
env:
PR_ID: ${{ github.event.pull_request.number }}
COMMIT_ID: ${{ github.event.pull_request.head.sha }}
BRANCH: develop
TASK: paddle-CI-${{ github.event.pull_request.number }}-npu
no_proxy: "bcebos.com,apiin.im.baidu.com,gitee.com,aliyun.com,.baidu.com,.tuna.tsinghua.edu.cn"

steps:
- name: Cleanup
run: |
rm -rf * .[^.]*

- name: Check docker image and run container
env:
GIT_PR_ID: ${{ github.event.pull_request.number || '0' }}
GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RUN_ID: ${{ github.run_id }}
cache_dir: /root/.cache
ccache_dir: /root/.ccache
docker_image: registry.baidubce.com/device/paddle-npu:cann80RC1-ubuntu20-aarch64-gcc84-py39
run: |
if [[ "${{ runner.name }}" == "IXUCA-2" ]];then
echo "IXUCA-2"
export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7
elif [[ "${{ runner.name }}" == "IXUCA-3" ]];then
echo "IXUCA-3"
export ASCEND_RT_VISIBLE_DEVICES=8,9,10,11
elif [[ "${{ runner.name }}" == "IXUCA-4" ]];then
echo "IXUCA-4"
export ASCEND_RT_VISIBLE_DEVICES=12,13,14,15
else
echo echo "IXUCA-1"
export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3
fi
echo ${ASCEND_RT_VISIBLE_DEVICES}

container_name=${TASK}-$(date +%Y%m%d-%H%M%S)
echo "container_name=${container_name}" >> ${{ github.env }}
docker run --shm-size=128G \
-d -t --name ${container_name} \
--privileged --network=host \
-e PR_ID \
-e COMMIT_ID \
-e BRANCH \
-e no_proxy \
-e USE_910B=1 \
-e FLAGS_npu_storage_format=0 \
-e ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES} \
-v ${cache_dir}:/root/.cache \
-v ${ccache_dir}:/root/.ccache \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/dcmi:/usr/local/dcmi \
-v ${{ github.workspace }}/../../..:${{ github.workspace }}/../../.. \
-v ${{ github.workspace }}:/PaddleCustomDevice -w /PaddleCustomDevice \
${docker_image} /bin/bash

- name: Download Code
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
source ${{ github.workspace }}/../../../proxy
set -x
wget -q --tries=5 --no-proxy https://paddle-github-action.bj.bcebos.com/PaddleCustomDevice/PR/${PR_ID}/${COMMIT_ID}/PaddleCustomDevice.tar.gz --no-check-certificate
echo "Extracting PaddleCustomDevice.tar.gz"
tar -xf PaddleCustomDevice.tar.gz --strip-components=1
git config --global --add safe.directory "*"
git remote add upstream https://github.com/PaddlePaddle/PaddleCustomDevice.git
git merge ${BRANCH} --no-edit
git --no-pager log --pretty=oneline -5'

- name: RUN 910B-ARM
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
# !!!!! SKIP IF NO NPU CHANGE !!!!
echo "=========== Checking PR Changes If NPU FULL CI Needed ==========="
change_numbers=$(git diff --name-only remotes/origin/${BRANCH} | wc -l)
change_backend=$(git diff --name-only remotes/origin/${BRANCH} | grep "backends/"| wc -l)
change_npu_only=$(git diff --name-only remotes/origin/${BRANCH} | grep "backends/npu"| wc -l)
git diff --name-only remotes/origin/${BRANCH}

if [ $change_numbers -ne $change_backend ]; then
echo "Common file changed, continue to run NPU FULL CI test ..."
elif [ $change_npu_only -eq 0 ] ; then
echo "NO NPU backend changes found, skip NPU FULL CI ...."
exit 0
fi

echo "============ CANN Version ============="
source ~/.bashrc
echo "============ Install PaddlePaddle CPU ============="
PATH=/usr/local/bin:${PATH}
ln -sf $(which python3.9) /usr/local/bin/python
ln -sf $(which pip3.9) /usr/local/bin/pip

pip install -U numpy==1.26.4 safetensors
python -m pip install --pre paddlepaddle -i https://www.paddlepaddle.org.cn/packages/nightly/cpu/
python -c "import paddle; print(paddle.__version__)"
python -c "import paddle; print(paddle.version.commit)"
# Install python packages required for incremental coverage
python -m pip install PyGithub
export LD_PRELOAD=/lib/aarch64-linux-gnu/libgomp.so.1:$LD_PRELOAD

bash backends/npu/tools/pr_ci_npu.sh'

- name: Terminate and delete the container
if: always()
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
rm -rf * .[^.]*
'
docker stop ${container_name}
docker rm ${container_name}
143 changes: 143 additions & 0 deletions .github/workflows/_910B-X86.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
name: PR-CI-910B-X86

on:
workflow_call:
inputs:
workflow-name:
type: string
required: false
clone_dir:
type: string
required: false
default: 'PaddlecustomDevice'
is_pr:
type: string
required: false
default: 'true'

jobs:
check-bypass:
name: Check bypass
uses: ./.github/workflows/check-bypass.yml
with:
workflow-name: '910b-x86'
secrets:
github-token: ${{ secrets.GITHUB_TOKEN }}

PR-CI-910B-X86:
name: PR-CI-910B-X86
needs: check-bypass
if: ${{ needs.check-bypass.outputs.can-skip != 'true' }}
runs-on:
group: NPU
env:
PR_ID: ${{ github.event.pull_request.number }}
COMMIT_ID: ${{ github.event.pull_request.head.sha }}
BRANCH: develop
TASK: paddle-CI-${{ github.event.pull_request.number }}-npu
no_proxy: "bcebos.com,apiin.im.baidu.com,gitee.com,aliyun.com,.baidu.com,.tuna.tsinghua.edu.cn"

steps:
- name: Cleanup
run: |
rm -rf * .[^.]*

- name: Check docker image and run container
env:
GIT_PR_ID: ${{ github.event.pull_request.number || '0' }}
GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RUN_ID: ${{ github.run_id }}
cache_dir: /root/.cache
ccache_dir: /root/.ccache
docker_image: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann80RC2-ubuntu20-npu-base-x86_64-gcc84
run: |
if [[ "${{ runner.name }}" == "npu-2" ]];then
echo "IXUCA-2"
export ASCEND_RT_VISIBLE_DEVICES=4,5,6,7
elif [[ "${{ runner.name }}" == "npu-3" ]];then
echo "IXUCA-3"
export ASCEND_RT_VISIBLE_DEVICES=8,9,10,11
elif [[ "${{ runner.name }}" == "npu-4" ]];then
echo "IXUCA-4"
export ASCEND_RT_VISIBLE_DEVICES=12,13,14,15
else
echo echo "npu-1"
export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3
fi
echo ${ASCEND_RT_VISIBLE_DEVICES}

container_name=${TASK}-$(date +%Y%m%d-%H%M%S)
echo "container_name=${container_name}" >> ${{ github.env }}
docker run --shm-size=128G \
-d -t --name ${container_name} \
--privileged --network=host \
-e PR_ID \
-e COMMIT_ID \
-e BRANCH \
-e no_proxy \
-e USE_910B=1 \
-e FLAGS_npu_storage_format=0 \
-e ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES} \
-v ${cache_dir}:/root/.cache \
-v ${ccache_dir}:/root/.ccache \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
-v /usr/local/dcmi:/usr/local/dcmi \
-v ${{ github.workspace }}/../../..:${{ github.workspace }}/../../.. \
-v ${{ github.workspace }}:/PaddleCustomDevice -w /PaddleCustomDevice \
${docker_image} /bin/bash

- name: Download Code
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
source ${{ github.workspace }}/../../../proxy
set -x
wget -q --tries=5 --no-proxy https://paddle-github-action.bj.bcebos.com/PaddleCustomDevice/PR/${PR_ID}/${COMMIT_ID}/PaddleCustomDevice.tar.gz --no-check-certificate
echo "Extracting PaddleCustomDevice.tar.gz"
tar -xf PaddleCustomDevice.tar.gz --strip-components=1
git config --global --add safe.directory "*"
git remote add upstream https://github.com/PaddlePaddle/PaddleCustomDevice.git
git merge ${BRANCH} --no-edit
git --no-pager log --pretty=oneline -5'

- name: RUN 910B-X86
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
# !!!!! SKIP IF NO NPU CHANGE !!!!
echo "=========== Checking PR Changes If NPU FULL CI Needed ==========="
change_numbers=$(git diff --name-only remotes/origin/${BRANCH} | wc -l)
change_backend=$(git diff --name-only remotes/origin/${BRANCH} | grep "backends/"| wc -l)
change_npu_only=$(git diff --name-only remotes/origin/${BRANCH} | grep "backends/npu"| wc -l)
git diff --name-only remotes/origin/${BRANCH}

if [ $change_numbers -ne $change_backend ]; then
echo "Common file changed, continue to run NPU FULL CI test ..."
elif [ $change_npu_only -eq 0 ] ; then
echo "NO NPU backend changes found, skip NPU FULL CI ...."
exit 0
fi

echo "============ CANN Version ============="
source ~/.bashrc
echo "============ Install PaddlePaddle CPU ============="
PATH=/usr/local/bin:${PATH}
ln -sf $(which python3.10) /usr/local/bin/python
ln -sf $(which pip3.10) /usr/local/bin/pip

pip install -U numpy==1.26.4 safetensors
python -m pip install --pre paddlepaddle -i https://www.paddlepaddle.org.cn/packages/nightly/cpu/
python -c "import paddle; print(paddle.__version__)"
python -c "import paddle; print(paddle.version.commit)"
# Install python packages required for incremental coverage
python -m pip install PyGithub

bash backends/npu/tools/pr_ci_npu.sh'

- name: Terminate and delete the container
if: always()
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
rm -rf * .[^.]*
'
docker stop ${container_name}
docker rm ${container_name}
Loading
Loading