Skip to content

[CI] Add build and static-check #72537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
May 7, 2025
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
167 changes: 167 additions & 0 deletions .github/workflows/Build-develop.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
name: Build-develop

on:
push:
branches: [develop]

permissions: read-all

defaults:
run:
shell: bash

env:
dockerfile: Dockerfile.cuda11.2_cudnn8_gcc82_trt8
docker_image: d5fc7ddda08c
COMMIT_ID: ${{ github.sha }}
work_dir: /paddle
PADDLE_ROOT: /paddle
TASK: paddle-CI-bbild-develop
ci_scripts: /paddle/ci
BRANCH: ${{ github.base.ref }}
CI_name: build-develop
no_proxy: "bcebos.com,apiin.im.baidu.com,gitee.com,aliyun.com,.baidu.com,.tuna.tsinghua.edu.cn"

jobs:
build:
name: Build
runs-on:
group: GZ_BD-CPU
steps:
- name: Checkout paddle
uses: actions/checkout@v4
with:
submodules: 'recursive'
fetch-depth: 1000

- name: Merge PR to test branch
run: |
git config --unset http.https://github.com/.extraheader
git submodule foreach --recursive sh -c "git config --local --unset-all 'http.https://github.com/.extraheader'"
git submodule foreach --recursive sh -c "git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'"
bash ${ci_scripts}/third_party_tag.sh

- name: Check docker image and run container
env:
PADDLE_CUDA_INSTALL_REQUIREMENTS: "ON"
FLAGS_fraction_of_gpu_memory_to_use: 0.15
CTEST_OUTPUT_ON_FAILURE: 1
CTEST_PARALLEL_LEVEL: 2
WITH_GPU: "ON"
CUDA_ARCH_NAME: Volta
WITH_COVERAGE: "OFF"
PADDLE_FRACTION_GPU_MEMORY_TO_USE: 0.15
CUDA_VISIBLE_DEVICES: 0,1
PRECISION_TEST: "ON"
WITH_CINN: "ON"
INFERENCE_DEMO_INSTALL_DIR: /home/data/cfs/.cache/build
WITH_INCREMENTAL_COVERAGE: "OFF"
WITH_AVX: "ON"
WITH_TESTING: "OFF"
COVERALLS_UPLOAD: "OFF"
GIT_PR_ID: ${{ github.event.pull_request.number }}
PADDLE_VERSION: 0.0.0
WITH_DISTRIBUTE: "ON"
WITH_UNITY_BUILD: "ON"
PY_VERSION: "3.10"
WITH_SHARED_PHI: "ON"
CCACHE_MAXSIZE: 150G
CCACHE_LIMIT_MULTIPLE: 0.8
CCACHE_STATSLOG: /paddle/build/.stats.log
CCACHE_SLOPPINESS: clang_index_store,time_macros,include_file_mtime
CACHE_DIR: /home/data/cfs/.cache/build
CCACHE_DIR: /home/data/cfs/.ccache/build
run: |
container_name=${TASK}-$(date +%Y%m%d-%H%M%S)
echo "container_name=${container_name}" >> ${{ github.env }}
docker run -d -t --name ${container_name} \
-v "/home/data/cfs:/home/data/cfs" \
-v "/home/data/cfs/.cache:/root/.cache" \
-v "/home/data/cfs/.ccache:/root/.ccache" \
-v "/dev/shm:/dev/shm" \
-v ${{ github.workspace }}/../../..:${{ github.workspace }}/../../.. \
-v ${{ github.workspace }}:/paddle \
-e BRANCH \
-e COMMIT_ID \
-e work_dir \
-e PADDLE_ROOT \
-e ci_scripts \
-e PADDLE_CUDA_INSTALL_REQUIREMENTS \
-e FLAGS_fraction_of_gpu_memory_to_use \
-e CTEST_OUTPUT_ON_FAILURE \
-e CTEST_PARALLEL_LEVEL \
-e WITH_GPU \
-e CUDA_ARCH_NAME \
-e WITH_COVERAGE \
-e PADDLE_FRACTION_GPU_MEMORY_TO_USE \
-e CUDA_VISIBLE_DEVICES \
-e PRECISION_TEST \
-e WITH_CINN \
-e INFERENCE_DEMO_INSTALL_DIR \
-e WITH_INCREMENTAL_COVERAGE \
-e WITH_SHARED_PHI \
-e WITH_TESTING \
-e COVERALLS_UPLOAD \
-e GIT_PR_ID \
-e PADDLE_VERSION \
-e WITH_DISTRIBUTE \
-e WITH_UNITY_BUILD \
-e PY_VERSION \
-e CCACHE_MAXSIZE \
-e CCACHE_LIMIT_MULTIPLE \
-e WITH_AVX \
-e CACHE_DIR \
-e CCACHE_DIR \
-e CCACHE_SLOPPINESS \
-e CCACHE_STATSLOG \
-e no_proxy \
-e CI_name \
-w /paddle --network host ${docker_image}

- name: Set git config
env:
work_dir: ${{ github.workspace }}
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
git config --global --add safe.directory ${work_dir}
git submodule foreach "git config --global --add safe.directory \$toplevel/\$sm_path"
git config --global user.name "PaddleCI"
git config --global user.email "[email protected]"
git remote add upstream https://github.com/PaddlePaddle/Paddle.git
'

- name: Build
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
mkdir -p /root/.cache/build
mkdir -p /root/.ccache/build
source ${{ github.workspace }}/../../../proxy
bash ${ci_scripts}/run_setup.sh bdist_wheel
'

- name: Upload paddle_whl
env:
home_path: ${{ github.workspace }}/..
bos_file: ${{ github.workspace }}/../bos/BosClient.py
paddle_whl: paddlepaddle_gpu-0.0.0-cp310-cp310-linux_x86_64.whl
run: |
docker exec -t ${{ env.container_name }} /bin/bash -c '
export AK=paddle
export SK=paddle
if [ ! -f "${{ env.bos_file }}" ]; then
wget -q --no-proxy -O ${{ env.home_path }}/bos_new.tar.gz https://xly-devops.bj.bcebos.com/home/bos_new.tar.gz --no-check-certificate
mkdir ${{ env.home_path }}/bos
tar xf ${{ env.home_path }}/bos_new.tar.gz -C ${{ env.home_path }}/bos
fi
mv /paddle/dist/${{ env.paddle_whl }} .
echo "Uploading paddle_whl to bos"
python ${{ env.bos_file }} ${{ env.paddle_whl }} paddle-github-action/PR/build/0/${{ env.COMMIT_ID }}
'

- name: Terminate and delete the container
if: always()
run: |
set +e
docker exec -t ${{ env.container_name }} /bin/bash -c 'rm -rf * .[^.]*'
docker stop ${{ env.container_name }}
docker rm ${{ env.container_name }}
14 changes: 14 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,17 @@ jobs:
name: Distribute-stable
uses: ./.github/workflows/_Distribute-stable.yml
needs: clone

build:
name: Linux-build
uses: ./.github/workflows/_Linux-build.yml
needs: build-docker
with:
docker_build_image: ${{ needs.build-docker.outputs.docker_build_image }}

static-check:
name: Static-Check
uses: ./.github/workflows/_Static-Check.yml
needs: build
with:
docker_build_image: ${{ needs.build.outputs.docker_build_image }}
1 change: 1 addition & 0 deletions .github/workflows/_Linux-CPU.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ jobs:
-e ci_scripts \
-e WITH_AVX \
-e CCACHE_DIR \
-e CI_name \
-e no_proxy \
-w /paddle --network host ${docker_image}

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/_Linux-NPU.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ jobs:
-e FLAGS_npu_storage_format \
-e TEST_IMPORTANT \
-e PADDLE_BRANCH \
-e CI_name \
-w /paddle --network host ${docker_image} /bin/bash
- name: Install Paddle-CPU
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/_Linux-XPU.yml
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,6 @@ jobs:
group: Kunlun

steps:

- name: Download build.tar.gz
run: |
sudo rm -rf * .[^.]*
Expand Down
Loading
Loading