Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "Attempt to fix FBGEMM CPU build (#3499)" #3528

Closed
wants to merge 17 commits into from
Closed

Revert "Attempt to fix FBGEMM CPU build (#3499)" #3528

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
59157bf
Revert "Attempt to fix FBGEMM CPU build (#3499)"
huydhn Dec 23, 2024
b01e940
Use system gcc compiler
huydhn Dec 24, 2024
f67daf9
Use CUDA installation from Nova
huydhn Dec 24, 2024
69b0f7a
Set CUDNN_INCLUDE_DIR and CUDNN_LIBRARY
huydhn Dec 24, 2024
7b20a82
Try one more time
huydhn Dec 24, 2024
8cae232
Fix bash typo
huydhn Dec 24, 2024
644ab0a
Building is too slow
huydhn Dec 24, 2024
78c3f5e
Try vision approach with LD_LIBRARY_PATH
huydhn Dec 24, 2024
852c2fe
Clean up
huydhn Dec 24, 2024
5bd9fe4
Clean up
huydhn Dec 24, 2024
000cf05
NOVA CUDNN is at /usr/local/cuda/lib64
huydhn Dec 24, 2024
a4c5de6
Fix libcuda path
huydhn Dec 25, 2024
2f6f76c
Add a CUDA fix for aarch64
huydhn Dec 25, 2024
373e9f9
Writing correct bash script in one go is hard
huydhn Dec 25, 2024
933102c
Swap the order?
huydhn Dec 25, 2024
e78ab24
Also fix libnvidia-ml.so
huydhn Dec 25, 2024
04f3ad2
Ready to land
huydhn Dec 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/scripts/nova_dir.bash
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@ if [[ "$working_dir" == "$FBGEMM_REPO" ]]; then cd fbgemm_gpu || echo "Failed to
## Build clean/wheel will be done in pre-script. Set flag such that setup.py will skip these steps in Nova workflow
export BUILD_FROM_NOVA=1

## TODO (huydhn): Remove this when Nova is moving to manywheel 2.28
export SET_GLIBCXX_PRELOAD=1

## Overwrite existing ENV VAR in Nova
if [[ "$CONDA_ENV" != "" ]]; then export CONDA_RUN="conda run --no-capture-output -p ${CONDA_ENV}" && echo "$CONDA_RUN"; fi

Expand Down
21 changes: 7 additions & 14 deletions .github/scripts/nova_prescript.bash
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,6 @@ runtime=$((end_time-start_time))
start_time=${end_time}
echo "[NOVA] Time taken to display GPU Info: ${runtime} seconds"

# Install C/C++ Compilers
install_cxx_compiler "${BUILD_ENV_NAME}"
end_time=$(date +%s)
runtime=$((end_time-start_time))
start_time=${end_time}
echo "[NOVA] Time taken to install C/C++ Compilers: ${runtime} seconds"

# Install Build Tools
install_build_tools "${BUILD_ENV_NAME}"
end_time=$(date +%s)
Expand All @@ -64,13 +57,13 @@ start_time=${end_time}
echo "[NOVA] Time taken to collect PyTorch environment information: ${runtime} seconds"

if [[ $CU_VERSION = cu* ]]; then
# Extract the CUDA version number from CU_VERSION
cuda_version=$(echo "[NOVA] ${CU_VERSION}" | cut -c 3-)
install_cudnn "${BUILD_ENV_NAME}" "$(pwd)/build_only/cudnn" "${cuda_version}"
end_time=$(date +%s)
runtime=$((end_time-start_time))
start_time=${end_time}
echo "[NOVA] Time taken to install cudnn: ${runtime} seconds"
# shellcheck disable=SC2155
env_prefix=$(env_name_or_prefix "${env_name}")

# Use Nova CUDA installation
echo "[INSTALL] Set environment variables CUDNN_INCLUDE_DIR and CUDNN_LIBRARY ..."
# shellcheck disable=SC2086
print_exec conda env config vars set ${env_prefix} CUDNN_INCLUDE_DIR="${CUDA_HOME}/include" CUDNN_LIBRARY="${CUDA_HOME}/lib"

echo "[NOVA] -------- Finding NVML_LIB_PATH -----------"
if [[ ${NVML_LIB_PATH} == "" ]]; then
Expand Down
Loading