Skip to content

Paraview recipe #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 70 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
cb18ea0
[email protected] build on hohgant with new EGL support
biddisco Mar 20, 2023
bdad018
osmesa debugging
biddisco Mar 21, 2023
7f40a51
Enable build-cache mirror
biddisco Mar 23, 2023
a89f166
Add job script for paraview build
biddisco Mar 23, 2023
6363977
More tweaks to try to fix osmesa on hohgant
biddisco Aug 18, 2023
2dd8e38
Moving/renaming files to fit alps recipes repo structure
biddisco Aug 18, 2023
7f42c17
Update convenience build scripts for debugging
biddisco Aug 18, 2023
4a79c2d
Use spack release version, with paraview customizations in stackinato…
biddisco Aug 18, 2023
05e84e9
use architecture dirs (eg a100) for different recipes
biddisco Aug 20, 2023
88a4c7c
Recipe for minimal paraview build on clariden
biddisco Aug 23, 2023
38f37da
Use symlinks for repos to avoid duplication, use copy.yaml for patch …
biddisco Aug 25, 2023
e2b5e64
Add stackinator build and paraview client config and pvserver launch …
biddisco Aug 25, 2023
894297e
Paraview build recipe updated with EGL, experimental plugin support
biddisco Sep 5, 2023
fbf3235
Sync paraview package with current EGL branch in spack
biddisco Sep 5, 2023
1d0396c
Bump paraview to latest 5.11.2 version
biddisco Oct 19, 2023
2091f50
Cleanup scripts
biddisco Oct 24, 2023
3127686
Paraview EGL now part of spack master, delete local repos, update rec…
biddisco Jan 17, 2024
a2fc56e
Add santis recipe and build script
biddisco Feb 10, 2024
383f10b
New hybrid approach to paraview install using cmake directly
biddisco Feb 26, 2024
0d5923c
libtree, clean build and permissions updates
biddisco Mar 6, 2024
93a73b1
Remove variant gl=egl in favour of virtual gl=egl
biddisco Apr 4, 2024
6ce9e87
replace santis with todi
biddisco Jul 1, 2024
5a371b6
Disable modules by default for gh200 paraview builds
biddisco Jul 4, 2024
259fc2c
Fix version of stdexec to pika compatible sha
biddisco Jul 4, 2024
95bbd17
[email protected]
biddisco Jul 5, 2024
12449ca
Lots of random fixes for paraview + scripts
biddisco Aug 19, 2024
c5c4ba9
Fix eiger paraview stack using osmesa and lib cdi support
biddisco Aug 21, 2024
c3a81a2
Fix link error for ispc
biddisco Aug 21, 2024
786b5d5
bump paraview to 5.13, disable raytracing on gh200 for now
biddisco Aug 22, 2024
326224f
Cleanup scripts/paths after rebase, add pvserver setup file etc
biddisco Aug 22, 2024
b66373a
Fix TBB find problem by setting TBB_ROOT
biddisco Aug 23, 2024
996a8c6
Add cdi from albestro/alby/paraview instesad of c2sm to avoid version…
biddisco Aug 27, 2024
9f927c8
Use jfrog for license. Fix cdi, adios, plugin builds and submit scripts
biddisco Aug 27, 2024
fa3f8eb
Fix paraview python view setup for numpy etc
biddisco Aug 28, 2024
902ca61
rename amd-zen2 to plain zen2, fix scripts paths, names, etc
biddisco Aug 28, 2024
6bcdd31
Add cluster name to RC launch script and cleanup script comments etc
biddisco Aug 29, 2024
5caeeeb
delete old/unused c2sm libs
biddisco Aug 29, 2024
f7d4518
Rename todi to daint
biddisco Aug 30, 2024
cef1092
change directory structure
simonpintarelli Aug 30, 2024
8a564a1
update config.yaml
simonpintarelli Aug 30, 2024
6f47326
Fix build script paths after version added to recipe location
biddisco Sep 2, 2024
b6cf934
Set spack commit to recent SHA and get paraview source from jfrog
biddisco Sep 2, 2024
10542cc
update paraview to 5.13.1, add adios2, h5py and conduit
biddisco Oct 18, 2024
d0e6469
Clean up build-env for plugins, add h5py for pv-python
biddisco Nov 20, 2024
6f9ce34
Remove h5hut from gh200 recipe
biddisco Nov 25, 2024
d6cb91d
Remove ispc customizations and set spack develop tag
biddisco Nov 26, 2024
a8ceafb
fix for spack@develop
simonpintarelli Nov 26, 2024
6dbdf83
paraview: cleanup views in environment setup
biddisco Jan 10, 2025
a5a6b88
paraview: update 5.13.1 -> 5.13.2
biddisco Jan 14, 2025
5ab8023
Improve user auto-config scripts for rc-server setup etc
biddisco Feb 4, 2025
3f2a87c
Update zen2 recipe with recent changes to install scripts
biddisco Feb 4, 2025
752bd88
Cleanup commits to ensure we always use a fixed version/commit
biddisco Feb 4, 2025
dad72d9
Fix import from spack packages in env-to-package script
biddisco Feb 5, 2025
de2e2ac
Fix libtheora hash as using @master introduced new build error
biddisco May 20, 2025
871c946
Update recipes/paraview/5.13.2/gh200/config.yaml
biddisco May 21, 2025
f791dbc
Remove cache-config from recipe repo
biddisco May 21, 2025
cdc4a19
Add clariden/santis to paraview gh200 build list
biddisco May 21, 2025
cf494cf
Remove nvindex config yaml file as it is in jfrog now
biddisco May 21, 2025
373db54
Remove old sync/update script
biddisco May 21, 2025
f569f9f
Rename environments for simplicity
biddisco May 21, 2025
026a184
Add note about git spack SHA used for recipe
biddisco May 21, 2025
dd3b47a
Remove some packages used in debugging and building other plugins etc
biddisco May 21, 2025
1d092b4
More cleanup of build/src folders after install
biddisco May 21, 2025
a166c20
remove modules.yaml as it is not used
biddisco May 23, 2025
860e2ed
bump boootstrap to gcc@12
biddisco May 23, 2025
cc5c218
Fix gcc and env setup for post install scripts broken by last few com…
biddisco May 23, 2025
03996e1
Add set-e to post install scripts to make sure that if build fails, u…
biddisco May 24, 2025
1b74f9f
Fixes to post install to ensure build completes ok
biddisco May 25, 2025
e3b4799
Change nvindex plugin tarball to aarch64 version for gh200
biddisco May 27, 2025
e3c4370
Fix jfrog nvidia index access for aarch64 and regular linux files
biddisco Jun 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -404,3 +404,14 @@ uenvs:
clariden: [gh200]
eiger: [zen2]
mount: "/user-tools"
paraview:
"5.13.2":
recipes:
gh200: 5.13.2/gh200
zen2: 5.13.2/zen2
deploy:
daint: [gh200]
clariden: [gh200]
santis: [gh200]
eiger: [zen2]
develop: True
5 changes: 5 additions & 0 deletions recipes/paraview/5.13.2/gh200/compilers.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
bootstrap:
spec: [email protected]
gcc:
specs:
- [email protected]
8 changes: 8 additions & 0 deletions recipes/paraview/5.13.2/gh200/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# NB. git sha for spack is required for some vtk external packages, but newer sha breaks other stuff
name: ParaView
store: /user-environment
spack:
commit: 195b869e1c5f0f850fe16ca1cc0c77fc71f5fcd6
repo: https://github.com/spack/spack.git
description: ParaView for Grace-Hopper using EGL
modules: false
127 changes: 127 additions & 0 deletions recipes/paraview/5.13.2/gh200/environments.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
paraview-cuda:
compiler:
- toolchain: gcc
spec: gcc
mpi:
spec: cray-mpich
gpu: cuda
unify: true
views:
develop:
exclude: []
uenv:
add_compilers: true
prefix_paths:
LD_LIBRARY_PATH: [lib, lib64]
paraview:
link: run
exclude: []
uenv:
add_compilers: true
prefix_paths:
LD_LIBRARY_PATH: [lib, lib64]
paraview-python:
link: run
exclude: []
uenv:
prefix_paths:
LD_LIBRARY_PATH: [lib, lib64]

specs:

# cuda
- [email protected]

# Build tools
# - git-lfs
- cmake
- ninja
- direnv
- ninja
- libtree
- llvm@18 ~gold ~cuda

# maths
- blaspp
- eigen
- fftw
- lapackpp
- openblas
- proj
Comment on lines +45 to +50
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are blaspp, lapackpp, openblas, ... paraview dependencies? Or are there for another legit reason?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proj is needed, eigen is needed, the others were snuck in as part of my testing, but they don't do any harm. They can be removed, but IMHO they are harmless and useful for people creating plugins that need them.


# c++ helpers
- fmt@10
- boost +atomic +chrono +container +context +coroutine +date_time +filesystem +graph +json +mpi +multithreaded +program_options +regex +serialization +shared +system +test +thread

# IO and parallelism
- hdf5 +mpi +cxx +hl +threadsafe +shared ~java
- netcdf-c +mpi
- lz4
- adios2 +python +hdf5

# memory management
# - jemalloc
# - mimalloc

# # system
# - numactl

# # profiling/testing
# - gperftools
# - googletest

# multithreading
- tbb

# in-situ support
- libcatalyst +mpi +python

# vtk external deps
- [email protected]
- [email protected]
- gl2ps
- glew
- jpeg
- jsoncpp
- libharu
- libtiff
- nlohmann-json
- [email protected]=master
- pugixml
- pegtl
- protobuf@:3.21
- seacas ~fortran ~applications ~legacy ~tests ~x11
- utf8cpp

# climate/weather
- cdi

# python
- [email protected]
- py-numpy
- py-pandas
- py-matplotlib
- py-mpi4py
- py-cftime
- py-h5py

# raytracing in VTK/ParaView
- [email protected] ~mpi +denoiser +volumes ~apps ~glm
- [email protected]
- openvkl
- embree
- rkcommon
- openimagedenoise

variants:
- +cuda
- cuda_arch=90
- ^[virtuals=gl] egl
- build_type=Release
- cxxstd=17
- +mpi
- ~fortran
- ~x11
- ~examples
- ~tests
- +cxx
11 changes: 11 additions & 0 deletions recipes/paraview/5.13.2/gh200/extra/reframe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
default:
features:
- cuda
- mpi
- osu-micro-benchmarks
- openmp
- serial
cc: mpicc
cxx: mpic++
ftn: mpifort
activation: /user-environment/env/default/activate.sh
Comment on lines +1 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not completely sure these are useful 🤔 It doesn't hurt, but there's no paraview tag and maybe no reasonable way to test that paraview is working correctly?

Just thinking out loud. It doesn't hurt to leave this here, but this will basically just enable testing of non-paraview stuff from the uenv.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reframe and CI stuff was just left in from the template I was using for the recipe. Simon added some CI related things in the config yaml. I'd prefer to not touch this stuff and let someone else add/remove whatever is needed.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on reflection - is there info anywhere on what the correct tags to put in here are? the osu_benchmarks/openmp etc mentioned there is clearly not correct and should be removed, but I do not know what is supposed to go in there

11 changes: 11 additions & 0 deletions recipes/paraview/5.13.2/gh200/packages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
packages:
all:
providers:
gl: [egl]
egl:
externals:
- spec: [email protected]
prefix: /usr
buildable: false
Comment on lines +5 to +9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can do, this was the original source of the egl settings, so it stayed here.

llvm:
require: llvm ~gold ~cuda
Comment on lines +10 to +11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not already set in the environment for the llvm spec? Or does it concretize wrong without this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a lot of problems with llvm initially, probably the settings do not need to be duplicated anywhere, but every time one of these is changed, so much stuff breaks that I don't want to touch it. If it is harmless I'd prefer to just leave the settings as they are and only change them if there is a future breakage/problem and someone files an issue.

Loading