Merge branch 'main' into main

Author: elenafuengar

.github/ISSUE_TEMPLATE/bug-report.md

@@ -9,40 +9,25 @@ assignees: elenafuengar
 ##  Bug report 🐛
 Create a bug report to help us improve Wakis robustness🦾💖
 
-**💬 Describe the bug**  
+#### 💬 Describe the bug
 A clear and concise description of what the bug is. What did you expect to happen, and what happened instead?
 
----
-
-**🔁 To Reproduce**  
-Steps to reproduce the behavior:
-
-1. This version'...'
-2. This script '...'
-3. This line of code '...'
-4. See the error
-
----
 
-**✅ Expected behavior**  
-What did you expect to happen?
+#### 🔁 To Reproduce  
+Steps to reproduce the behavior or paste your Python code here:
 
----
-
-**🖼️ Screenshots (optional)**  
-If applicable, add screenshots or logs to help explain your problem.
+```python
+# Your code
+```
 
----
-
-**🖥️ System Info**  
-Please complete the following:
+#### 🖥️ System Info  
+If possible complete the following:
 
 - 🧭 **OS**: [e.g. Ubuntu 22.04, macOS Ventura, Windows 11]
 - 🧩 **Python version**: [e.g. 3.10]
-- 💾 **Wakis version**: [e.g. 0.5.2]
-- ⚙️ **Environment**: [e.g. Conda, pip, Jupyter Notebook, MPI-enabled]
+- 💾 **Wakis version**: [e.g. 0.6.0]
+- ⚙️ **Environment**: [e.g. Running on CPU/GPU from Script, Jupyter Notebook, MPI-enabled...]
 
----
 
-**📎 Additional context**  
+#### 📎 Additional context
 Any other info that might help us reproduce or understand the issue. Logs, configurations, or even your simulation setup!

.github/ISSUE_TEMPLATE/feature-request.md

@@ -9,31 +9,23 @@ assignees: elenafuengar
 ## Feature request 💡
 Suggest an idea that should be implemented in Wakis ⚙️💖
 
-**🧩 Is your feature request related to a problem? Please describe.**  
-A clear and concise description of the underlying problem or limitation.  
-_Example: "I'm always frustrated when [...]"_
 
----
-
-**🔍 Describe the solution you'd like**  
+#### 🔍 Describe the solution you'd like  
 What would the ideal solution look like? Be specific!  
 _Example: "I wish I could [...], and the solver would automatically [...]"_
 
----
 
-**🔄 Describe alternatives you've considered**  
-Have you tried other workarounds, external tools, or existing features?
+#### 🔄 Describe alternatives you've considered 
+Have you tried other workarounds, external tools, or existing features? You can include some Python code/pseudo-code here if you don't want to go through the Pull-Request hassle:
+
+```python
+# your code idea
+```
 
----
 
-**📎 Additional context**  
-Include any extra information, screenshots, mockups, or use cases that help us understand your idea better.
 
----
 
-**🙏 Thank you!**  
-We're excited to hear your suggestions. Your feedback makes Wakis better for everyone! 🌟
 
+>**🙏 PS: Thank you!**  
+>We're excited to hear your suggestions. Your feedback makes Wakis better for everyone! 🌟
 
-**Additional context**
-Add any other context or screenshots about the feature request here.

.github/workflows/pull_request_CPU.yml

@@ -0,0 +1,42 @@
+# This workflow runs on pull requests and nightly
+name: pull_request_CPU_python11
+
+on:
+  # Run tests for every pull request targeting main
+  pull_request:
+    branches:
+      - main
+    types: [opened, synchronize, reopened]
+
+jobs:
+  run_tests:
+    runs-on: ubuntu-latest
+
+    defaults:
+      run:
+        shell: bash -el {0}
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+        with:
+          path: wakis
+
+      - name: Setup Miniforge
+        uses: conda-incubator/setup-miniconda@v2
+        with:
+          miniforge-version: latest
+          python-version: "3.11"
+
+      - name: Install wakis
+        run: |
+          cd wakis
+          pip install .['notebook']
+
+      - name: Print installed packages
+        run: conda list
+
+      - name: Run pytest
+        run: |
+          cd wakis
+          python -m pytest --color=yes -v -s tests

README.md

@@ -45,6 +45,12 @@ Check 📁 `examples/` and `notebooks/` for different physical applications:
 * Custom perturbation interacting with PEC geometry 
 * Wakefield simulation of accelerator cavity on CPU, GPU and with MPI
 
+Check 🌐📁 [`SWAN_tutorial/`](https://github.com/ImpedanCEI/SWAN_tutorial) for hands-on notebook examples ready to run on CERN's SWAN service's GPUs (A100, Tesla T4):
+
+[<img class="open_in_swan" data-path="your_submodule_name" alt="Open this Gallery in SWAN" src="https://swanserver.web.cern.ch/swanserver/images/badge_swan_white_150.png">][gallery_url]
+
+[gallery_url]:https://cern.ch/swanserver/cgi-bin/go?projurl=https://github.com/ImpedanCEI/SWAN_tutorial.git
+
 Check 🌐📁 [`wakis-benchmarks/`](https://github.com/ImpedanCEI/wakis-benchmarks) for beam-coupling impedance calculations & comparisons to the commercial tool *CST® Wakefield solver*:
 * PEC cubic cavity below cutoff (mm) and above cutoff (cm)
 * Conductive cubic cavity below cutoff

docs/installation.md

@@ -139,41 +139,80 @@ mamba install cudatoolkit=11.8.0
 
 You can find the official CUDA Toolkit in the [NVIDIA development website](https://developer.nvidia.com/cuda-downloads?target_os=Linux&target_arch=x86_64&Distribution=Fedora&target_version=41&target_type=rpm_local)
 
-## Python Notebooks troubleshooting
+### HTCondor with GPU submission
 
-### Matplotlib interactive plots
-Within jupyter notebooks, in order to be able to zoom and interact with matplotlib figures, one needs to use notebook magic commands `%`. 
-* The recommended one for Jupyter notebooks on the web is `%matplotlib widget`
-* The recommended one for Jupyter notebooks on VS Code in `%matplotlib ipympl`
+Your `config_condor.sub` file needs to request a GPU
+```
+if !defined FNAME
+    FNAME               = outputs
+endif
 
-The package `ipympl`can be easily installed using `pip install ipympl`
+ID                      = $(Cluster).$(Process)
 
-### PyVista interactive plots
-To be able to render 3D interactive plots in Jupyter notebooks, it is recommended to use the `wakis['notebook']` pip installation. 
+output                  = ./logs/$(FNAME).$(ID).out
+error                   = ./logs/$(FNAME).$(ID).err
+log                     = ./logs/$(FNAME).$(Cluster).log
 
-Some driver problems may arise depending on pre-installed versions. One way of solving common errors like `MESA-loader` or `libGL error` is installing a new driver within your conda environment with:
+should_transfer_files   = YES
+when_to_transfer_output = ON_EXIT
+
+request_GPUs            = 1
+request_CPUs            = 1
++MaxRunTime             = 100
++AccountingGroup        = your_acct_group
 
+executable              = simulation_run.sh
 ```
-conda install conda-forge::libstdcxx-ng
+
+Your `simulation_run.sh` file could optionally include:
+
 ```
+#!/bin/bash
+# source the miniforge
+source /afs/cern.ch/work/u/user/miniforge3/etc/profile.d/conda.sh
+conda activate /afs/cern.ch/work/u/user/SimulationProjects/wakis_simulation/wakis-env
 
-If you're in a headless environment (e.g., remote server, openstack machine), forcing OSMesa (Off-Screen Mesa) rendering might help:
+# Fix for UCX errors
+export UCX_TLS=rc,tcp,cuda_copy,cuda_ipc
+
+# Fix for HDF5 file locking issues, when overwriting to same directory
+export HDF5_USE_FILE_LOCKING=FALSE
+
+python3.11 -m wakis_simulation.main
+```
+
+An example python project for simulating on HTCondor, is shown here [BLonD Simulation Template](https://gitlab.cern.ch/blond/blond-simulation-template). 
+The submission files are based on the approach used in this example project [^3], modified for GPU. The project can be modified for the wakis environment, it currently is set up 
+for simulations with BLonD, the longitudinal beam dynamics code.
+
+## CPU Multithreading
+To achieve multithreading for Wakis `sparse matrix x vector` operations, the Intel-MKL backend has been implemented as an alternative to single-threaded `scipy.sparse.dot`. To install it in your conda environment simply do:
+
+```
+pip install mkl mkl-service sparse_dot_mkl
+```
+
+Wakis will detect that the package is installed and use it as default backend. To control the number of threads and memory pinning, add the following lines to your python script **before** the imports:
 
 ```python
+# [!] Set before importing numpy/scipy/mkl modules
 import os
-os.environ['PYVISTA_USE_OSMESA'] = 'True'
+os.environ["OMP_NUM_THREADS"] = str(os.cpu_count())       # Number of OpenMP threads
+os.environ["KMP_AFFINITY"] = "balanced,granularity=fine"  # Thread pinning
 ```
 
-## MPI setup
-To run multi-CPU parallelized simulations, Wakis needs the following packages:
+* 🔜 A domain decomposition multithreading is envisioned for future resleases
+
+## CPU/GPU MPI setup
+To run multi-node CPU parallelized simulations, Wakis needs the following packages:
 
 * OpenMPI installed in your operating system:
 * Python package [`mpi4py`](https://mpi4py.readthedocs.io/en/stable/)
 * Python package [`ipyparallel`](https://ipyparallel.readthedocs.io/en/latest/tutorial/intro.html) to start a MPI kernel inside notebooks
 
 The preferred install method is through `conda-forge`:
 
-```
+```bash
 # All at once [recommended]
 conda install -c conda-forge mpi4py openmpi
 ```
@@ -205,6 +244,7 @@ CC=mpicc pip install --no-cache-dir mpi4py
 # For working on jupyter notebooks:
 pip install ipyparallel
 ```
+### Checking multi-GPU compatibility
 
 * `OpenMPI` is built in Linux with multi-GPU compatibility (provided `cupy` is correctly setup):
 ```bash
@@ -224,6 +264,13 @@ Before launching an MPI simulation, make sure to set the environment variable `O
 # Set environment variable to use CUDA-aware before launching your MPI processes
 export OMPI_MCA_opal_cuda_support=true
 ```
+Or from python, before importing `mpi4py`:
+```python
+import os
+os.environ['OMPI_MCA_opal_cuda_support']='true'
+```
+
+### Running multi-GPU simulation
 
 To run MPI simulations from the terminal:
 ```bash
@@ -236,6 +283,52 @@ mpiexec --mca btl_smcuda_cuda_ipc_max 0 -n 4 python your_wakis_gpu_script.py
 mpiexec --mca opal_cuda_support 1 -n 4 python your_wakis_gpu_script.py
 ```
 
+It is also possible to run multi-GPU from **python notebooks** using `ipyparallel` and the notebook magic `%%px`:
+```python
+import ipyparallel as ipp
+cluster = ipp.Cluster(engines="mpi", n=2).start_and_connect_sync()
+```
+Once the cluster has initialized:
+```python
+%%px
+import os
+os.environ['OMPI_MCA_opal_cuda_support']='true' # multi GPU
+
+from mpi4py import MPI
+comm = MPI.COMM_WORLD
+rank = comm.Get_rank()
+size = comm.Get_size()
+print(f"Process {rank} of {size} is running")
+
+import cupy
+cupy.cuda.Device(rank).use()
+```
+
+## Python Notebooks troubleshooting
+
+### Matplotlib interactive plots
+Within jupyter notebooks, in order to be able to zoom and interact with matplotlib figures, one needs to use notebook magic commands `%`. 
+* The recommended one for Jupyter notebooks on the web is `%matplotlib widget`
+* The recommended one for Jupyter notebooks on VS Code in `%matplotlib ipympl`
+
+The package `ipympl`can be easily installed using `pip install ipympl`
+
+### PyVista interactive plots
+To be able to render 3D interactive plots in Jupyter notebooks, it is recommended to use the `wakis['notebook']` pip installation. 
+
+Some driver problems may arise depending on pre-installed versions. One way of solving common errors like `MESA-loader` or `libGL error` is installing a new driver within your conda environment with:
+
+```
+conda install conda-forge::libstdcxx-ng
+```
+
+If you're in a headless environment (e.g., remote server, openstack machine), forcing OSMesa (Off-Screen Mesa) rendering might help:
+
+```python
+import os
+os.environ['PYVISTA_USE_OSMESA'] = 'True'
+```
+
 ----
 
 [PyVista](https://github.com/pyvista/pyvista) a python package for 3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK) [^1]. 
@@ -245,3 +338,4 @@ We rely on [PyVista](https://github.com/pyvista/pyvista) to import CAD/STL geome
 
 [^1]: Sullivan and Kaszynski, (2019). PyVista: 3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK). Journal of Open Source Software, 4(37), 1450, https://doi.org/10.21105/joss.01450
 [^2]: Anaconda Software Distribution. (2020). Anaconda Documentation. Anaconda Inc. Retrieved from https://docs.anaconda.com/
+[^3]: S. Lauber, (2025). Blond Simulation Template. Retrieved from https://gitlab.cern.ch/blond/blond-simulation-template

docs/usersguide.md

@@ -44,7 +44,7 @@ from wakis import WakeSolver      # Wake and Impedance calculation
 
 ### Simulation domain, geometry and materials setup
 
-`wakis` is a numerical electromagnetic solver that uses the Finite Integration Technique. The grid used is a structured grid composed by rectangular cells. The simulation domain is a rectangular box that will be broken into cells where the electromagnetic fields will be computed.
+`wakis` is a numerical electromagnetic solver that uses the Finite Integration Technique. The grid used is a structured grid composed by rectangular cells. The simulation domain is a rectangular box that will be broken into cells where the electromagnetic fields will be computed. Notice that all the input parameters for Wakis must be in SI Base Units.
 
 #### Number of mesh cells
 
@@ -63,7 +63,7 @@ Note that the number of cells will heavily affect the simulation time (in partic
 ```
 
 ### `STL` geometry importing
-In beam-coupling impedance simulations one is usually interested in the geometric impedance, together with the impedance coming from material properties. In `wakis`, the geometry to simulate (sometimes referred as Embedded boundaries) can be imported from a `.stl` file containing a CAD model.
+In beam-coupling impedance simulations one is usually interested in the geometric impedance, together with the impedance coming from material properties. In `wakis`, the geometry to simulate (sometimes referred as Embedded boundaries) can be imported from a `.stl` file containing a CAD model, and the units should be meters [m].
 
 ```python
 # stl geometry files (add path to them if necessary)
@@ -119,7 +119,7 @@ stl_materials = {'cavity': 'vacuum',       # equivalent to [1.0, 1.0, 0]
 ```
 
 ### Defining the `grid` object
-After this, the user needs to indicate the domain bounds to simulate. They can be calculated from the imported geometry or just hardcoded:
+After this, the user needs to indicate the domain bounds to simulate, in [m]. They can be calculated from the imported geometry or just hardcoded with `float` values:
 
 ``` python
 # Domain bounds
@@ -136,8 +136,8 @@ grid = GridFIT3D(xmin, xmax, ymin, ymax, zmin, zmax,
                 stl_solids=stl_solids, 
                 stl_materials=stl_materials,
                 stl_translate=stl_translate,
-                stl_scale=stl_scale
-                stl_rotate=stl_rotate
+                stl_scale=stl_scale,
+                stl_rotate=stl_rotate,
                 )
 ```