simulateur

Description

This PREESM project models the degridding step of the DDFacet radio-astronomy simulator. It uses a dataflow approach to process an image in the spatial domain:

1. The input image is first transformed into the frequency domain using an FFT.
2. The degridding operation is then applied.
3. To validate the FFT processing, an inverse FFT (iFFT) is executed in parallel, and the result is compared with the original input.

The actors for setup, degridding, and FFT are adapted from the generic-framework branch of the SEP project:
→ https://gitlab.insa-rennes.fr/SKA/ska_sep_preesm

Structure

Before running the project, make sure the following directories exist inside the Code/ folder:
Code/output/ – for storing output results (generated by the application)
Code/data/ – for input data files (follow the instruction below)

Data

The input image data, stored in .csv format, can be generated with SEP.

The input image data used for simulation can be generated via SEP.
The measurement sets can be downloaded directly here:
GLEAM :
Dataset (SKA project, documentation):
→ https://github.com/ska-telescope/sim-datasets

Download Measurement Set (GLEAM):
→ https://drive.google.com/drive/folders/1tq8jF0myYyk2BRgAaAyFlb4PcGzWXUtB

Folder structure:

-Code/
--data/
---input/
----cycle_0_deconvolved_*
---sim_*

Compilation

cd Code/build
cmake ..
make

Execution

./degridder_pipeline

Parallel Version

A parallel version of the degridding actor is implemented. Several scenario configurations are provided to target multiple architectures.

Python Scripts – Utilities

csv_to_image.py
Reconstructs an image from a CSV (with real and imaginary parts).
Usage: python3 csv_to_image.py <grid_size> <path/to/file.csv>

csvtoimage_all.py
Converts all CSV files in a given directory into FITS files. The delimiter is often ","
Usage: python3 csvtoimage_all.py <input_directory> <output_directory>

image_compare.py
Verifies that the output image is close to the input.

image_compare_mean.py
Computes the mean difference between the points of two images.

fft_reconstruction.py
Reconstructs an image from visibilities (quick but low-quality).
Usage: python3 fft_reconstruction.py <file.csv> <grid_size>

plot_vis.py
Plots the visibilities.
Usage: python3 plot_vis.py |

Installation – Required Dependencies (for reading .ms files with casacore):

• casacore
• bison
• flex
• wcslib
• gsl

Typical usage : CPU

Build the container

sudo singularity build simulator.sif simulator.def To run it : singularity shell simulator.sif All following commands must be run inside this container.

Generate csv input image

This line generates a .fits and corresponding .csv file with size 32 grid, 1 source and a sigma of 0.1 : python3 fits_to_csv.py 32 1 0.1 fits.fits input.csv This line converts data/sim_small.ms into csv files to avoid use of casacore python3 ms_to_csv.py

Launch simulator

You may have to first rename the previously generate "outputs.csv" file to "cycle_0_deconvolved_2560.csv" (depends on the scenario, check the hardcoded file paths in src/top's config_struct_set_up function. Do not forget to rebuild if you change the paths.). Move it to "data/input", and create an "output" folder in "Code/" to store the results. Then :

CPU code

mkdir build && cd build cmake .. && make ./degridder_pipeline it will generate csv files and save them to "Code/output". .The scenario parameter "num_scenario" in the preesm algorithm sets the files used as input

Heterogeneous code

Display generated image

python csv_to_image.py <grid_size> output/reconstructed.csv In this example we set grid size to 32 earlier.

Heterogeneous simulator

---

Bind vitis tools (shouldn't be necessary but good to know) : Singularity shell --bind /tools/Xilinx/:/tools/Xilinx/ simulator.sif

Setup

build fftw3 for arm64 cortex a53 : sudo apt install gfortran-arm-linux-gnueabihf gcc-arm-none-eabi gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu g++-aarch64-linux-gnu -y Compilation : ./configure --prefix (common_image_path)/sysroots/cortexa72-cortexa53-amd-linux/ --host=aarch64-linux --with-sysroot=(common_image_path)/sysroots/cortexa72-cortexa53-amd-linux --enable-float CC="(vitis_path)/gnu/aarch64/lin/aarch64-linux/bin/aarch64-linux-gnu-gcc -march=armv8-a -mcpu=cortex-a53" && make && make install (voir https://www.fftw.org/fftw3_doc/Installation-on-Unix.html) Si besoin de double précision (fftw3), la ligne de commande du dessus suffit. Si besoin de simble précision (fftw3f), ajouter --enable-float. Le simulateur utilise de la précision SIMPLE, donc --enable-float nécessaire

Build vitis project

make all/step1/step2/step3 VERSION=<2024.1/2025.1> VITIS_DIR=<path to vitis (with the right version)> COMMON_IMAGE=<path to common image> TARGET=kr260 Should fail by the end of step 3 but it's normal. Open vitis with the "system_project" folder as workspace and apply these modifications to the application : config_project.py : cpp_version = "17" includes = ["/home/jamorin/xilinx-zynqmp-common-v2025.1/sysroots/cortexa72-cortexa53-amd-linux/include"] libs = ["fftw3f"] libs_paths = ["/home/jamorin/xilinx-zynqmp-common-v2025.1/sysroots/cortexa72-cortexa53-amd-linux/lib/"] flags = "-fpermissive"

• possiblement supprimer <#include "cblas_mangling.h"> de cblas.h dans cortexa53(...)/usr/include, ça résout un problème d'inclusion et on peut espérer que ça n'en pose pas de nouveau
• virer de preesm_gen.h les inclusions de *.hpp qui cause des conflits avec la librairie standard de vitis
• pas besoin de md5.c, possible de le virer des sources
• possible besoin d'ajouter #include aux kernels hls si problèmes de compil avec std::complex

If you target hardware emulation, go to the system project and change the target platform to zcu104.