WIP: Pydarshan file_based sorting#954

Closed

Yanlilyu wants to merge 37 commits intodarshan-hpc:mainfrom

Yanlilyu:pydarshan-file-based-sorting

Commits on Apr 25, 2025

darshan report changes to enable name filtering
Shane Snyder
committed
updates job summary to handle name exclude/include
Shane Snyder
committed
keep helpful heatmap warning messages
Shane Snyder
committed
fix some typing hints
Shane Snyder
committed
more tweaks to make sure warnings are printed
Shane Snyder
committed

Commits on Apr 26, 2025

testing additions
Shane Snyder
committed

Commits on Apr 29, 2025

Log-based sorting

Yanlilyu
authored and
Shane Snyder
committed
Use a path glob pattern from the user input

Yanlilyu
authored and
Shane Snyder
committed
File_based sorting

Yanlilyu
authored and
Shane Snyder
committed
Improve codes for command line arguments

Yanlilyu
authored and
Shane Snyder
committed
Test for job_stats.py

Yanlilyu
authored and
Shane Snyder
committed
File based sorting tool

Yanlilyu
authored and
Shane Snyder
committed
Test for file_stats.py

Yanlilyu
authored and
Shane Snyder
committed
File based sorting tool

Yanlilyu
authored and
Shane Snyder
committed
cleanup job_stats argparse usage
Shane Snyder
committed
cleanup functions and docstrings
Shane Snyder
committed
support csv output in job_stats
Shane Snyder
committed
only read necessary module data in job_stats
Shane Snyder
committed
add job metadata to output
Shane Snyder
committed
more human-readable output in job_stats
Shane Snyder
committed
gracefully handle missing data
Shane Snyder
committed
add exe and total_files columns to job_stats
Shane Snyder
committed
simplify order_by column names
Shane Snyder
committed
add job_stats pretty printing with rich
Shane Snyder
committed
job_stats tweaks
Shane Snyder
committed
lots of revamping of file_stats
Shane Snyder
committed
add rich as pydarshan dependency
Shane Snyder
committed
add support for pydarshan record name filters
Shane Snyder
committed
parallelize job_stats using ProcessPoolExecutor
Shane Snyder
committed
add partial_flag and uid to job_stats
Shane Snyder
committed
add optional manifest file for large inputs
Shane Snyder
committed
updates to file_stats
Shane Snyder
committed
updated pytests for job_stats/file_stats
Shane Snyder
committed
skip new dlio_logs for some test cases
Shane Snyder
committed
add more in-depth testing for job_stats/file_stats
Shane Snyder
committed
updated pydarshan docs
Shane Snyder
committed
skip tests when no logs repo
Shane Snyder
committed