Permalink
Comparing changes
Choose two branches to see what’s changed or to start a new pull request.
If you need to, you can also or
learn more about diff comparisons.
Open a pull request
Create a new pull request by comparing changes across two branches. If you need to, you can also .
Learn more about diff comparisons here.
...
- 9 commits
- 425 files changed
- 9 contributors
Commits on Jul 22, 2024
Commits on Dec 3, 2024
-
Sync develop to stable (v.0.6) (#216)
* Performace improvement for vpicio (#211) * Paritial fix for the region transfer/wait performance issue * Committing clang-format changes * Improve the async processing for vpicio_mts_all, also fix a few compile issues * Committing clang-format changes * Minor change * Continue to optimize start_all performance for vpicio, add a few time related convinient functions * Committing clang-format changes * Fix hanging issue in CI testing * Committing clang-format changes * Disable debug prints * Revert back for non-all ops * Better pthread management * Better pthread management * Fix timeout issue with CI testing and clang-formatting * Committing clang-format changes * Test * Trigger test * Committing clang-format changes * Trigger CI * Committing clang-format changes * Trigger CI * Switch to static partition for vpicio * Replace vpicio_mts with new implementation --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Multi-thread fix and request merging (#205) * Update getting_started.rst (#184) * Removing gres option for ctest (#182) * Removing gres option from scripts * Update check for core --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix an issue with region transfer request * Committing clang-format changes * Merge small requests when they are contiguous and 1D, change srun commands in run scripts to detect Perlmutter compute nodes * Merge only for REGION_LOCAL partition * Committing clang-format changes * Fix a bug that causes some tests to fail * Fix a couple of issues with start/wait all * Committing clang-format changes * Add aggregation support for contiguous read operations * Committing clang-format changes * Fix compile issue when multithread is enabled * Committing clang-format changes * minor change with test code * Committing clang-format changes * Remove metadata mutex for multi threading * Committing clang-format changes * Fix mutex * Committing clang-format changes * Fix an issue when closing an obj * Sync develop to stable (v.0.5) (#201) * Update getting_started.rst (#184) * Removing gres option for ctest (#182) * Removing gres option for ctest * Removing gres option from scripts * Update check for core --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * enable cache by default (#187) * Removing PDC macro (#189) * Removing gres option for ctest * Removing gres option from scripts * Update check for core * Remove PDC macro * Committing clang-format changes --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * BDCATS fix (#193) * Fix issues with bdcats_batch * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Update mpi_test.sh (#197) * Update .gitlab-ci.yml (#195) * Updates for latest integration with Jacamar and Gitlab tokens in CI * VPICIO bugfix (#196) * Fix VPICIO bug * Add more checks and error out when no server is selected * Committing clang-format changes * Add VPICIO and BDCATS to MPI test --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix vpicio_mts (#199) --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Committing clang-format changes * Fix rebase issue * Add timers * Committing clang-format changes * Add explict transfer start (all) with MPI communicator * Committing clang-format changes * MPI fix * remove debug msg * Committing clang-format changes * Add function comment for doc * Revert script changes * Committing clang-format changes * Revert script changes * Committing clang-format changes * Revert script setting --------- Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Update CMakeLists.txt to bump version number (#202) * Update CMakeLists.txt to bump version number * Update clang-format-check.yml * IDIOMS Update & BULKI v0.1 (#203) * fix cmake mercury_util not found issue * update for Julia support * fix hdf5.h not found for src/tools * update container config * add libhdf5-dev for Github Actions * update CMake for HDF5 in tools * update logic for finding HDF5 * update * remove use system hdf5 * delete useless find library * update findHDF5 * Feature/dart (#11) Update to avoid fixing compilation issue on src/tools (due to : HDF5 cannot be found) * Use cc on Perlmutter (#161) Dr. Tang fixed a compilation issue in NERSC CI where HDF5 cannot be detected even if the cray-parallel-hdf5 module is loaded on Perlmultter. * update with fixes on tools and llsm example * add gitignore for llsm * update gitignore * Feature/dart (#12) * fix formatting * update clangformat10 * update base dockerfile * Add clang-format10 to docker container. Also fixed clang-format. Add clang-format10 to docker container. Also fixed clang-format. * Fix pdc ls (#154) * pdc import, export, ls compiled successfully * removed requested files * formatting issues * changed install tools * gets checkpoint files * grabbing checkpoint files from within sub-directories, minor comments * Committing clang-format changes * Committing clang-format changes * Fix a few issues with pdc_ls * Committing clang-format changes --------- Co-authored-by: nickaruwang <nickwan0318@gmail.com> Co-authored-by: Nick Wang <66816536+nickaruwang@users.noreply.github.com> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * update documentation * update document update document * sync branch sync branch * no UUID module is required * update document and make UUID an optional package * update docker repo name * updating docker repo name and make UUID optional * Complete support for Docker and Github Codespace (#157) Include support for Docker and Github Codespace so we can run our dev environment with the support of Docker. * SQLite and RocksDB support for KVtags (#165) SQLite and RocksDB support for KVtags * fix round for tag delete * update test * bulki update * BULKI base type worked * BULKI all tests done * new index code * update * update new test * update csv bench * update * update script * adding python scripts for generating large metadata set for LLSM application * update json schema * better json validator * update importer * update code for non-MPI compatibility * update llsm converter * update LLSM data converter * split files * update .gitignore * update * add timing info * update * update tag size * detect object creation failure * update * update object name with date * update for robustness * update * update JMD_DEBUG option * update output for overall output * update inttypes.h * update * update extractor * update inttypes.h * update converter * update importer information * Update getting_started.rst (#184) * Removing gres option for ctest (#182) * Removing gres option for ctest * Removing gres option from scripts * Update check for core --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * fix issue * fixed search issues * update for infix * update * index persistence still needs improvement * update * enable cache by default (#187) * Removing PDC macro (#189) * Removing gres option for ctest * Removing gres option from scripts * Update check for core * Remove PDC macro * Committing clang-format changes --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * update * range query done' * range query local test passed * multi-condition in progress * clean up code * add comments * new benchmark * update * update range query test * update cmake: * update * update * update * update * someta range query * someta range query * someta range query * fix value serialization * update * update double free * update * update * update * fixed pointer issue * rb_tree delete fixed, now need to check index persistence * update * update * update * update * update * update * update * update * update * update * update * update * update * BDCATS fix (#193) * Fix issues with bdcats_batch * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * update * clean up code * update test sh * IDIOMS persistence DONE * update * remove old kvtag benchmarks * update * update * update changes * dart info * update * multi data type for the same key, supported now * Monitoring changes from feature/dart to develop (#18) Major changes: * IDIOMS -> affix-based query benchmark * IDIOMS -> Simulation Test * IDIOMS -> Multi data type supported for the same key * IDIOMS -> Range Query and Exact Query for Numeric Values * IDIOMS -> benchmark for numeric values (exact search and range query) * IDIOMS -> Index Persistence * BULKI -> A data serialization and deserialization mechanism. * fix CMakeLists.txt * update * update format * update BULKI interface order * BULKI API sorted * add idioms ci test * Feature/dart (#20) 1. add documentation about BULKI and IDIOMS query conditions 2. add ci test for IDIOMS 3. optimized BULKI to save space on its metadata fields. * Feature/dart (#22) update version * update * update * update * remove unnecessary .bin file * update * update --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: nickaruwang <nickwan0318@gmail.com> Co-authored-by: Nick Wang <66816536+nickaruwang@users.noreply.github.com> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> * Fix region transfer with object static partitioning (#214) * Update pdc_region_transfer.c * Committing clang-format changes * Update .gitlab-ci.yml Fix issue with Perlmutter CI libfabric module --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * EQSIM benchmark code and fixes (#213) * Update getting_started.rst (#184) * Removing gres option for ctest (#182) * Removing gres option for ctest * Removing gres option from scripts * Update check for core --------- Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * enable cache by default (#187) * Benchmark code for EQSIM data * Committing clang-format changes * Minor adjustments * Committing clang-format changes * Updates * Committing clang-format changes * Change vpicio to use local server partitioning, add some debug prints * Committing clang-format changes * Add metadata query to benchmark code * Committing clang-format changes * Add ZFP compression for read and write * Committing clang-format changes * Add a option to use more ranks to read data so total data of each rank is less than the 4GB chunk limit * Committing clang-format changes * Add a data query code for EQSIM data * Committing clang-format changes * Minor adjustments for the HDF5 read code * Committing clang-format changes * Fix an issue with periodic data flush, minor changes to benchmark code * Committing clang-format changes * fix an issue with 3d read segfault * Committing clang-format changes * Fix compile issue * Update .gitlab-ci.yml * Update sleep time * Replace function * Replace function * Minor updates and doc changes * Committing clang-format changes * Update --------- Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Hyunju Oh <oh.693@osu.edu> Co-authored-by: Hyunju Oh <hjoh16@login15.chn.perlmutter.nersc.gov> Co-authored-by: Zhang Wei <zhangwei217245@lbl.gov> Co-authored-by: nickaruwang <nickwan0318@gmail.com> Co-authored-by: Nick Wang <66816536+nickaruwang@users.noreply.github.com>
8 people authoredDec 3, 2024
Commits on Jan 10, 2025
Commits on Feb 4, 2025
-
Fix a thread race issue (#224)
* Fix a thread race issue that may cause memory error when larger than cache max size data is transferred * Add a test that writes more data than server cache size * Fix CI run command
Commits on Apr 1, 2025
-
Synchronize with develop (#242)
* Fix cache flush (#226) * Fix a thread race issue that may cause memory error when larger than cache max size data is transferred * Add a test that writes more data than server cache size * Fix CI run command * Update nersc.yml (#238) * Since PDCinit returns a uint64_t, 0 should indicate failure (#233) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Check the return value of `PDC_Client_init` in `PDC_init` (#230) * Check that return value of PDC_Client_init in PDC_init * Change return to 0 This will make is simpler when merging #233 (comment) --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Change `printf` to PDC logger (#232) * Changed all printf to use pdc logger Also removed large blocks of comments and chanegd the pdc logger to print the file name, function, and line number. * Change typo of LOG_INFO to LOG_ERROR * Correct grammar from fail -> failed * update grammer succesfully close -> successfully closed * switch type of LOG_INFO to LOG_ERROR * Add logging docs and fix some LOG_INFO->LOG_JUST_PRINT * update clang formatting --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Malloc correct size for pdc_obj_metadata_pkg (#237) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * PDCregion_transfer_create validate client buf, local region, and remote regions (#236) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: Noah Lewis <47840925+TheAssembler1@users.noreply.github.com>
3 people authoredApr 1, 2025 -
* install latest cmake * fix mercury version to match the docs * try fix
Commits on Jul 21, 2025
-
Synchronize with develop (#256)
* Add pdc_logger.h to installation (#245) * sync with gitlab (#248) * Fix restart issue (#228) * Fix cache flush (#226) * Fix a thread race issue that may cause memory error when larger than cache max size data is transferred * Add a test that writes more data than server cache size * Fix CI run command * Fix restart issue * Update nersc.yml (#238) * Since PDCinit returns a uint64_t, 0 should indicate failure (#233) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Check the return value of `PDC_Client_init` in `PDC_init` (#230) * Check that return value of PDC_Client_init in PDC_init * Change return to 0 This will make is simpler when merging #233 (comment) --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Change `printf` to PDC logger (#232) * Changed all printf to use pdc logger Also removed large blocks of comments and chanegd the pdc logger to print the file name, function, and line number. * Change typo of LOG_INFO to LOG_ERROR * Correct grammar from fail -> failed * update grammer succesfully close -> successfully closed * switch type of LOG_INFO to LOG_ERROR * Add logging docs and fix some LOG_INFO->LOG_JUST_PRINT * update clang formatting --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Malloc correct size for pdc_obj_metadata_pkg (#237) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * PDCregion_transfer_create validate client buf, local region, and remote regions (#236) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: Noah Lewis <47840925+TheAssembler1@users.noreply.github.com> * Fix return metadata dtype (#246) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Region info transfer struct type and helper functions (#247) * Fix cache flush (#226) * Fix a thread race issue that may cause memory error when larger than cache max size data is transferred * Add a test that writes more data than server cache size * Fix CI run command * checkpoint * Switch variables such as count_0, start_0, and size0... to arrays This will reduce code duplication, reduce bugs, and make it easier to switch to support n-dimnesional data. * clang format * checkpoint * created better function names and documentation * remove * Committing clang-format changes * clang format * remove file * change for use helper function * fix bug with incorrect helper function call --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix issues with PDC tools (#249) * Fix issues with PDC tools * Correct LOG_ERROR to LOG_INFO * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix printing in `PGOTO_ERROR` and `PGOTO_ERROR_VOID` (#250) Print new line by default in `PGOTO_ERROR` and `PGOTO_ERROR_VOID` Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Group Tests Into Folders (#252) * Fix cache flush (#226) * Fix a thread race issue that may cause memory error when larger than cache max size data is transferred * Add a test that writes more data than server cache size * Fix CI run command * Grouped commons tests into folders This commit also changes the src/tests/CmakeLists.txt to build tests within their new folders * add deprecated folder remove buf_map folder * Update run_multiple_mpi_test.sh * Update dependencies-macos.sh * Update dependencies-macos.sh --------- Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: Jean Luca Bez <jeanlucabez@gmail.com> * Return the same obj_id if the obj is just created or already opened (#254) * Return the same obj_id if the obj is just created or already opened * Committing clang-format changes * Update doc * Update dependencies-macos.sh --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * add option to choose interface (#255) * add option to connect to a given network interface * Committing clang-format changes * fix conflict * include header * enable output on failure --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Fix multithreading compilation (#259) * fix multhreading compilation * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix segmentation fault of calling `PDCobj_create_mpi` twice with duplicate object name (#262) * Validate sucess of PDC_obj_create and PDC_find_id in PDCobj_create_mpi * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Use `PDC_malloc`, `PDC_free`, `PDC_calloc`, and `PDC_realloc` (#260) * checkpoint * replace free with PDC_free and calloc with PDC_calloc * Committing clang-format changes * fix more mallocs to PDC_malloc * more PDC_free fixes * Committing clang-format changes * Update ubuntu-cache.yml * remove eno1 * fix realloc * Committing clang-format changes * Update ubuntu-no-cache.yaml * Fix several bugs with error checking with object dim allocation * Committing clang-format changes * fix bug * Committing clang-format changes * Update ubuntu-no-cache.yaml * Update ubuntu-cache.yml * Set default value of ndim to 1 in PDCprop_create when using PDC_OBJ_CREATE * Committing clang-format changes * Malloc when defaulting to ndim size 1. Only free hostname when we PDC_malloc the memory because pointers returned by getenv are not malloced and could point to static memory. * Committing clang-format changes * Update README.md minor change to trigger the pipeline --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> Co-authored-by: Jean Luca Bez <jeanlucabez@gmail.com> * Fix Sphinx documentation errors and warnings (#265) * Fix all sphinx warnings and errors. Removed repeat declarations of functions. * Committing clang-format changes * remove def of EXTENSION_MAPPING * gitignore for docs and fix c structs * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Replace `docs/README.md` -> steps to build docs (#268) * Replace docs/README.md -> steps to build docs * Update README.md --------- Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Use `FUNC_ENTER` and `FUNC_LEAVE` (#270) * use func enter and func leave in all functions * Committing clang-format changes * fix infinite recursion between memory managment, hash table, and per function timing * Committing clang-format changes * add profiling to CI --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * New test macros and code cleanup (#261) * checkpoint * Committing clang-format changes * some tests * Committing clang-format changes * checkpoint * open_obj uses new test macros * Committing clang-format changes * read_obj uses TASSERT * read_obj uses TASSERT * Committing clang-format changes * cont_del and cont_getid use test macros * convert more tests to use macros * convert more tests to macros * Committing clang-format changes * Committing clang-format changes * clang format * use test helper in cont_info and cont_add_del * more tests use macros * Committing clang-format changes * use tests macros in more tests * use PGOTO* macros instead of goto * clang format * more log fixes * logging cleanup and more usage of test macros * Committing clang-format changes * clang format and fix CMakeLists for tests * use tests macros in transfer overlap 2D/3D * use TASSERT in more tests * Committing clang-format changes * use test asserts * all tests on the CI use TASSERT * fix printing and newlines in tests * print time, file name, function name, and line number in debug print --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Tests logging typo fix (#273) * Fixed logging typos * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Rename pdc_server.exe to pdc_server for consistency (#275) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Update vpicio_mts.c (#276) Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Client Propogate `HG_Finalize` error on `PDCclose` (#263) * all but 4 close errors are fixed * Committing clang-format changes * client side HG_Finalize now passes on serial tests * Committing clang-format changes * cleanup * Committing clang-format changes * Update pdc_region_transfer.c * free bulk handles during region transfer close --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Standardize ID Lookup Null Checks and Error Handling (#281) * cleanup finding id's * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> * Obj open fix (#279) * Fix seg fault for PDCobj_open on non-existent object * Committing clang-format changes * Remove log from NULL check * Log message when object metadata isn't found. --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix multithread (#274) * move hash table mutex to hashtable source filse * Committing clang-format changes * add multithread compile test --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> * Fix seg fault when mercury initialization fails (#283) * check for NULL paramterse in hash table * Committing clang-format changes --------- Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com> Co-authored-by: Jean Luca Bez <jlbez@lbl.gov> --------- Co-authored-by: Noah Lewis <47840925+TheAssembler1@users.noreply.github.com> Co-authored-by: Houjun Tang <htang4@lbl.gov> Co-authored-by: github-actions <github-actions[bot]@users.noreply.github.com>
4 people authoredJul 21, 2025
Loading
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff 0.5...stable