Skip to content

OP dataset generator (for use in Kaggle Dataset publishing) - #491

Merged
comcon1 merged 10 commits into
mainfrom
datasets-geenerator
May 19, 2026
Merged

OP dataset generator (for use in Kaggle Dataset publishing)#491
comcon1 merged 10 commits into
mainfrom
datasets-geenerator

Conversation

@comcon1

@comcon1 comcon1 commented Apr 25, 2026

Copy link
Copy Markdown
Member
  • Generate OP dataset from experiments.
  • Generate OP dataset from simulations.

The PR is a shorter version of the all-dataset version #476. It's much shorter for review.
This PR: NMRLipids/BilayerData#388 already uses this branch.

It has additional dependencies: tables and unidecode, and it has very simple usage:

(Databank) comcon1@tanteliss:~/repo/Databank/developer$ ./gen-op-dataset.py --help
fairmd-lipids 1.5.1.dev4+g37cd713e8 by NMRlipids Open Collaboration - GPL-3.0-or-later
FAIRMD Lipids is initialized from the folder: /home/comcon1/repo/Databank/Data
---------------------------------------------------------------
usage: gen-op-dataset.py [-h] [--exps] [--sims]

options:
  -h, --help  show this help message and exit
  --exps      Run experiments
  --sims      Run simulations

It will generate one or another dataset in HDF5 format.


📚 Documentation preview 📚: https://databank--491.org.readthedocs.build/

@comcon1 comcon1 changed the title Datasets geenerator OP dataset generator (for use in Kaggle Dataset publishing) Apr 25, 2026
@comcon1
comcon1 marked this pull request as ready for review April 25, 2026 14:19
@comcon1 comcon1 added enhancement New feature or request github_actions Pull requests that update GitHub Actions code labels Apr 25, 2026
@comcon1 comcon1 self-assigned this Apr 25, 2026
@comcon1
comcon1 requested a review from MagnusSletten April 25, 2026 14:27

@batukav batukav left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth fixing before merge is the dataframe preallocation in _prepare_df_common, which can leave NaN/-1 rows in the output when not every opdict entry has a SMILEIDX mapping. Beyond that there's a fair amount of duplication between the two store_to_hdf5 implementations that the abstract base could absorb. Minor cleanups noted inline.

Comment thread developer/gen-op-dataset.py Outdated
Comment thread developer/gen-op-dataset.py Outdated
Comment thread developer/gen-op-dataset.py
Comment thread developer/gen-op-dataset.py Outdated
Comment thread developer/gen-op-dataset.py Outdated
Comment thread developer/gen-op-dataset.py Outdated
Comment thread developer/gen-op-dataset.py
@comcon1
comcon1 requested a review from batukav May 14, 2026 12:19
@comcon1
comcon1 merged commit e6d5bdd into main May 19, 2026
7 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants