Skip to content

Modify apl storage - #467

Merged
comcon1 merged 2 commits into
NMRLipids:mainfrom
comcon1:modify-apl-storage
Feb 24, 2026
Merged

Modify apl storage#467
comcon1 merged 2 commits into
NMRLipids:mainfrom
comcon1:modify-apl-storage

Conversation

@comcon1

@comcon1 comcon1 commented Feb 20, 2026

Copy link
Copy Markdown
Member
  • changes schema of apl.json
    From:
{
 "0.0": 62.4,
 "20.0": 63.1
}

to:

[
   [0, 62.4],
   [20.0, 63.1]
]
  • remove code duplication in data-retrieving API
  • add get_density functionality

NOTE! Can be merged only together with corresponding schema changes in BilayerData.


📚 Documentation preview 📚: https://databank--467.org.readthedocs.build/

@comcon1

comcon1 commented Feb 20, 2026

Copy link
Copy Markdown
Member Author

Here is a script for conversion of Simulations folder in the BilayerData repository:

#!/usr/bin/env python3
"""
Convert apl.json dicts to sorted Nx2 matrix arrays.
Searches subfolders, saves as apl_matrix.json.
"""

import json
from pathlib import Path

from fairmd.lipids.auxiliary import CompactJSONEncoder

def convert_apl_file(input_path):
    """Convert single apl.json to matrix."""
    with open(input_path, "r") as f:
        data = json.load(f)

    # Sort by price key (float), convert to [price, value] pairs
    matrix = [[float(k), v] for k, v in sorted(data.items(), key=lambda x: float(x[0]))]

    with open(input_path, "w") as f:
        json.dump(matrix, f, indent=2, cls=CompactJSONEncoder)

    print(f"✓ Converted {input_path}{input_path} ({len(matrix)} rows)")


def main():
    root = Path(".")
    apl_files = list(root.rglob("apl.json"))

    if not apl_files:
        print("No apl.json files found in subfolders.")
        return

    print(f"Found {len(apl_files)} apl.json files:")
    for f in apl_files:
        print(f"  {f}")

    for apl_file in apl_files:
        try:
            convert_apl_file(apl_file)
        except Exception as e:
            print(f"✗ Error in {apl_file}: {e}")


if __name__ == "__main__":
    main()

Should be applied after the script is accepted.

@comcon1
comcon1 marked this pull request as ready for review February 20, 2026 20:37

@batukav batukav left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not apply a schema like

{"time": [0.0, 20.0], "apl": [62.4, 63.1]}

This would make accessing the time series values much easier. If @comcon1 schema is required for the other parts of the project like the website etc, I am fine with the original request. I just wanted to know.

@comcon1

comcon1 commented Feb 22, 2026 via email

Copy link
Copy Markdown
Member Author

@comcon1

comcon1 commented Feb 22, 2026

Copy link
Copy Markdown
Member Author

Can we not apply a schema like

If to apply this schema, then probably to all data of this kind? To densitites and FormFactors? Will it be comfortable to see the values like this. I'm not sure. I want to see by eyes that density at 0 is 200 etc.

@batukav

batukav commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

The schema would be the same for all time/spatial series data, including density profile, form factors etc.

After some thoughts, I agree that we should go with the initial schema suggestion with the condition that each data point will be a single line as shown below. It's good for the eyes, easy on git, easy to audit and easy to load into numpy.

[
  [0.0, 62.4],
  [20.0, 63.1],
  [40.0, 63.5]
]

@batukav batukav left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider making each entry a single line, for example

[
  [
    100000.0078125, 61.4804
  ],
  [
...

It will make using CLI commands like grep also easier.

@comcon1

comcon1 commented Feb 23, 2026

Copy link
Copy Markdown
Member Author

You think it

[
  [
    100000.0078125, 61.4804
  ],
  [
   ...

is better than

[
  [100000.0078125, 61.4804],
  [...
```?
It is three times more lines! It still has one list entity on the line.

@comcon1

comcon1 commented Feb 23, 2026

Copy link
Copy Markdown
Member Author

A.. I see!
I have one number per line in my edits. It's quite stupid. Should fix it definitely, thx!

@comcon1

comcon1 commented Feb 23, 2026

Copy link
Copy Markdown
Member Author

I have used now the compact encoder to get a nice format. The formatter-snippet is updated so it can be applied directly to the database.

@comcon1
comcon1 requested a review from batukav February 23, 2026 13:12
comcon1 added a commit to comcon1/BilayerData that referenced this pull request Feb 24, 2026
@comcon1
comcon1 merged commit 52ebae5 into NMRLipids:main Feb 24, 2026
10 of 11 checks passed
comcon1 added a commit to NMRLipids/BilayerData that referenced this pull request Feb 24, 2026
@comcon1
comcon1 deleted the modify-apl-storage branch March 1, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants