Modify apl storage - #467
Conversation
|
Here is a script for conversion of #!/usr/bin/env python3
"""
Convert apl.json dicts to sorted Nx2 matrix arrays.
Searches subfolders, saves as apl_matrix.json.
"""
import json
from pathlib import Path
from fairmd.lipids.auxiliary import CompactJSONEncoder
def convert_apl_file(input_path):
"""Convert single apl.json to matrix."""
with open(input_path, "r") as f:
data = json.load(f)
# Sort by price key (float), convert to [price, value] pairs
matrix = [[float(k), v] for k, v in sorted(data.items(), key=lambda x: float(x[0]))]
with open(input_path, "w") as f:
json.dump(matrix, f, indent=2, cls=CompactJSONEncoder)
print(f"✓ Converted {input_path} → {input_path} ({len(matrix)} rows)")
def main():
root = Path(".")
apl_files = list(root.rglob("apl.json"))
if not apl_files:
print("No apl.json files found in subfolders.")
return
print(f"Found {len(apl_files)} apl.json files:")
for f in apl_files:
print(f" {f}")
for apl_file in apl_files:
try:
convert_apl_file(apl_file)
except Exception as e:
print(f"✗ Error in {apl_file}: {e}")
if __name__ == "__main__":
main()Should be applied after the script is accepted. |
batukav
left a comment
There was a problem hiding this comment.
Can we not apply a schema like
{"time": [0.0, 20.0], "apl": [62.4, 63.1]}
This would make accessing the time series values much easier. If @comcon1 schema is required for the other parts of the project like the website etc, I am fine with the original request. I just wanted to know.
|
We can apply this to everything, yes. But will it be comfortable to compare in git diffs?
Best regards, Alex
Sent from [Proton Mail](https://proton.me/mail/home) for Android.
…-------- Original Message --------
On Sunday, 02/22/26 at 15:13 Batuhan Kav ***@***.***> wrote:
@batukav commented on this pull request.
Can we not apply a schema like
{"time": [0.0, 20.0], "apl": [62.4, 63.1]}
This would make accessing the time series values much easier. If ***@***.***(https://github.com/comcon1) schema is required for the other parts of the project like the website etc, I am fine with the original request. I just wanted to know.
—
Reply to this email directly, [view it on GitHub](#467 (review)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AGBXJOH6FKUH47KLJ4YRZNT4NG2QRAVCNFSM6AAAAACV2SG7E2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTQMZXG43TIOBUGY).
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
If to apply this schema, then probably to all data of this kind? To densitites and FormFactors? Will it be comfortable to see the values like this. I'm not sure. I want to see by eyes that density at 0 is 200 etc. |
|
The schema would be the same for all time/spatial series data, including density profile, form factors etc. After some thoughts, I agree that we should go with the initial schema suggestion with the condition that each data point will be a single line as shown below. It's good for the eyes, easy on git, easy to audit and easy to load into numpy. |
batukav
left a comment
There was a problem hiding this comment.
Please consider making each entry a single line, for example
[
[
100000.0078125, 61.4804
],
[
...
It will make using CLI commands like grep also easier.
|
You think it is better than |
|
A.. I see! |
35056be to
fc5cbe6
Compare
|
I have used now the compact encoder to get a nice format. The formatter-snippet is updated so it can be applied directly to the database. |
From:
{ "0.0": 62.4, "20.0": 63.1 }to:
get_densityfunctionalityNOTE! Can be merged only together with corresponding schema changes in BilayerData.
📚 Documentation preview 📚: https://databank--467.org.readthedocs.build/