-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
auto-migrateconsistencyAspect requiring special treatment/logic outside of generic common principlesAspect requiring special treatment/logic outside of generic common principles
Description
Inspired by looking into duckdb's feature of "hive partitioning" (https://duckdb.org/docs/stable/data/partitioning/hive_partitioning) very reminiscent of the BIDS folders structure, see e.g. duckdb example:
orders
├── year=2021
│ ├── month=1
│ │ ├── file1.parquet
│ │ └── file2.parquet
│ └── month=2
│ └── file3.parquet
└── year=2022
├── month=11
│ ├── file4.parquet
│ └── file5.parquet
└── month=12
└── file6.parquet
and unfortunately duckdb not yet allowing to use different separator (- instead of =; and _ instead of / ;) ) so could be applied directly to BIDS datasets:
potentially it originates from Arrow:
@CodyCBakerPhD even shown stronger positive feeling for this as it is indeed a better semantic alignment to "variable=value".
Cons are imho singular:
- huge breaking change
Pros:
- "=" is more natural there
- any tool with hive partitioning would be able to understand BIDS!
- "auto-migrate"able (added new label) - datasets could migrate easily
- would open
-for more adequate use/allowance within labels
CodyCBakerPhDyarikoptic
Metadata
Metadata
Assignees
Labels
auto-migrateconsistencyAspect requiring special treatment/logic outside of generic common principlesAspect requiring special treatment/logic outside of generic common principles