Skip to content

Commit bbf529f

Browse files
committed
Light refactoring
I realized that _split_fname_procid_suffix was no longer used for anything. I think it's nice to have a function that explicitly describes the relationship between the filename passed into yt.load and the filename template. Consequently I replaced this function with a new function called `_infer_fname_template` that we call from `_determine_data_layout` (the contents didn't change dramatically)
1 parent 640f729 commit bbf529f

1 file changed

Lines changed: 31 additions & 40 deletions

File tree

yt/frontends/cholla/data_structures.py

Lines changed: 31 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -74,15 +74,7 @@ def _determine_data_layout(f: _h5py.File) -> tuple[np.ndarray, _BlockDiskMapping
7474
# replace ``{blockid}`` with ``0``)
7575
# 2. "root.h5": is the standard format used by Cholla's concatenation scripts
7676
# (older versions of Cholla without MPI also used this format to name outputs)
77-
_dir, _base = os.path.split(filename)
78-
_sep_i = _base.rfind(".")
79-
no_suffix = (_sep_i == -1) or (_base[_sep_i:] == "") or (_base[:_sep_i] == "")
80-
if no_suffix or not _base[_sep_i + 1 :].isdecimal():
81-
inferred_fname_template = filename # filename doesn't change based on blockid
82-
cur_filename_suffix = None
83-
else:
84-
inferred_fname_template = os.path.join(_dir, _base[:_sep_i]) + ".{blockid}"
85-
cur_filename_suffix = int(_base[_sep_i + 1 :])
77+
inferred_fname_template, cur_filename_suffix = _infer_fname_template(filename)
8678

8779
# STEP 2: Check whether the hdf5 file has a flat structure
8880
# ========================================================
@@ -150,48 +142,47 @@ def _get_common_idx():
150142
return blockid_location_arr, mapping
151143

152144

153-
def _split_fname_procid_suffix(filename: str):
154-
"""Splits ``filename`` at the '.' separating the beginning part of the
145+
def _infer_fname_template(filename: str) -> tuple[str, int | None]:
146+
"""Infers the template for all Cholla data-files based on the filename
147+
passed to ``yt.load``.
148+
155149
string from the process-id suffix, and returns both parts in a 2-tuple.
156150
157151
There are 2 conventions for the names of Cholla's data-files:
158-
159-
1. "root.h5.{procid}" is the standard format used when Cholla directly
160-
writes out data-files. When outputting a snaphsot, each MPI-process
161-
writes a separate file, named using this template (``{procid}`` is
162-
replaced with MPI-rank). Modern versions of Cholla without MPI replace
163-
``{procid}`` with ``0``. When passed a name of this form, the function
164-
returns ``("root.h5", "{procid}")``
165-
2. "root.h5": is the standard format used by Cholla's concatenation scirpts
166-
(older versions of the code compiled without MPI also used this format
167-
to name outputs). When passed a name of this form, the function returns
168-
``("root.h5", "")``
169-
170-
Notes
171-
-----
172-
The following examples illustrate how the behavior differs from that of
173-
``os.path.splitext``
174-
175-
- For "root.h5.3": ``os.path.splitext`` returns ``("root.h5", ".3")``,
176-
while this function returns ``("root.h5", "3")``
177-
- For "root.h5": os.path.splitext`` returns ``("root", ".h5")``, while this
178-
function returns ``("root.h5", "")``
152+
1. "root.h5.{blockid}" is the standard format Cholla uses when writing
153+
files storing a single snapshot. Each MPI-rank will write a separate
154+
file and replace ``{blockid}`` with MPI-rank (Modern Cholla versions
155+
without MPI replace ``{blockid}`` with ``0``)
156+
2. "root.h5": is the standard format used by Cholla's concatenation
157+
scripts (older versions of Cholla without MPI also used this format
158+
to name outputs)
159+
160+
Returns
161+
-------
162+
template: str
163+
The path to the file containing a blockid is given by calling
164+
``template.format(blockid=<blockid>)``. (This will work whether
165+
all blocks are stored in 1 file or blocks are distributed across
166+
files)
167+
cur_blockid_suffix: int or None
168+
The blockid specified in the suffix of ``filename``. If there isn't a
169+
suffix, then this will be None.
179170
"""
180171

181172
# at this time, we expect the suffix to be the minimum number of characters
182173
# that are necessary to represent the process id. For flexibility, we will
183174
# allow extra zero-padding
184175

185-
match filename.rpartition("."):
186-
case ("", "", _) | (_, ".", ""):
187-
return (filename, "")
188-
case (prefix, ".", _) if prefix == "" or prefix[-1] == "/":
176+
_dir, _base = os.path.split(filename)
177+
match _base.rpartition("."):
178+
case ("", ".", _): # Cholla never writes a file like this
189179
raise ValueError(
190-
f"can't split a process-suffix off of {filename!r} "
191-
"since the remaining filename would be empty"
180+
f"1st character in {filename!r} is the only '.' in the file's name"
192181
)
193-
case (prefix, ".", suffix):
194-
return (prefix, suffix) if suffix.isdecimal() else (filename, "")
182+
case (prefix, ".", suffix) if suffix.isdecimal():
183+
return os.path.join(_dir, f"{prefix}.{{blockid}}"), int(suffix)
184+
case _:
185+
return (filename, None)
195186

196187

197188
class ChollaGrid(AMRGridPatch):

0 commit comments

Comments
 (0)