Skip to content

Abstract reader/writer design #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
be22083
added objid functions
jreadey Feb 4, 2025
28dcfc6
fix flake8 errors
jreadey Feb 4, 2025
54c83d5
merge hsds hdf5dtype changes
jreadey Feb 8, 2025
3a2b084
patch flake8 error
jreadey Feb 8, 2025
133e962
patch flake8 error
jreadey Feb 8, 2025
856ee65
keep backward compatibility for enum members key
jreadey Feb 9, 2025
eec4efc
first pass at abstrct db class
jreadey Feb 12, 2025
2f546b9
first pass at h5py reader
jreadey Feb 18, 2025
4b9cb68
added h5json_writer
jreadey Feb 23, 2025
bad4012
create reader and writer packages
jreadey Feb 23, 2025
c5c28a4
basic dataset read/write methods added
jreadey Feb 26, 2025
c0a6cc3
update h5tojson script
jreadey Feb 26, 2025
48d43e4
added h5json read
jreadey Feb 27, 2025
06b5a6f
added h5py writer
jreadey Feb 27, 2025
8fceb5f
added filters.py
jreadey Feb 27, 2025
af4d46a
updates for h5py_writer to write dataset values
jreadey Mar 4, 2025
7c393b6
revert to using members for dtype enums
jreadey Mar 5, 2025
825fc89
add support for reference types
jreadey Mar 7, 2025
88fa1eb
support for h5py and json readers and writers
jreadey Mar 25, 2025
541b966
fix for vlen encoding
jreadey Mar 28, 2025
398e2d3
fix for reference types
jreadey Apr 2, 2025
9978c45
fix flake8 errors
jreadey Apr 2, 2025
436d921
fix flake8 error
jreadey Apr 2, 2025
51063f6
update testall script
jreadey Apr 2, 2025
d14599a
fix flake8 error
jreadey Apr 2, 2025
e4be33c
make tmp dir in testall
jreadey Apr 2, 2025
8af6508
fix for h5json writer on windows
jreadey Apr 2, 2025
d519d8b
require python >= 3.9
jreadey Apr 2, 2025
4169d5c
remove redundant stripId function
jreadey Apr 3, 2025
7840ca4
add test for incremental updates
jreadey Apr 3, 2025
deb501f
fix flake8 errors
jreadey Apr 3, 2025
1bf10b1
added dset writes to h5py_writer test
jreadey Apr 3, 2025
bfd6cdd
fix for array types
jreadey Apr 10, 2025
d1e2b39
fix for scalar json to arr conversion
jreadey Apr 16, 2025
c6d77f8
support jsontoarray for all byte strings
jreadey Apr 17, 2025
cb3419a
fix errors in jsonToArray function
jreadey Apr 18, 2025
1513334
added extra jsonToArray test
jreadey Apr 21, 2025
289bacb
support setting single element compounds with a list
jreadey Apr 21, 2025
135d88f
handle assigning sequence to multi-dim array
jreadey Apr 21, 2025
13ea473
clean up debug print messages
jreadey Apr 21, 2025
3b87203
fix jsonToArray for single element compoound values
jreadey May 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
python-version: ["3.9", "3.10", "3.11", "3.12"]
runs-on: ${{ matrix.os }}

steps:
Expand Down
Binary file modified data/hdf5/dset_creationprop.h5
Binary file not shown.
34 changes: 0 additions & 34 deletions data/json/nullspace_dset.json

This file was deleted.

13 changes: 7 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,27 +9,26 @@ classifiers = [
"Topic :: Software Development :: Build Tools",
"License :: OSI Approved :: BSD License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
]
authors = [{ "name" = "The HDF Group", "email" = "[email protected]" }]
keywords = ["json", "hdf5", "multidimensional array", "data", "datacube"]
requires-python = ">=3.8"
requires-python = ">=3.9"
dependencies = [
"h5py >=3.10",
"h5py >= 3.10",
"numpy >= 2.0; python_version>='3.9'",
"jsonschema >=4.4.0",
"tomli; python_version<'3.11'",
"numpy >=1.20,<2.0.0; python_version=='3.8'",
]

dynamic = ["version"]

[project.urls]
Homepage = "https://hdf5-json.readthedocs.io"
Documentation = "https://hdf5-json.readthedocs.io"
Homepage = "https://support.hdfgroup.org/documentation/hdf5-json/latest/"
Documentation = "https://support.hdfgroup.org/documentation/hdf5-json/latest/"
Source = "https://github.com/HDFGroup/hdf5-json"
"Bug Reports" = "https://github.com/HDFGroup/hdf5-json/issues"
Social = "https://twitter.com/hdf5"
Expand All @@ -52,6 +51,8 @@ build-backend = "setuptools.build_meta"
package-dir = { "" = "src" }
packages = [
"h5json",
"h5json.reader",
"h5json.writer",
"h5json.h5tojson",
"h5json.jsontoh5",
"h5json.schema",
Expand Down
6 changes: 6 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[flake8]
max-line-length = 120
# E402: module level import not at top of file
# C901: too complex
# F401: unused exports are necessary in __init__.py
ignore = E402, C901, F401
8 changes: 8 additions & 0 deletions src/h5json/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@
from .hdf5dtype import getTypeResponse
from .hdf5dtype import getItemSize
from .hdf5dtype import createDataType
from .objid import createObjId
from .objid import getCollectionForId
from .objid import isObjId
from .objid import isS3ObjKey
from .objid import getS3Key
from .objid import getObjId
from .objid import isSchema2Id
from .objid import isRootObjId
from .hdf5db import Hdf5db
from . import _version

Expand Down
Loading