From 6ea3f68657319c97691bfd07aebd23b262e20421 Mon Sep 17 00:00:00 2001
From: nhz2
Date: Tue, 22 Apr 2025 17:56:02 -0400
Subject: [PATCH] Add docs on how to run tests and add format table
---
.gitignore | 3 ++-
CONTRIBUTING.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++
README.md | 38 ++++++++++++++----------------
3 files changed, 80 insertions(+), 22 deletions(-)
create mode 100644 CONTRIBUTING.md
diff --git a/.gitignore b/.gitignore
index 2bab4b9..462bd1f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -8,4 +8,5 @@ Manifest.toml
fixture.tar.gz
fixture
.CondaPkg
-*.cov
\ No newline at end of file
+*.cov
+/temp/
\ No newline at end of file
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
new file mode 100644
index 0000000..bcc7600
--- /dev/null
+++ b/CONTRIBUTING.md
@@ -0,0 +1,61 @@
+# ChunkCodecs Dev Notes
+
+## Repository directories
+
+This monorepo holds a number of different Julia packages:
+
+- `ChunkCodecCore`: defines the interface.
+- `ChunkCodecTests`: defines tests for the interface.
+- `test`: contains slower compatibility tests.
+
+There are also a number of packages with glue code to implement the interface for various C libraries and formats.
+
+## Running basic tests
+
+First install Julia 1.12 or later from .
+
+Then instantiate the workspace:
+
+```sh
+julia --project=. -e 'import Pkg; Pkg.update()'
+```
+
+Each package contains basic tests in its `"test"` sub directory.
+
+For example here is how to run the tests for `LibZlib`.
+
+Run test script:
+
+```sh
+julia --project=LibZlib/test LibZlib/test/runtests.jl
+```
+
+If you are on a machine with more than 24 GB of RAM you can also run:
+
+```sh
+julia --project=LibZlib/test --heap-size-hint=15G LibZlib/test/big-mem-tests.jl
+```
+
+This will use local versions of other packages in this repo.
+
+To test the package as it would be when installed:
+copy the package to a temporary directory before testing.
+
+```sh
+julia -e 'mkdir("temp"); cp("LibZlib", "temp/LibZlib")'
+julia --project=temp/LibZlib -e 'import Pkg; Pkg.update()'
+julia --project=temp/LibZlib/test temp/LibZlib/test/runtests.jl
+```
+
+## Running compatibility tests
+
+The main `"test"` directory contains more tests.
+These tests have complex dependencies and are more fragile.
+
+For example to run
+[imagecodecs](https://github.com/cgohlke/imagecodecs) compatibility tests.
+
+```sh
+julia --project=test -e 'import Pkg; Pkg.update()'
+julia --project=test test/imagecodecs-compat.jl
+```
diff --git a/README.md b/README.md
index a2578d2..5418479 100644
--- a/README.md
+++ b/README.md
@@ -3,30 +3,26 @@
[](https://github.com/JuliaIO/ChunkCodecs.jl/actions/workflows/CI.yml)
[](https://github.com/JuliaTesting/Aqua.jl)
-## Warning: ChunkCodecs is currently a WIP and its API may drastically change at any time.
-## Warning: There is not enough compatibility testing, so do not save important data using this package yet.
+## Warning: ChunkCodecs is currently a WIP. Suggestions for major API changes are welcome.
A consistent Julia interface for lossless encoding and decoding of bytes in memory.
-## Repository directories
-
-This monorepo holds a number of different Julia packages:
-
-- `ChunkCodecCore`: defines the interface.
-- `ChunkCodecTests`: defines tests for the interface.
-
-There are also a number of packages with glue code to implement the interface for various C libraries.
-
-- `LibBlosc`
-- `LibBzip2`
-- `LibLz4`
-- `LibSnappy`
-- `LibZlib`
-- `LibZstd`
-
-Each package contains basic tests.
-
-There is also a `test` directory for slower compatibility tests.
+## Available Formats
+
+| Name | Other Names | Package | Encoding | Decoding |
+|---|---|---|---|---|
+| Zstd | .zst RFC8878 | ChunkCodecLibZstd | ✅ | ✅ |
+| Zlib | RFC1950 | ChunkCodecLibZlib | ✅ | ✅ |
+| Snappy | | ChunkCodecLibSnappy | ✅ | ✅ |
+| Shuffle | | ChunkCodecCore | ✅ | ✅ |
+| Noop | | ChunkCodecCore | ✅ | ✅ |
+| LZ4Numcodecs | | ChunkCodecLibLz4 | ✅ | ✅ |
+| LZ4Frame | .lz4 | ChunkCodecLibLz4 | ✅ | ✅ |
+| LZ4Block | | ChunkCodecLibLz4 | ✅ | ✅ |
+| Gzip | .gz RFC1952 | ChunkCodecLibZlib | ✅ | ✅ |
+| Deflate | RFC1951 | ChunkCodecLibZlib | ✅ | ✅ |
+| BZ2 | .bz2 bzip2 | ChunkCodecLibBzip2 | ✅ | ✅ |
+| Blosc | | ChunkCodecLibBlosc | ✅ | ✅ |
## Simple encoding and decoding