HerdDB depends on two artifacts that are not on Maven Central:
- jvector — the
eolivelli/jvectorfork (mainbranch). Built on the default JDK (currently 25). - BookKeeper 4.17.4-SNAPSHOT — the
eolivelli/bookkeeperfork on branchfix-concurrentlonghashmap-record-4.17, carrying a temporary fix for thePendingAddOpleak /ConcurrentLongHashMaprecord issue tracked in issue #435. Thecirce-checksummodule bundles a Lombok 1.18.30 annotation processor that does not run on JDK 25 (TypeTag :: UNKNOWN), so this build must use JDK 17. CI follows the same pattern (see.github/workflows/ci.yml).
Bootstrap them once into your ~/.m2/repository — they live there permanently
and can be reused for every PR / agent run:
# jvector — default JDK is fine
git clone https://github.com/eolivelli/jvector ~/dev/jvector
mvn -f ~/dev/jvector/pom.xml -DskipTests -Drat.skip=true clean install
# BookKeeper — must build on JDK 17
git clone --branch fix-concurrentlonghashmap-record-4.17 \
https://github.com/eolivelli/bookkeeper ~/dev/bookkeeper
JAVA_HOME=$(sdk home java 17.0.19-tem) PATH=$JAVA_HOME/bin:$PATH \
mvn -f ~/dev/bookkeeper/pom.xml -DskipTests \
-Drat.skip=true -Dspotbugs.skip=true \
-Dcheckstyle.skip=true -Dlicense.skip=true \
clean install
After bootstrap, the artifacts you care about are under:
~/.m2/repository/io/github/jbellis/(jvector)~/.m2/repository/org/apache/bookkeeper/(incl.bookkeeper-server,bookkeeper-common,bookkeeper-stats-api,circe-checksum, all 4.17.4-SNAPSHOT)
For agent workflows that use an isolated Maven repo
(-Dmaven.repo.local=$MAVEN_REPO), do not re-clone and re-build these
dependencies on every run. Instead, copy the relevant subtrees from
~/.m2/repository into $MAVEN_REPO once at the start of the run:
mkdir -p "$MAVEN_REPO/io/github" "$MAVEN_REPO/org/apache"
cp -r ~/.m2/repository/io/github/jbellis "$MAVEN_REPO/io/github/"
cp -r ~/.m2/repository/org/apache/bookkeeper "$MAVEN_REPO/org/apache/"
Re-run the bootstrap commands above whenever the upstream eolivelli/jvector
or eolivelli/bookkeeper branch heads move.
Never push commits directly to the master branch. Always create a feature branch and
open a pull request so that CI runs and changes can be reviewed before merging.
Spotless replaced Maven Checkstyle in issue #544 / PR #554 — do not invoke
mvn checkstyle:check directly; with no project-level configuration the
plugin falls back to the Sun defaults and reports thousands of bogus
violations. Use Spotless instead.
First auto-fix formatting (unused imports, trailing whitespace, trailing newline, tabs → spaces) with Spotless, then run the code validation checks locally before opening a pull request:
mvn -B spotless:apply -DskipTests
mvn -B spotless:check apache-rat:check install -DskipTests spotbugs:check -Pci
This matches what .github/workflows/ci.yml runs in CI (spotless, Apache RAT
license headers, SpotBugs). The goal order is significant: spotbugs:check
must run after install because it analyses the freshly-compiled .class
files that install writes — running spotbugs first leaves it inspecting
stale classes (or no classes at all on a clean checkout).
For any change related to indexes, checkpoints, or concurrency, also run every
subclass of DirectMultipleConcurrentUpdatesSuite (in
herddb-core/src/test/java/herddb/server/hammer/), the BLink-level
concurrent search/insert hammer (BLinkConcurrentSearchInsertTest in
herddb-utils/src/test/java/herddb/index/blink/), and — for any change
that touches the lazy data-page read path or the value cache — every
subclass of LazyValueCacheConcurrentUpdatesSuite (in
herddb-remote-file-service/src/test/java/herddb/remote/hammer/); make
sure they all pass:
mvn -pl herddb-core -Dtest='DirectMultipleConcurrentUpdatesSuiteNoIndexesTest,DirectMultipleConcurrentUpdatesSuiteWithNonUniqueIndexesTest,DirectMultipleConcurrentUpdatesSuiteWithUniqueIndexesTest' test
mvn -pl herddb-utils -Dtest='BLinkConcurrentSearchInsertTest' test
mvn -pl herddb-remote-file-service -Dtest='LazyValueCacheConcurrentUpdatesSuiteNoIndexesTest,LazyValueCacheConcurrentUpdatesSuiteWithNonUniqueIndexesTest,LazyValueCacheConcurrentUpdatesSuiteWithUniqueIndexesTest' test
The DirectMultipleConcurrentUpdatesSuite tests exercise concurrent DML with
periodic checkpoints and recovery against the file DSM, so they are the main
regression gate for the primary-key index, the checkpoint pipeline, and
transaction isolation. BLinkConcurrentSearchInsertTest hammers the
lower-level BLink with many concurrent searches against fewer concurrent
writers on overlapping keys, catching regressions in the per-node and anchor
lock primitives that drive every primary-key lookup.
LazyValueCacheConcurrentUpdatesSuite mirrors the file-DSM hammer's workload
shape (insert + concurrent UPDATE/GET + post-restart consistency check) but
routes every read through RemoteFileDataStorageManager + LazyValueCache,
so it is the regression gate for the lazy data-page read path, the value
cache's refcount discipline, and post-restart recovery against remote
storage. Run them multiple times when the first pass is green to reduce the
chance of a flake masking a real bug.
Tests that require ZooKeeper/BookKeeper infrastructure (cluster mode) must be annotated with
@Category(ClusterTest.class) (import herddb.core.ClusterTest and org.junit.experimental.categories.Category).
This includes any test that:
- Uses
ZKTestEnv - Extends
ReplicatedLogtestcase,MultiServerBase, orBookkeeperFailuresBase - Starts multiple
Serverinstances with ZooKeeper coordination
CI runs cluster and core tests in separate workflows. Forgetting the annotation means the test will only run in the core workflow and will likely fail there due to missing infrastructure.
Always cover code changes with unit or integration tests. Run the new tests locally and ensure they pass before opening a PR. Also run the existing tests related to the code touched by the current work to catch regressions.
Never run the full herddb-core test suite — it takes a very long time. Instead, run single
tests or small batches using Maven's -Dtest=... selector, for example:
mvn -pl herddb-core -Dtest=MyChangedTest test
mvn -pl herddb-core -Dtest='Foo*Test,BarTest#someMethod' test
Never catch Throwable or bare Exception unless strictly necessary. Catch the
narrowest exception type(s) the code in the try block can actually throw. If a broad
catch is unavoidable (e.g. isolating a plugin boundary, preventing a background thread
from dying, top-level request handlers), add a comment explaining why the broad catch
is required.
VECTOR.md at the repository root is the operator-facing contract for the
vector search subsystem — architecture, configuration keys, compaction
pipeline, optimizer-pod protocol, on-disk format, recovery semantics.
Whenever a change touches the indexing service, PersistentVectorStore,
VectorIndexCompactor, RemoteSegmentGraphMerger, the
index-optimizer pod, the Remote File Service vector path, or any
operator-visible configuration key, update VECTOR.md in the same PR
so it continues to describe the code as it stands today. The file is
written as a present-tense reference: state what the code does now, not
what it used to do — do not add change-history bullets, do not link to
GitHub issues or PRs, and avoid phrases like "removed", "legacy", or
"used to". If a behaviour is being replaced, rewrite the affected
paragraphs in place so they describe only the current behaviour.
Monitoring and supervision guidance for the k3s-local vector benchmark lives in the
herddb-k3s-bench agent definition at .claude/agents/herddb-k3s-bench.md.