v1.4.0-RC1

Changelog

[v1.4.0-RC1] - 2022-10-04

Bug Fixes

• Makefile: add fips build flag to test/fips
• Makefile: remove interactive flag from dist/fips command
• ci: fix regex pattern for fips binaries
• cmd/immuadmin: set correct data-type for replication-sync-followers flag
• embedded/store: fix size calculation of precommitted txs
• embedded/store: Fix checking for closed store when syncing TXs
• embedded/store: avoid attempts to commit in wrong order
• embedded/store: expose durable precommitted state
• embedded/store: include allowPrecommitted into tx reader construction
• embedded/store: ensure tx is released upon error
• embedded/store: aht up to precommited tx
• github: Update github actions after migration of Dockerfile's
• pkg/database: return master commit state if failing to read follower precommitted one
• pkg/database: Fix mutex lock in ExportTx
• pkg/database: set follower states holder when changing replication status
• pkg/server: add logs when replicator does not start

Changes

• add dependabot config
• Add empty line between license header and package
• Dockerfile.fips: add fips build changes
• cmd/immuadmin: add new replication flags
• cmd/immuadmin: revert default replication-master-port
• cmd/immuadmin: use default immudb port as default value for replication-master-port flag
• cmd/immuclient: flag replication-sync-enabled to enable sync replication
• cmd/immudb: deprecate replication-enabled towards replication-is-replica
• docker: Move main Dockerfile's to build folder
• docker: Simplify the main Dockerfile
• embedded/store: resolve pre-committed using clogbuf
• embedded/store: wip reduce allocations in exportTx
• embedded/store: mutexless export-tx
• embedded/store: enhanced tx discarding logic
• embedded/store: wip load precommitted txs
• embedded/store: method to dynamically switch to external allowance
• embedded/store: wip wait for precommitted txs
• embedded/store: explicit allowPrecommitted and restricted access to precommitted txs
• embedded/store: minor code simplification
• embedded/store: possibility to read tx header of precommitted txs
• embedded/store: support for concurrent replicated precommits
• embedded/store: tx parsing with sanity checks
• embedded/store: handle commit case when there is nothing new to commit
• embedded/store: tolerate partial data or inconsistencies when loading pre-committed txs
• embedded/store: explanatory comments added
• embedded/store: waits for durable precommitted txs
• embedded/store: minor renaming and comment additions
• embedded/store: add integrity checks when reading precommitted txs
• pkg/api: currentState endpoint includes precommitted info
• pkg/api: explicit sync replication setting
• pkg/api/schema: reformat schema.proto file
• pkg/database: minor typo in comment
• pkg/database: sync exportTx
• pkg/database: improve error comparison
• pkg/database: follower commit progress without additional waits
• pkg/database: handle special case related to sql initialization
• pkg/database: disable automatic sql init on older databases
• pkg/integration: add synchronous replication integration tests
• pkg/replication: allowPreCommitted only with sync replication enabled
• pkg/replication: speed up follower reconnection
• pkg/replication: use session-based authentication
• pkg/replication: handling a particular case in an optimized manner
• pkg/replication: backward compatible replication
• pkg/replication: check committedTxID from master
• pkg/replication: wip optimize concurrency in replicators
• pkg/replication: handle case when follower precommit state is up-to-date but commit state is lies behind
• pkg/replication: sync replication using follower state
• pkg/replication: configurable prefetchTxBufferSize and replicationCommitConcurrency
• pkg/replication: improve error comparison
• pkg/replication: graceful closing
• pkg/replication: further progress in sync replication
• pkg/replication: replicator with backward compatibility mode
• pkg/replicator: wip precommitted tx discarding when follower diverged from master
• pkg/server: explicit sync replication
• pkg/server: handle admin user creation with sync replication enabled
• pkg/server: support for systemdb with session-based auth
• pkg/server: display all replication settings
• pkg/server: include sync replication settings in options
• pkg/server: use replication settings

Features

• cmd/immuadmin: flag to set the number of sync followers
• cmd/immudb: flag to set the number of sync followers for systemdb and defaultdb
• embedded/store: functionality to discard precommitted txs
• embedded/store: core support for sync replication
• pkg/api: api extensions to support sync replication
• pkg/database: wip sync replication logic
• pkg/replication: mode to allow tx discarding on followers
• pkg/replication: wip replicator with support for sync replication
• pkg/server: sync replication logic
• pkg/server: Add ability to inject custom database management object

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

immudb Binaries

File	SHA256
immudb-v1.4.0-RC1-darwin-amd64	e2133e437897911b48ba4b7ec40c5a7a31b10522ce36f4f971d1964576642a59
immudb-v1.4.0-RC1-darwin-arm64	a036f56e5465f5337edb56b45cff5c5ad23a65bb2d46efea536eeda43f9bc856
immudb-v1.4.0-RC1-freebsd-amd64	6eabed369b715bc6f1b5c0b3656a5b72878d664214d65ca2071e2240fbcbc235
immudb-v1.4.0-RC1-linux-amd64	ac2543dc3952abc96199f40aa805ecfac335f7a76788a02e641bed111b35280c
immudb-v1.4.0-RC1-linux-amd64-fips	a7842f83df2637118696c8c248977a465413696fa704631d2f1ab355250d307b
immudb-v1.4.0-RC1-linux-amd64-static	bc03f2235255311fd52c1a69c99864f7db31a792a6a312c74df9991ac8f05a91
immudb-v1.4.0-RC1-linux-arm64	e548ef5a4976e1925a08c6401688641af1c7da67dce2c401908e921a0d06169a
immudb-v1.4.0-RC1-linux-s390x	04c9b9820e79f52b757fd8ff462e9d65f785bb781a92bfd04c44c7cb2b94f07f
immudb-v1.4.0-RC1-windows-amd64.exe	fecba3d6894bdb341de5b0e3c826ddd7cb390026b2e903935c3266d6f4beaa50

immuclient Binaries

File	SHA256
immuclient-v1.4.0-RC1-darwin-amd64	ae8977e1b07b923649b0cca1a20680e0501a04605daaf5c64dd9dc2b50478932
immuclient-v1.4.0-RC1-darwin-arm64	ca39fd74a0818719908a7f3c46e8bb19ffc93678c953ae573d1d03b40b1a4965
immuclient-v1.4.0-RC1-freebsd-amd64	c71787595d2da1ead1d6710d31a7859d2b5ef08efe7a9158c3933a1f3d1c4a60
immuclient-v1.4.0-RC1-linux-amd64	295f2457a3e271bdee1db1dc0b8349355661eeddece932c7bd22b44c132c4b88
immuclient-v1.4.0-RC1-linux-amd64-fips	f48de7b92695437bd3f7dc7e7ab8589b0dd4e96e4f0877c7e87770349c2d57bf
immuclient-v1.4.0-RC1-linux-amd64-static	f3c6f4c08244e8e84bbefcb5f3e053a8612fa63c820015493bca87489773f765
immuclient-v1.4.0-RC1-linux-arm64	695f9c7f9ec5a147265b4489ae2e2282b144338a383d779037c58a8efed380fe
immuclient-v1.4.0-RC1-linux-s390x	4d7f0ff5eade6c0e29adf1467007481c14ed53ab8b606191244f5266683d1483
immuclient-v1.4.0-RC1-windows-amd64.exe	851494f9e7b64356dce80220d41dbdd54954183b46430e77fd03779b60cd49e0

immuadmin Binaries

File	SHA256
immuadmin-v1.4.0-RC1-darwin-amd64	4352e28868ac037b4e78478862a20460c88976b6d35562cea1c7e9df19d3810f
immuadmin-v1.4.0-RC1-darwin-arm64	245b9644d1c74b24c612c651b644f87c182946b21877b73631513f8b8d250ac3
immuadmin-v1.4.0-RC1-freebsd-amd64	85c15624f3399570c9254e3e575fbcbdf62...

v1.3.2

Release notes

Performance was the main focus of this immudb release. The way durability guarantees were implemented until now i.e. fsync was called synchronously as the final commit step, resulted in a lot of overhead for each transaction.

A significant improvement in write performance (up to x10 faster compared to previous immudb release v1.3.1) was achieved by grouping transactions in the last step of the commit process. The gains are greatest when there are fewer key-value entries per transaction and multiple writers are simultaneously working. The cost of the sync operation pays for itself as the number of key-value pairs in each transaction increases, although some noticeable speed can still be achieved (up to x2 compared to previous immudb release v1.3.1).

Read performance was also significantly improved due to the introduction of reusable pools of objects, which reduced allocations needed during query resolution. The gains are quite significant in both single and multi-get operations.

Benchmarking

Stay tuned, benchmarks done using stresser2 tool (https://github.com/codenotary/immudb-tools) will be available soon, we're still collecting some results.

JSON Logging

Logs can also be generated in json format for easier ingestion by logging platforms.
It's quite simple to enable this useful feature, just run immudb with the logformat flag set to json e.g. ./immudb --logformat=json

Go 1.18

immudb binaries and Docker images are now built with Go 1.18.

Changelog

[v1.3.2] - 2022-08-25

Bug Fixes

• access tls value in global scope within ingress annotations
• company name in webconsole and other files
• build: Fix go-acc and goveralls invocations
• build: update go version to 1.18 in Dockerfiles
• build/RELEASING.md: Add note about updating playground
• embedded: use tmp folder for unit test cases
• embedded/sql: Support single BEGIN statement.
• embedded/store: Check precommitted state when replicating
• embedded/store: Optionally preallocate Tx pools
• embedded/store: Ensure ordering of transaction timestamps
• embedded/store: Assign blTxID within locked tx state
• embedded/store: ensure tx is released upon error
• embedded/store: Improved check for replicated transaction
• embedded/store: Return correct error on key length exceeded
• embedded/store: Protect against simultaneous replicators
• embedded/store: Reduce the amount of allocations for tx object
• embedded/tools/stress_tool: Fix compilation after recent update to tx holder pool
• getRandomTable: increase RNG range for table generation
• github: Remove unnecessary /test/ path when uploading perf results to s3
• github: Do not use yaml anchors in github workflows
• pkg/client: Invalid client state after connection refused
• pkg/client/clienttest: enforce mock client to interface
• pkg/database: Fix calculation of proof for VerifiableTxByID
• pkg/database: Correct revision for Scan requirests
• server: Show info text with a logger
• servertest: Allow accessing Server object before starting the server
• stdlib/rows: add colums to row response
• test/performance: Cleanup test directory

Changes

• pin google.golang.org/protobuf to v1.27.1 (currently used version for generated code).
• Introduce separate TxHolder pools
• update github.com/spf13/viper to v1.12.0.
• makefile formatting.
• update build constraint to new & future-proof syntax.
• format tools.go.
• deprecate ImmuClient.HealthCheck in favour of ServerInfo.
• reimplement ImmuClient.HealthCheck using rpc ServerInfo instead of (deprecated) Health.
• refactor TestServerInfo.
• Update main go versin to 1.18
• generate gRPC stubs.
• ignore schema_grpc.pb.go in coveralls.
• use go.mod version of github.com/grpc-ecosystem/grpc-gateway when building codegen.
• regenerate with correct version of protoc-gen-go.
• ignore schema_grpc.pb.go in code coverage.
• pin github.com/pseudomuto/protoc-gen-doc to 1.4.1 (currently used version for generated code).
• Makefile: Update webconsole to 1.0.16
• build: Update RELEASING.md doc
• build: Improve generation of build checksums
• cmd/immuadmin: Add support for max-commit-concurrency option
• cmd/immuadmin: Add support for read-tx-pool-size option
• cmd/immudb: Add support for max-sessions command line option
• database/sql: Delay txholder allocation on VerifiableSQLGet
• embedded/ahtree: threshold-based sync
• embedded/ahtree: use bigger default write buffer size
• embedded/ahtree: improve error handling
• embedded/ahtree: flushless append
• embedded/ahtree: improve validations and error handling
• embedded/ahtree: support newst appendable implementation
• embedded/ahtree: improve error message consistency
• embedded/ahtree: minor error message change
• embedded/appendable: autosync when write buffer is full
• embedded/appendable: autosync support in multi-appendable
• embedded/appendable: flush when no more writes are done in appendable
• embedded/appendable: improve explanatory comment inside sync method
• embedded/appendable: improve singleapp validation and error handling
• embedded/appendable: error tolerant seek
• embedded/appendable: improve validations and error handling
• embedded/appendable: inmem buffer offset
• embedded/appendable: wip remoteapp validation
• embedded/appendable: return io.EOF when offset is out of range
• embedded/appendable: upgrade mocked and remote appendable based on new flushing assumptions
• embedded/appendable: auto-sync options
• embedded/appendable: multi-appendable shared write buffer
• embedded/htree: improve error handling
• embedded/sql: Remove unnecessary tx holder buffer from SQLTx
• embedded/store: Better errors returned during replication error
• embedded/store: aht options
• embedded/store: Use dedicated error for replication conflicts
• embedded/store: sync AHT before tx commit log
• embedded/store: in-mem clog buffer written when synced
• embedded/store: wrap internal already closed errors
• embedded/store: handle appendable already close error
• embedded/store: Optimize ReadTxHeader method
• embedded/store: Do not write values if concurrency limit is reached
• embedded/store: Add dedicated error for tx pool exhaustion
• embedded/store: add TODO comment
• embedded/store: parametrize write buffer size
• embedded/store: wip retryable sync
• embedded/store: flush-less precommit
• embedded/store: multi-tx syncs
• embedded/store: use smaller default buffer size
• embedded/store: Add txDataReader to process transaction data
• embedded/store: avoid sync waiting if there are no new transactions
• embedded/store: improve error comparison with errors.Is(...)
• embedded/store: wip error declaration
• embedded/store: Add explicit ReadTxEntry method
• embedded/store: Add explicit ReadTxHeader
• embedded/store: Optimize ReadTxEntry method
• embedded/store: Add txPoolOptions to setup pool parameters upon creation
• embedded/store: set new default write buffer values
• embedded/store/txpool: Allocate pool entries separately
• embedded/store/txpool: Make txPoolOptions members private
• embedded/tbtree: improve error handling
• embedded/tbtree: use non-retryable sync
• embedded/tbtree: define using generic errors towards errors.Is(...) usage
• embedded/watchers: improve error handling
• github: Update ACTIONS_SECRETS.md file
• github: Allow selection of runner to run perf test
• github: Run perf test suite on pull requests
• github: Run performance test suite on push to master
• github: Add simple documentation of PERF_TEST_xxx secrets
• github: Install qemu using docker/setup-qemu-action
• github: Allow using multiple runners for perf test suite
• github: Upload perf results to AWS s3
• pkg/api: Add tx pool size to GRPC and stored db options
• pkg/api: milliseconds message type
• pkg/api: expose aht settings
• pkg/api: export syncFrequency database parameter
• pkg/api: deprecate rpc Health in favour of ServerInfo.
• pkg/database: Add tx pool size to db options
• pkg/database: Remove txHolder from get operation
• pkg/database: Do not allocate txholder for history scans
• pkg/database: allocate tx buffer before doing verified writes
• pkg/logger: Add memory logger
• pkg/logger: add json logger
• pkg/server: simplify ImmuServer.Health.
• pkg/server: Add pprof option
• test/performance: Add basic flags to the benchmark process
• test/performance: Add separate Write KV/s test.
• test/performance: Split benchmark list and run code
• test/performance: Move test seed out of configuration
• test/performance: Move random generator and key tracker to common coode
• test/performance: Add CPU time / memory stats gathering
• test/performance: Allow customized name for the benchmark
• test/performance: Add basic IO stats
• test/performance: Improve live IO display
• test/performance: Better logging and output
• test/performance: Correctly close random data generator

Features

• revert usages of ServerInfo that would break backwards compatibility.
• add test for HealthCheck.
• cmd/immuadmin: expose syncFrequency and WriteBufferSize db parameters
• cmd/immuclient: add info command to immuclient.
• pkg/api: expose write buffer parameter
• pkg/api: improve documentation of ServerInfo.
• pkg/api: remove ServerInfoResponse.status field.
• pkg/api: add ServerInfo rpc to deprecate Health.
• pkg/client: revert WaitForHealthCheck change to...

v1.3.2-RC1

Changelog

[v1.3.2-RC1] - 2022-08-24

Bug Fixes

• access tls value in global scope within ingress annotations
• company name in webconsole and other files
• build: Fix go-acc and goveralls invocations
• build: update go version to 1.18 in Dockerfiles
• build/RELEASING.md: Add note about updating playground
• embedded: use tmp folder for unit test cases
• embedded/sql: Support single BEGIN statement.
• embedded/store: Check precommitted state when replicating
• embedded/store: Optionally preallocate Tx pools
• embedded/store: Ensure ordering of transaction timestamps
• embedded/store: Assign blTxID within locked tx state
• embedded/store: ensure tx is released upon error
• embedded/store: Improved check for replicated transaction
• embedded/store: Return correct error on key length exceeded
• embedded/store: Protect against simultaneous replicators
• embedded/store: Reduce the amount of allocations for tx object
• embedded/tools/stress_tool: Fix compilation after recent update to tx holder pool
• getRandomTable: increase RNG range for table generation
• github: Remove unnecessary /test/ path when uploading perf results to s3
• github: Do not use yaml anchors in github workflows
• pkg/client: Invalid client state after connection refused
• pkg/client/clienttest: enforce mock client to interface
• pkg/database: Fix calculation of proof for VerifiableTxByID
• pkg/database: Correct revision for Scan requirests
• server: Show info text with a logger
• servertest: Allow accessing Server object before starting the server
• stdlib/rows: add colums to row response
• test/performance: Cleanup test directory

Changes

• pin google.golang.org/protobuf to v1.27.1 (currently used version for generated code).
• Introduce separate TxHolder pools
• update github.com/spf13/viper to v1.12.0.
• makefile formatting.
• update build constraint to new & future-proof syntax.
• format tools.go.
• deprecate ImmuClient.HealthCheck in favour of ServerInfo.
• reimplement ImmuClient.HealthCheck using rpc ServerInfo instead of (deprecated) Health.
• refactor TestServerInfo.
• Update main go versin to 1.18
• generate gRPC stubs.
• ignore schema_grpc.pb.go in coveralls.
• use go.mod version of github.com/grpc-ecosystem/grpc-gateway when building codegen.
• regenerate with correct version of protoc-gen-go.
• ignore schema_grpc.pb.go in code coverage.
• pin github.com/pseudomuto/protoc-gen-doc to 1.4.1 (currently used version for generated code).
• Makefile: Update webconsole to 1.0.16
• build: Update RELEASING.md doc
• build: Improve generation of build checksums
• cmd/immuadmin: Add support for max-commit-concurrency option
• cmd/immuadmin: Add support for read-tx-pool-size option
• cmd/immudb: Add support for max-sessions command line option
• database/sql: Delay txholder allocation on VerifiableSQLGet
• embedded/ahtree: threshold-based sync
• embedded/ahtree: use bigger default write buffer size
• embedded/ahtree: improve error handling
• embedded/ahtree: flushless append
• embedded/ahtree: improve validations and error handling
• embedded/ahtree: support newst appendable implementation
• embedded/ahtree: improve error message consistency
• embedded/ahtree: minor error message change
• embedded/appendable: autosync when write buffer is full
• embedded/appendable: autosync support in multi-appendable
• embedded/appendable: flush when no more writes are done in appendable
• embedded/appendable: improve explanatory comment inside sync method
• embedded/appendable: improve singleapp validation and error handling
• embedded/appendable: error tolerant seek
• embedded/appendable: improve validations and error handling
• embedded/appendable: inmem buffer offset
• embedded/appendable: wip remoteapp validation
• embedded/appendable: return io.EOF when offset is out of range
• embedded/appendable: upgrade mocked and remote appendable based on new flushing assumptions
• embedded/appendable: auto-sync options
• embedded/appendable: multi-appendable shared write buffer
• embedded/htree: improve error handling
• embedded/sql: Remove unnecessary tx holder buffer from SQLTx
• embedded/store: Better errors returned during replication error
• embedded/store: aht options
• embedded/store: Use dedicated error for replication conflicts
• embedded/store: sync AHT before tx commit log
• embedded/store: in-mem clog buffer written when synced
• embedded/store: wrap internal already closed errors
• embedded/store: handle appendable already close error
• embedded/store: Optimize ReadTxHeader method
• embedded/store: Do not write values if concurrency limit is reached
• embedded/store: Add dedicated error for tx pool exhaustion
• embedded/store: add TODO comment
• embedded/store: parametrize write buffer size
• embedded/store: wip retryable sync
• embedded/store: flush-less precommit
• embedded/store: multi-tx syncs
• embedded/store: use smaller default buffer size
• embedded/store: Add txDataReader to process transaction data
• embedded/store: avoid sync waiting if there are no new transactions
• embedded/store: improve error comparison with errors.Is(...)
• embedded/store: wip error declaration
• embedded/store: Add explicit ReadTxEntry method
• embedded/store: Add explicit ReadTxHeader
• embedded/store: Optimize ReadTxEntry method
• embedded/store: Add txPoolOptions to setup pool parameters upon creation
• embedded/store: set new default write buffer values
• embedded/store/txpool: Allocate pool entries separately
• embedded/store/txpool: Make txPoolOptions members private
• embedded/tbtree: improve error handling
• embedded/tbtree: use non-retryable sync
• embedded/tbtree: define using generic errors towards errors.Is(...) usage
• embedded/watchers: improve error handling
• github: Update ACTIONS_SECRETS.md file
• github: Allow selection of runner to run perf test
• github: Run perf test suite on pull requests
• github: Run performance test suite on push to master
• github: Add simple documentation of PERF_TEST_xxx secrets
• github: Install qemu using docker/setup-qemu-action
• github: Allow using multiple runners for perf test suite
• github: Upload perf results to AWS s3
• pkg/api: Add tx pool size to GRPC and stored db options
• pkg/api: milliseconds message type
• pkg/api: expose aht settings
• pkg/api: export syncFrequency database parameter
• pkg/api: deprecate rpc Health in favour of ServerInfo.
• pkg/database: Add tx pool size to db options
• pkg/database: Remove txHolder from get operation
• pkg/database: Do not allocate txholder for history scans
• pkg/database: allocate tx buffer before doing verified writes
• pkg/logger: Add memory logger
• pkg/logger: add json logger
• pkg/server: simplify ImmuServer.Health.
• pkg/server: Add pprof option
• test/performance: Add basic flags to the benchmark process
• test/performance: Add separate Write KV/s test.
• test/performance: Split benchmark list and run code
• test/performance: Move test seed out of configuration
• test/performance: Move random generator and key tracker to common coode
• test/performance: Add CPU time / memory stats gathering
• test/performance: Allow customized name for the benchmark
• test/performance: Add basic IO stats
• test/performance: Improve live IO display
• test/performance: Better logging and output
• test/performance: Correctly close random data generator

Features

• revert usages of ServerInfo that would break backwards compatibility.
• add test for HealthCheck.
• cmd/immuadmin: expose syncFrequency and WriteBufferSize db parameters
• cmd/immuclient: add info command to immuclient.
• pkg/api: expose write buffer parameter
• pkg/api: improve documentation of ServerInfo.
• pkg/api: remove ServerInfoResponse.status field.
• pkg/api: add ServerInfo rpc to deprecate Health.
• pkg/client: revert WaitForHealthCheck change to maintain backwards-compatibility.
• pkg/client: implement ImmuClient.ServerInfo.
• pkg/server: implement ImmuServer.ServerInfo.

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

immudb Binaries

File	SHA256
immudb-v1.3.2-RC1-darwin-amd64	9c8373eb161e27f1e3e887725167bb58c37b9e395b102e50ba52efff1cf1a5f9
immudb-v1.3.2-RC1-darwin-arm64	5017bc5a24776d4f952a53cabe12efe0f9f90fb23fc21fe7f9cd7d6feabfe76e
immudb-v1.3.2-RC1-freebsd-amd64	5925e6ea56bc070904ca1755aba8889e84b4baf225376b22126594fcfd1000b3
immudb-v1.3.2-RC1-linux-amd64	832fd0bbe3ce1d83dca47b814cecdf9161f63362d0e7c1c8601d5bb30eb56101
immudb-v1.3.2-RC1-linux-amd64-static	8eb21972c3d89a7e365da23d254a2fa62bc09b2ef3d1a1b9d44113306129f4cf
immudb-v1.3.2-RC1-linux-arm64	cb8e63b556b59dbeac822a5f3d190077a68bacfce46fe166256767e7eced0e79
[immudb-v1.3.2-RC1-linux-...

v1.3.1

Release notes

Let us introduce another immudb release: 1.3.1. This time we focused mostly on stability and bug fixing, however some small features still managed to slip through into our changelog.

Reworked documentation structure

The documentation hosted on https://docs.immudb.io/ gained a brand new menu structure making it much cleaner for newcomers and much easier to navigate by more experienced immudb users. Make sure to check it out and remember that your feedback here is always very welcome.

S3 compatibility

One of the most important issues we've started observing was the incompatibility with various S3 storage implementations. Based on our internal tests and community feedback we managed to greatly improve support 3rd party S3 service providers such as elest.io.

SQL Snapshot leak fixes

When performing read operations on immudb, an internal snapshot of B-tree is created. We found few places (mostly in case of query errors) that would result in snapshots not being freed once the SQL operation is finished. Immudb has internal limit of active open snapshots at a time thus such leaked snapshots were causing significant issues during application development.

Idempotent NOW() function

The NOW() function available in SQL is now idempotent within a single transaction. Before it was returning the exact time when the NOW() function was called. The new implementation guarantees that it always returns the time when the current transaction was started.

Experimental support for OFFSET

The new OFFSET keyword can be used in SQL statements to instruct the server that it should skip some initial set of entries. Similarly, the GRPC methods for scanning KV entries got additional offset parameter. Internally offsets are implemented by skipping initial entries from the result on the server side thus it may come with performance penalty when the value of such offset is large.

Changelog

[v1.3.1] - 2022-06-30

Bug Fixes

• embedded/store: filter evaluation after valRef resolution
• README: Update readme to show examples for 1.3.0 version
• cmd/immuadmin: use StreamChunkSize as max chunk size during tx replication
• cmd/immudb: include metrics endpoint related flags
• embedded/remotestorage: Fix invalid comment
• embedded/remotestorage/s3: Correctly url decode entry names
• embedded/remotestorage/s3: Simplify the code for scan
• embedded/remotestorage/s3: Avoid using HEAD requests
• embedded/remotestorage/s3: Fix s3 object name validation
• embedded/sql: Use defer to cleanup unclosed readers on error
• embedded/sql: Fix snapshot leak on query initialization failure
• embedded/sql: Properly close readers in joint row reader
• embedded/sql: Fix reader leaks during initialization failures
• embedded/sql: Fix snapshot leaks in union readers
• embedded/sql: ensure timestamp is evaluated with microsecond precision
• pkg/client: ensure connection is closed and session can be re-established
• pkg/database: Do not panic if incorrect number of pk values is given to VerifiableSQLGet
• pkg/server: Fix remote storage test after recent changes
• pkg/server/sessions: Avoid deadlock when closing session manager
• pkg/server/sessions: Session manager test fixes
• pkg/server/sessions: Handle short buffer read when generating session id
• pkg/server/sessions: Correctly start session guard
• pkg/server/sessions: Use strong random source for session ID

Changes

• embedded/store: offset handling at keyreader
• Update dependencies
• build: Update RELEASING.md file
• embedded/remotestorage: Improve error reporting
• embedded/remotestorage: More detailed errors
• embedded/remotestorage: Improve testing of remotestorage
• embedded/remotestorage/s3: Improved s3 object name checks
• embedded/sql: Do not return error from conditional and limit readers
• embedded/sql: fixed-timed tx
• github: Update push action
• github: Run tests with minio service
• github: On macOS run client only test on pull requests
• github: Run coverage tests with minio enabled
• github: Update minimal supported go version to 1.15
• pkg/client: Better detection of tests that require external immudb
• pkg/server: Add missing copyright headers
• pkg/server/session: Move options normalization into options struct
• pkg/server/sessions: Improve options handling
• pkg/server/sessions: Add MaxSessions option
• pkg/server/sessions: Simplify session handling code
• remotestorage: Add prometheus metrics for remote storage kind
• tools: Remove old stream tool

Features

• embedded/sql: offset clause
• embedded/store: offset in key scanning
• pkg/api: offset attribute in scan and zscan endpoints

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.3.1-darwin-amd64	be0c602551f801e3971360a6146d7d3edfd5d226381b95e5e1411c04f6662bfe
immudb-v1.3.1-darwin-arm64	c3c60ad8842f27765480c0ff21ed948e827f912fe75f97ac99a21b99116cd490
immudb-v1.3.1-freebsd-amd64	447021f9bc00b1ad08fd5f76550bc6240f18a762164178b8afa9197e6254df85
immudb-v1.3.1-linux-amd64	731d8de18e3d32e102a8aad70fd47234b46b2da0c5d8e04d33032af7624f37a5
immudb-v1.3.1-linux-amd64-static	b0e22d11bfe3c1c4d05e37c22f30f1dc4072320e6b70f6db76f21a485d789075
immudb-v1.3.1-linux-arm64	8da9ca4bdb478eee408f5a37d8af9ab56e0f468476e959051cfdbf1cedd905ae
immudb-v1.3.1-linux-s390x	14e6ade9a657b333de52010ae4fcd714a2e48e9f38aecd965a22f6349814758e
immudb-v1.3.1-windows-amd64.exe	1396d0c61d215bcdbf43605808a5a81f419689f0e626a2f5b1b456e8dcc302e4

Immuclient Binaries

File	SHA256
immuclient-v1.3.1-darwin-amd64	ce167f3b4569ab5fb4872120281211f9febf442a9443c03ddded38c8f14a62ac
immuclient-v1.3.1-darwin-arm64	16790e1ca4789c8bbb1ace95501915c17f1713e4e2d6d3a7d532054042ebdfe4
immuclient-v1.3.1-freebsd-amd64	e2f5d6c8e46ade303383c9a8d4094cc4fea0e3b7120bd6eac3ee135a5c5f45fe
immuclient-v1.3.1-linux-amd64	015aa7eb91fc011562db27a2b0172cf64f318dbf21e7588fedef43e5f62d1ee4
immuclient-v1.3.1-linux-amd64-static	526f942a711ccf644b402d7ad3599a32f00bbbd826bbc046cc2720aa61fc3686
immuclient-v1.3.1-linux-arm64	71314a0528df3d692f8789ade535f19db7e25319e377d8d0c32236dac4e6b4bf
immuclient-v1.3.1-linux-s390x	3d24102b76572931e91b1b06d35ae355cbc4c57fe669972096d0377f782f1fa4
immuclient-v1.3.1-windows-amd64.exe	233cb79db6c4317860138848aeb41fe5cb6ad2adaceeccc0e5ec4e0e8c17874c

Immuadmin Binaries

File	SHA256
immuadmin-v1.3.1-darwin-amd64	a62b8942bb7fa9bde7b4905869ccf451ddd28ec477e1ad5d920a39f59afedff1
immuadmin-v1.3.1-darwin-arm64	ad3025ce49baec3008379648d760a2b2f93811a7f460132cfcd9ef5634a39ab0
immuadmin-v1.3.1-freebsd-amd64	bf935f03102eca534e466e3329f0434443bb41940b6590525a4d7796967ba2d1
immuadmin-v1.3.1-linux-amd64	3c5e343b408ff9600004022d02900b63e89bb888677ad843ec47cca7713bc431
immuadmin-v1.3.1-linux-amd64-static	135648491f5bc60c38733c270f0c448734f7406280da0aeadd60f4a71d3fb3e7
immuadmin-v1.3.1-linux-arm64	8c158fe6097a481276b71bec1777185e57f43fd6f16fd79cb78f9fe0ef397297
immuadmin-v1.3.1-linux-s390x	855f792bdc594551bf2e044e52b38f60fddc831684e2e5a5213138ec441952a4
immuadmin-v1.3.1-windows-amd64.exe	078e0f8b4d6cf4eca79e44a9df1b9b9869da22f24cb57d1f024f17d0bed08805

v1.3.1-RC1

Changelog

[v1.3.1-RC1] - 2022-06-30

Bug Fixes

• README: Update readme to show examples for 1.3.0 version
• cmd/immuadmin: use StreamChunkSize as max chunk size during tx replication
• cmd/immudb: include metrics endpoint related flags
• embedded/remotestorage: Fix invalid comment
• embedded/remotestorage/s3: Correctly url decode entry names
• embedded/remotestorage/s3: Simplify the code for scan
• embedded/remotestorage/s3: Avoid using HEAD requests
• embedded/remotestorage/s3: Fix s3 object name validation
• embedded/sql: Use defer to cleanup unclosed readers on error
• embedded/sql: Fix snapshot leak on query initialization failure
• embedded/sql: Properly close readers in joint row reader
• embedded/sql: Fix reader leaks during initialization failures
• embedded/sql: Fix snapshot leaks in union readers
• embedded/sql: ensure timestamp is evaluated with microsecond precision
• pkg/client: ensure connection is closed and session can be re-established
• pkg/database: Do not panic if incorrect number of pk values is given to VerifiableSQLGet
• pkg/server: Fix remote storage test after recent changes
• pkg/server/sessions: Avoid deadlock when closing session manager
• pkg/server/sessions: Session manager test fixes
• pkg/server/sessions: Handle short buffer read when generating session id
• pkg/server/sessions: Correctly start session guard
• pkg/server/sessions: Use strong random source for session ID

Changes

• Update dependencies
• build: Update RELEASING.md file
• embedded/remotestorage: Improve error reporting
• embedded/remotestorage: More detailed errors
• embedded/remotestorage: Improve testing of remotestorage
• embedded/remotestorage/s3: Improved s3 object name checks
• embedded/sql: Do not return error from conditional and limit readers
• embedded/sql: fixed-timed tx
• github: Update push action
• github: Run tests with minio service
• github: On macOS run client only test on pull requests
• github: Run coverage tests with minio enabled
• github: Update minimal supported go version to 1.15
• pkg/client: Better detection of tests that require external immudb
• pkg/server: Add missing copyright headers
• pkg/server/session: Move options normalization into options struct
• pkg/server/sessions: Improve options handling
• pkg/server/sessions: Add MaxSessions option
• pkg/server/sessions: Simplify session handling code
• remotestorage: Add prometheus metrics for remote storage kind
• tools: Remove old stream tool

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.3.1-RC1-darwin-amd64	a06254ce724fe9be3374c88664d0a27cd732a0eae0aa58451fe4d04f05f43c44
immudb-v1.3.1-RC1-darwin-arm64	45c996fca79126d409ba1e9520cf3f6172c04142916741643583fed77f840fca
immudb-v1.3.1-RC1-freebsd-amd64	aa5c35695d84b38e2f086ffde50558e9f8a958cc2ea8ce46b21dd44969aba969
immudb-v1.3.1-RC1-linux-amd64	2a7b1f28fd0e7bb9e5531a0efb2f0fc7c9c6ea0159053ca7878d5d82019e56bd
immudb-v1.3.1-RC1-linux-amd64-static	bc459e15c8ea5722f9724c29566fb65c49f7894af2fc8e33a51ebcdb710a007c
immudb-v1.3.1-RC1-linux-arm64	797822fc74a8137278c0de376966d9cfe239451920e68f102e1c77a3bcada40c
immudb-v1.3.1-RC1-linux-s390x	058e02a0e86fbce1609ed99792553f45bb7e2355f698137574d53fe201356a49
immudb-v1.3.1-RC1-windows-amd64.exe	d22b6f3db757997dca1f55500547dc989c700f1e795cc695e54f75545167750b

Immuclient Binaries

File	SHA256
immuclient-v1.3.1-RC1-darwin-amd64	dd5fd916b1ca62593cf99dcc868485c32d32ff09f6a1d0215013bb9906470a0b
immuclient-v1.3.1-RC1-darwin-arm64	e80590acfc4bd3c31802caecf5317351cc65dcaf9758fa2c31a1bccf63006736
immuclient-v1.3.1-RC1-freebsd-amd64	513e3cff2f8dbde3c416eb3a6dc44c4c7d03c2eb5fd9505917f49e1c1aadff53
immuclient-v1.3.1-RC1-linux-amd64	b97cba3d4b5b124b5df21e6847469d440cdf35d8ca78f5ad00b4e431fa9f832d
immuclient-v1.3.1-RC1-linux-amd64-static	b0b6bc7f68557b6192cce4328f6ab7398ef4dd4ca55a6aafb909c7eb69dfc5fd
immuclient-v1.3.1-RC1-linux-arm64	18a6a72394e07e3e6fb25507ead0698bdcf12f8bff79dd23fd29a9167fc971a0
immuclient-v1.3.1-RC1-linux-s390x	d95107808813e430474489e2e7c3302c2e46972595c92afaed22a16e42c9435a
immuclient-v1.3.1-RC1-windows-amd64.exe	e1cc8758e76fb8db1b08018e60da8608619faa7806ce10c36b4fdf272b20ae8a

Immuadmin Binaries

File	SHA256
immuadmin-v1.3.1-RC1-darwin-amd64	168ec88456844898cba47e9eb6cd848d0f298b9af58956ec4cdd4fc6162f5c0b
immuadmin-v1.3.1-RC1-darwin-arm64	6ef6511c1f4e3bc0ab7d2953aaae228bcb242a207d043a639d18fbf2ead1382a
immuadmin-v1.3.1-RC1-freebsd-amd64	b56a2c00fad79114369ed13b5dbe1f937b54df475c5596622f7e22eac6d99f86
immuadmin-v1.3.1-RC1-linux-amd64	e057a9eb4744d80039f7f6b55ce7ab74111d67fa5eb933ae53e4500f0d02ea26
immuadmin-v1.3.1-RC1-linux-amd64-static	47e44c628a466e254180647730e856ed7a8b06b34153592d81eae98bb4708d2f
immuadmin-v1.3.1-RC1-linux-arm64	03f049db9313c2cf89fc60fec6f08f6bc16c4f82a7e36073ff1db741f56ee0ab
immuadmin-v1.3.1-RC1-linux-s390x	86c300e376ad02a472540734086c12ca6dbe9d2ad210dcc50934f44ba6ad043f
immuadmin-v1.3.1-RC1-windows-amd64.exe	43dfef022ad6f6d07afbe1ecddfc6663e808c6e97fbe28a10ab7b69c1a4a4662

v1.3.0

Release notes

We're pleased to announce immudb 1.3. This version brings some new notable changes, a lot of improvements and some bug fixes.

Revision-based queries

In this version we've extended queries on the KV level by adding value revisions. This brings up an easy to use and elegant way to access historical values for keys such as getting the first version (revision 1) or a previous value (revision -1). You can quickly check how it works with newest immuclient by following our updated User quickstart guide or by writing an application using revision using our SDKs.

Initial support for ALTER TABLE

This version also brings the initial support for ALTER TABLE statements in the SQL layer. We've added support for ADD COLUMN and RENAME COLUMN operations.

Temporal SQL queries based on time

When dealing with historical values on the SQL layer, it is now possible to use time expressions next to transaction ids for specifying historical values. Queries such as SELECT * FROM sales SINCE '2022-01-06 11:38' UNTIL '2022-01-06 12:00' are now perfectly valid. You can find more info in the SQL reference.

Catalog queries

Information about the SQL catalog can now be read directly using SQL. For that purpose we've added few built-in functions into our SQL dialect such as DATABASES(). Read more about this feature in the SQL reference

Documentation updates

In this release we also worked on the immudb documentation available at https://docs.mmudb.io/. Next to hundreds of smaller changes, the new menu structure will help finding useful resources. Most notably, the developer documentation is now split between the Key Value and SQL operations.

Other improvements

There are many more features added in this release. Most notably immudb now supports the USE [DATABASE] <dbname> and UNION SELECT SQL statements increasing the SQL compatibility. In addition to that, the Scan operation on the KV layer has now extended set of parameters allowing much greater filtering of scanned results.

Important notes

Starting with 1.3 release, immudb will now return an error if the amount of rows returned from the Scan, TxScan and ZScan will be greater than the allowed maximum (1000). Before it would return the first 1000 entries without reporting the error. We've found out that the previous behavior lead to a very dangerous and hidden bugs if the caller made an assumption that those calls returned all available results if the limit was not given.

To retrieve results larger than 1000 elements, pagination with a proper limit value should be used. This is the same requirement as before however it currently requires explicit limit value.

In order to handle arbitrary result sizes, future releases of immudb will focus on streaming APIs where entries will be retrieved from the server in an incremental way.

Changelog

[v1.3.0] - 2022-05-23

Bug Fixes

• embedded/sql: return invalid value when using aggregated col selector in temporal queries
• pkg/client: enhance client-side validations in verified methods
• cmd/immuclient: Do not crash on login prompt
• embedded/sql: selector resolution using valuesRowReader
• embedded/sql: continue stmt execution on handler after changing db in use
• embedded/sql: increase auto_increment pk once per row
• embedded/sql: typo in error message
• embedded/sql: adjust named parameter parsing
• github: Run sonarcloud code analysis after cove coverate
• pkg/database: avoid silent returns when the scan limit is reached
• pkg/database: Fix detection of incorrect revision numbers
• pkg/database: Correctly interpret negative revision for getAt

Changes

• Dockerfile: Add EXPOSE 5432 and IMMUDB_PGSQL_SERVER to all immudb images
• README.md: Switch to github badge
• build: Update the RELEASING.md documentation
• cmd/immuclient: Remove unnecessary sleep for set commands
• cmd/immuclient: Move history command to a separate file
• cmd/immuclient: Extract separate immuclient options
• embedded/sql: validate current database as first step
• embedded/sql: check tx range edge cases
• embedded/sql: not showing unexistent db name as part of error message
• embedded/sql: fully non-transactional db creation and selection
• embedded/sql: re-include ttimestamp conversions in tx periods
• embedded/sql: de-duplicate error handling
• embedded/sql: database selection without multidb handler is still transactional
• embedded/sql: database selection as non-transactional
• embedded/sql: postpone period evaluation so to support parameters type inference
• embedded/sql: param substitution in functional datasource
• embedded/sql: functional-style catalog queries
• embedded/sql: quoted identifiers
• embedded/sql: ensure db selection is the last operation
• embedded/sql: unit testing db selection
• embedded/sql: include short database selection stmt
• embedded/sql: ensure context propagation with multiple txs
• embedded/sql: implicit time expression
• embedded/sql: functional catalog api
• embedded/sql: non-functional catalog access
• embedded/sql: wip grammar extensions to enrich temporal queries
• embedded/sql: sql tx with context
• embedded/sql: multi-db handler
• embedded/sql: detailed error messages
• embedded/store: minor refactoring time-based tx lookup
• github: Speedup push github actions
• grpc: Extend Scan API with endKey, inclusiveSeek, inclusiveEnd
• pkg/api: extend database creation response to indicate db already existed
• pkg/database: minor error renaming
• pkg/database: Rename getAt to getAtTx
• pkg/database: provide query parameters during resolution
• pkg/database: Add missing copyright header in scan_test.go
• pkg/database: contextual sql tx
• pkg/database: set multi-db handler after db initialization
• pkg/database: Improved checking of KeyRequest constraints
• pkg/database: databases catalog query yet unsupported
• pkg/database: maintain MaxKeyScanLimit for backward compatibility
• pkg/integration: Add SQL verify tests after ALTER TABLE
• pkg/integration: Cleanup and restructure SQL tests
• pkg/server: upgrade database method signature
• pkg/server: contextual sql tx

Features

• Add revision number when getting DB entries
• Calculate revision number when scanning key history
• api/schema: Add revision-based option to key query
• cmd/immuclient: Add restore operation
• cmd/immuclient: Add revision numbers when looking up key history
• cmd/immuclient: Add support for revision-based get in immuclient
• cmd/immuclient: Better error messages for invalid revision for restore command
• embedded/sql: create database if not exists
• embedded/sql: queries with temporal ranges
• embedded/sql: temporal row ranges
• embedded/sql: catalog queries
• embedded/sql: WIP - UNION operator
• embedded/sql: Implement ALTER TABLE ADD COLUMN
• embedded/store: time-based tx lookup
• embedded/store: ranged key update reading
• pkg/client: Add revision-based get request on the go client
• pkg/database: Add revision-based get request on the GRPC level
• pkg/server: support database creation from sql
• pkg/server: support database selection from sql stmt

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.3.0-darwin-amd64	4dd1ffe15682edf57b5b3ee539808e140f4a3ae6f544b8b95ad875772c5776d2
immudb-v1.3.0-darwin-arm64	4dd6c91a59fd868b8b641b7557a9ccebf8f458d6ea5c3c3f6baf96e336bbd810
immudb-v1.3.0-freebsd-amd64	27b3533a1e665dbf74298ac76a67b37058974723f1b214e808cd7d511479c23b
immudb-v1.3.0-linux-amd64	1aaab3892cc1f1ef8bd8e5f02e25e09c76d76cd1f10f57397329bc9040e38a9a
immudb-v1.3.0-linux-amd64-static	2f404dd618798ece9b3fc183a623b13c2bb646d914dbb82f56e1a317a47d0239
immudb-v1.3.0-linux-arm64	8192640ecd581fab6630034fe059a48ad7b1fa5fa0f62abdbfc2bbf219cd04ac
immudb-v1.3.0-linux-s390x	03751514079e98aaacd4bfdcfc26bd79a0abe6d0462ff030d2ecc429dc4da970
immudb-v1.3.0-windows-amd64.exe	232ea264f32197288bea374a76ccb002ab24d3fbbff94aae79cbacdf18880701

Immuclient Binaries

File	SHA256
immuclient-v1.3.0-darwin-amd64	5e9def61732ef67c9c78c84a41fe377beb00c81a19b098213ed4e3e80c5d9042
[immuclient-v1.3.0-darwin-arm64](https://github.com/vchain-us/immudb/releases/download/v1.3.0/immuclient-v1.3.0-d...

v1.3.0-RC1

Changelog

[v1.3.0-RC1] - 2022-05-19

Bug Fixes

• cmd/immuclient: Do not crash on login prompt
• embedded/sql: selector resolution using valuesRowReader
• embedded/sql: continue stmt execution on handler after changing db in use
• embedded/sql: increase auto_increment pk once per row
• embedded/sql: typo in error message
• embedded/sql: adjust named parameter parsing
• github: Run sonarcloud code analysis after cove coverate
• pkg/database: avoid silent returns when the scan limit is reached
• pkg/database: Fix detection of incorrect revision numbers
• pkg/database: Correctly interpret negative revision for getAt

Changes

• Dockerfile: Add EXPOSE 5432 and IMMUDB_PGSQL_SERVER to all immudb images
• README.md: Switch to github badge
• build: Update the RELEASING.md documentation
• cmd/immuclient: Remove unnecessary sleep for set commands
• cmd/immuclient: Move history command to a separate file
• cmd/immuclient: Extract separate immuclient options
• embedded/sql: validate current database as first step
• embedded/sql: check tx range edge cases
• embedded/sql: not showing unexistent db name as part of error message
• embedded/sql: fully non-transactional db creation and selection
• embedded/sql: re-include ttimestamp conversions in tx periods
• embedded/sql: de-duplicate error handling
• embedded/sql: database selection without multidb handler is still transactional
• embedded/sql: database selection as non-transactional
• embedded/sql: postpone period evaluation so to support parameters type inference
• embedded/sql: param substitution in functional datasource
• embedded/sql: functional-style catalog queries
• embedded/sql: quoted identifiers
• embedded/sql: ensure db selection is the last operation
• embedded/sql: unit testing db selection
• embedded/sql: include short database selection stmt
• embedded/sql: ensure context propagation with multiple txs
• embedded/sql: implicit time expression
• embedded/sql: functional catalog api
• embedded/sql: non-functional catalog access
• embedded/sql: wip grammar extensions to enrich temporal queries
• embedded/sql: sql tx with context
• embedded/sql: multi-db handler
• embedded/sql: detailed error messages
• embedded/store: minor refactoring time-based tx lookup
• github: Speedup push github actions
• grpc: Extend Scan API with endKey, inclusiveSeek, inclusiveEnd
• pkg/api: extend database creation response to indicate db already existed
• pkg/database: minor error renaming
• pkg/database: Rename getAt to getAtTx
• pkg/database: provide query parameters during resolution
• pkg/database: Add missing copyright header in scan_test.go
• pkg/database: contextual sql tx
• pkg/database: set multi-db handler after db initialization
• pkg/database: Improved checking of KeyRequest constraints
• pkg/database: databases catalog query yet unsupported
• pkg/database: maintain MaxKeyScanLimit for backward compatibility
• pkg/integration: Add SQL verify tests after ALTER TABLE
• pkg/integration: Cleanup and restructure SQL tests
• pkg/server: upgrade database method signature
• pkg/server: contextual sql tx

Features

• Add revision number when getting DB entries
• Calculate revision number when scanning key history
• api/schema: Add revision-based option to key query
• cmd/immuclient: Add restore operation
• cmd/immuclient: Add revision numbers when looking up key history
• cmd/immuclient: Add support for revision-based get in immuclient
• cmd/immuclient: Better error messages for invalid revision for restore command
• embedded/sql: create database if not exists
• embedded/sql: queries with temporal ranges
• embedded/sql: temporal row ranges
• embedded/sql: catalog queries
• embedded/sql: WIP - UNION operator
• embedded/sql: Implement ALTER TABLE ADD COLUMN
• embedded/store: time-based tx lookup
• embedded/store: ranged key update reading
• pkg/client: Add revision-based get request on the go client
• pkg/database: Add revision-based get request on the GRPC level
• pkg/server: support database creation from sql
• pkg/server: support database selection from sql stmt

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.3.0-RC1-darwin-amd64	cbdabc2bae86a489744cd51ca035a3994129b1a25470985957c21b4dab227a57
immudb-v1.3.0-RC1-darwin-arm64	a80ce4c541a1d122ce0a9ddf3648cf57e969a27f109451eddd68f8d05d30f8f3
immudb-v1.3.0-RC1-freebsd-amd64	d81651387ccdd18fca470e77faa1fcdc5c4568ca5e45c2c336cec03006fa890d
immudb-v1.3.0-RC1-linux-amd64	5abe683d0147b69c37eaa0ee14d688c02aee9be7d5583c711bbcf64066add226
immudb-v1.3.0-RC1-linux-amd64-static	7bed6d577981f8fa67b9a845c8a0451961a670ffcf23698a97ffbc35d9fbe61b
immudb-v1.3.0-RC1-linux-arm64	eedc7fbe102ead28c3801229063ac80f5e57fe7c60b5eb621fe0f4b8889b482e
immudb-v1.3.0-RC1-linux-s390x	4a8ef792a658d3003b84b57d29292ec490dee1fb430e598342d41cae0c9db92b
immudb-v1.3.0-RC1-windows-amd64.exe	7c453663e3aa01985a86e60de24e08532fd2c383b874105a0f158452d33101b4

Immuclient Binaries

File	SHA256
immuclient-v1.3.0-RC1-darwin-amd64	fe0606d638bbf80e1b20b74252952d71880e03e9b1042a4b9823453f4624d40c
immuclient-v1.3.0-RC1-darwin-arm64	d847e0022f748c58772cf5721dad8c86de55d94cb897811a64d90e41a177e3a7
immuclient-v1.3.0-RC1-freebsd-amd64	7b9f3b71c90bcd2e0cea3a1bddd89450fe821eec8613262ad028caf281c50959
immuclient-v1.3.0-RC1-linux-amd64	348dd55bf2d0c3eca0c899f950f2d86afca7ce0a595f975a0c00434bb9397e9e
immuclient-v1.3.0-RC1-linux-amd64-static	b53a25a03e8a63f5fb2421d70e29d6b10d602d102a04284b8aa5339b1ef213a1
immuclient-v1.3.0-RC1-linux-arm64	cccd2453b11316103a125d37b8ea2731fcff777b3a3aaf3b9454b0ec024b939c
immuclient-v1.3.0-RC1-linux-s390x	30cdec8ab0c1a3c896f47b5c098241578caf5fa807d4649006fbda3ddb89f0e9
immuclient-v1.3.0-RC1-windows-amd64.exe	2c8f3454528c3060f3be860060cec39feff8d7dc42bd3e5f226983e89a4038ec

Immuadmin Binaries

File	SHA256
immuadmin-v1.3.0-RC1-darwin-amd64	44b6f682f690b915020cf46f891068416639d03e46e1d0ac69ba146606984031
immuadmin-v1.3.0-RC1-darwin-arm64	ea8bba35520c6f573102a5019d6f5d227f828d61a0868e0fbef381eb4f5e7c1d
immuadmin-v1.3.0-RC1-freebsd-amd64	0521ff358e99c0106742e5445a5044d13812131c61309e40ad820a9f41d8f87c
immuadmin-v1.3.0-RC1-linux-amd64	a3cef49bfff27103d4943cdf7601232a1851c98bbf8d50e741048e57422b2eb6
immuadmin-v1.3.0-RC1-linux-amd64-static	28b42d2a858ac93181bb9d66358c19d3be711fc44c98881786cf2ad6079e5314
immuadmin-v1.3.0-RC1-linux-arm64	b28a1c183107c239156840dab0e275eea5304175f0108736d7d40c963de496f7
immuadmin-v1.3.0-RC1-linux-s390x	12817e873188d4a8b56...

v1.2.4

immudb v1.2.4

This is a small bugfix immudb release aimed at fixing issues found in the previous 1.2.3 version.

• fix issue in btree node split that would cause panics for workloads using very large and small keys at the same time
• add new option to transaction scan to avoid dereferencing reference keys - this option can be used to avoid unnecessary IO reads when reading all entries within a single transaction
• fix issue with online compaction in scenarios with large read workloads
• fix automatic relogin in immuclient binary
• release process improvements
• improvements to podman compatibility (contributed by Benjamin Gentil)

Changelog

[v1.2.4] - 2022-04-28

Bug Fixes

• Dockerfile: Fix HOME variable for podman
• cmd/immuclient: upgrade not logged in error handling
• embedded/tbtree: create nodes with the right number of children
• embedded/tbtree: Better logging in btree flush
• embedded/tbtree: Fix cleanupPercentage in SnapshotSince call
• embedded/tbtree: ensure node split is evaluated
• embedded/tbtree: split into multiple nodes
• github/push: Fix notarization of binaries
• pkg/auth: Clarify comments about token injection
• pkg/auth: Do not send duplicated authorization header
• pkg/server: include db name in flush index result

Changes

• CHANGELOG.md: remove bogus liist tag entry
• build/RELEASING.md: Update releasing docs
• cmd/immuclient: include db name when printing current state
• embedded/store: index settings validations
• embedded/tbtree: ensure node size is consistent with key and value sizes
• embedded/tbtree: rename function that calculates node size lower bound
• github: Use smaller 5-days retention for master builds
• github: Update github workflow on master / version push
• github: Update github action versions
• github/push: Build docker images after tests
• github/push: Build, test and notarize for release/v* branches
• github/push: Calcualte sha256 checksums for binaries in github
• github/push: Add quick test linux-amd64 binaries
• github/push: Add quick test for Mac x64 binaries
• github/push: Add quick test for linux-arm64 binaries through qemu
• github/push: Add quick test for linux-s390x binaries through qemu
• github/push: Run stress test before notarizing binaries
• pkg/api: txbyid with keepReferencesUnresolved option
• tools/testing: Add stress tool
• tools/testing: Add randomized key length mode for stress test tool

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.2.4-darwin-amd64	ee43bff63c87eb29a981ec366203092933bf1d13b791d62e26c7fe36637f9f31
immudb-v1.2.4-darwin-arm64	c00b37c6cd154247b2c58c76517ddf74fbbbe7abddade138c3b78b855fc605a8
immudb-v1.2.4-freebsd-amd64	a9e3187226b5daea62ea94d836851a910db957ea06c7cc0dcb0a81802676b2ce
immudb-v1.2.4-linux-amd64	3689e4d8bca0ed6c0785e3ec82169d07e35988534e68741c4f0335c7efb5eb73
immudb-v1.2.4-linux-amd64-static	eb9436b1dbebdc4002d459c24b56396ae3c832b7648cba2bd394cd853f956f0b
immudb-v1.2.4-linux-arm64	f915c394643bade7d0744ccc2a835080d991f3fdb97f8e20781c5c1f99751734
immudb-v1.2.4-linux-s390x	b5cb02e62b99c701aece553caa1e4cdd741e4d65697d67c1d57a93e74cb9f65b
immudb-v1.2.4-windows-amd64.exe	ff1ca6fb6ca0d8b32a429c3466a3099b136d43fc346d9f41511dd0ec623188fe

Immuclient Binaries

File	SHA256
immuclient-v1.2.4-darwin-amd64	389fbb52ada28027a943c27bebe7c962b2af47d1a1d9e79a8d47a17494790d50
immuclient-v1.2.4-darwin-arm64	d0c376392703eeaa1822a0e35530c23f3be871dd471e30ece8398d1402a8ef9f
immuclient-v1.2.4-freebsd-amd64	47d38ece837e558fd99bc5375b61082eb427ab69688da30fc200d098532c6ace
immuclient-v1.2.4-linux-amd64	afaa1345dbd345a32bb18c53ecfcff3d1da804c3a57c4dee88f52c81de46d0ac
immuclient-v1.2.4-linux-amd64-static	1157c8ff346510d6ce072a50818933d07d802df3e166d1090ebb446ef436451f
immuclient-v1.2.4-linux-arm64	054b7383a6b9b09937c26fb5493a90e2c8eeb41095807d2bcba396e8a0fc370a
immuclient-v1.2.4-linux-s390x	9b7d127817c43f9c81dff72b4abe4f1cb4155b5b728ab4ea9b1f59e9e6b45b48
immuclient-v1.2.4-windows-amd64.exe	5324f22567bf7b190936ab0205a03a25360c84b478f83cf0863d93c9f4fa9053

Immuadmin Binaries

File	SHA256
immuadmin-v1.2.4-darwin-amd64	4992a91a70148892f7677a1f35a1f74b00a113dae21247ea8919bb27fa496ba9
immuadmin-v1.2.4-darwin-arm64	8677d51e433304393db6b622e8496f7039c6f35c1c213c06a86464f253a28940
immuadmin-v1.2.4-freebsd-amd64	82c5e50b555b8e2dafa4868837762ff631bb56cfe1f1afd3e622bf4628d2eaca
immuadmin-v1.2.4-linux-amd64	41858833eab18d6379146d5d15c041c6ee001a3fff9cd5dcf086ad89c6b13664
immuadmin-v1.2.4-linux-amd64-static	8637c12c5a3aafe57431c602e45e327b06d000071b8cee1097bf58de62a679a0
immuadmin-v1.2.4-linux-arm64	9ae5928100431589a22d659dd49255195e16a52819ac580b8eabe84745b4d126
immuadmin-v1.2.4-linux-s390x	3dff9d772a21d5c25b77e7437ef7f8984905801050c3fbe776c55533a0e22d18
immuadmin-v1.2.4-windows-amd64.exe	2e1ac9b1d333e23121c497fea3e81e24e197550d621c682399c699f874e60c6c

v1.2.4-RC1

Changelog

[v1.2.4-RC1] - 2022-04-27

Bug Fixes

• Dockerfile: Fix HOME variable for podman
• cmd/immuclient: upgrade not logged in error handling
• embedded/tbtree: create nodes with the right number of children
• embedded/tbtree: Better logging in btree flush
• embedded/tbtree: Fix cleanupPercentage in SnapshotSince call
• embedded/tbtree: ensure node split is evaluated
• embedded/tbtree: split into multiple nodes
• github/push: Fix notarization of binaries
• pkg/auth: Clarify comments about token injection
• pkg/auth: Do not send duplicated authorization header
• pkg/server: include db name in flush index result

Changes

• CHANGELOG.md: remove bogus liist tag entry
• build/RELEASING.md: Update releasing docs
• cmd/immuclient: include db name when printing current state
• embedded/store: index settings validations
• embedded/tbtree: ensure node size is consistent with key and value sizes
• embedded/tbtree: rename function that calculates node size lower bound
• github: Use smaller 5-days retention for master builds
• github: Update github workflow on master / version push
• github: Update github action versions
• github/push: Build docker images after tests
• github/push: Build, test and notarize for release/v* branches
• github/push: Calcualte sha256 checksums for binaries in github
• github/push: Add quick test linux-amd64 binaries
• github/push: Add quick test for Mac x64 binaries
• github/push: Add quick test for linux-arm64 binaries through qemu
• github/push: Add quick test for linux-s390x binaries through qemu
• github/push: Run stress test before notarizing binaries
• pkg/api: txbyid with keepReferencesUnresolved option
• tools/testing: Add stress tool
• tools/testing: Add randomized key length mode for stress test tool

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.2.4-RC1-darwin-amd64	1e5e4ccc150acd679b1d16d9b228dc04e9021976c515ce6d947151d5c7207098
immudb-v1.2.4-RC1-darwin-arm64	2bdd12cadd094400a29fdceb6dc2706932dbb5806cb8a3a5e70ae27de5aaafbf
immudb-v1.2.4-RC1-freebsd-amd64	025b1b7993ba2b267e695d13c01f4485d900420b0475679ab17f33002f8849ab
immudb-v1.2.4-RC1-linux-amd64	7007babf447e117d8b3e4d9d67ae75fa40f52880657be09ea8139007082aba8a
immudb-v1.2.4-RC1-linux-amd64-static	46f10bef27b04c0727d142cd6f51ab826db00a8f18223e8d13cc210f016e36df
immudb-v1.2.4-RC1-linux-arm64	48b6def03a92214e8ac95549ceac07e1c5dc231a70b88de1fbfefc49de2db9dc
immudb-v1.2.4-RC1-linux-s390x	306f6c652ba4c247572a23efe576b377bad072f0445d12e4259cb4f85f60afbc
immudb-v1.2.4-RC1-windows-amd64.exe	17362575f6a2f245512ccc2faea7710c0dd4c011c874ca931a7ef047ac7713e8

Immuclient Binaries

File	SHA256
immuclient-v1.2.4-RC1-darwin-amd64	121b9bb08e25b964cd405362337d2b03174f823e3fa3c880452662a33e69dee9
immuclient-v1.2.4-RC1-darwin-arm64	cf8862b57a76adecd9f24980106d0080f8cfbbd8c0ae82364906dffec044af35
immuclient-v1.2.4-RC1-freebsd-amd64	3fb24fa6c7feff3fdbc83b9783bfa2afc1fa9580f3e48072b6adaccb1c7f60fd
immuclient-v1.2.4-RC1-linux-amd64	5d54afb6be8930d948356e740186e17b5ba35e14378488b4478fe979ea90419c
immuclient-v1.2.4-RC1-linux-amd64-static	7624eb69fbce57d11122341aae5f4b41218c9f44919ec55f99c1bb03ae7f4d5f
immuclient-v1.2.4-RC1-linux-arm64	1ba38a8193c178fa29f304b88c288565d9acca2ab2a21bd0407b2e6420360aa4
immuclient-v1.2.4-RC1-linux-s390x	31b7cd8bfd50f8d9279bd49ca5e3680f703d4ea1b43b9dd47edd949bf359344b
immuclient-v1.2.4-RC1-windows-amd64.exe	70b6af3acda3c2b3236c2f890a88d9bd89a77d767e0af5ad4991481bfb72d21d

Immuadmin Binaries

File	SHA256
immuadmin-v1.2.4-RC1-darwin-amd64	3d8fb0f8065ce4d29f9e3aa5c167fc6f9e3c0163ba3db427891b529b24ff009b
immuadmin-v1.2.4-RC1-darwin-arm64	117f0319e7511b329d3e1466a18de9c0de4ecd5cad9dc81f4935824fa82c9692
immuadmin-v1.2.4-RC1-freebsd-amd64	c6e3bf10286c5ee0ef40425fb69239edfe354adf33d1d2b48ed1934df636e6d5
immuadmin-v1.2.4-RC1-linux-amd64	eaf830c2cc97a4d89462ac93b1da05c5797ec72481aba9676c4cf5b15acd83af
immuadmin-v1.2.4-RC1-linux-amd64-static	0f749d9d9965b4eda0991b814211d575fda2f4d20462f7c9f8660ff840ee2571
immuadmin-v1.2.4-RC1-linux-arm64	509fadd4053343c7e3960465243822e6dd49ec35bb94e8f32f15d81173ed6157
immuadmin-v1.2.4-RC1-linux-s390x	061f365eca4e8b3d672a0711621d8894c3c49c0bbb1eeaf4dcc34eaaf112b57f
immuadmin-v1.2.4-RC1-windows-amd64.exe	537be6fe451928493a650b4dd24fccb2e1c4eb364f3c3cf4b0c1d6f7165d6171

v1.2.3

immudb 1.2.3

immudb v1.2.3

• Introduction of instrumentation panel via Prometheus for enhanced monitoring
• Tons of optimizations and simplified long-run maintenance functionality (self-compacting index)
• Run-time database loading/unloading
• Physical database deletion

Fixes:

• improved error handling when webconsole can't be started
• fix sorted set scans in descending order
• fix displayed name of signing key environment variable
• fixes for session handling

Improvements:

• much improved prometheus metrics related to internal btree
  • immudb_btree_cache_size, immudb_btree_cache_hit, immudb_btree_cache_miss, immudb_btree_cache_evict - for monitoring btree node cache usage
  • immudb_btree_depth, immudb_btree_inner_node_entries, immudb_btree_leaf_node_entries for monitoring btree depth and internal data distribution
  • immudb_btree_flushed_nodes_last_cycle, immudb_btree_flushed_nodes_total, immudb_btree_flushed_entries_last_cycle, immudb_btree_flushed_entries_total, immudb_btree_compacted_nodes_last_cycle, immudb_btree_compacted_nodes_total, immudb_btree_compacted_entries_last_cycle, immudb_btree_compacted_entries_total - for monitoring flush and compaction progress
  • immudb_btree_nodes_data_begin, immudb_btree_nodes_data_end for monitoring new online compaction
• SQL syntax improvements (support for <> as an alternative to !=)
• Many btree improvements:
  • faster btree insertions:
    • reduction of insertion time by 22% in our test setup
  • much better protection against unclean shutdowns (minimize the risk of full index rebuild)
  • smaller IO pressure by reducing the number of sync operations
  • more balanced distribution of entries in btree nodes
  • reduced node sizes
    • reduction of disk space occupied by index by 22% in our test setup
  • online index compaction - https://docs.immudb.io/1.2.3/develop/management.html#index-cleaning
• Non-indexable entries
• More tests and log messages
• Many database options are now configurable through the GRPC api, list of options: https://docs.immudb.io/master/develop/management.html#configuration-options
• Improved database health check endpoint
• Compatibility mode with 1.1 proofs - https://docs.immudb.io/1.2.3/operations/backwards-compatibility.html
• Conditional KV writes - https://docs.immudb.io/1.2.3/develop/reading.html#conditional-writes
• On-demand database load / unload and deletion - https://docs.immudb.io/1.2.3/develop/management.html#database-loading-unloading

Important Notes:

Updating to this version will cause regeneration of the btree index. For large databases this may take some time.

Changelog

[v1.2.3] - 2022-04-14

Bug Fixes

• cmd/immuadmin: simplify logging when flushing and compacting current db
• pkg/database: return key not found when resolving a deleted entry
• pkg/database: Return correct error for verifiedGet on deleted entries
• CI/CD: Golang compiler is not needed for building docker images
• CI/CD: Use CAS for notarization
• embedded/store: Ensure up-to-date index on constrained writes
• embedded/store: Fix early precondition checks
• embedded/tbtree: copy-on-write when increasing root ts
• immudb: Fix the name of signing key env var
• pkg: Fix tests after recent changes in API
• pkg/api: Remove unused Sync field from IndexOptions
• pkg/api: typo in kv metadata message
• pkg/api/schema: Use correct id for preconditions in SetRequest
• pkg/auth: Avoid unguarded read from user tokens map
• pkg/client: Adopt to EncodeReference changes
• pkg/client: Prevent updates with incorrect database settings
• pkg/client: Use correct response for UpdateDatabaseV2
• pkg/client/errors: Ensure FromErrors works with ImmuError instance
• pkg/client/errors: Update the list of error codes
• pkg/database: Better handling of invalid constraints
• pkg/database: Improve test coverage for KV constraints
• pkg/database: automatically set max score if not specified in desc order
• pkg/errors: Correct GRPC error mapping for precondition failure
• pkg/server: Use buffered channel for catching OS signals
• pkg/server: adjust time to millis convertion
• pkg/server: ensure sessions locks get released
• pkg/server: override default settings with existent values
• pkg/server: typo in log message
• tools/monitoring: Update grafana dashboards

Changes

• Fix typo in a comment
• Rename Constraints to Preconditions
• cleanup percentage as float value
• Update copyright to 2022
• Dockerfile: Improve dockerfile builds
• build: improve release instructions (#1100)
• cmd/immuadmin: add safety flag in delete database command
• cmd/immuclient: health command description
• embedded/ahtree: fix error message
• embedded/appendable: appendable checksum calculation
• embedded/appendable: return io.EOF if there are not enough data for checksum calculation
• embedded/appendable: discard capability
• embedded/appendable: fix typo in error message
• embedded/store: syncThld at store options
• embedded/store: Add missing Copyright header
• embedded/store: declare constants for all the options
• embedded/store: parametrized index write buffer size
• embedded/store: add synced setting in index options
• embedded/store: sync aht when syncing the store
• embedded/store: verbose data corruption error
• embedded/store: index one tx per iteration
• embedded/store: use store layer for constraint validations
• embedded/store: constraint validations with deletion and expiration support
• embedded/store: do not skip expired entries when indexing
• embedded/store: verbose logging during compaction
• embedded/store: skip expired entries during indexing
• embedded/store: improve compaction logging
• embedded/store/options: Simplify validation tests
• embedded/tbree: only insert nodes in cache when they were mutated
• embedded/tbree: use shared writeOpts
• embedded/tbree: remove obsolete property
• embedded/tbree: bump index version
• embedded/tbtree: minOffset only for non-mutated nodes
• embedded/tbtree: Add metrics for index data size
• embedded/tbtree: reduce allocations when flushing
• embedded/tbtree: use double for min offset calculation
• embedded/tbtree: data discarding with opened and older snapshots
• embedded/tbtree: improve snapshot loading and discarding
• embedded/tbtree: synced flush if cleanup percentage is greater than zero
• embedded/tbtree: Add more internal metrics
• embedded/tbtree: reduce allocs during flush
• embedded/tbtree: ensure current snapshot is synced for compaction
• embedded/tbtree: validate input kv pairs before sorting
• embedded/tbtree: Use KV entries count for sync threshold
• embedded/tbtree: no cache update during compaction reads
• embedded/tbtree: discard unreferenced data when flushing index
• embedded/tbtree: discard unreferenced data
• embedded/tbtree: middle node split
• embedded/tbtree: min offset handling
• embedded/tbtree: validate compaction target path
• embedded/tbtree: positive compaction threshold
• embedded/tbtree: discard unreferenced data after sync
• embedded/tbtree: ensure sync on gracefully closing
• embedded/tbtree: checksum-based snapshot consistency validation
• embedded/tbtree: self-healing index
• embedded/tbtree: set initial offsets during initialization
• embedded/tbtree: validate data-format version
• embedded/tbtree: Extend buckets for child node count histogram
• embedded/tbtree: reduce fixed records length
• embedded/tbtree: open-ranged nodes
• embedded/tbtree: wip reduce node size
• embedded/tbtree: use binary search during key lookups
• embedded/tbtree: fully replace sync with syncThld
• embedded/tbtree: rebase non-indexed on kv syncthreshold
• embedded/tbtree: explicit error validation before loading
• embedded/tbtree: sort kv pairs in bulkInsert
• makefile: fix cas sign instructions
• metrics: Add better flush / compaction metrics for btree
• pkg/api: prepare flushindex endpoint for future extensions
• pkg/api: use entries spec in verified and scan tx endpoints
• pkg/api: parametrized index cleanup percentage
• pkg/api: non-indexable entries
• pkg/api: use nullable prefix in db settings message
• pkg/api: add synced param to flushindex endpoint
• pkg/api: change proto schema toward db loading/unloading
• pkg/api: uniform v2 endpoints
• pkg/api: db loading and unloading
• pkg/client: optional client connection
• pkg/client: synced flushing to enable physical data deletion
• pkg/client: use txRequest in TxByIDWithSpec method
• pkg/database: tx entries excluded by default if non-null spec is provided
• pkg/database: optional tx value resolution
• pkg/database: remove db name from options
• pkg/database: use shared tx holder when resolving tx entries
• pkg/database: parameters to resolve references at tx
• pkg/integration: integrate non-indexed into grpc apis
• pkg/server: Dump used db options when loading databases
• pkg/server: replication options for systemdb and defaultdb
• pkg/server: use syncThreshold
• pkg/server: use previous store default values
• pkg/server: endpoint to retrieve settings of selected database
• pkg/server: log web-console error on boot
• pkg/server: start/stop replicator when loading/unloading db
• pkg/server: expose max opened files for btree indexing
• pkg/server: minor changes
• **pkg/server:*...