v1.2.3-RC1

Changelog

[v1.2.3-RC1] - 2022-04-13

Bug Fixes

• CI/CD: Golang compiler is not needed for building docker images
• CI/CD: Use CAS for notarization
• embedded/store: Ensure up-to-date index on constrained writes
• embedded/store: Fix early precondition checks
• embedded/tbtree: copy-on-write when increasing root ts
• immudb: Fix the name of signing key env var
• pkg: Fix tests after recent changes in API
• pkg/api: Remove unused Sync field from IndexOptions
• pkg/api: typo in kv metadata message
• pkg/api/schema: Use correct id for preconditions in SetRequest
• pkg/auth: Avoid unguarded read from user tokens map
• pkg/client: Adopt to EncodeReference changes
• pkg/client: Prevent updates with incorrect database settings
• pkg/client: Use correct response for UpdateDatabaseV2
• pkg/client/errors: Ensure FromErrors works with ImmuError instance
• pkg/client/errors: Update the list of error codes
• pkg/database: Better handling of invalid constraints
• pkg/database: Improve test coverage for KV constraints
• pkg/database: automatically set max score if not specified in desc order
• pkg/errors: Correct GRPC error mapping for precondition failure
• pkg/server: Use buffered channel for catching OS signals
• pkg/server: adjust time to millis convertion
• pkg/server: ensure sessions locks get released
• pkg/server: override default settings with existent values
• pkg/server: typo in log message
• tools/monitoring: Update grafana dashboards

Changes

• Fix typo in a comment
• Rename Constraints to Preconditions
• cleanup percentage as float value
• Update copyright to 2022
• Dockerfile: Improve dockerfile builds
• build: improve release instructions (#1100)
• cmd/immuadmin: add safety flag in delete database command
• cmd/immuclient: health command description
• embedded/ahtree: fix error message
• embedded/appendable: appendable checksum calculation
• embedded/appendable: return io.EOF if there are not enough data for checksum calculation
• embedded/appendable: discard capability
• embedded/appendable: fix typo in error message
• embedded/store: syncThld at store options
• embedded/store: Add missing Copyright header
• embedded/store: declare constants for all the options
• embedded/store: parametrized index write buffer size
• embedded/store: add synced setting in index options
• embedded/store: sync aht when syncing the store
• embedded/store: verbose data corruption error
• embedded/store: index one tx per iteration
• embedded/store: use store layer for constraint validations
• embedded/store: constraint validations with deletion and expiration support
• embedded/store: do not skip expired entries when indexing
• embedded/store: verbose logging during compaction
• embedded/store: skip expired entries during indexing
• embedded/store: improve compaction logging
• embedded/store/options: Simplify validation tests
• embedded/tbree: only insert nodes in cache when they were mutated
• embedded/tbree: use shared writeOpts
• embedded/tbree: remove obsolete property
• embedded/tbree: bump index version
• embedded/tbtree: minOffset only for non-mutated nodes
• embedded/tbtree: Add metrics for index data size
• embedded/tbtree: reduce allocations when flushing
• embedded/tbtree: use double for min offset calculation
• embedded/tbtree: data discarding with opened and older snapshots
• embedded/tbtree: improve snapshot loading and discarding
• embedded/tbtree: synced flush if cleanup percentage is greater than zero
• embedded/tbtree: Add more internal metrics
• embedded/tbtree: reduce allocs during flush
• embedded/tbtree: ensure current snapshot is synced for compaction
• embedded/tbtree: validate input kv pairs before sorting
• embedded/tbtree: Use KV entries count for sync threshold
• embedded/tbtree: no cache update during compaction reads
• embedded/tbtree: discard unreferenced data when flushing index
• embedded/tbtree: discard unreferenced data
• embedded/tbtree: middle node split
• embedded/tbtree: min offset handling
• embedded/tbtree: validate compaction target path
• embedded/tbtree: positive compaction threshold
• embedded/tbtree: discard unreferenced data after sync
• embedded/tbtree: ensure sync on gracefully closing
• embedded/tbtree: checksum-based snapshot consistency validation
• embedded/tbtree: self-healing index
• embedded/tbtree: set initial offsets during initialization
• embedded/tbtree: validate data-format version
• embedded/tbtree: Extend buckets for child node count histogram
• embedded/tbtree: reduce fixed records length
• embedded/tbtree: open-ranged nodes
• embedded/tbtree: wip reduce node size
• embedded/tbtree: use binary search during key lookups
• embedded/tbtree: fully replace sync with syncThld
• embedded/tbtree: rebase non-indexed on kv syncthreshold
• embedded/tbtree: explicit error validation before loading
• embedded/tbtree: sort kv pairs in bulkInsert
• makefile: fix cas sign instructions
• metrics: Add better flush / compaction metrics for btree
• pkg/api: prepare flushindex endpoint for future extensions
• pkg/api: use entries spec in verified and scan tx endpoints
• pkg/api: parametrized index cleanup percentage
• pkg/api: non-indexable entries
• pkg/api: use nullable prefix in db settings message
• pkg/api: add synced param to flushindex endpoint
• pkg/api: change proto schema toward db loading/unloading
• pkg/api: uniform v2 endpoints
• pkg/api: db loading and unloading
• pkg/client: optional client connection
• pkg/client: synced flushing to enable physical data deletion
• pkg/client: use txRequest in TxByIDWithSpec method
• pkg/database: tx entries excluded by default if non-null spec is provided
• pkg/database: optional tx value resolution
• pkg/database: remove db name from options
• pkg/database: use shared tx holder when resolving tx entries
• pkg/database: parameters to resolve references at tx
• pkg/integration: integrate non-indexed into grpc apis
• pkg/server: Dump used db options when loading databases
• pkg/server: replication options for systemdb and defaultdb
• pkg/server: use syncThreshold
• pkg/server: use previous store default values
• pkg/server: endpoint to retrieve settings of selected database
• pkg/server: log web-console error on boot
• pkg/server: start/stop replicator when loading/unloading db
• pkg/server: expose max opened files for btree indexing
• pkg/server: minor changes
• pkg/server: synced db runtime operations
• pkg/server: expose flush index endpoint
• pkg/server: increase default max number of active snapshots
• pkg/server: tolerate failed user-created db loading
• pkg/server: convert time to milliseconds
• pkg/serverr: validate request in deprecated database creation endpoint
• stats: Add btree cache prometheus stats

Features

• Entries-independent constraints in GRPC api
• Improved API for database creation and update
• Move KV write constraints to OngoingTX member
• Improved validation of kv constraints
• KV: Add constrained KV writes for ExecAll operation
• KV: Do not create unnecessary snapshots when checking KV constraints
• KV: Move constraints validation to OngoingTx
• KV: Add constrained KV writes for Set operation
• KV: Add constrained KV writes for Reference operation
• embedded/cache: dynamic cache resizing
• embedded/store: Fail to write metadata if proof version does not support it
• embedded/store: Add max header version used during writes
• embedded/store: non-indexable entries
• embedded/store: Add tests for generation of entries with metadata
• embedded/store: Allow changing tx header value using GRPC api.
• embedded/tbtree: decouple flush and sync by introducing syncThreshold attribute
• immuadmin: Allow changing proof compatibility from immuadmin
• kv: Update grpc protocol with KV set constraints
• pkg/api: delete database endpoint
• pkg/api: tx api with entry filtering capabilities
• pkg/client: new method to fetch tx entries in a single call
• pkg/database: Updated GRPC protocol for constrained writes
• pkg/database: Update code for new constrained write protocol
• pkg/database: add noWait attribute in get request
• pkg/server: database health endpoint
• pkg/server: support database loading/unloading
• pkg/server: new endpoint databaseSettings
• pkg/server: expose all database settings
• tools/monitoring: Add immudb Grafana dashboard
• tools/monitoring: added datasource selection, added instance selection, labels include instance, fixed calculations

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.2.3-RC1-darwin-amd64	c2a0d651602862d4113adaf43708175cb9979e042e0705ade5481a773f214088
immudb-v1.2.3-RC1-darwin-arm64	cfb782e84be293935adfc5c32d6bd1d617e8c8f2356b250c3753bd2db0347a0e
immudb-v1.2.3-RC1-freebsd-amd64	72fbc59b9386d3dc1840210c7813854f9f18e00631378f8e83ad525955e95f10
[immudb-v1.2.3-RC1-...

1.2.2

All binaries are notarized and can now be authenticated against the Community Attestation Service with SignerID [email protected].

After downloading the CAS client simply run cas authenticate binaryname --signerID [email protected]

Changelog

[v1.2.2] - 2022-01-17

Bug Fixes

• registering connection in order to make possible conn recycling
• Dockerfile: Add ca-certificates.crt file to immudb image
• client/file_cache: Fix storing immudb state in file cache
• embedded/immustore: Avoid deadlock when acquire vLog lock
• embedded/sql: max key len validations
• embedded/sql: consider not null flag is on for auto incremental column
• pkg/server: validate if db is not replica then other replication attributes are not set
• pkg/stdlib: fix last insert id generation

Changes

• create code of conduct markdown file (#1051)
• cmd/immuclient: return actual login error
• embedded/sql: wip client provided auto-incremental values
• embedded/sql: add first and last insert pks retrivial methods
• embedded/sql: change column constraints ordering
• metrics: Add indexer metrics
• metrics: Add more s3-related metrics
• pkg/database: pre-validation of duplicated entries in execAll operation
• pkg/database: temporarily disable execall validations
• pkg/database: instantiate tx holder only in safe mode
• pkg/database: self-contained noWait execAll
• pkg/database: descriptive error messages
• pkg/replication: delay replication after failure
• pkg/stdlib: clean connection registration and leftovers

Features

• hot backup and restore
• embedded/sql: support for basic insert conflict handling
• s3: Add support for AWS V4 signatures

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.2.2-darwin-amd64	641853c5fc3c091f5e744f29282921e7c4463eeae76f169cda9d88f787f01b6d
immudb-v1.2.2-darwin-arm64	b3c867513a1682d2b4c927eaab919eced899fd5e8bca07e8e94415478e36d3a9
immudb-v1.2.2-freebsd-amd64	81c5e00e50595358e4ae1e3a6fbf32ba9aa4a75136c40f1291d57568579ba44d
immudb-v1.2.2-linux-amd64	efffe43ae1a95cfa1acd340221621a63b16fb5be2a79728e9d893d22f27f116a
immudb-v1.2.2-linux-amd64-static	93e49b7952eca6b5cdb8b58fe49fee7f841c49b74187492fa93f11d8aea0c8d6
immudb-v1.2.2-linux-arm64	f6fa5f3895731299fe4d0aec77e4313146c306df406365548fe880a9057f1cbf
immudb-v1.2.2-linux-s390x	16fa90bd42b736e40e241a90fe42eb3f4c6d849c7603f9fe71c815bcac141245
immudb-v1.2.2-windows-amd64.exe	42e49263ea0285958996e4965a60db46231d682d0f2bcb550a82dd4d639a9af6

Immuclient Binaries

File	SHA256
immuclient-v1.2.2-darwin-amd64	e57512dcba9f5cbf072bf7a53f2d06dce9d8106b06534b665583359c0f82a9e4
immuclient-v1.2.2-darwin-arm64	7ad3bee87b5438884111b4efe14a1736ea1afaa1a36c069efa69c7deafb052e4
immuclient-v1.2.2-freebsd-amd64	4e869bb48f9d2fde61584892fd7bcf779f997c90cd2d073777191b70718b0865
immuclient-v1.2.2-linux-amd64	c3361d4bea4ed52484138aef3d2e2a92bea824ab5c26ed7c26a3665c000d6519
immuclient-v1.2.2-linux-amd64-static	3e72fbba186a50432dab6827cc40e02d08915ff84b891bb85b534717b9754d10
immuclient-v1.2.2-linux-arm64	b57a4996b099b0c96b7d5f3883d7db6295b24799e43795466fe718cc709695ea
immuclient-v1.2.2-linux-s390x	bf9130d4b4ce2d2c74adf03d40f66d2c669e8a8bbeed3ee6898b63a8f2736511
immuclient-v1.2.2-windows-amd64.exe	f140e8fe37f84e4c45122b0291442edd11157bdce0a8b8cbd9f12a336571ace2

Immuadmin Binaries

File	SHA256
immuadmin-v1.2.2-darwin-amd64	5d181bc75c0bd60fd7f377e33d6b9f08e0ddb85266893e14436b7f26a0683a73
immuadmin-v1.2.2-darwin-arm64	4714c15e7e08b22b00e50501112bea7b61f03fccfce3f14eafaf14fd6ed5a5e7
immuadmin-v1.2.2-freebsd-amd64	a7f2b5566f2b691c3a36fc7d348a0f38e52a288becda05d329152eab4a39bce1
immuadmin-v1.2.2-linux-amd64	6fb772d83b19324940bb4b76d1a8c08d62dd1ae03d9d54df3600ac9d808d7cd6
immuadmin-v1.2.2-linux-amd64-static	79d707b0b63d8df1c543dce7d20fa77082ebf0773d4fc8d275d4efc41eb32a42
immuadmin-v1.2.2-linux-arm64	d0dbaad3fbe743a3f5ce0417df5d365b1be8142cdb1e4c732675bc7f5d6faed4
immuadmin-v1.2.2-linux-s390x	93e9e6bd009072b493c442703dbc5451777fedbde00314f1452443c5b84e9aa2
immuadmin-v1.2.2-windows-amd64.exe	636d76033514c7813a8be72ec3089ffe445c951aa9f238b254990bc0062c1220

v1.2.1 (ACID support, GDPR, Golang stdlib support and improved SQL compatibility)

Immudb 1.2.1 highlights

This release of immudb comes with many new features and improvements. The most important ones are highlighted below.

Improved transaction support

Starting with this version of immudb, SQL clients can now use true SQL transactions. It is now possible to start a transaction, perform multiple read and write operations and then commit or rollback changes. Before that each transaction had to be executed within one command sent to the server.

Logical deletion and expiration

KV and SQL records can now be deleted. This was enabled by introduction of metadata attached to entries that can store information about deletion or expiration. Such deleted entries are no longer available when trying to read KV or SQL entries but one can still lookup the history of a key or perform an SQL statement using time travel to fetch historical values before deletion.

In adddition to deletion, KV entries also support setting data expiration. Once expired, values of KV entries can no longer be retrieved. Expiration is a much stronger concept than deletion and once expired, immudb will protect against reading of the value for given key even if the history of an entry is retrieved.

Much improved SQL support

Due to introduction of metadata, immudb now supports the DELETE statement. Deleted rows do not appear in standard SQL queries anymore, however one can still query them using time travel feature.

This release also add support for a standard UPDATE statement that enables modifying only some columns of a row, before that the only way to update an existing entry was with the UPSERT statement that required values for all columns and could only be used with a primary key.

An improvement to the UPSERT allows updating rows for tables that have secondary indexes.

There are many more smaller improvements in the SQL engine for this release such as: indexes for columns with NULL values, native TIMESTAMP type, SELECT DISTINCT, IS [NOT] NULL, IN, basic CAST, LIKE NOT. We also improved SQL compatibility for table aliases and COUNT function.

Important notes

Breaking changes on the SQL layer

In order to support nullable columns, a breaking change for the on-disk format was introduced. After upgrading to immudb 1.2.1, the old data will not be available through the SQL interface even though immudb still contains the low-level KV representation of those entries. If you need to migrate the old data to the new format, please contact us on our discord channel.

Breaking changes in the SDK

One of the strengths of immudb is that immudb can prove to the client that the record was immutably stored in the database and that it was not tampered with. Verification of such proof is done on the client side.

Introduction of the entry metadata requires a slightly modified version of the proof that also needs to consider such metadata. An updated version of the SDK will be needed to correctly verify proofs for new entries.

There are also many changes in the API related to new transaction support. Because transaction requires server-side state, the client can now operate in a context of a server-side session. There are also many new commands related to transaction support.

Avoiding downtime during the upgrade

To avoid downtime during the upgrade, components should be upgraded in the following order:

• immudb clients
• immudb follower replica instances
• immudb main database

Changelog

[v1.2.1] - 2021-12-13

Bug Fixes

• protobuf: Fix compatibility with 1.1 version
• database: Internal consistency check on data reads
• database/meta: Do not crash on history with deleted items
• pkg/database: history skipping not found entries

Changes

• cmd/immuadmin/command: add super user login hint
• embedded/sql: use sql standard escaping with single quotes
• embedded/sql: support for escaped strings
• embedded/store: prevent value reading of expired entries
• embedded/store: fix expiration error declaration
• embedded/store: dedicated expiration error
• embedded/store: improve metadata serialization/deserialization methods
• embedded/store: validations during metadata deserialization
• embedded/store: return data corrupted error when deserialization cannot proceed
• embedded/store: use fixed time during the lifespan of a tx
• embedded/store: easily extendable meta attributes
• embedded/store: mandatory expiration filter
• embedded/store: reduce attribute code size
• pkg/auth: require admin permission to export and replicate txs
• pkg/integration: remove useless compilation tag on tests
• pkg/server: deprecate GetAuth and WithAuth
• pkg/server/sessions: session max inactivity time set to 3m and minor stat collecting fix
• pkg/server/sessions: tuning sessions params
• pkg/server/sessions: session timeout set to 2 min

Features

• embedded/store: logical entries expiration
• pkg/api: logical entries expiration
• pkg/client: expirable set

[v1.2.0-RC1] - 2021-12-07

Bug Fixes

• Update jaswdr/faker to v1.4.3 to fix build on 32-bit systems
• Makefile: Fix building immudb for specific os/arch
• Makefile: Use correct version of the grpc-gateway package
• embedded/sql: ensure determinism and no value overlaps distinct rows
• embedded/sql: fix inserting calculated null values
• embedded/sql: correct max key length validation based on specified col max length
• embedded/sql: fix rollback stmt
• embedded/sql: normalize parameters with lower case identifiers
• embedded/sql: param substitution in LIKE expression
• embedded/sql: Use correct statement for subquery
• embedded/sql: Do not modify value returned by colsBySelector
• embedded/sql: Fix SELECT * when joining with subquery
• embedded/sql: distinct row reader with limit argument
• embedded/store: release lock when tx has a conflict
• embedded/store: read conflict validation
• embedded/store: typo in error message
• pkg/auth: Fix password tests
• pkg/client: fix database name saving on token service
• pkg/database: sql exec on provided tx
• pkg/server: fix keep alive session interceptor
• testing: using pointers for job channels
• webconsole: Fix html of the default missing page.

Changes

• refining sdk client constructor and add readOnly tx guard
• Update build/RELEASING.md documentation.
• fix more tests
• decoupled token service
• token is handled internally by sdk. Remove useless code
• remove token service from client options and fix tests
• cmd/immuadmin/command: fix immuadmin token name on client creation
• cmd/immuclient: deleteKeys functioin and updates after metadata-related changes
• cmd/immuclient: temporary disable displaying hash in non-verified methods
• embeddded/tbtree: leverage snapshot id to identify it's the current unflushed one
• embedded/multierr: minor code simplification
• embedded/sql: set INNER as default join type
• embedded/sql: minor update after rebasing
• embedded/sql: Alter index key prefixes
• embedded/sql: wip rw transactions
• embedded/sql: remove opt_unique rule to ensure proper error message
• embedded/sql: minor code simplification
• embedded/sql: kept last snapshot open
• embedded/sql: Simplify row_reader key selection
• embedded/sql: Better error messages when (up|in)serting data
• embedded/sql: standardized datasource aliasing
• embedded/sql: wip sqlTx
• embedded/sql: de-duplicate tx attributes using tx header struct
• embedded/sql: fix nullable values handling
• embedded/sql: rollback token
• embedded/sql: set parsing verbose mode when instantiating sql engine
• embedded/sql: unsafe snapshot without flushing
• embedded/sql: reusable index entries and ignore deleted index entries
• embedded/sql: bound stmt execution to a single sqltx
• embedded/sql: delay index sync until fetching row by its pk
• embedded/sql: changes on tx closing
• embedded/sql: leverage metadata for logical deletion
• embedded/sql: use order type in scanSpecs
• embedded/sql: cancel non-closed tx
• embedded/sql: expose Cancel method
• embedded/sql: wip interactive sqltx
• embedded/sql: limit row reader
• embedded/sql: return map with last inserted pks
• embedded/sql: use int type for limit arg
• embedded/sql: defer execution of onClose callback
• embedded/sql: sql engine options and validations
• embedded/sql: standard count(*)
• embedded/sql: ddl stmts not counted in updatedRows
• embedded/sql: method to return sql catalog
• embedded/sql: non-thread safe tx
• embedded/sql: use current db from ongoing tx
• embedded/sql: postponing short-circuit evaluation for safetiness
• embedded/sql: wip sql tx preparation
• embedded/store: entryDigest calculation including key len
• embedded/store: conservative read conflict validation
• embedded/store: non-thread safe ongoing tx
• embedded/store: wip tx header versioning
• embedded/store: expose ExistKeyWithPrefix in OngoingTx
• embedded/store: set header version at commit time
• embedded/store: set tx as closed even on failed attempts
• embedded/store: strengthen tx validations
• embedded/store: GetWith method accepting filters
• embedded/store: handle watchersHub closing error
• embedded/store: remove currentShapshot method
• embedded/store: tx header version validations and increased max number of entries
• embedded/store: threadsafe tx
• embedded/store: filter out entries when filter evals to t...

v1.2.0-RC1

release: v1.2.0-RC1

Signed-off-by: Michele Meloni <[email protected]>

v1.1.0

Immudb 1.1.0 highlights

This release of immudb comes with many new features and improvements. The most important ones are highlighted below.

Follower replication

Immudb follower replication is now stable. This enables configuring a cluster of immudb nodes where follower replicas replicate transactions committed on the leader node.

S3 storage backend

S3 storage backend is now marked stable. With the S3 backend, immudb automatically uploads the data into an S3-compatible server reducing the amount of data stored locally.

Much improved SQL backend

Following new features were added to the SQL backend:

• Multi-column indexes
• Multi-column primary keys
• Unique indexes
• Support for negative values in INTEGER columns and calculations using negative values
• Huge improvements of query optimization with indexes
• AUTO_INCREMENT primary key

Improved docker images

New release comes with changes to how our docker images are built.

The main codenotary/immudb image is now built using a scratch base to reduce the image size and increase its security.

There are two more versions of the immudb image - one based on debian (codenotary/immudb:1.1.0-bulllseye-small) and one based on almalinux (codenotary/immudb:1.1.0-almalinux-8-minimal).

Important notes

Breaking changes on the SQL layer

This release is using completely new format for storing SQL data. This change was necessary to support new features but it is not compatible with the old format.
After upgrading to immudb 1.1.0, the old data will not be available through the SQL interface even though immudb still contains the low-level KV representation of those entries. If you need to migrate the old data to the new format, please contact us on our discord channel.

Go version support

Starting with the next release our official builds and docker images will be compiled and tested using go 1.17. In order to support older systems where go 1.17 is not easily available, we are providing statically-linked binaries. We also ensure that binaries can be manually compiled with older go versions down to 1.13 (excluding tests).

Changelog

[v1.1.0] - 2021-09-22

Bug Fixes

• Minor updates to build/RELEASING.md
• Update Dockerfile.alma maintainer field
• CI: Fix building and releasing almalinux images
• Dockerfile: Fix compiling version information in docker images
• Dockerfile.rndpass: Fix building rndpass docker image
• embedded/sql: suffix endKey when scan over all entries
• embedded/sql: in-mem catalog rollback and syncing fixes
• embedded/sql: exclude length from maxKey
• embedded/sql: return error when joint table doest not exist
• embedded/sql: support edge case of table with just an auto-increment column
• embedded/sql: limit auto-increment to single-column pks
• embedded/sql: take into account table aliasing during range calculations
• embedded/sql: improve error handling and parameters validation
• embedded/sql: adjust selector ranges calculation
• embedded/sql: consider boolean type in maxKeyVal
• embedded/sql: set type any to nil parameters
• embedded/sql: fix table aliasing with implicit selectors
• embedded/sql: enforce ordering by grouping column
• embedded/store: fix constraint condition
• embedded/store: error handling when setting offset fails
• pkg: improve signature verification during audit
• pkg/stdlib: fix driver connection releasing

Changes

• Update RELEASING.md with documentation step.
• remove wip warning for fully implemented features
• Add documentation badge to README.md
• Add documentation link at the beginning of README.md
• Add documentation link to command line help outputs
• Update codenotary maintainer info
• Remove experimental S3 warning from README
• CI: Build almalinux-based immudb image
• CI: Explicitly require bash in gh action building docker images
• CI: Use buildkit when building docker images
• Dockerfile: Build a debian-based image for immudb next to the scratch one
• Dockerfile: Use scratch as a base for immudb image
• Dockerfile: Remove unused IMMUDB_DBNAME env var
• Dockerfile: Update base docker images
• Makefile: More explicit webconsole version
• Makefile: Add darwin/amd64 target
• build.md: Add info about removing webconsole/dist folder
• cmd/immuadmin: remove replication flag shortcut
• cmd/immuadmin: parse all db flags when preparing settings
• cmd/immuadmin: improve flag description and rollback args spec
• cmd/immuclient: display number of updated rows as result of sql exec
• cmd/immudb: use common replication prefix
• docker: Update generation of docker tags
• embedded: leverage kv constraint to enforce upsert over auto-incremental pk requires row to already exist
• embedded/multierr: enhace multi error implementation
• embedded/sql: fix primary key supported types error message
• embedded/sql: get rid of limited joint implementation
• embedded/sql: ignore null values when encoding row
• embedded/sql: include constraint only when insert occurs without auto_incremental pk
• embedded/sql: wip scan optimizations based on query condition and sorting
• embedded/sql: partial progress on selector range calculation
• embedded/sql: partial progress on selector range calculation
• embedded/sql: expose primary key index id
• embedded/sql: leverage endKey to optimize indexing scanning
• embedded/sql: minor code refactoring
• embedded/sql: mark catalog as mutated when using auto incremental pk
• embedded/sql: fix max key length validation
• embedded/sql: catalog loading requires up to date data store indexing
• embedded/sql: optional parenthesis when specifying single-column index
• embedded/sql: disable TIMESTAMP data-type
• embedded/sql: move index selection closer to data source in query statements
• embedded/sql: optimize integer key mapping
• embedded/sql: use plain big-endian encoding for integer values
• embedded/sql: include support for int64 parameters
• embedded/sql: minor refactoring to simplify code
• embedded/sql: minor code simplification
• embedded/sql: use int64 as value holder for INTEGER type
• embedded/sql: add further validations when encoding values as keys
• embedded/sql: move index spec closer to ds
• embedded/sql: reserve byte to support multi-ordered indexes
• embedded/sql: index prefix function
• embedded/sql: use Cols as a replacement for ColsByID
• embedded/sql: changed identifiers length in catalog
• embedded/sql: validate non-null pk when decoding index entry
• embedded/sql: limit upsert to tables without secondary indexes
• embedded/sql: remove join constraints
• embedded/sql: convert unmapIndexedRow into unmapRow with optional indexed value
• embedded/tbtree: typo in log message
• embedded/tbtree: compaction doesn't need snapshots to be closed
• embedded/tbtree: return kv copies
• embedded/tbtree: adjust seekKey based on prefix even when a value is set
• embedded/tools: update sql stress tool with exec summary
• pkg/api: changed db identifiers type
• pkg/api: use fresh id in proto message
• pkg/api: use a map for holding latest auto-incremental pks
• pkg/api: use int64 as value holder for INTEGER type
• pkg/api: use follower naming for replication credentials
• pkg/api: include updated rows and last inserted pks in sql exec result
• pkg/api: delete deprecated clean operation
• pkg/client: changed db identifiers type
• pkg/client: move unit testing to integration package to avoid circular references
• pkg/database: warn about data migration needed
• pkg/database: minor refactoring coding conventions
• pkg/database: remove active replication options from database
• pkg/database: create sql db instance if not present
• pkg/database: minor adjustments based on multi-column indexing
• pkg/database: update integration to exec summary
• pkg/database: display as unique if there is a single-column index
• pkg/database: include updated rows and last inserted pks in sql exec result
• pkg/database: warn about data migration needed
• pkg/database: minor renaming after rebase
• pkg/pgsql/server: adds pgsql server maxMsgSize 32MB limit
• pkg/pgsql/server: add a guard on payload message len
• pkg/replication: handle disconnection only within a single thread
• pkg/replication: use new context for each client connection
• pkg/replication: use info log level for network failures
• pkg/server: followers management
• pkg/server: validate replication settings
• pkg/server: nil tlsConfig on default options
• pkg/server: change max concurrency per database to 30
• pkg/server: changed default db file size and make it customizable at db creation time
• pkg/server: use replica wording
• pkg/stdLib: implementing golang standard sql interfaces
• pkg/stdlib: remove pinger interface implementation and increase code coverage
• pkg/stdlib: increase code coverage and fix blob results scan
• pkg/stdlib: immuclient options identifier(uri) is used to retrieve cached connections
• pkg/stdlib: simplified and hardened uri handling

Features

• Dockerfile for almalinux based image
• cmd/immuadmin: add replication flags
• cmd/immuadmin: add flag to exclude commit time
• embedded/multierr: implement stardard error Is & As methods
• embedded/sql: wip unique multi-column indexes
• embedded/sql: inner join with joint table and subqueries
• embedded/sql: towards more powerful joins
• embedded/sql: value expressions in row spe...

v1.0.5

Changelog

[v1.0.5] - 2021-08-02

Bug Fixes

• bind psql port to the same IP address as grpc interface (#867)
• Update crypto, sys dependencies
• consistent reads of recently written data
• embedded/ahtree: fix the full revert corner case
• embedded/store: Truncate aht before commit
• embedded/store: Don't fail to open on corrupted commit log
• embedded/store: revert change so to prevent nil assigments
• embedded/store: handle missing error case during commit phase
• embedded/store: use reserved concurrency slot for indexing
• embedded/tbtree: ensure clog is the last one being synced
• embedded/tbtree: garbage-less nodes log
• embedded/tbtree: use padding to ensure stored snapshots are named following lex order
• embedded/tbtree: flush logs containing compacted index
• embedded/tbtree: ensure proper data flushing and syncing
• pkg/client/auditor: fix and enhance state signature verification
• pkg/pgsql/server: fix boolean and blob extended query handling
• pkg/pgsql/server: hardened bind message parsing and fix leftovers
• pkg/pgsql/server/fmessages: use a variable size reader to parse fe messages
• pkg/server: initialize db settings if not present
• pkg/server: lock userdata map read
• s3: Use remote storage for index
• s3: Use remote storage for new databases
• sql/engine: Harden DecodeValue
• store/indexer: Ensure indexer state lock is always unlocked

Changes

• Better logging when opening databases
• increased coverage handling failure branches (#861)
• remove unused interceptors and add missing error code prefixes
• move sqlutils package to schema
• group user methods in a dedicated file
• Update dependencies
• appendable: Expose validation functions of appendable options
• appendable/multiapp: Introduce appendableLRUCache
• appendable/multiapp: Add hooks to the MultiFileAppender implementation
• cmd/immuclient: fix panic in immuclient cli mode
• cmd/immuclient: update error comparisson
• embedded: col descriptor with attributes
• embedded/ahtree: minor changes towards code redabilitiy
• embedded/ahtree: minor refactoring improving readability
• embedded/ahtree: auto-truncate partially written commit log
• embedded/cache: Add Pop and Replace methods to LRUCache.
• embedded/sql: initial type specialization in place
• embedded/sql: dump catalog with a different database name
• embedded/sql: Remove linter warnings
• embedded/sql: explicit catalog reloading upon failed operations
• embedded/sql: parameters type inference working with aggregations
• embedded/sql: towards non-blocking sql initialization
• embedded/sql: remove public InferParameters operations from sql statements
• embedded/sql: several adjustments and completion in type inference functions
• embedded/sql: cancellable wait for catalog
• embedded/sql: expose InferParameters function in RowReader interface
• embedded/sql: type specialization
• embedded/sql: validate either named or unnamed parameters are used within the same stmt
• embedded/store: tx metatada serialization/deserialization
• embedded/store: minor code simplification
• embedded/store: validate replicated tx against current store
• embedded/store: minor refactoring improving readability
• embedded/store: edge-case validation with first tx
• embedded/store: auto-truncate partially written commit log
• embedded/tbtree: nodes and commit prefix renaming
• embedded/tbtree: auto-truncate partially written commit log
• embedded/tbtree: use setOffset for historical data overwriting
• embedded/tbtree: enable snapshot generation while compaction is in progress
• embedded/tbtree: expose current number of snapshots
• embedded/tbtree: full snapshot recovery
• embedded/tbtree: warn if an error is raised while discarding snapshots
• pkg/api: kept simple db creation api to guarantee backward compatibility
• pkg/api: kept same attribute id in TxEntry message
• pkg/api: fix typo inside a comment
• pkg/api: comment on deprecated and not yet supported operations
• pkg/api: remove information not required to cryptographically prove entries
• pkg/auth: list of supported operations in maintenance mode
• pkg/database: support the case where database tx is not the initial one due to migration
• pkg/database: sql operations after catalog is created
• pkg/database: single-store databases
• pkg/database: method to retrieve row cursor based on a sql query stament
• pkg/database: no wait for indexing during tx replication
• pkg/database: use fixed database name
• pkg/database: sql catalog reloading on replica
• pkg/database: migrate systemdb catalog to fixed database naming
• pkg/database: internal method renaming
• pkg/database: sql catalog per database. migration from shared catalog store when required
• pkg/database: replace fixed naming with current database
• pkg/database: replace fixing naming with current database
• pkg/database: expose catalog loading operation
• pkg/database: catalog reloading by replicas
• pkg/database: parameter inference for parsed statements
• pkg/database: re-construct sql engine once catalog is ready
• pkg/database: wait for sql engine initialization before closing
• pkg/database: add IsReplica method
• pkg/database: gracefully stop by cancelling sql initialization
• pkg/database: log when a database is sucessfully opened
• pkg/database: log warning about WIP feature when using replication capabilities
• pkg/errors: immuerrors use an internal map to determine code from the message
• pkg/errors: add more error codes and add Cod prefix to codes var names
• pkg/errors: fix user operations error codes with pgsql official ones, increase coverage
• pkg/errors: add comments
• pkg/pgsql: increase server coverage
• pkg/pgsql/server: protect parameters description message from int16 overflown
• pkg/pgsql/server: increase code coverage
• pkg/pgsql/server: handle positional parameters
• pkg/pgsql/server: add max parameters value size guard and move error package in a higher level to avoid cycle deps
• pkg/pgsql/server: add bind message negative value size guards
• pkg/pgsql/server: handle empty statements
• pkg/pgsql/server: hardened INTEGER parameters conversion
• pkg/pgsql/server: increase multi inserts tests number in simple and extended query
• pkg/pgsql/server: some polishments in the state machine
• pkg/pgsql/server: simplify query machine
• pkg/pgsql/server: add a guard to check max message size and handle default case in parsing format codes in bind messages
• pkg/pgsql/server/fmessages: uniform malformed bind messages
• pkg/server: systemdb renaming
• pkg/server: disable user mgmt operations in maintenance mode
• pkg/server: move userdata lock in the inner method getLoggedInUserDataFromUsername
• pkg/server: minor updates after rebasing
• pkg/server: remove duplicated property
• pkg/server: minor adjustments after rebasing from master branch
• pkg/server: fix typo in error message
• pkg/server: remove methods moved to user file
• pkg/stream: inject immu errors
• pkg/stream: fix namings on stream api objects
• pkg/stream: use io.Reader interface
• pkg/stream: use wrapped errors

Features

• immuclient running as auditor - replace "prometheus-port" and "prometheus-host" CLI flags with "audit-monitoring-host" and "audit-monitoring-port" (int) and start a single HTTP server which exposes all the needed endpoints (GET /metrics, /initz, /readyz, /livez and /version)
• add /healthz and /version endpoints for immudb and immuclient auditor
• add immudb error package
• appendable: Add remote s3 backend
• cmd/immuadmin: update database command
• cmd/immuadmin: upgrade database creation with settings
• cmd/immuadmin: add flag to create database as a replica
• cmd/immuclient: upgrade database creation with settings
• embedded/sql: support for named positional parameters
• embedded/sql: towards leveraging readers for type inference
• embedded/sql: catalog dumping
• embedded/sql: adding method to infer typed parameters from sql statements
• embedded/sql: support for unnamed parameters
• embedded/store: WIP replicatedCommit
• embedded/store: passive waiting for transaction commit
• embedded/store: tx export and commit replicated
• pkg/api: enhanced createDatabase endpoint to specify database replication
• pkg/api: new endpoint to update database settings
• pkg/api: endpoints for exporting and replicating txs
• pkg/client: replica creation and replication API
• pkg/client: implements update database settings operation
• pkg/client: deprecate CleanIndex operation
• pkg/database: suppport runtime replication settings changes
• pkg/database: implement passive wait for committed tx
• pkg/database: parameters type inference exposed in database package
• pkg/database: db as replica and replication operations
• pkg/error: add improved error handling
• pkg/pgsql/server: add extended query messages and inner logic
• pkg/server: initial handling of database replication settings
• pkg/server: replicas and replication endpoints
• pkg/server: implements update database settings endpoint
• pkg/server: leverage maintenance mode to recover systemdb and defaultdb databases
• pkg/server: stream of committed txs
• pkg/server: enable simultaneous replication of systemdb and defaultdb
  ...

v1.0.1

Changelog

[v1.0.1] - 2021-06-02

Bug Fixes

• go mod tidy/vendor with statik module (#796)
• cmd/immuclient: remove warnings on sql commands in interactive mode
• cmd/immuclient: improve immuclient tx and safetx error message
• embedded/sql: interprete binary prefix if followed by a quote
• pkg/server: always create system db (even when auth is off)

Changes

• enable webconsole in docker image
• remove mtls evironments var from dockerfile
• enhance Makefile so to automatically download latest webconsole if not already present
• README/doc updates (#791)
• embedded/store: apply synced settings to indexing data
• embedded/store: sync values once all entries are written
• pkg/database: retry database selection after registration
• pkg/database: auto-registration when not present in the catalog

Features

• embedded/sql: support NULL syntax
• pkg/database: enhace table description by adding nullable constraint
• webconsole: default web console page (#786)

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v1.0.1-darwin-amd64	6e31ae0bb3272dc1b423db3f8b173e8805768737b0c5b73748cff0f46c8761ce
immudb-v1.0.1-freebsd-amd64	0961a3c522e98228637cb9c708da201a43d549cd40a65a56addb36cc1e992d32
immudb-v1.0.1-linux-amd64	026723f2f78ddcb978e1621cadf1b907bdd20fa89ce2ed055829add5b1fe3bb3
immudb-v1.0.1-linux-amd64-static	c41c1a87d3b6a7f4c0ffdce5536611ec989cc9a92ed6c37c096549178d5186aa
immudb-v1.0.1-linux-arm64	15cd77d6bfe3edae45c815bcb2a620f1ad182661f3cd54a5fa1dbf1babbf00d6
immudb-v1.0.1-linux-s390x	419f14721ce90033040bd395877a7ee9d48ead76e748ff065d48c58207abf596
immudb-v1.0.1-windows-amd64.exe	f6c394ab088c1d3c9ad3fc8c9ecc9e34b5533777f4b099aaeca85ff9b7115df2
immudb-v1.0.1-zos-s390x	c3337370b3186ec569c9eba1e5111034beaf543f6bb9de619bb96dc46df6e26d

Immuclient Binaries

File	SHA256
immuclient-v1.0.1-darwin-amd64	ee60a2c26bf02f863d94abd78ed5cce7822733981a937634a8f6b19a52a76049
immuclient-v1.0.1-freebsd-amd64	ef86dc8210e506188d92866245e4aba5e3fc1414546976d16a2c5d1d9ce934b9
immuclient-v1.0.1-linux-amd64	d639e29cdf8435e45eb87d6aed89e87f4a6916b6897a6c0de6be9e4a26bd524f
immuclient-v1.0.1-linux-amd64-static	b1d59a06945a7333db920063bf850d62c91791a2ccd7ece69983784e23f941e4
immuclient-v1.0.1-linux-arm64	06fc02a57cdc2eb39008d599205c6feb5b03f6d86d3afc896d2adeb615277b7b
immuclient-v1.0.1-linux-s390x	32899bb2a9cfa241d9e2aa072f79de2a60fd871fb51b75d7e6e1d13a061f0d5b
immuclient-v1.0.1-windows-amd64.exe	e9991880454417fd9fc8940836b4944c5ad2168a1692c65b0a70c56427c9ba52
immuclient-v1.0.1-zos-s390x	1d794eca0dd84af118fd6dfb9be82759dbb3d5df575718129192cfc8a7d7ae01

Immuadmin Binaries

File	SHA256
immuadmin-v1.0.1-darwin-amd64	d1db4636da70fd6abbe72de217db75229b8b16ef1bbaa226a0dcd86d96e2276f
immuadmin-v1.0.1-freebsd-amd64	8e95ce05039481aa1e4a1d0ace808c7575ea37ee6d63d3dad48d933df910673c
immuadmin-v1.0.1-linux-amd64	640d73fc01697a75d41ea8ef78e35f938a6e6fbbfefa7295ff2c708f692b7f4a
immuadmin-v1.0.1-linux-amd64-static	37edd3e2829aa40a2aae19a26ad3f3614330e130e177b006c18d34e1b270b22e
immuadmin-v1.0.1-linux-arm64	eff637e71c14de3b44a8e3b82fdd95d9a8343427e311215f607bec80f3e96d3a
immuadmin-v1.0.1-linux-s390x	af02523e082cca45317ea343601ce65743ce0d4f721a647939c961362bdd6a37
immuadmin-v1.0.1-windows-amd64.exe	e1a22fb27c0423399cd770939afa42f054127b17b53bbdcc5a6a0d3da30fc38d

1.0.0 (SQL)

Highlights

Major new features:

• Relational (SQL) Support
  immudb can be used both as a key-value store and as a relational database (SQL), making it the first immutable database with relational capabilities. immudb also expands tampering verification capabilities to rows.
• TimeTravel(™)
  immudb is the first database which allows you to do queries across time. You can pin a table into a point in time inside a query or subquery, allowing for powerful comparisons.
• immudb Web Console
  immudb includes an embedded web console with administration and query capabilities. Create users, manage databases, and query relational data with no additional downloads or tools.
• PostgreSQL wire protocol Compatibility
  immudb SQL features can be used with libraries compatible with the pgsql protocol. This means popular database access frameworks already available in Ruby, C, Java JDBC, PHP, etc., can be used to perform operations on immudb.

Changelog

[v1.0.0] - 2021-05-20

Bug Fixes

• fix windows installer service and missing flags
• make prequisites fixes introduced in #726 (#732)
• tlsConfig is always non-nil
• cmd/immuclient/immuclienttest: fix options injection in client test helper (#749)
• embedded: ensure readers get properly closed
• embedded/sql: fix selector aliasing
• embedded/sql: add missing error handling
• embedded/sql: close reader after loading catalog
• embedded/sql: prevent side effects in conditional clauses
• embedded/store: fix issue when resuming indexing
• embedded/store: notified latest committed tx when opening store
• embedded/store: fix indexing data race
• pkg/client: row verification with nullable values
• pkg/client/cache: fix lock file cache issue on windows
• pkg/client/cache: clean state file when re-writing old stetes
• pkg/database: use SQLPrefix when reopening database
• pkg/database: unwrap parameter before calling sqlexec
• pkg/pgsql/server: handle data_row message with text format
• pkg/server: complete error handling
• pkg/server: disable pgsql server by default and fix previous server tests
• pkg/sql: columns resolved with aliases
• pkg/sql: resolve shift/reduce conflict in SELECT stmt

Changes

• blank line needed after tag or interpreted as comment
• improved make dist script
• add pgsql related flags
• increase coverage and minor fix
• README SDK description and links (#717)
• github workflow to run stress_tool (#714)
• make roadmap about pgsql wire more explicit (#723)
• fix immugw support
• dblist interface is moved to database package and extended
• expose missing methods to REST (#725)
• fix rebase leftovers
• fix acronym uppercase
• revert 3114f92 in order to allow insecure connection on grpc server
• bundle webconsole inside dist binaries
• reword wire compatibility
• inject immudb user authentication
• move concrete class dblist to database package
• cmd/immuclient: render raw values
• cmd/immuclient: query result rendering
• cmd/immudb: add debug info env var details
• cmd/immudb/command: restore missing pgsql cmd flag
• cmd/immudb/command: enabled pgsql server only in command package
• cmd/immudb/command: remove parsing path option in unix
• cmd/immudb/command: handle tls configuration errors
• embedded/cache: thread-safe lru-cache
• embedded/sql: set 'x' as blob prefix
• embedded/sql: store non-null values and only col ids on encoded rows
• embedded/sql: case insensitive functions
• embedded/sql: validate table is empty before index creation
• embedded/sql: case insensitive identifiers
• embedded/sql: minor refactoring to expose functionality needed for row verification
• embedded/sql: keep one snapshot open and close when releasing
• embedded/sql: resolve query with current snapshot if readers are still open
• embedded/sql: improved nullables
• embedded/sql: skip tabs
• embedded/sql: move sql engine under embedded package
• embedded/sql: safer support for selected database
• embedded/sql: expose functionality needed for row verification
• embedded/store: pausable indexer
• embedded/store: commitWith callback using KeyIndex interface
• embedded/store: use specified sync mode also for the incremental hash tree
• embedded/store: log during compaction
• embedded/store: use indexer state to terminate indexing goroutine
• embedded/store: index info to return latest indexed tx
• embedded/tbree: postpone reader initialization until first read
• embedded/tbtree: make snapshot thread-safe
• embedded/tbtree: remove dots denoting progress when flushing is not needed
• embedded/tbtree: index compaction if there is not opened snapshot, open snapshot if compaction is not in already in progress
• embedded/watchers: cancellable wait
• pkg/api: render varchar as quoted strings
• pkg/api: render varchar as raw string value
• pkg/api: include data needed for row verification
• pkg/api: sql api spec
• pkg/api: render varchar as quoted string
• pkg/api/schema: Handle tools via modules (#726)
• pkg/auth: add SQL-related permissions
• pkg/auth: perm spec for row verification endpoint
• pkg/client: use to fetch current database name
• pkg/client: remove deprecated operations
• pkg/client: improved sql API
• pkg/client: auto convert numeric values to uint64
• pkg/client/cache: release lock only if locked file is present, and wait for unlock when already present
• pkg/client/clienttest: add VerifiedGetAt mock method
• pkg/database: upgrade wait for indexing api
• pkg/database: use MaxKeyScanLimit to limit query results
• pkg/database: set implicit database using UseDatabase method
• pkg/database: support for nullable values
• pkg/database: add missing copy
• pkg/database: make use of UseDatabase operation
• pkg/database: improved parameters support
• pkg/database: ensure rowReader get closed upon completion
• pkg/database: typed-value proto conversion
• pkg/database: row verification against kv-entries
• pkg/database: return mapped row values
• pkg/database: upgrade ExecAll to use KeyIndex interface
• pkg/database: use store-level snapshots
• pkg/database: close sql engine when db gets closed
• pkg/database: support multi-selected columns
• pkg/database: towards prepared sql query support
• pkg/database: support index compaction with sql engine in place
• pkg/embedded: introduce Snapshot at Store level
• pkg/pgsql: handle parameter status and terminate messages
• pkg/pgsql: bind immudb sql engine
• pkg/pgsql: fix filename format
• pkg/pgsql: add pgsql server wire protocol stub
• pkg/pgsql: use options flag to determine if pgsql server need to be launched
• pkg/pgsql: handle empty response and command complete message
• pkg/pgsql/client: add jackc/pgx pgsql client for testing purpose
• pkg/pgsql/server: move sysdb in session constructor
• pkg/pgsql/server: handle version statement
• pkg/pgsql/server: handle an ssl connection request if no certificate is present on server
• pkg/pgsql/server: improve error handling when client message is not recognized
• pkg/pgsql/server: minor fixes and leftovers
• pkg/pgsql/server: enforce reserved statements checks
• pkg/pgsql/server: fix connection upgrade pgsql protocol messages
• pkg/pgsql/server: protect simple query flow with a mutex
• pkg/pgsql/server: limit number of total connections and do not stop server in case of errors
• pkg/pgsql/server: add missing copyright
• pkg/pgsql/server: add debug logging messages, split session handling in multiple files
• pkg/pgsql/server: default error in simplequery loop has error severity
• pkg/pgsql/server: remove host parameter
• pkg/server: fix db mock
• pkg/server: remove unused options
• pkg/server: inject sqlserver in main immudb server
• pkg/server: load systemdb before any other db
• pkg/server: use systemdb as a shared catalog store
• pkg/server: renamed server reference
• pkg/server: remove tls configuration in server
• pkg/sql: using new KeyReaderSpec
• pkg/sql: grammar adjustments to support aggregated columns
• pkg/sql: swap LIMIT and ORDER BY parse ordering
• pkg/sql: row reader to support GROUP BY behaviour
• pkg/sql: make row values externally accessible
• pkg/sql: support for SUM aggregations
• pkg/sql: unify augmented and grouped row readers
• pkg/sql: refactored AggregatedValue and TypedValue interfaces
• pkg/sql: column descriptors in row readers
• pkg/sql: improve error handling
• pkg/sql: return ErrNoMoreRows when reading
• pkg/sql: row reader used to close the snapshot once reading is completed
• pkg/sql: remove alter column stmt
• pkg/sql: many internal adjustments related to name binding
• pkg/sql: upgrade to new store commit api
• pkg/sql: joins limited to INNER type and equality comparison against ref table PK
• pkg/sql: add comment about nested joins
• pkg/sql: mapping using ids, ordering and renaming support
• pkg/sql: composite readers
• pkg/sql: wip multiple readers
• pkg/sql: catch store indexing errors
• pkg/sql: add generated sq...

release: v0.9.2

Highlights

The main highlights of this release are:

• immudb gRPC API now support streams.
  The Go SDK provides the methods Stream* to perform streaming operations. Eg. StreamSet, StreamGet
• In the past, immudb returned by default the last indexed transaction. As the index was asynchronous, it may not be the last insertion, unless SinceTx was specified with a value bigger than the current transaction.
  immudb now defaults to a more intuitive behavior, where operations will wait until indexing is completed and return the last transaction, unless NoWait=true is specified.
• immudb now performs index compaction asynchronously, without blocking insertions.

Changelog

[v0.9.2] - 2021-04-07

Bug Fixes

• fix StreamVerifiedSet and Get and add an (integration) test for them
• fix inclusion proofs in StreamVerifiedSet and Get
• password reader 'inappropriate ioctl for device' from stdin (#658)
• include AtTx in StreamZScan response
• embedded: fix data races
• embedded: use mutex to sync ops at tx lru-cache
• embedded/store: sync ReadTx operation
• embedded/store: fix indexing sync and error retrieval
• embedded/store: ensure waitees get notified when store is restarted
• embedded/store: remove checking for closed store when fetching any vlog
• embedded/store: continue indexing once index is replaced with compacted index
• embedded/store: set delay with duration in ms
• embedded/store: ensure watchers get notified when indexing is up-to-date
• embedded/tbtree: release lock when compaction thld was not reached
• embedded/tbtree: insertion delay while compacting not affecting compaction
• embedded/tbtree: set lastSnapshot once flushed is completed
• pkg/auth: add missing stream write methods to permissions
• pkg/client: fix minor leftover
• pkg/client: fix security issue: if client local state became corrupted an error is returned
• pkg/client: ensure dual proof verification is made when there is a previously verified state
• pkg/database: wrap seekKey with prefix only when seekKey is non-empty
• pkg/server: use latest snapshot when listing users

Changes

• add max tx values length guard and remove code duplication
• move stream service to a proper package
• improve serverside stream error handling
• add video streamer command
• renaming stream methods
• updating copyright
• renaming stream methods, add stubs and stream service factory
• fix binary notation
• in server store creation max value entry is fixed to 32Mb
• mocked server uses the inner immudb grpc server and can be gracefully stopped
• set stream supports multiple key values
• fixed minimum chunk size at 4096 bytes
• refactor code quality issues
• increase stream coverage and add a guard if key is present on a stream but no value is found
• remove fake proveSinceTxBs key send in streamVerifiableSet
• polish streams methods and add comments
• embedded: remove unused cbuffer package
• embedded: add logger
• embedded: fix some typos with comments
• embedded: log indexing notifications
• embedded: descriptive logs on indexing and already closed errors
• embedded: add logs into relevant operations
• embedded: compaction and snapshot handling
• embedded/appendable: thread-safe multi-appendable
• embedded/appendable: sync before copying appendable content
• embedded/appendable: multi-appendable fine-grained locking
• embedded/store: remove duplicated logging
• embedded/store: stop indexing while commiting with callback
• embedded/store: leverage fine-grained locking when reading tx data
• embedded/store: general improvements on snapshot management
• embedded/store: log number of transactions yet to be indexed
• embedded/store: use buffered channel instead of a circular buffer
• embedded/store: set a limit on indexing iteration
• embedded/store: set max file size to 2Gb (#649)
• embedded/store: remove conditional locking before dumping index
• embedded/store: lock-less readTx
• embedded/tbtree: terminate reader if prefix won't match any more
• embedded/tbtree: optimize seek position
• embedded/tbtree: revert seek key setting
• embedded/tbtree: sync key-history log during compaction
• embedded/tbtree: sync before dumping
• embedded/tbtree: optimize seek position
• embedded/watchers: broadcasting optimisation
• embedded/watchers: minor renaming
• embedded/watchers: accept non-continuous notification
• pkg/client: add a guard to check for min chunk size
• pkg/client: add stream service factory on client and increase stream coverage
• pkg/client: maps server error on client package
• pkg/client: remove local files tests
• pkg/client: add GetKeyValuesFromFiles helper method
• pkg/client: integration test is skipped if immudb server is not present
• pkg/database: return error while waiting for index to be up to date
• pkg/database: use in-mem current snapshot in execAll operation
• pkg/database: leverage lightweight waiting features of embedded store
• pkg/database: return error while waiting for index to be up to date
• pkg/database: ensure scan runs over fully up-to-date snapshot
• pkg/database: illegal state guard is added to verifiable get and getTx methods
• pkg/server: max recv msg size is set to 32M
• pkg/server: add a guard to check for min chunk size
• pkg/server: revert quit chan exposure
• pkg/server: add server error mapper interceptor
• pkg/server: add small delay for indexing to be completed
• pkg/server: exposes Quit chan
• pkg/stream: remove bufio.reader when not needed
• pkg/stream: remove duplicated code
• pkg/stream: add ErrNotEnoughDataOnStream error and ImmuServiceReceiver_StreamMock
• pkg/stream: renamed stream test package
• pkg/stream: add a guard to detect ErrNotEnoughDataOnStream on client side
• pkg/stream: remove bufio and add ventryreceiver unit test
• pkg/stream: add some comments to mesasge receiver
• pkg/stream: add more corner cases guards
• pkg/stream/streamtest: add dummy file generator
• tools: fix copyright
• tools/stream: get stream content directly from immudb
• tools/stream/benchmark: add stream benchmark command
• tools/stream/benchmark/streamb: add SinceTx value to getStream

Code Refactoring

• stream kvreceiver expose Next method to iterate over key values
• stream receiver implements reader interface
• use of explicit messages for stream request
• pkg/stream: use ParseValue func in zreceiver and remove the redundant readSmallMsg func
• pkg/stream: refactor receiver to increase simplicity

Features

• add flusher to stream data to client
• add Stream Scan and client stream ServiceFactory
• Add StreamVerifiedSet and StreamVerifiedGet
• add client->server stream handler
• increase default store max value length to 32MB
• refactors and implement server->client stream handler
• Remove unnecessary dependencies (#665)
• add support for user, password and database flags in immuclient (#659)
• chunk size is passed as argument in client and server
• embedded/store: expose insertion delay while compacting
• embedded/store: configurable compaction threshold to set the min number of snapshots for a compaction to be done
• embedded/store: integrate watchers to support indexing synchronicity
• embedded/store: tx header cache to speed up indexing
• embedded/tbtree: configurable insertion delay while compaction is in progress
• embedded/tbtree: automatically set seekKey based on prefixKey when it's not set
• embedded/watchers: lightweight watching center
• embedded/watchers: fetchable current state
• pkg/client: handle illegal state error
• pkg/database: non-blocking, no history compaction
• pkg/database: non-blocking index compaction
• pkg/database: default scan parameters using up-to-date snapshot
• pkg/server: add signature on stream verifiable methods and tests
• pkg/stream: add exec all stream

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v0.9.2-darwin-amd64	2cd830d7f4d042121348f60b8595c3c9052a2e8d417c1a2dd4001bee40e22490
immudb-v0.9.2-linux-amd64	7481ae5dfaede6d99849a88e0202bac4ae559db6868ea7499606d9328b110307
immudb-v0.9.2-linux-amd64-static	dcbf510939a27ccf0322b0e79348f5761f7cba265f0c5e299f616db78e0aa11e
immudb-v0.9.2-linux-s390x	0dcde091392400f5aabd4fae9ff80dd19112d3cd593fa28504e2f2cb84749001
immudb-v0.9.2-windows-amd64.exe	969186c033ebb1cfbc95b9cfc0bc24d1d8d318f42797bf8db204101971544e11
immudb-v0.9.2-freebsd-amd64	c342ebb3784f6845b2250b720390f9f4d25add7927ec55a155751e217326b4c3
immudb-v0.9.2-linux-arm64	528967d2edcb55d3dfdb8d48e852fbe574e815f6e1911b2ad84250b6bb827906

Immuclient Binaries

...

release: v0.9.1

Changelog

[v0.9.1] - 2021-02-08

Bug Fixes

• cmd/sservice: fix group creation linux cross command
• cmd/sservice: fix services management and add permissions guard
• embedded/history: read history log file to set initial offset
• embedded/store: mutex on txlog
• embedded/store: continued indexing
• embedded/store: fix indexing sync (#621)
• embedded/store: copy key inside TxEntry constructor
• embedded/tbtree: use minkey for desc scan
• embedded/tbtree: prevNode nil comparisson
• embedded/tbtree: determine entry by provided seekKey
• embedded/tbtree: fix key history ordering (#619)
• pkg/client: fix verifiedGetAt
• pkg/client/auditor: hide auditor password in logs
• pkg/client/cache: return an error if no state is found
• pkg/database: check key does not exists in latest state
• pkg/server: set default settings within DefaultStoreOptions method

Changes

• update acknowledgments
• cmd/sservice: minor fixes
• embeddded/tbtree: reduce mem allocs
• embedded: expose store opts
• embedded: refactor TxEntry
• embedded/store: validates targetTx is consistent with proof len
• embedded/store: sync access to commit and tx logs
• embedded/store: move TxReader code to its own file
• embedded/store: renamed reader as KeyReader
• embedded/store: minor KeyReader renaming
• embedded/store: adapt after History changes
• embedded/store: use conditional locking in indexing thread
• embedded/store/options.go: increase DefaultMaxKeyLen
• embedded/tbtree: reduce mem consumption
• embedded/tbtree: offset map per branch
• embedded/tbtree: change history file extension
• embedded/tbtree: history log file
• embedded/tbtree: return ErrOffsetOutOfRange if invalid offset was provided
• embedded/tbtree: configurable max key length
• pkg: current db included in signed state
• pkg: unit testing index cleanup, use selected db
• pkg/api: include server uuid and db name into state message
• pkg/api: history limit as int32
• pkg/api: minor changes in TxScan message
• pkg/client: bound reference if atTx is provided in VerifiedSetReferenceAt
• pkg/client: validate returned entries from metadata
• pkg/client: add state service lock and unlock capabilities
• pkg/client: set bound on SetReference and ZAdd
• pkg/client: strip prefix from returned keys in txById and verifiedTxById
• pkg/client: use indexing specified in GetRequest
• pkg/database: return empty list if offset is out of range
• pkg/database: catch NoMoreEntries error and return empty list on scan and zscan operations
• pkg/database: unsafe read tx inside CommitWith callback
• pkg/database: initial implementation of ExecAll with CommitWith
• pkg/server: include uuid and db as result of verifiable operations
• pkg/server: server mock wrapping default server implementation
• pkg/server: change server default options. Max key value to 10kb
• pkg/server: set default max value lenght to 1Mb
• pkg/server: initialize mts options
• pkg/server: expose store opts
• pkg/server: naming conventions
• pkg/server: use server wrapper to enable post processing of results

Features

• cmd/immuadmin: db index cleanup
• embedded: history with offset and limit, key updates counting
• embedded/appendable: check no closed and flush before copying
• embedded/appendable: implements Copy function
• embedded/appendable: flush and seek to start before copying
• embedded/store: index cleanup
• embedded/store: TxScan asc/desc order
• embedded/store: commitWith callback receiving assigned txID
• embedded/store: allow increasing max value size after creation time
• embedded/tbtree: HistoryReader to iterate over key updates
• embedded/tbtree: complete history implementation
• embedded/tbtree: full dump using copy on history log
• embedded/tbtree: full dump of current snapshot
• pkg: index cleanup service
• pkg/client: add verifiedGetSince
• pkg/client: implementation of TxScan operation
• pkg/client: add state file locker
• pkg/database: TxScan implementation
• pkg/database: support for free and bound references
• pkg/database: history with offset and limit
• pkg/database: KeyRequest retrieves key at a specific tx or since a given tx
• pkg/server: sign new state within verifiable operations
• pkg/server: use exposed synced mode

Downloads

Docker image
https://hub.docker.com/r/codenotary/immudb

Immudb Binaries

File	SHA256
immudb-v0.9.1-darwin-amd64	c681fdbd52d912fa51cf47cd15bd45f26ca776ab439cecab4cef09a44b5968fa
immudb-v0.9.1-linux-amd64	d9e1d609d6a9cd68be7f8b2a8c1140de68b3772c77d4617257b964a2f00f3f73
immudb-v0.9.1-linux-amd64-static	529a9200376cff7093f15adfad1086e6282031aa15783d8c7a851b7789f837a9
immudb-v0.9.1-linux-s390x	3c2f3e60f4f090ff37c17fa417d8096a233f68df270cfe50278d5fea50b0bbe8
immudb-v0.9.1-windows-amd64.exe	cdf679879e4fc1bbe653a78b9d7819424421b91c7b758e7b537a37cab3ac1397

Immuclient Binaries

File	SHA256
immuclient-v0.9.1-darwin-amd64	20d59dd8d087586b6842897479f59e3f311412b951208a96f8f2e21cb7c0983d
immuclient-v0.9.1-linux-amd64	ceafb2f80ff26f8accdb2b49a45c1cd9129187e094807ae9d3c85876c3c4636e
immuclient-v0.9.1-linux-amd64-static	29c897a8fde369acea9cce2d9d9730c5e7316881f627445a525aa5e69fa5bd38
immuclient-v0.9.1-linux-s390x	ceb9700f519ae9eba02fb2470fcbd86e58730a163aba34d1f24bd64ea5a41c6f
immuclient-v0.9.1-windows-amd64.exe	750fb5e0c00c3f191a067a92383e5f5b1bd564594112c95a5947066a0598ae0d

Immuadmin Binaries

File	SHA256
immuadmin-v0.9.1-darwin-amd64	76a549b6b6e29bd9193a0c75663fee5c766ac412c98f1b57da280befccd0c05b
immuadmin-v0.9.1-linux-amd64	95164fe01012e18f13f78d0371fd7bf59b77988f7b5de27256fc90b29526c1de
immuadmin-v0.9.1-linux-amd64-static	c974770c94b41e8df0fcd3721d10012706e6f119eff646085cab5c545d1bbfa8
immuadmin-v0.9.1-linux-s390x	47fa8c9ff19e9a4e2cd5ef4f6738dc448085e4c382106836c4b26580eed63d6c
immuadmin-v0.9.1-windows-amd64.exe	36122ba55648c84d609c8715c47e370c4738a58b8b36f1e53976b5db751fa16a