v1.4.1

This release mostly contains refactoring and dependency updates, and some improvements in log operation cycle.

Improved log operation

• Add undeletetree command [#2638]
• Query information_schema compatible with MySQL 8 [#2652]
• Gracefully exit the program when lease expires [#2655]
• Fix MySQL world-writable config warning [#2713]
• Update provision tree script in k8s to use createtree [#2676]

Merkle refactoring

• Packages under merkle are deprecated and to be removed in the next releases [#2636, #2646, #2715].
• Use https://github.com/transparency-dev/merkle repository instead.

Minor refactoring

• Check the tile height invariant stricter [#2726]
• Remake dumplib into integration test [#2710]
• Remove unused code and data [#2709, #2711]

Some changes technically break semver, but most of this code was not intended for external use. E.g.:

• Unexport helper [#2720]

Dependency updates

• bitbucket.org/creachadair/shell: 0.0.6 -> 0.0.7 [#2637]
• cloud.google.com/go/spanner: 1.25.0 -> 1.31.0 [#2630, #2639, #2650, #2697]
• contrib.go.opencensus.io/exporter/stackdriver: 0.13.8 -> 0.13.12 [#2634, #2699, #2723]
• github.com/apache/beam: 2.32.0+incompatible -> 2.33.0+incompatible [#2620]
• github.com/fullstorydev/grpcurl: 1.8.2 -> 1.8.6 [#2621, #2674]
• github.com/google/go-cmp: 0.5.7 -> 0.5.8 [#2729]
• github.com/prometheus/client_golang: 1.11.0 -> 1.12.1 [#2662]
• github.com/pseudomuto/protoc-gen-doc: 1.5.0 -> 1.5.1 [#2677]
• github.com/transparency-dev/merkle: v0.0.1 [#2734]
• go.etcd.io/etcd/*/v3: 3.5.0 -> 3.5.4 [#2631, #2667, #2716]
• golang.org/x/tools: 0.1.5 -> 0.1.10 [#2615, #2618, ..., #2684]
• google.golang.org/api: 0.54.0 -> 0.77.0 [#2613, #2619, ..., #2728]
• google.golang.org/grpc: 1.40.0 -> 1.44.0 [#2617, #2635, #2643, #2661]
• google.golang.org/grpc/cmd/protoc-gen-go-grpc: 1.1.0 -> 1.2.0 [#2645]
• google.golang.org/protobuf: 1.27.1 -> 1.28.0 [#2686]
• protoc: 3.12.4 -> 3.20.1 [#2731]

Full Changelog: v1.4.0...v1.4.1

v1.4.0

v1.4.0

• Recommended go version for development: 1.17
  This is the version used by the cloudbuild presubmits. Using a different version can lead to presubmits failing due to unexpected diffs.
• GCP terraform script updated. GKE 1.19 and updated CPU type to E2

Dependency updates

Many dep updates, including:

• Upgraded to etcd v3 in order to allow grpc to be upgraded (#2195)
• etcd was v0.5.0-alpha.5, now v3.5.0
• grpc upgraded from v1.29.1 to v1.40.0
• certificate-transparency-go from v1.0.21 to v1.1.2-0.20210512142713-bed466244fa6
• protobuf upgraded from v1 to v2
• MySQL driver from 1.5.0 to 1.6.0

Cleanup

• Removed signatures from LogRoot and EntryTimestamps returned by RPCs (reflecting that there should not be a trust boundary between Trillian and the personality.)
• Removed the deprecated crypto.NewSHA256Signer function.
• Finish removing the LogMetadata.GetUnsequencedCounts() method.
• Removed the following APIs:
  • TrillianLog.GetLeavesByHash
  • TrillianLog.GetLeavesByIndex
  • TrillianLog.QueueLeaves
• Removed the incomplete Postgres storage backend (#1298).
• Deprecated LogRootV1.Revision field.
• Moved rfc6962 hasher one directory up to eliminate empty leftover package.
• Removed unused log_client tool.
• Various tidyups and improvements to merke & proof generation code.
• Remove some remnants of experimental map.

Storage refactoring

• NodeReader.GetMerkleNodes does not accept revisions anymore. The implementations must use the transaction's ReadRevision
  instead.
• TreeStorage migrated to using compact.NodeID type suitable for logs.
• Removed the tree storage ReadRevision and WriteRevision methods.
  Revisions are now an implementation detail of the current storages. The change allows log implementations which don't need revisions.
• Removed Rollback methods from storage interfaces, as Close is enough to cover the use-case.
• Removed the unused IsOpen and IsClosed methods from transaction interfaces.
• Removed the ReadOnlyLogTX interface, and put its only used GetActiveLogIDs method to LogStorage.
• Inlined the LogMetadata interface to ReadOnlyLogStorage.
• Inlined the TreeStorage interfaces to LogStorage.
• Removed the need for the storage layer to return ephemeral node hashes. The application layer always requests for complete subtree nodes comprising the compact ranges corresponding to the requests.
• Removed the single-tile callback from SubtreeCache, it uses only GetSubtreesFunc now.
• Removed SetSubtreesFunc callback from SubtreeCache. The tiles should be written by the caller now, i.e. the caller must invoke the callback.

v1.3.13

Cleanup

• Removed the experimental map API.

v1.3.12

Misc improvements

• Removed unused PeekTokens method from the quota.Manager interface.
• Ensure goroutines never block in the subtree cache (#2272).
• Breaking unnecessary dependencies for Trillian clients:
  • Moved verifiers from merkle into merkle/{log,map}verifiersub-pacakges,
    reducing the amount of extra baggage inadvertently pulled in by clients.
• Concrete hashers have been moved into subpackages, separating them from their
  registration code, allowing clients to directly pull just the hasher they're
  interested in and avoid the Trillian/hasher registry+protobuf deps.
• Moved some packages intended for internal-only use into internal packages:
  • InMemoryMerkleTree (indended to only be used by Trillian tests)
• Removed wrapper for etcd client (#2288).
• Moved --quota_system and --storage_system flags to main.go so that they
  are initialised properly. It might break depending builds relying on these
  flags. Suggested fix: add the flags to main.go.
• Made signer tolerate mastership election failures [#1150].
• testdb no longer accepts the --test_mysql_uri flag, and instead honours the
  TEST_MYSQL_URI ENV var. This makes it easier to blanket configure tests to use a
  specific test DB instance.
• Removed experimental Skylog folder (#2297).
• Fixed a race condition in the operation manager that should only affect tests
  (#2302).
• Run gofumpt formatter on the whole repository (#2315).
• Refactor signer operation loop (#2294).

Upgrades

• Dockerfiles are now based on Go 1.13 image.
• The etcd is now pinned to v3.4.12.
• The golangci-lint suite is now at v1.36.0.
• CI/CD has migrated from Travis to Google Cloud Build.
• prometheus from 1.7.1 to 1.9.0 (#2239, #2270).
• go-cmp from 0.5.2 to 0.5.4 (#2262).
• apache/beam from 2.26.0+incompatible to 2.27.0+incompatible (#2273).
• lib/pq from 1.8.0 to 1.9.0 (#2264).
• go-redis from 6.15.8+incompatible to 6.15.9+incompatible (#2215).

Process

• Recognise that we do not follow strict semantic versioning practices.

v1.3.11

Documentation

Added docs which describe the Claimant Model of transparency, a useful
framework for reasoning about the design and architecture of transparent
systems.

Misc improvements

• Fixed int to string conversion warnings for golang 1.15
• Metric improvements for fetched leaf counts
• Move tools.go into its own directory to help with dependencies

Dependency updates

• go-grpc-middleware from 1.2.0 to 1.2.2 (#2219, #2229)
• stackdriver from 0.13.2 to 0.13.4 (#2220, #2223)
• Google api from 0.28.0 to 0.29.0 (#2193)

Improvements to cloudspanner

New database-integration tests, #1740 have identified and help resolve inconsistencies and missing features between the various storage implementations.

Storage TX Interfaces

• QueueLeaves has been removed from the LogTreeTX interface because
  QueueLeaves is not transactionaal. All callers use the
  QueueLeaves function in the LogStorage interface.
• AddSequencedLeaves has been removed from the LogTreeTX for the same reason.

MySQL Dequeueing Change #2159

mysql will now remove leaves from the queue inside of UpdateLeaves rather than directly inside of Dequeue.
This change brings the behavior of the mysql storage implementation into line with the spanner implementation
and makes consistent testing possible.

Update Dependencies

Selected Dependency Updates

• etcd from v3.3.18 to 3.4.7 (#2090)
• etcd-operator from v0.9.1 to v0.9.4
• upgraded protoc version to latest (#2088)
• github.com/golang/protobuf to v1.4.1 (#2111)
• google.golang.org/grpc from v1.26 to 1.29.1 (#2108)

See all changes

v1.3.8

HTTP APIs

The HTTP/JSON APIs have been removed in favor of a pure gRPC intereface.
grpcurl is the recommended way
of interacting with the gRPC API from the commandline.

v1.3.7

Server Binaries

The trillian_log_server, trillian_log_signer and trillian_map_server
binaries have moved from github.com/google/trillian/server/ to
github.com/google/trillian/cmd. A subset of the server package has also
moved and has been split into cmd/internal/serverutil, quota/etcd and
quota/mysqlqm packages.

v1.3.6

Deployments

the kubernetes configs will now provision 5 nodes for trillian's etcd cluster,
instead of 3 nodes.
this makes the etcd cluster more resilient
to nodes becoming temporarily unavailable, such as during updates (it can now
tolerate 2 nodes being unavailable, instead of just 1).

Monitoring & Metrics

A count of the total number of individual leaves the logserver attempts to
fetch via the GetEntries.* API methods has been added.