Skip to content

Commit 8d697d1

Browse files
committed
README: add information about v2 and format upgrade
1 parent e768f25 commit 8d697d1

File tree

2 files changed

+38
-25
lines changed

2 files changed

+38
-25
lines changed

README.md

+38-23
Original file line numberDiff line numberDiff line change
@@ -86,16 +86,16 @@ differences.
8686

8787
## RocksDB Compatibility
8888

89-
Pebble strives for forward compatibility with RocksDB 6.2.1 (the latest version
90-
of RocksDB used by CockroachDB). Forward compatibility means that a DB generated
91-
by RocksDB 6.2.1 can be upgraded for use by Pebble. Pebble versions in the `v1`
92-
series may open DBs generated by RocksDB 6.2.1. Since its introduction, Pebble
93-
has adopted various backwards-incompatible format changes that are gated behind
94-
new 'format major versions'. The Pebble `master` branch does not support opening
95-
DBs generated by RocksDB. DBs generated by RocksDB may only be used with recent
96-
versions of Pebble after migrating them through format major version upgrades
97-
using previous versions of Pebble. See the below section of format major
98-
versions.
89+
Pebble `v1` strives for forward compatibility with RocksDB 6.2.1 (the latest
90+
version of RocksDB used by CockroachDB). Forward compatibility means that a DB
91+
generated by RocksDB 6.2.1 can be upgraded for use by Pebble. Pebble versions in
92+
the `v1` series may open DBs generated by RocksDB 6.2.1. Since its introduction,
93+
Pebble has adopted various backwards-incompatible format changes that are gated
94+
behind new 'format major versions'. Pebble `v2` and newer does not support
95+
opening DBs generated by RocksDB. DBs generated by RocksDB may only be used with
96+
recent versions of Pebble after migrating them through format major version
97+
upgrades using previous versions of Pebble. See the below section of format
98+
major versions.
9999

100100
Even the RocksDB-compatible versions of Pebble only provide compatibility with
101101
the subset of functionality and configuration used by CockroachDB. The scope of
@@ -157,10 +157,13 @@ range of Pebble versions support that format.
157157
| FormatMinTableFormatPebblev1 | 9 | No | v1 |
158158
| FormatPrePebblev1Marked | 10 | Background | v1 |
159159
| FormatSSTableValueBlocks | 12 | No | v1 |
160-
| FormatFlushableIngest | 13 | No | v1, master |
161-
| FormatPrePebblev1MarkedCompacted | 14 | Blocking | v1, master |
162-
| FormatDeleteSizedAndObsolete | 15 | No | v1, master |
163-
| FormatVirtualSSTables | 16 | No | v1, master |
160+
| FormatFlushableIngest | 13 | No | v1, v2, master |
161+
| FormatPrePebblev1MarkedCompacted | 14 | Blocking | v1, v2, master |
162+
| FormatDeleteSizedAndObsolete | 15 | No | v1, v2, master |
163+
| FormatVirtualSSTables | 16 | No | v1, v2, master |
164+
| FormatSyntheticPrefixSuffix | 17 | No | v2, master |
165+
| FormatFlushableIngestExcises | 18 | No | v2, master |
166+
| FormatColumnarBlocks | 19 | No | v2, master |
164167

165168
Upgrading to a format major version with 'Background' in the migration
166169
column may trigger background activity to rewrite physical file
@@ -171,18 +174,30 @@ writes if upgrading a live database through
171174
`RatchetFormatMajorVersion`, but the method call will not return until
172175
the migration is complete.
173176

177+
Upgrading existing stores can be performed via the `RatchetFormatMajorVersion`
178+
method. If the database does not use a custom comparer, merger, or block
179+
property collectors, the `pebble` tool can also be used, at the latest version
180+
that supports the format. For example:
181+
```
182+
# WARNING: only use if no custom comparer/merger/property collector are necessary.
183+
go run github.com/cockroachdb/pebble/cmd/[email protected] db upgrade <db-dir>
184+
```
185+
174186
For reference, the table below lists the range of supported Pebble format major
175187
versions for CockroachDB releases.
176188

177-
| CockroachDB release | Earliest supported | Latest supported |
178-
|---------------------|------------------------------------|---------------------------|
179-
| 20.1 through 21.1 | FormatMostCompatible | FormatMostCompatible |
180-
| 21.2 | FormatMostCompatible | FormatSetWithDelete |
181-
| 21.2 | FormatMostCompatible | FormatSetWithDelete |
182-
| 22.1 | FormatMostCompatible | FormatSplitUserKeysMarked |
183-
| 22.2 | FormatMostCompatible | FormatPrePebblev1Marked |
184-
| 23.1 | FormatSplitUserKeysMarkedCompacted | FormatFlushableIngest |
185-
| 23.2 | FormatSplitUserKeysMarkedCompacted | FormatVirtualSSTables |
189+
| CockroachDB release | Earliest supported | Latest supported |
190+
|---------------------|------------------------------------|-----------------------------|
191+
| 20.1 through 21.1 | FormatMostCompatible | FormatMostCompatible |
192+
| 21.2 | FormatMostCompatible | FormatSetWithDelete |
193+
| 21.2 | FormatMostCompatible | FormatSetWithDelete |
194+
| 22.1 | FormatMostCompatible | FormatSplitUserKeysMarked |
195+
| 22.2 | FormatMostCompatible | FormatPrePebblev1Marked |
196+
| 23.1 | FormatSplitUserKeysMarkedCompacted | FormatFlushableIngest |
197+
| 23.2 | FormatPrePebblev1Marked | FormatVirtualSSTables |
198+
| 24.1 | FormatFlushableIngest | FormatSyntheticPrefixSuffix |
199+
| 24.2 | FormatVirtualSSTables | FormatSyntheticPrefixSuffix |
200+
| 24.3 | FormatSyntheticPrefixSuffix | FormatColumnarBlocks |
186201

187202
## Pedigree
188203

format_major_version.go

-2
Original file line numberDiff line numberDiff line change
@@ -197,8 +197,6 @@ const (
197197
// block.
198198
FormatColumnarBlocks
199199

200-
// TODO(msbutler): add major version for synthetic suffixes
201-
202200
// -- Add new versions here --
203201

204202
// FormatNewest is the most recent format major version.

0 commit comments

Comments
 (0)