Skip to content

Commit 0b88d9a

Browse files
atovpekoatovpeko
authored
Merge branch 'atovpeko1/edu-628-timescale-db-2290-release' into atovpeko1/edu-628-timescale-db-2290-release-recompress-unordered
Signed-off-by: Anastasiia Tovpeko <114177030+atovpeko@users.noreply.github.com>
2 parents 8f3d788 + d8ba723 commit 0b88d9a

10 files changed

Lines changed: 178 additions & 28 deletions

File tree

‎astro.config.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1375,6 +1375,8 @@ export default defineConfig({
13751375
items: [
13761376
{ label: "add_columnstore_policy()", link: "/reference/timescaledb/hypercore/add_columnstore_policy" },
13771377
{ label: "remove_columnstore_policy()", link: "/reference/timescaledb/hypercore/remove_columnstore_policy" },
1378+
{ label: "add_compaction_policy()", link: "/reference/timescaledb/hypercore/add_compaction_policy" },
1379+
{ label: "remove_compaction_policy()", link: "/reference/timescaledb/hypercore/remove_compaction_policy" },
13781380
],
13791381
},
13801382
{

‎src/content/docs/get-started/choose-your-path/install-timescaledb.mdx‎

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -135,10 +135,9 @@ If you have previously installed {C.PG}, you may encounter errors following thes
135135

136136
The latest {C.TIMESCALE_DB} releases for {C.PG}:
137137

138-
- [{C.PG} 18](https://github.com/timescale/timescaledb/releases/download/2.28.1/timescaledb-postgresql-18-windows-amd64.zip)
139-
- [{C.PG} 17](https://github.com/timescale/timescaledb/releases/download/2.28.1/timescaledb-postgresql-17-windows-amd64.zip)
140-
- [{C.PG} 16](https://github.com/timescale/timescaledb/releases/download/2.28.1/timescaledb-postgresql-16-windows-amd64.zip)
141-
- [{C.PG} 15](https://github.com/timescale/timescaledb/releases/download/2.28.1/timescaledb-postgresql-15-windows-amd64.zip)
138+
- [{C.PG} 18](https://github.com/timescale/timescaledb/releases/download/2.29.0/timescaledb-postgresql-18-windows-amd64.zip)
139+
- [{C.PG} 17](https://github.com/timescale/timescaledb/releases/download/2.29.0/timescaledb-postgresql-17-windows-amd64.zip)
140+
- [{C.PG} 16](https://github.com/timescale/timescaledb/releases/download/2.29.0/timescaledb-postgresql-16-windows-amd64.zip)
142141

143142
For release information, see the [GitHub releases page](https://github.com/timescale/timescaledb/releases).
144143

‎src/content/docs/reference/timescaledb/hypercore/add_compaction_policy.mdx‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
title: add_compaction_policy()
3+
description: Set a policy to automatically compact unordered chunks in the columnstore
4+
products: [cloud, self_hosted]
5+
keywords: [columnstore, hypercore, policies, compaction, direct compress]
6+
---
7+
8+
import * as C from "@constants";
9+
10+
import SinceRelease from '@components/SinceRelease.astro';
11+
12+
import { Callout } from '@stainless-api/docs/components';
13+
14+
<SinceRelease version="2.29.0" product="tsdb" />
15+
16+
Create a [job](/reference/timescaledb/jobs-automation/add_job) that automatically compacts unordered {C.CHUNK}s in the
17+
{C.COLUMNSTORE}. Compaction combines overlapping batches within a {C.CHUNK} so queries no longer need an extra sort step
18+
to restore order.
19+
20+
<Callout variant="note">
21+
22+
You usually don't need to add this policy yourself. [Direct compress](/build/data-management/write-data/insert#improve-performance-with-direct-compress)
23+
creates it for you, and it is the main reason {C.CHUNK}s end up unordered. Use `add_compaction_policy` when you want to
24+
tune the schedule or bound how much work each run does. Direct compress is a tech preview, so this policy is too.
25+
26+
</Callout>
27+
28+
The policy only processes {C.CHUNK}s that are fully in the {C.COLUMNSTORE} and marked as unordered. It skips partial
29+
{C.CHUNK}s, which the [{C.COLUMNSTORE} policy](/reference/timescaledb/hypercore/add_columnstore_policy) handles, and frozen {C.CHUNK}s. If your {C.HYPERTABLE}
30+
doesn't use direct compress, it is unlikely to have unordered {C.CHUNK}s and this policy has nothing to do.
31+
32+
To view the policies that you set or the policies that already exist, see [informational views](/reference/timescaledb/informational-views/jobs).
33+
34+
## Samples
35+
36+
- **Add a compaction policy with the default 5 minute schedule**:
37+
38+
```sql
39+
SELECT add_compaction_policy('metrics');
40+
```
41+
42+
- **Run less often, bound the work per run, and skip {C.CHUNK}s still being written to**:
43+
44+
```sql
45+
SELECT add_compaction_policy('metrics',
46+
schedule_interval => INTERVAL '15 minutes',
47+
max_chunks => 10,
48+
max_batches => 500,
49+
inactive_for => INTERVAL '30 minutes');
50+
```
51+
52+
## Arguments
53+
54+
The syntax is:
55+
56+
```sql
57+
SELECT add_compaction_policy(
58+
hypertable = '<hypertable_name>',
59+
if_not_exists = true | false,
60+
schedule_interval = <interval>,
61+
initial_start = <timestamptz>,
62+
timezone = '<timezone>',
63+
max_chunks = <integer>,
64+
max_batches = <integer>,
65+
inactive_for = <interval>
66+
);
67+
```
68+
69+
| Name | Type | Default | Required | Description |
70+
|---------------------|-------------|--------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
71+
| `hypertable` | REGCLASS | - | ✔ | Name of the {C.HYPERTABLE} to run this [job](/reference/timescaledb/jobs-automation/add_job) on. |
72+
| `if_not_exists` | BOOLEAN | `false` | ✖ | Set to `true` so this job fails with a warning rather than an error if a compaction policy already exists on `hypertable`. |
73+
| `schedule_interval` | INTERVAL | `5 minutes` | ✖ | Set the interval between the finish time of the last execution of this policy and the next start. When direct compress creates this policy, it uses `1 minute` instead. |
74+
| `initial_start` | TIMESTAMPTZ | `NULL` | ✖ | Set the time this job is first run. |
75+
| `timezone` | TEXT | `NULL` | ✖ | Set to a valid time zone to mitigate DST shifting. If `initial_start` is set, subsequent executions of this policy are aligned on `initial_start`. |
76+
| `max_chunks` | INTEGER | `NULL` | ✖ | Set the maximum number of {C.CHUNK}s to process in a single run, including {C.CHUNK}s that fail. Leave unset to process every eligible {C.CHUNK}. |
77+
| `max_batches` | INTEGER | `NULL` | ✖ | Set the maximum number of batches to combine in each {C.CHUNK}. Leave unset for no limit. |
78+
| `inactive_for` | INTERVAL | `NULL` | ✖ | Only compact {C.CHUNK}s that have not been written to for this interval. Leave unset to compact every eligible {C.CHUNK} regardless of when it was last written to. |
79+
80+
## Returns
81+
82+
|Column|Type|Description|
83+
|-|-|-|
84+
|`job_id`|INTEGER|{C.TIMESCALE_DB} background job ID created to implement this policy|

‎src/content/docs/reference/timescaledb/hypercore/alter_table.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,7 @@ ALTER TABLE <table_name> SET (timescaledb.enable_columnstore,
8080
timescaledb.compress_segmentby = '<column_name> [, ...]',
8181
timescaledb.sparse_index = '<index>(<column_name>), <index>(<column_name>)',
8282
timescaledb.compress_chunk_time_interval='interval',
83+
timescaledb.direct_compress = true | false,
8384
ALTER <column name> SET NOT NULL,
8485
ADD CONSTRAINT <constraint_name> UNIQUE (<column name>, ... )
8586
);
@@ -95,6 +96,7 @@ ALTER TABLE <table_name> SET (timescaledb.enable_columnstore,
9596
|`timescaledb.sparse_index`| TEXT | {C.TIMESCALE_DB} evaluates the columns you already have indexed, checks which data types are a good fit for sparse indexing, then creates a sparse index as an optimization. | ✖ | Configure the sparse indexes for {C.COLUMNSTORE} {C.CHUNK}s. Requires setting `timescaledb.compress_orderby`. Aliases: `timescaledb.index`, `timescaledb.compress_index`, `timescaledb.compress_sparse_index`. Supported index types are `bloom(<column>, …)` for equality filters and `minmax(<column>)` for range filters; use a comma-separated list to set multiple. For when each is best, configuration patterns, and restrictions, see [Sparse indexes on the columnstore](/build/performance-optimization/indexing#sparse-indexes-on-the-columnstore). Set to an empty string to disable sparse indexes; call `ALTER TABLE your_table_name RESET (timescaledb.sparse_index);` to revert to the default selection. |
9697
| `timescaledb.compress_chunk_time_interval` | INTERVAL | - | ✖ | EXPERIMENTAL: reduce the total number of {C.CHUNK}s in the {C.COLUMNSTORE} for `table`. If you set `compress_chunk_time_interval`, {C.CHUNK}s added to the {C.COLUMNSTORE} are merged with the previous adjacent {C.CHUNK} within `chunk_time_interval` whenever possible. These {C.CHUNK}s are irreversibly merged. If you call [convert\_to\_rowstore](/reference/timescaledb/hypercore/convert_to_rowstore), merged {C.CHUNK}s are not split up. You can call `compress_chunk_time_interval` independently of other compression settings; `timescaledb.enable_columnstore` is not required. |
9798
| `interval` | TEXT | - | ✖ | Set to a multiple of the [chunk\_time\_interval](/reference/timescaledb/hypertables/set_chunk_time_interval) for `table`. |
99+
| `timescaledb.direct_compress` | BOOLEAN | `false` | ✖ | Compress data in memory during ingestion and write compressed batches directly to the {C.COLUMNSTORE}, instead of writing uncompressed rows first. Turns on the {C.COLUMNSTORE} if it isn't already enabled. If you set `timescaledb.enable_columnstore = false` in the same statement, `direct_compress` is silently ignored. See [Improve performance with direct compress](/build/data-management/write-data/insert#improve-performance-with-direct-compress). |
98100
| `ALTER` | TEXT | | ✖ | Set a specific column in the columnstore to be `NOT NULL`. |
99101
| `ADD CONSTRAINT` | TEXT | | ✖ | Add `UNIQUE` constraints to data in the columnstore. |
100102

‎src/content/docs/reference/timescaledb/hypercore/index.mdx‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,9 @@ import HCPolicyWorkflow from '@partials/_hypercore_policy_workflow.mdx';
3131
- [`add_columnstore_policy()`](/reference/timescaledb/hypercore/add_columnstore_policy): set a policy to automatically move {C.CHUNK}s in a {C.HYPERTABLE}
3232
to the {C.COLUMNSTORE} when they reach a given age
3333
- [`remove_columnstore_policy()`](/reference/timescaledb/hypercore/remove_columnstore_policy): remove a {C.COLUMNSTORE} policy from a {C.HYPERTABLE}
34+
- [`add_compaction_policy()`](/reference/timescaledb/hypercore/add_compaction_policy): set a policy to automatically compact unordered {C.CHUNK}s in
35+
the {C.COLUMNSTORE}
36+
- [`remove_compaction_policy()`](/reference/timescaledb/hypercore/remove_compaction_policy): remove a compaction policy from a {C.HYPERTABLE}
3437

3538
### Configuration
3639

‎src/content/docs/reference/timescaledb/hypercore/remove_compaction_policy.mdx‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
title: remove_compaction_policy()
3+
description: Remove a compaction policy from a hypertable
4+
products: [cloud, self_hosted]
5+
keywords: [hypercore, columnstore, policies, compaction, remove]
6+
---
7+
8+
import * as C from "@constants";
9+
10+
import SinceRelease from '@components/SinceRelease.astro';
11+
12+
import ReturnsVoid from '@partials/_returns-void.mdx';
13+
14+
<SinceRelease version="2.29.0" product="tsdb" />
15+
16+
Remove a [compaction policy](/reference/timescaledb/hypercore/add_compaction_policy) from a {C.HYPERTABLE}.
17+
18+
Unordered {C.CHUNK}s are no longer compacted automatically, so queries against them keep the extra sort step needed to
19+
restore order. To restart automatic compaction, call [add\_compaction\_policy](/reference/timescaledb/hypercore/add_compaction_policy) again. Turning off
20+
`direct_compress` also removes the policy it created.
21+
22+
## Samples
23+
24+
You see the compaction policies in the [informational views](/reference/timescaledb/informational-views/jobs).
25+
26+
- **Remove the compaction policy from the `metrics` {C.HYPERTABLE}**:
27+
28+
```sql
29+
SELECT remove_compaction_policy('metrics');
30+
```
31+
32+
## Arguments
33+
34+
The syntax is:
35+
36+
```sql
37+
SELECT remove_compaction_policy(
38+
hypertable = '<hypertable_name>',
39+
if_exists = true | false
40+
);
41+
```
42+
43+
| Name | Type | Default | Required | Description |
44+
|--|--|--|--|-|
45+
|`hypertable`|REGCLASS|-|✔| Name of the {C.HYPERTABLE} to remove the policy from|
46+
| `if_exists` | BOOLEAN | `false` |✖| Set to `true` so this job fails with a warning rather than an error if a compaction policy does not exist on `hypertable` |
47+
48+
## Returns
49+
50+
<ReturnsVoid />

0 commit comments

Comments
 (0)