Skip to content

Commit 670ee45

Browse files
grdumasgrdumas
authored
Merge pull request #23 from redhat-performance/feature/opensearch-backup-tool
Add OpenSearch backup and restore utility
2 parents df01a7b + 45504ad commit 670ee45

3 files changed

Lines changed: 1357 additions & 0 deletions

File tree

‎scripts/BACKUP_EXAMPLE.md‎

Lines changed: 271 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,271 @@
1+
# OpenSearch Backup Script - Example Output
2+
3+
## Example: Backing Up Both Indices
4+
5+
```bash
6+
$ ./scripts/opensearch_backup.py backup --index both --output backups/
7+
8+
======================================================================
9+
BACKUP ESTIMATION
10+
======================================================================
11+
12+
Index: zathras-results
13+
Documents: 12,345
14+
Index size: 2.34 GB
15+
16+
Index: zathras-timeseries
17+
Documents: 456,789
18+
Index size: 15.67 GB
19+
20+
----------------------------------------------------------------------
21+
Total documents: 469,134
22+
Total index size: 18.01 GB
23+
Estimated backup size: 4.50 GB
24+
(with gzip compression)
25+
======================================================================
26+
27+
Proceed with backup? [y/N]: y
28+
29+
INFO: Starting backup of index 'zathras-results'...
30+
INFO: Backed up 10,000 / 12,345 documents...
31+
INFO: ✓ Backup complete: 12,345 documents written to backups/zathras-results_20260612_143022.ndjson.gz
32+
INFO: Backup file size: 587.23 MB
33+
34+
INFO: Starting backup of index 'zathras-timeseries'...
35+
INFO: Backed up 10,000 / 456,789 documents...
36+
INFO: Backed up 20,000 / 456,789 documents...
37+
...
38+
INFO: Backed up 450,000 / 456,789 documents...
39+
INFO: ✓ Backup complete: 456,789 documents written to backups/zathras-timeseries_20260612_143022.ndjson.gz
40+
INFO: Backup file size: 3.92 GB
41+
42+
======================================================================
43+
BACKUP SUMMARY
44+
======================================================================
45+
46+
Index: zathras-results
47+
Documents: 12,345
48+
File: backups/zathras-results_20260612_143022.ndjson.gz
49+
Size: 587.23 MB
50+
51+
Index: zathras-timeseries
52+
Documents: 456,789
53+
File: backups/zathras-timeseries_20260612_143022.ndjson.gz
54+
Size: 3.92 GB
55+
======================================================================
56+
```
57+
58+
## Example: Listing Backups
59+
60+
```bash
61+
$ ./scripts/opensearch_backup.py list --directory backups/
62+
63+
======================================================================
64+
AVAILABLE BACKUPS
65+
======================================================================
66+
67+
zathras-results_20260612_143022.ndjson.gz
68+
Size: 587.23 MB
69+
Path: backups/zathras-results_20260612_143022.ndjson.gz
70+
71+
zathras-timeseries_20260612_143022.ndjson.gz
72+
Size: 3.92 GB
73+
Path: backups/zathras-timeseries_20260612_143022.ndjson.gz
74+
75+
zathras-results_20260611_091500.ndjson.gz
76+
Size: 585.10 MB
77+
Path: backups/zathras-results_20260611_091500.ndjson.gz
78+
79+
======================================================================
80+
```
81+
82+
## Example: Restoring from Backup
83+
84+
```bash
85+
$ ./scripts/opensearch_backup.py restore --input backups/zathras-results_20260612_143022.ndjson.gz
86+
87+
======================================================================
88+
RESTORE CONFIRMATION
89+
======================================================================
90+
91+
Source file: backups/zathras-results_20260612_143022.ndjson.gz
92+
Target index: zathras-results
93+
File size: 587.23 MB
94+
95+
======================================================================
96+
97+
Proceed with restore? [y/N]: y
98+
99+
INFO: Starting restore to index 'zathras-results' from backups/zathras-results_20260612_143022.ndjson.gz...
100+
INFO: Index 'zathras-results' exists
101+
INFO: Restored 10,000 documents...
102+
INFO: ✓ Restore complete: 12,345 documents restored
103+
104+
======================================================================
105+
RESTORE SUMMARY
106+
======================================================================
107+
108+
Index: zathras-results
109+
Documents restored: 12,345
110+
======================================================================
111+
```
112+
113+
## Example: User Cancels Backup
114+
115+
```bash
116+
$ ./scripts/opensearch_backup.py backup
117+
118+
======================================================================
119+
BACKUP ESTIMATION
120+
======================================================================
121+
122+
Index: zathras-results
123+
Documents: 12,345
124+
Index size: 2.34 GB
125+
126+
Index: zathras-timeseries
127+
Documents: 456,789
128+
Index size: 15.67 GB
129+
130+
----------------------------------------------------------------------
131+
Total documents: 469,134
132+
Total index size: 18.01 GB
133+
Estimated backup size: 4.50 GB
134+
(with gzip compression)
135+
======================================================================
136+
137+
Proceed with backup? [y/N]: n
138+
Backup cancelled.
139+
```
140+
141+
## Example: Backup Without Compression
142+
143+
```bash
144+
$ ./scripts/opensearch_backup.py backup --index zathras-results --no-compress
145+
146+
======================================================================
147+
BACKUP ESTIMATION
148+
======================================================================
149+
150+
Index: zathras-results
151+
Documents: 12,345
152+
Index size: 2.34 GB
153+
154+
----------------------------------------------------------------------
155+
Total documents: 12,345
156+
Total index size: 2.34 GB
157+
Estimated backup size: 2.34 GB
158+
======================================================================
159+
160+
Proceed with backup? [y/N]: y
161+
162+
INFO: Starting backup of index 'zathras-results'...
163+
INFO: Backed up 10,000 / 12,345 documents...
164+
INFO: ✓ Backup complete: 12,345 documents written to backups/zathras-results_20260612_150000.ndjson
165+
INFO: Backup file size: 2.28 GB
166+
167+
======================================================================
168+
BACKUP SUMMARY
169+
======================================================================
170+
171+
Index: zathras-results
172+
Documents: 12,345
173+
File: backups/zathras-results_20260612_150000.ndjson
174+
Size: 2.28 GB
175+
======================================================================
176+
```
177+
178+
## Example: Verbose Mode
179+
180+
```bash
181+
$ ./scripts/opensearch_backup.py backup --index zathras-results -v
182+
183+
======================================================================
184+
BACKUP ESTIMATION
185+
======================================================================
186+
187+
Index: zathras-results
188+
Documents: 12,345
189+
Index size: 2.34 GB
190+
191+
----------------------------------------------------------------------
192+
Total documents: 12,345
193+
Total index size: 2.34 GB
194+
Estimated backup size: 587.23 MB
195+
(with gzip compression)
196+
======================================================================
197+
198+
Proceed with backup? [y/N]: y
199+
200+
DEBUG: Request to /zathras-results/_search?scroll=5m succeeded
201+
INFO: Starting backup of index 'zathras-results'...
202+
DEBUG: Request to /_search/scroll succeeded
203+
DEBUG: Request to /_search/scroll succeeded
204+
INFO: Backed up 10,000 / 12,345 documents...
205+
DEBUG: Request to /_search/scroll succeeded
206+
DEBUG: Request to /_search/scroll succeeded
207+
INFO: ✓ Backup complete: 12,345 documents written to backups/zathras-results_20260612_151000.ndjson.gz
208+
INFO: Backup file size: 587.23 MB
209+
...
210+
```
211+
212+
## Example: Error - Config File Not Found
213+
214+
```bash
215+
$ ./scripts/opensearch_backup.py backup
216+
217+
ERROR: Failed to load config from config/export_config.yml: [Errno 2] No such file or directory: 'config/export_config.yml'
218+
```
219+
220+
**Solution**: Copy the example config and fill in your credentials:
221+
222+
```bash
223+
cp config/export_config_example.yml config/export_config.yml
224+
# Edit config/export_config.yml with your OpenSearch credentials
225+
```
226+
227+
## Example: Error - Connection Failed
228+
229+
```bash
230+
$ ./scripts/opensearch_backup.py backup
231+
232+
ERROR: Failed to get stats for 'zathras-results': Connection error: [Errno -2] Name or service not known
233+
```
234+
235+
**Solution**: Check that:
236+
1. The OpenSearch URL in `config/export_config.yml` is correct
237+
2. The OpenSearch server is accessible from your network
238+
3. Credentials are correct
239+
240+
## Example: Restore to Different Index
241+
242+
```bash
243+
$ ./scripts/opensearch_backup.py restore \
244+
--input backups/zathras-results_20260612_143022.ndjson.gz \
245+
--index zathras-results-restored
246+
247+
======================================================================
248+
RESTORE CONFIRMATION
249+
======================================================================
250+
251+
Source file: backups/zathras-results_20260612_143022.ndjson.gz
252+
Target index: zathras-results-restored
253+
File size: 587.23 MB
254+
255+
======================================================================
256+
257+
Proceed with restore? [y/N]: y
258+
259+
INFO: Starting restore to index 'zathras-results-restored' from backups/zathras-results_20260612_143022.ndjson.gz...
260+
INFO: Creating index 'zathras-results-restored'...
261+
INFO: Restored 10,000 documents...
262+
INFO: ✓ Restore complete: 12,345 documents restored
263+
264+
======================================================================
265+
RESTORE SUMMARY
266+
======================================================================
267+
268+
Index: zathras-results-restored
269+
Documents restored: 12,345
270+
======================================================================
271+
```

0 commit comments

Comments
 (0)