Skip to content

Issue 10898 #11224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
+43 −1
Open

Issue 10898 #11224

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5a791c4
add content for registering searchable snapshot repository using hdfs…
etgraylog Oct 13, 2025
2d12a12
add clarification to step 2
etgraylog Oct 13, 2025
4fadf42
add clarification to step 3
etgraylog Oct 13, 2025
e6e5a45
Style job: 'Don't define acronyms in headings' and 'is a heading and …
etgraylog Oct 14, 2025
e3e7ec3
Add conf.<key> parameter in PUT _snapshot/searchable_snapshots
etgraylog Oct 14, 2025
d3d21c7
Revise optional step for hadoop kerberos authentication
etgraylog Oct 14, 2025
e3005e3
Add missing front-slash
etgraylog Oct 14, 2025
1148e85
Add missing frontslash
etgraylog Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 43 additions & 1 deletion _tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
sudo ./bin/opensearch-plugin install repository-s3
```

If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker#working-with-plugins). Your `Dockerfile` should look something like this:
If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins). Your `Dockerfile` should look something like this:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins). Your `Dockerfile` should look something like this:
If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins). Your `Dockerfile` should look similar to the following:


```
FROM opensearchproject/opensearch:{{site.opensearch_version}}
Expand Down Expand Up @@ -239,6 +239,48 @@

You will most likely not need to specify any parameters except for `bucket` and `base_path`. For allowed request parameters, see [Register or update snapshot repository API]({{site.url}}{{site.baseurl}}/api-reference/snapshots/create-repository/).

### HDFS

To use HDFS as a snapshot repository, follow these steps:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To use HDFS as a snapshot repository, follow these steps:
To use Hadoop Distributed File System (HDFS) as a snapshot repository, follow these steps:


1. Create an HDFS directory for snapshots and ensure that the OpenSearch user has read and write permissions to it (ex: `/opensearch/repositories/searchable_snapshots`).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Create an HDFS directory for snapshots and ensure that the OpenSearch user has read and write permissions to it (ex: `/opensearch/repositories/searchable_snapshots`).
1. Create an HDFS directory for snapshots (for example, `/opensearch/repositories/searchable_snapshots`) and ensure that the OpenSearch user has read and write permissions to it.


1. Install the `repository-hdfs` plugin on all nodes:

```bash
sudo ./bin/opensearch-plugin install repository-hdfs
```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
```
```
{% include copy.html %}


If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins}). Your `Dockerfile` should look something like this:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins}). Your `Dockerfile` should look something like this:
If you're using the Docker installation, see [Working with plugins]({{site.url}}{{site.baseurl}}/opensearch/install/docker/#working-with-plugins}). Your `Dockerfile` should look similar to the following:


```
FROM opensearchproject/opensearch:{{site.opensearch_version}}
RUN /usr/share/opensearch/bin/opensearch-plugin install --batch repository-hdfs
```

1. (Optional) If you have Kerberos authentication enabled on your HDFS cluster:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. (Optional) If you have Kerberos authentication enabled on your HDFS cluster:
1. (Optional) If your HDFS cluster uses Kerberos, you may need to distribute the keytab file to all nodes and ensure that the OpenSearch user has read access.


If using Kerberos you may need to distribute the keytab file to all nodes and ensure that the OpenSearch user has read access to it. Otherwise, skip this step.

Check failure on line 264 in _tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Spelling] Error: keytab. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.

Raw Output:
{"message": "[OpenSearch.Spelling] Error: keytab. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md", "range": {"start": {"line": 264, "column": 54}}}, "severity": "ERROR"}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If using Kerberos you may need to distribute the keytab file to all nodes and ensure that the OpenSearch user has read access to it. Otherwise, skip this step.


1. Restart all nodes in the OpenSearch cluster.

1. Register the repository using the OpenSearch Snapshot API:

```json
PUT _snapshot/searchable_snapshots
{
"type": "hdfs",
"settings": {
// If using kerberos authentication, uncomment and replace YOURREALM with your realm name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably just have 2 examples with and without kerberos, content itself LGTM though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. @etgraylog Could you split this into 2 requests and remove c-style comments? Otherwise, if you copy or copy as curl, the request won't work as is.

// "security.principal": "opensearch@YOURREALM"
"uri": "hdfs://namenode:8020/",
"path": "/opensearch/repositories/searchable_snapshots",
"conf.<key>": "<value>" // (Optional) Additional HDFS configuration settings
}
}
```
{% include copy-curl.html %}

### Registering a Microsoft Azure storage account using Helm

Expand Down
Loading