Add local Protobuf conversion instructions to README (opensearch-project#362)

* Add local Protobuf conversion instructions to README

Signed-off-by: xil <fridalu66@gmail.com>

* update changelog

Signed-off-by: xil <fridalu66@gmail.com>

* rephrase

Signed-off-by: xil <fridalu66@gmail.com>

* update

Signed-off-by: xil <fridalu66@gmail.com>

* Add Prerequisites

Signed-off-by: xil <fridalu66@gmail.com>

---------

Signed-off-by: xil <fridalu66@gmail.com>

Author: lucy66hw

CHANGELOG.md

@@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ## [Unreleased]
 ### Added
+- Add local Protobuf conversion instructions to README ([#362](https://github.com/opensearch-project/opensearch-protobufs/pull/362))
 
 ### Changed
 

DEVELOPER_GUIDE.md

@@ -4,23 +4,41 @@ Install bazel, using the version in .bazelversion.
 # Compile protos and grpc
 
 ## All artifacts
-```
-bazel build //...
-```
+Generate protobuf libraries for your preferred language:
 
-## Java
-```
+```bash
+# Java
 bazel build //:java_protos_all
-```
 
-## Python
-```
+# Python
 bazel build //:python_protos_all
-```
 
-## Go
-```
+# Go
 bazel build //:go_protos_all
+
+# All languages
+bazel build //:java_protos_all //:python_protos_all //:go_protos_all
+```
+
+### Docker Build Options
+
+Use Docker to build and test protobuf libraries:
+
+```bash
+# Java
+docker build --target build-bazel-java .
+docker build --target package-bazel-java .
+docker build --target test-bazel-java .
+
+# Python
+docker build --target build-bazel-python .
+docker build --target package-bazel-python .
+docker build --target test-bazel-python .
+
+# Go
+docker build --target build-bazel-go .
+docker build --target package-bazel-go .
+docker build --target test-bazel-go .
 ```
 
 # Proto generated code
@@ -83,44 +101,61 @@ bazel build //:opensearch_protos_wheel
 pip install bazel-bin/opensearch_protos-*-py3-none-any.whl
 ```
 
-# Protobuf Convert Process
+# Protobuf Local Convert Guide
 
-**ProtoConvertProcess** consists of the following steps:
-- **Preprocessing** Preprocess downloaded [OpenSearch API Specification](https://github.com/opensearch-project/opensearch-api-specification) before convert to Protobuf-schema.
-- **Conversion** The prepared API specification is transformed into a Protobuf schema using [openapi-generator](https://github.com/OpenAPITools/openapi-generator).
-- **Postprocessing** The resulting Protobuf files are refined and adjusted to meet the standards after the conversion.
+To generate Protobuf definitions from the latest OpenSearch API specification, follow these steps. All commands are intended to be run from the project root directory.
 
-The [Spec Preprocessing](tools/proto-convert/src/PreProcessing.ts) includes two steps:
+## Prerequisites
 
-1. **Filter**
-    - Filters only the target APIs defined in [spec-filter.yaml](tools/proto-convert/src/config/spec-filter.yaml).
-    - Extract a single API per group from the OpenSearch spec.
+- **Node.js** >= v22
+- **Java** >= 17
 
-2. **Sanitizer**
-    - Normalizes schema and property names to be compatible with Protobuf naming rules.
+1. **Download the latest OpenSearch API Specification**
 
-**Setup**
 
-1. Install [Node.js](https://nodejs.org/en/learn/getting-started/how-to-install-nodejs)
-2. Install project dependencies:
+   ```bash
+   curl -L -o opensearch-openapi.yaml \
+     https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml
+   ```
 
+2. **Run Preprocessing**
 
-    npm run preprocessing -- --help
-**Arguments**
+   ```bash
+   npm ci && npm run preprocessing
+   ```
 
-- `--input <path>`: The path read downloaded opensearch-api-specification yaml file, defaults to `<repository-root>/build/opensearch-openapi.yaml`.
-- `--output <path>`: The path to write the final preprocessed spec to, defaults to `<repository-root>/build/processed-opensearch-openapi.yaml`.
+3. **Download OpenAPI Generator CLI**
 
-**Example**
+   ```bash
+   curl -L -f -o openapi-generator-cli.jar \
+     https://github.com/opensearch-project/opensearch-protobufs/releases/download/openapi-generator-tool/openapi-generator-cli.jar
+   ```
 
-```bash
- npm run preprocessing  -i <input_path> -o <output_path>
-```
+4. **Convert to Protobuf**
+
+   ```bash
+   java -jar openapi-generator-cli.jar generate -c tools/proto-convert/src/config/protobuf-generator-config.yaml
+   ```
+
+5. **Run Postprocessing**
+
+   ```bash
+   npm run postprocessing
+   ```
+
+After these steps, you will find the generated Protobuf service definitions in the `generated/services/default_service.proto`. Note that service files need be manually created. You can use `generated/services/default_service.proto` as a reference for defining gRPC service definitions.
+
+### Additional Notes
+
+- **For Search/Bulk Requests:**
+  Protobufs for search and bulk operations are already provided in the repository. Some schemas are excluded from generation because they don't have gRPC supported. The exclusion list is defined in [`spec-filter.yaml`](tools/proto-convert/src/config/spec-filter.yaml) under the `excluded_schemas` section.
+
+- **For Other Requests:**
+  For other APIs, generate Protobuf definitions locally and review them to ensure they meet your requirements. We recommend implementing and testing a gRPC server with the generated Protobufs to verify correctness. Once validated, submit a Pull Request to add the new API path to [`spec-filter.yaml`](tools/proto-convert/src/config/spec-filter.yaml). A maintainer will merge the PR and run the workflow to automatically generate and incorporate the Protobufs.
+
+**Note:** Make sure you are running all commands from the project root folder.
 
-**openapi-generator**
 
-OpenAPI Generator offers a range of configuration options. The configuration is specified in the [protobuf-generator-config.yaml](tools/proto-convert/src/config/protobuf-generator-config.yaml).
-OpenAPI Generator supports the customization of mustache templates to generate the desired output, with the templates located in [protobuf-schema-template](tools/proto-convert/src/config/protobuf-schema-template)
 # Ignored files
 
 All generated files are excluded from version control via the `.gitignore` file. This includes:

README.md

@@ -6,47 +6,6 @@ The [opensearch-api-specification repo](https://github.com/opensearch-project/op
 
 This repository will also include a variety of tooling and CI, linters and validators, and generated code, which is described in more detail below.
 
-## Quick Start
-
-### Build Protobuf Libraries
-
-Generate protobuf libraries for your preferred language:
-
-```bash
-# Java
-bazel build //:java_protos_all
-
-# Python
-bazel build //:python_protos_all
-
-# Go
-bazel build //:go_protos_all
-
-# All languages
-bazel build //:java_protos_all //:python_protos_all //:go_protos_all
-```
-
-### Docker Build Options
-
-Use Docker to build and test protobuf libraries:
-
-```bash
-# Java
-docker build --target build-bazel-java .
-docker build --target package-bazel-java .
-docker build --target test-bazel-java .
-
-# Python
-docker build --target build-bazel-python .
-docker build --target package-bazel-python .
-docker build --target test-bazel-python .
-
-# Go
-docker build --target build-bazel-go .
-docker build --target package-bazel-go .
-docker build --target test-bazel-go .
-```
-
 ## Releases
 
 Each OpenSearch Protobufs release includes: