[META] Productionalizing Client/Server GRPC in OpenSearch

[amberzsy]

Please describe the end goal of this project

Please comment on this issue with any feedback on our recent experimental release.
This is a meta issue for all work relating to Client to Server GRPC.

Supporting References

• [RFC] Protobuf in OpenSearch
• [META] Leverage protobuf for serializing select node-to-node objects
• [META] Protobuf for Search API
• gRPC support for client Search API (initial client<>server benchmark result for Search api)

Issues

Core Work

V3.0.0 Experimental feature release

Feature	GH	Finished
Server Core	gRPC Server #16556	✔️
Search API - matchAll/Term/Terms/MatchNone/ScriptScore	[Feature] GRPC Search APIs #16783	✔️
Document API - Bulk	[Feature] GRPC Document APIs #16784	✔️
Bulk Benchmark	#16711	✔️
Search API Benchmark	#16710	✔️
API documentation	opensearch-project/documentation-website#9591	✔️

GRPC Server Core

• [META] gRPC Server #16556

Search APIs

• [RFC] gRPC-based API for Search  #15190 (proposal)
• [Feature] GRPC Search APIs - 3.0.0 Experimental feature release #16783

Document APIs

• [Feature] GRPC Document APIs - 3.0.0 Experimental feature release #16784

API Documentation

• [DOC] Create experimental GRPC APIs documentation documentation-website#9591

Benchmarking

• GRPC Search API benchmark - 3.0.0 Experimental feature release #16710
• GRPC Document API benchmark - 3.0.0 Experimental feature release #16711

V3.1.0 feature release

Feature	GH	Finished
Server Core
Query DSL
Document API

Core

• [Feature Request] Security plugin integration for grpc-transport plugin  #16905

Query DSL

• [Feature] Term-level queries
• [Feature] Full-text queries
• [Feature] Boolean, Boosting queries
• [Feature] KNN

Document APIs

• [Feature] Index Document API
• [Feature] Update Document API
• [Feature] Delete Document API
• [Feature] Get Document API (? TBD)

V3.2.0 feature release or TBD
Index APIs

• [Feature Request] GRPC Index APIs #16785
  Other APIs
• [Feature] GRPC Cluster APIs
• [Feature] GRPC Nodes APIs
• [Feature] GRPC CAT APIs
• [Feature] GRPC Ingest APIs
  TODO: Other APIs from the public doc are omitted for brevity, and can be added later as we become closer to working on them.

Proto conversion

Protobuf support

• [PROPOSAL] Publish protobuf files. opensearch-api-specification#532
• [PROPOSAL] Spec to Protobuf Conversion Rules and Tooling opensearch-api-specification#677
• [FEATURE] Spec Conversion to Protobufs opensearch-api-specification#707

CI/CD

• [PROPOSAL] Core <> Spec/Protobuf CI/CD opensearch-api-specification#653
• [FEATURE] Support Java jar publishing to remote Maven repository

Related

• [META] GRPC Ecosystem Integration #16786

Related component

Other