Protobuf schema change detected (opensearch-project#271)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: lucy66hw <178352792+lucy66hw@users.noreply.github.com>
Preprocessing - handle unamed additionalproperties (opensearch-project#272)

Signed-off-by: xil <fridalu66@gmail.com>
Protobuf schema change detected (opensearch-project#273)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: lucy66hw <178352792+lucy66hw@users.noreply.github.com>
Eliminate file name dependencies in Python generated protobuf code

Signed-off-by: Karen X <karenxyr@gmail.com>

Merge branch 'main' into pythonfile

Signed-off-by: Karen X <karenxyr@gmail.com>
simplify by reading proto files instead of generated py files

Signed-off-by: Karen X <karenxyr@gmail.com>

add unit tests

Signed-off-by: Karen X <karenxyr@gmail.com>

Address security vulnerabilities (opensearch-project#277)

Update dependency @eslint/eslintrc to v3.2.0 (opensearch-project#278)

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>
Update dependency @eslint/eslintrc to v3.3.0 (opensearch-project#279)

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>
Update dependency @eslint/eslintrc to v3.3.1 (opensearch-project#281)

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>
More security vulnerabilities (opensearch-project#284)

Signed-off-by: karenx <karenx@uber.com>
Co-authored-by: karenx <karenx@uber.com>
Merge remote-tracking branch 'upstream/main' into test123

Author: karenyrx

.github/workflows/build-protobufs-python.yml

@@ -10,9 +10,36 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+
       - uses: actions/setup-java@v4
         with:
           distribution: temurin # Temurin is a distribution of adoptium
           java-version: 21
-      - name: build protobufs python
-        run: bazel build //:python_schemas //:python_services
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+
+      - name: Install Python dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install pytest
+
+      - name: Run unit tests
+        run: |
+          cd tools
+          python -m pytest test_generate_init_files.py -v --tb=short
+
+      - name: Build Python protobufs
+        run: bazel build //:python_protos_all
+
+      - name: Test Python imports
+        run: |
+          # Create virtual environment for testing
+          python -m venv test_env
+          source test_env/bin/activate
+          pip install grpcio protobuf
+
+          # Run the dedicated test script
+          export PYTHONPATH=bazel-bin
+          python tools/python/print_modules.py

BUILD.bazel

@@ -97,8 +97,13 @@ genrule(
 )
 
 genrule(
-    name = "fix_python_imports",
-    srcs = [":python_schemas", ":python_services", ":generate_pyi_files"],
+    name = "python_protos_all",
+    srcs = [
+        ":python_schemas",
+        ":python_services",
+        ":generate_pyi_files",
+        "tools/generate_init_files.py",
+    ],
     outs = [
         "opensearch/protobufs/schemas/common_pb2.py",
         "opensearch/protobufs/schemas/common_pb2.pyi",
@@ -112,6 +117,8 @@ genrule(
         "opensearch/protobufs/services/search_service_pb2.py",
         "opensearch/protobufs/services/search_service_pb2.pyi",
         "opensearch/protobufs/services/search_service_pb2_grpc.py",
+        "opensearch/__init__.py",
+        "opensearch/protobufs/__init__.py",
         "opensearch/protobufs/schemas/__init__.py",
         "opensearch/protobufs/services/__init__.py",
     ],
@@ -127,8 +134,8 @@ genrule(
         # Find and replace "from protos" with "from opensearch.protobufs" in all files
         find $(RULEDIR)/opensearch/protobufs/ -type f -exec sed -i 's/from protos/from opensearch.protobufs/g' {} +
 
-        touch $(RULEDIR)/opensearch/protobufs/schemas/__init__.py
-        touch $(RULEDIR)/opensearch/protobufs/services/__init__.py
+        python3 tools/generate_init_files.py $(RULEDIR)/opensearch/protobufs/schemas $(RULEDIR)/opensearch/protobufs/services
+
     """,
 )
 
@@ -157,6 +164,6 @@ py_wheel(
     platform = "any",
     python_requires = ">=3.10",
     deps = [
-        ":fix_python_imports",
+        ":python_protos_all",
     ],
 )

CHANGELOG.md

@@ -9,6 +9,8 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Add terms aggregation protos  ([#268](https://github.com/opensearch-project/opensearch-protobufs/pull/268))
 - Preprocessing: Handle unnamed additionalProperties.([#272](https://github.com/opensearch-project/opensearch-protobufs/pull/272))
 
+- Support importing without proto file name knowledge in Python generated protobuf code ([#275](https://github.com/opensearch-project/opensearch-protobufs/pull/275))
+
 ### Changed
 - Update preprocessing for x-protobuf-excluded ([#266](https://github.com/opensearch-project/opensearch-protobufs/pull/266))
 - Fix aggregations protos ([#270](https://github.com/opensearch-project/opensearch-protobufs/pull/270))

README.md

@@ -17,7 +17,7 @@ Generate protobuf libraries for your preferred language:
 bazel build //:java_protos_all
 
 # Python
-bazel build //:python_schemas //:python_services
+bazel build //:python_protos_all
 
 # Go
 bazel build //:go_protos_all
@@ -109,16 +109,16 @@ SearchResponse response = client.search(request);
 ### Python
 
 ```python
-from opensearch.protobufs.schemas import search_pb2
-from opensearch.protobufs.services import search_service_pb2
+from opensearch.protobufs.schemas import SearchRequest, BulkRequest, IndexDocumentRequest
+from opensearch.protobufs.services import SearchServiceStub
 
 # Use generated message types
-request = search_pb2.SearchRequest()
+request = SearchRequest()
 request.query = "elasticsearch"
 request.size = 10
 
 # Use generated gRPC clients
-client = search_service_pb2.SearchServiceStub(channel)
+client = SearchServiceStub(channel)
 response = client.Search(request)
 ```
 
@@ -135,8 +135,8 @@ bazel-bin/protos/services/*_go_proto_pb/protos/services/*.pb.go
 bazel-bin/libjava_protos_all.jar
 
 # Python
-bazel-bin/protos/schemas/*_python_proto_pb/protos/schemas/*_pb2.py
-bazel-bin/protos/services/*_python_proto_pb/protos/services/*_pb2.py
+bazel-bin/opensearch/protobufs/schemas/
+bazel-bin/opensearch/protobufs/services/
 ```
 
 ## Intended usage of the repo