Implement backup & restore (#479)

## Problem

Implement backup & restore

## Solution

Added new methods to `Pinecone` and `PineconeAsyncio`:
- `create_index_from_backup`
- `create_backup`
- `list_backups`
- `describe_backup`
- `delete_backup`
- `list_restore_jobs`
- `describe_restore_job`

These can also be accessed with the new-style syntax, e.g.
`pc.db.index.create_from_backup`, `pc.db.backup.create`,
`pc.db.restore_job.list`.

More Details:

- Had to re-run codegen to pull in recent spec changes
- Organize implementation around resource-types
- Expose legacy-style names (`create_backup`,
`create_index_from_backups`) as well as new-style names
`pc.db.index.create_from_backup`. In the upcoming release, both styles
will be present. We still need to do some work to reorg methods for some
less-used parts of the client (bulk imports, etc) before transitioning
fully to the new style in examples and documentation.
- For new methods being added, begin enforcing keyword argument usage
with a new `@kwargs_required` decorator. I will probably follow up and
add this to all new methods added in the recent refactoring PR. Keyword
arguments are strongly preferred over positional arguments because the
keyword labels act as documentation and having the keyword labels makes
them order-independent. This gives a lot of flexibility to expand the
signature or change things from required to optional later without
creating breaking changes for callers.
- Wire up the code paths for new methods:
    - `Pinecone > DbControl > BackupResource`
    - `Pinecone > DbControl > IndexResource`
    - `Pinecone > DbControl > RestoreJobResource`
    - `PineconeAsyncio > DbControlAsyncio > AsyncioBackupResource`
    - `PineconeAsyncio > DbControlAsyncio > AsyncioIndexResource`
    - `PineconeAsyncio > DbControlAsyncio > AsyncioRestoreJobResource`
- Update interface classes so that docs will show information about the
new methods.

## Usage

### Initial setup

```python
from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key='key')

# First you need an index
pc.create_index(
    name='foo',
    dimension=2,
    metric='cosine',
    spec=ServerlessSpec(cloud='aws', region='us-east-1')
)

# Upsert some fake data just for demonstration purposes
import random

idx = pc.Index(name='foo')
idx.upsert(
    vectors=[
         (str(i), [random.random(), random.random()] for i in range(1000)
    ]
)
```

### Backups

```python
pc.create_backup(
    index_name='foo', 
    backup_name='bar', 
    description='an example backup'
)

# Describe a backup
pc.describe_backup(backup_id='7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1')
# {
#     "backup_id": "7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1",
#     "source_index_name": "foo",
#     "source_index_id": "4c292a8a-77cc-4a37-917d-51c6051a80bf",
#     "status": "Ready",
#     "cloud": "aws",
#     "region": "us-east-1",
#     "tags": {},
#     "name": "bar",
#     "description": "",
#     "dimension": 2,
#     "record_count": 1000,
#     "namespace_count": 1,
#     "size_bytes": 289392,
#     "created_at": "2025-05-13T14:15:16.908702Z"
# }


# List backups
pc.list_backups()
# [
#     {
#         "backup_id": "7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1",
#         "source_index_name": "foo",
#         "source_index_id": "4c292a8a-77cc-4a37-917d-51c6051a80bf",
#         "status": "Ready",
#         "cloud": "aws",
#         "region": "us-east-1",
#         "tags": {},
#         "name": "bar",
#         "description": "",
#         "dimension": 2,
#         "record_count": 1000,
#         "namespace_count": 1,
#         "size_bytes": 289392,
#         "created_at": "2025-05-13T14:15:16.908702Z"
#     }
# ]

# Delete backup
pc.delete_backup(backup_id='7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1')
```

### Creating an index from backup
```python
# Create index from backup
pc.create_index_from_backup(
  backup_id='7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1',
  name='foo2',
  deletion_protection='enabled',
  tags={'env': 'testing'}
)
# {
#     "name": "foo2",
#     "metric": "cosine",
#     "host": "foo2-dojoi3u.svc.aped-4627-b74a.pinecone.io",
#     "spec": {
#         "serverless": {
#             "cloud": "aws",
#             "region": "us-east-1"
#         }
#     },
#     "status": {
#         "ready": true,
#         "state": "Ready"
#     },
#     "vector_type": "dense",
#     "dimension": 2,
#     "deletion_protection": "enabled",
#     "tags": {
#         "env": "testing"
#     }
# }
```

### Restore job

```python
# List jobs
pc.list_restore_jobs()
# {'data': [{'backup_id': 'e5957dc2-a76e-4b72-9645-569fb7ec143f',
#            'completed_at': datetime.datetime(2025, 5, 13, 14, 56, 13, 939921, tzinfo=tzutc()),
#            'created_at': datetime.datetime(2025, 5, 13, 14, 56, 4, 534826, tzinfo=tzutc()),
#            'percent_complete': 100.0,
#            'restore_job_id': '744ea5bd-7ddc-44ce-81f5-cfb876572e59',
#            'status': 'Completed',
#            'target_index_id': '572130f9-cfdd-42bf-a280-4218cd112bf8',
#            'target_index_name': 'foo2'},
#           {'backup_id': '7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1',
#            'completed_at': datetime.datetime(2025, 5, 13, 16, 27, 10, 290234, tzinfo=tzutc()),
#            'created_at': datetime.datetime(2025, 5, 13, 16, 27, 6, 130522, tzinfo=tzutc()),
#            'percent_complete': 100.0,
#            'restore_job_id': '06aa5739-2785-4121-b71b-99b73c3e3247',
#            'status': 'Completed',
#            'target_index_id': 'd3f31cd1-b077-4bcf-8e7d-d091d408c82b',
#            'target_index_name': 'foo2'}],
#  'pagination': None}

# Describe jobs
pc.describe_restore_job(job_id='504dd1a9-e3cd-420f-8756-65d5411fcb10')
# {'backup_id': '7c8e6fcf-577b-4df5-9869-3c67f0f3d6e1',
#  'completed_at': datetime.datetime(2025, 5, 13, 15, 55, 10, 108584, tzinfo=tzutc()),
#  'created_at': datetime.datetime(2025, 5, 13, 15, 54, 49, 925105, tzinfo=tzutc()),
#  'percent_complete': 100.0,
#  'restore_job_id': '504dd1a9-e3cd-420f-8756-65d5411fcb10',
#  'status': 'Completed',
#  'target_index_id': 'b5607ee7-be78-4401-aaf5-ea20413f409d',
#  'target_index_name': 'foo4'}
```

## Type of Change

- [x] New feature (non-breaking change which adds functionality)

## Test Plan

Describe specific steps for validating this change.

Author: jhamon

.github/workflows/testing-integration.yaml

@@ -11,8 +11,17 @@ jobs:
       PINECONE_API_KEY: '${{ secrets.PINECONE_API_KEY }}'
       PINECONE_ADDITIONAL_HEADERS: '{"sdk-test-suite": "pinecone-python-client"}'
     strategy:
+      fail-fast: false
       matrix:
         python_version: [3.9, 3.12]
+        test_suite:
+          - tests/integration/control/index
+          - tests/integration/control/collections
+          - tests/integration/control/backup
+          - tests/integration/control/restore_job
+          - tests/integration/control_asyncio/index
+          - tests/integration/control_asyncio/backup
+          - tests/integration/control_asyncio/restore_job
     steps:
       - uses: actions/checkout@v4
       - name: 'Set up Python ${{ matrix.python_version }}'
@@ -23,11 +32,8 @@ jobs:
         uses: ./.github/actions/setup-poetry
         with:
           include_asyncio: true
-      - name: 'Run index tests'
-        run: poetry run pytest tests/integration/control/index --retries 5 --retry-delay 35 -s -vv --log-cli-level=DEBUG
-      - name: 'Run collection tests'
-        run: poetry run pytest tests/integration/control/collections --retries 5 --retry-delay 35 -s -vv --log-cli-level=DEBUG
-
+      - name: 'Run tests'
+        run: poetry run pytest ${{ matrix.test_suite }} --retries 2 --retry-delay 35 -s -vv --log-cli-level=DEBUG
 
   inference:
     name: Inference tests

codegen/apis

@@ -76,6 +76,10 @@
     "PodSpec": ("pinecone.db_control.models", "PodSpec"),
     "PodSpecDefinition": ("pinecone.db_control.models", "PodSpecDefinition"),
     "PodType": ("pinecone.db_control.enums", "PodType"),
+    "RestoreJobModel": ("pinecone.db_control.models", "RestoreJobModel"),
+    "RestoreJobList": ("pinecone.db_control.models", "RestoreJobList"),
+    "BackupModel": ("pinecone.db_control.models", "BackupModel"),
+    "BackupList": ("pinecone.db_control.models", "BackupList"),
 }
 
 _config_lazy_imports = {

codegen/python-oas-templates

@@ -36,6 +36,9 @@
 from pinecone.core.openapi.db_control.model.create_index_from_backup_request import (
     CreateIndexFromBackupRequest,
 )
+from pinecone.core.openapi.db_control.model.create_index_from_backup_response import (
+    CreateIndexFromBackupResponse,
+)
 from pinecone.core.openapi.db_control.model.create_index_request import CreateIndexRequest
 from pinecone.core.openapi.db_control.model.error_response import ErrorResponse
 from pinecone.core.openapi.db_control.model.index_list import IndexList
@@ -281,7 +284,7 @@ def __create_collection(
         def __create_index(self, create_index_request, **kwargs: ExtraOpenApiKwargsTypedDict):
             """Create an index  # noqa: E501
 
-            Create a Pinecone index. This is where you specify the measure of similarity, the dimension of vectors to be stored in the index, which cloud provider you would like to deploy with, and more.    For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/indexes/create-an-index#create-a-serverless-index).   # noqa: E501
+            Create a Pinecone index. This is where you specify the measure of similarity, the dimension of vectors to be stored in the index, which cloud provider you would like to deploy with, and more.    For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/index-data/create-an-index).   # noqa: E501
             This method makes a synchronous HTTP request by default. To make an
             asynchronous HTTP request, please pass async_req=True
 
@@ -352,7 +355,7 @@ def __create_index_for_model(
         ):
             """Create an index with integrated embedding  # noqa: E501
 
-            Create an index with integrated embedding.  With this type of index, you provide source text, and Pinecone uses a [hosted embedding model](https://docs.pinecone.io/guides/inference/understanding-inference#embedding-models) to convert the text automatically during [upsert](https://docs.pinecone.io/reference/api/2025-01/data-plane/upsert_records) and [search](https://docs.pinecone.io/reference/api/2025-01/data-plane/search_records).  For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/indexes/create-an-index#integrated-embedding).  # noqa: E501
+            Create an index with integrated embedding.  With this type of index, you provide source text, and Pinecone uses a [hosted embedding model](https://docs.pinecone.io/guides/index-data/create-an-index#embedding-models) to convert the text automatically during [upsert](https://docs.pinecone.io/reference/api/2025-01/data-plane/upsert_records) and [search](https://docs.pinecone.io/reference/api/2025-01/data-plane/search_records).  For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/index-data/create-an-index#integrated-embedding).  # noqa: E501
             This method makes a synchronous HTTP request by default. To make an
             asynchronous HTTP request, please pass async_req=True
 
@@ -418,7 +421,7 @@ def __create_index_for_model(
             callable=__create_index_for_model,
         )
 
-        def __create_index_from_backup(
+        def __create_index_from_backup_operation(
             self, backup_id, create_index_from_backup_request, **kwargs: ExtraOpenApiKwargsTypedDict
         ):
             """Create an index from a backup  # noqa: E501
@@ -427,7 +430,7 @@ def __create_index_from_backup(
             This method makes a synchronous HTTP request by default. To make an
             asynchronous HTTP request, please pass async_req=True
 
-            >>> thread = api.create_index_from_backup(backup_id, create_index_from_backup_request, async_req=True)
+            >>> thread = api.create_index_from_backup_operation(backup_id, create_index_from_backup_request, async_req=True)
             >>> result = thread.get()
 
             Args:
@@ -453,7 +456,7 @@ def __create_index_from_backup(
                 async_req (bool): execute request asynchronously
 
             Returns:
-                None
+                CreateIndexFromBackupResponse
                     If the method is called asynchronously, returns the request
                     thread.
             """
@@ -462,12 +465,12 @@ def __create_index_from_backup(
             kwargs["create_index_from_backup_request"] = create_index_from_backup_request
             return self.call_with_http_info(**kwargs)
 
-        self.create_index_from_backup = _Endpoint(
+        self.create_index_from_backup_operation = _Endpoint(
             settings={
-                "response_type": None,
+                "response_type": (CreateIndexFromBackupResponse,),
                 "auth": ["ApiKeyAuth"],
                 "endpoint_path": "/backups/{backup_id}/create-index",
-                "operation_id": "create_index_from_backup",
+                "operation_id": "create_index_from_backup_operation",
                 "http_method": "POST",
                 "servers": None,
             },
@@ -491,7 +494,7 @@ def __create_index_from_backup(
             },
             headers_map={"accept": ["application/json"], "content_type": ["application/json"]},
             api_client=api_client,
-            callable=__create_index_from_backup,
+            callable=__create_index_from_backup_operation,
         )
 
         def __delete_backup(self, backup_id, **kwargs: ExtraOpenApiKwargsTypedDict):
@@ -1192,6 +1195,8 @@ def __list_project_backups(self, **kwargs: ExtraOpenApiKwargsTypedDict):
 
 
             Keyword Args:
+                limit (int): The number of results to return per page. [optional] if omitted the server will use the default value of 10.
+                pagination_token (str): The token to use to retrieve the next page of results. [optional]
                 _return_http_data_only (bool): response data without head status
                     code and headers. Default is True.
                 _preload_content (bool): if False, the urllib3.HTTPResponse object
@@ -1226,13 +1231,19 @@ def __list_project_backups(self, **kwargs: ExtraOpenApiKwargsTypedDict):
                 "http_method": "GET",
                 "servers": None,
             },
-            params_map={"all": [], "required": [], "nullable": [], "enum": [], "validation": []},
+            params_map={
+                "all": ["limit", "pagination_token"],
+                "required": [],
+                "nullable": [],
+                "enum": [],
+                "validation": ["limit"],
+            },
             root_map={
-                "validations": {},
+                "validations": {("limit",): {"inclusive_maximum": 100, "inclusive_minimum": 1}},
                 "allowed_values": {},
-                "openapi_types": {},
-                "attribute_map": {},
-                "location_map": {},
+                "openapi_types": {"limit": (int,), "pagination_token": (str,)},
+                "attribute_map": {"limit": "limit", "pagination_token": "paginationToken"},
+                "location_map": {"limit": "query", "pagination_token": "query"},
                 "collection_format_map": {},
             },
             headers_map={"accept": ["application/json"], "content_type": []},
@@ -1519,7 +1530,7 @@ async def __create_collection(self, create_collection_request, **kwargs):
         async def __create_index(self, create_index_request, **kwargs):
             """Create an index  # noqa: E501
 
-            Create a Pinecone index. This is where you specify the measure of similarity, the dimension of vectors to be stored in the index, which cloud provider you would like to deploy with, and more.    For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/indexes/create-an-index#create-a-serverless-index).   # noqa: E501
+            Create a Pinecone index. This is where you specify the measure of similarity, the dimension of vectors to be stored in the index, which cloud provider you would like to deploy with, and more.    For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/index-data/create-an-index).   # noqa: E501
 
 
             Args:
@@ -1581,7 +1592,7 @@ async def __create_index(self, create_index_request, **kwargs):
         async def __create_index_for_model(self, create_index_for_model_request, **kwargs):
             """Create an index with integrated embedding  # noqa: E501
 
-            Create an index with integrated embedding.  With this type of index, you provide source text, and Pinecone uses a [hosted embedding model](https://docs.pinecone.io/guides/inference/understanding-inference#embedding-models) to convert the text automatically during [upsert](https://docs.pinecone.io/reference/api/2025-01/data-plane/upsert_records) and [search](https://docs.pinecone.io/reference/api/2025-01/data-plane/search_records).  For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/indexes/create-an-index#integrated-embedding).  # noqa: E501
+            Create an index with integrated embedding.  With this type of index, you provide source text, and Pinecone uses a [hosted embedding model](https://docs.pinecone.io/guides/index-data/create-an-index#embedding-models) to convert the text automatically during [upsert](https://docs.pinecone.io/reference/api/2025-01/data-plane/upsert_records) and [search](https://docs.pinecone.io/reference/api/2025-01/data-plane/search_records).  For guidance and examples, see [Create an index](https://docs.pinecone.io/guides/index-data/create-an-index#integrated-embedding).  # noqa: E501
 
 
             Args:
@@ -1640,7 +1651,7 @@ async def __create_index_for_model(self, create_index_for_model_request, **kwarg
             callable=__create_index_for_model,
         )
 
-        async def __create_index_from_backup(
+        async def __create_index_from_backup_operation(
             self, backup_id, create_index_from_backup_request, **kwargs
         ):
             """Create an index from a backup  # noqa: E501
@@ -1670,19 +1681,19 @@ async def __create_index_from_backup(
                     Default is True.
 
             Returns:
-                None
+                CreateIndexFromBackupResponse
             """
             self._process_openapi_kwargs(kwargs)
             kwargs["backup_id"] = backup_id
             kwargs["create_index_from_backup_request"] = create_index_from_backup_request
             return await self.call_with_http_info(**kwargs)
 
-        self.create_index_from_backup = _AsyncioEndpoint(
+        self.create_index_from_backup_operation = _AsyncioEndpoint(
             settings={
-                "response_type": None,
+                "response_type": (CreateIndexFromBackupResponse,),
                 "auth": ["ApiKeyAuth"],
                 "endpoint_path": "/backups/{backup_id}/create-index",
-                "operation_id": "create_index_from_backup",
+                "operation_id": "create_index_from_backup_operation",
                 "http_method": "POST",
                 "servers": None,
             },
@@ -1706,7 +1717,7 @@ async def __create_index_from_backup(
             },
             headers_map={"accept": ["application/json"], "content_type": ["application/json"]},
             api_client=api_client,
-            callable=__create_index_from_backup,
+            callable=__create_index_from_backup_operation,
         )
 
         async def __delete_backup(self, backup_id, **kwargs):
@@ -2333,6 +2344,8 @@ async def __list_project_backups(self, **kwargs):
 
 
             Keyword Args:
+                limit (int): The number of results to return per page. [optional] if omitted the server will use the default value of 10.
+                pagination_token (str): The token to use to retrieve the next page of results. [optional]
                 _return_http_data_only (bool): response data without head status
                     code and headers. Default is True.
                 _preload_content (bool): if False, the urllib3.HTTPResponse object
@@ -2364,13 +2377,19 @@ async def __list_project_backups(self, **kwargs):
                 "http_method": "GET",
                 "servers": None,
             },
-            params_map={"all": [], "required": [], "nullable": [], "enum": [], "validation": []},
+            params_map={
+                "all": ["limit", "pagination_token"],
+                "required": [],
+                "nullable": [],
+                "enum": [],
+                "validation": ["limit"],
+            },
             root_map={
-                "validations": {},
+                "validations": {("limit",): {"inclusive_maximum": 100, "inclusive_minimum": 1}},
                 "allowed_values": {},
-                "openapi_types": {},
-                "attribute_map": {},
-                "location_map": {},
+                "openapi_types": {"limit": (int,), "pagination_token": (str,)},
+                "attribute_map": {"limit": "limit", "pagination_token": "paginationToken"},
+                "location_map": {"limit": "query", "pagination_token": "query"},
                 "collection_format_map": {},
             },
             headers_map={"accept": ["application/json"], "content_type": []},

pinecone/__init__.py

@@ -30,10 +30,10 @@
 from typing import Dict, Literal, Tuple, Set, Any, Type, TypeVar
 from pinecone.openapi_support import PropertyValidationTypedDict, cached_class_property
 
-T = TypeVar("T", bound="DedicatedSpec")
+T = TypeVar("T", bound="ByocSpec")
 
 
-class DedicatedSpec(ModelNormal):
+class ByocSpec(ModelNormal):
     """NOTE: This class is @generated using OpenAPI.
 
     Do not edit the class manually.
@@ -102,7 +102,7 @@ def discriminator(cls):
     @classmethod
     @convert_js_args_to_python_args
     def _from_openapi_data(cls: Type[T], environment, *args, **kwargs) -> T:  # noqa: E501
-        """DedicatedSpec - a model defined in OpenAPI
+        """ByocSpec - a model defined in OpenAPI
 
         Args:
             environment (str): The environment where the index is hosted.
@@ -189,7 +189,7 @@ def _from_openapi_data(cls: Type[T], environment, *args, **kwargs) -> T:  # noqa
 
     @convert_js_args_to_python_args
     def __init__(self, environment, *args, **kwargs) -> None:  # noqa: E501
-        """DedicatedSpec - a model defined in OpenAPI
+        """ByocSpec - a model defined in OpenAPI
 
         Args:
             environment (str): The environment where the index is hosted.

pinecone/core/openapi/db_control/api/manage_indexes_api.py

@@ -61,9 +61,7 @@ class CreateBackupRequest(ModelNormal):
 
     allowed_values: Dict[Tuple[str, ...], Dict[str, Any]] = {}
 
-    validations: Dict[Tuple[str, ...], PropertyValidationTypedDict] = {
-        ("name",): {"max_length": 45, "min_length": 1}
-    }
+    validations: Dict[Tuple[str, ...], PropertyValidationTypedDict] = {}
 
     @cached_class_property
     def additional_properties_type(cls):
@@ -139,7 +137,7 @@ def _from_openapi_data(cls: Type[T], *args, **kwargs) -> T:  # noqa: E501
                                 Animal class but this time we won't travel
                                 through its discriminator because we passed in
                                 _visited_composed_classes = (Animal,)
-            name (str): The name of the index. Resource name must be 1-45 characters long, start and end with an alphanumeric character, and consist only of lower case alphanumeric characters or '-'.  [optional]  # noqa: E501
+            name (str): The name of the backup. [optional]  # noqa: E501
             description (str): A description of the backup. [optional]  # noqa: E501
         """
 
@@ -224,7 +222,7 @@ def __init__(self, *args, **kwargs) -> None:  # noqa: E501
                                 Animal class but this time we won't travel
                                 through its discriminator because we passed in
                                 _visited_composed_classes = (Animal,)
-            name (str): The name of the index. Resource name must be 1-45 characters long, start and end with an alphanumeric character, and consist only of lower case alphanumeric characters or '-'.  [optional]  # noqa: E501
+            name (str): The name of the backup. [optional]  # noqa: E501
             description (str): A description of the backup. [optional]  # noqa: E501
         """