Here's my local version of this implementation, it depends on S3 vectors client which is a class from AWS PHP SDK.

@chr-hertel @OskarStark

Wdyt, should it depend on AWS SDK or AWS async one, it currently doesn't have this class, maybe it can be autogenerated though using their tools, see async-aws/aws#2010

<?php

declare(strict_types=1);

namespace App\Service;

use Aws\S3Vectors\S3VectorsClient;
use Symfony\AI\Platform\Vector\Vector;
use Symfony\AI\Store\Document\Metadata;
use Symfony\AI\Store\Document\VectorDocument;
use Symfony\AI\Store\ManagedStoreInterface;
use Symfony\AI\Store\StoreInterface;
use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Component\Uid\Uuid;

/**
 * AWS S3 Vectors implementation of the Symfony AI vector store interface.
 *
 * This store provides vector similarity search capabilities using AWS S3 Vectors service.
 * It manages a vector bucket and index for storing and querying document embeddings.
 *
 * @see https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3vectors-2025-07-15.html
 */
final readonly class S3VectorStore implements StoreInterface, ManagedStoreInterface
{
    private const string DEFAULT_DISTANCE_METRIC = 'cosine';
    private const string DEFAULT_DATA_TYPE = 'float32';
    private const int DEFAULT_MAX_RESULTS = 10;

    public function __construct(
        private S3VectorsClient $client,

        private string $vectorBucketName,

        private string $vectorIndexName,
    ) {}

    /**
     * Sets up the vector bucket and index.
     *
     * @param array{
     *     dimension?: int,
     *     distance_metric?: 'cosine'|'euclidean',
     *     data_type?: 'float32',
     *     tags?: array<string, string>,
     *     metadata_configuration?: array{
     *         nonFilterableMetadataKeys: list<string>
     *     },
     *     encryption_configuration?: array{
     *         sseType?: 'AES256'|'aws:kms',
     *         kmsKeyArn?: string
     *     }
     * }|array<mixed> $options Configuration options:
     *
     * - dimension: The dimensions of the vectors to be inserted into the vector index (default: 768)
     * - distance_metric: The distance metric to be used for similarity search (default: 'cosine')
     * - data_type: The data type of the vectors to be inserted into the vector index (default: 'float32')
     * - tags: User-defined tags to apply to the vector bucket and index
     * - metadata_configuration: Configuration for non-filterable metadata keys
     * - encryption_configuration: Encryption settings for the vector bucket and index
     */
    public function setup(array $options = []): void
    {
        $dimension = $options['dimension'] ?? throw new \InvalidArgumentException('The "dimension" option is required for setting up the vector store.');
        $distanceMetric = $options['distance_metric'] ?? self::DEFAULT_DISTANCE_METRIC;
        $dataType = $options['data_type'] ?? self::DEFAULT_DATA_TYPE;
        $tags = $options['tags'] ?? [];
        $metadataConfiguration = $options['metadata_configuration'] ?? [
            'nonFilterableMetadataKeys' => [Metadata::KEY_TEXT],
        ];
        $encryptionConfiguration = $options['encryption_configuration'] ?? [];

        // Create vector bucket if it doesn't exist
        try {
            $this->client->getVectorBucket([
                'vectorBucketName' => $this->vectorBucketName,
            ]);
        } catch (\Aws\S3Vectors\Exception\S3VectorsException $e) {
            if ('NotFoundException' === $e->getAwsErrorCode()) {
                $bucketParams = [
                    'vectorBucketName' => $this->vectorBucketName,
                ];

                if ([] !== $tags) {
                    $bucketParams['tags'] = $tags;
                }

                if ([] !== $encryptionConfiguration) {
                    $bucketParams['encryptionConfiguration'] = $encryptionConfiguration;
                }

                $this->client->createVectorBucket($bucketParams);
            } else {
                throw $e;
            }
        }

        // Check if vector index exists, create if not found
        try {
            $this->client->getIndex([
                'vectorBucketName' => $this->vectorBucketName,
                'indexName' => $this->vectorIndexName,
            ]);
        } catch (\Aws\S3Vectors\Exception\S3VectorsException $e) {
            if ('NotFoundException' === $e->getAwsErrorCode()) {
                $indexParams = [
                    'vectorBucketName' => $this->vectorBucketName,
                    'indexName' => $this->vectorIndexName,
                    'dimension' => $dimension,
                    'distanceMetric' => $distanceMetric,
                    'dataType' => $dataType,
                    'metadataConfiguration' => $metadataConfiguration,
                ];

                if ([] !== $tags) {
                    $indexParams['tags'] = $tags;
                }

                if ([] !== $encryptionConfiguration) {
                    $indexParams['encryptionConfiguration'] = $encryptionConfiguration;
                }

                $this->client->createIndex($indexParams);
            } else {
                throw $e;
            }
        }
    }

    /**
     * Drops the vector index
     */
    public function drop(): void
    {
        try {
            $this->client->deleteIndex([
                'vectorBucketName' => $this->vectorBucketName,
                'indexName' => $this->vectorIndexName,
            ]);
        } catch (\Aws\S3Vectors\Exception\S3VectorsException $e) {
            // Index doesn't exist - acceptable
            if ('NotFoundException' !== $e->getAwsErrorCode()) {
                throw $e;
            }
        }
    }

    /**
     * Adds documents to the vector index.
     *
     * The number of vectors in a single request must not exceed the resource capacity.
     * Vector dimensions must match the dimension count configured for the vector index.
     * For the cosine distance metric, zero vectors (vectors containing all zeros) aren't allowed.
     * For both cosine and euclidean distance metrics, vector data must contain only valid floating-point values.
     *
     * @param VectorDocument ...$documents Documents to add with their vector embeddings and metadata
     */
    public function add(VectorDocument ...$documents): void
    {
        if ([] === $documents) {
            return;
        }

        $vectors = [];
        foreach ($documents as $document) {
            $vectorData = $document->vector->getData();

            $vector = [
                'key' => $document->id->toString(),
                'data' => [
                    'float32' => $vectorData,
                ],
            ];

            if (\count($document->metadata) > 0) {
                $metadata = $document->metadata->getArrayCopy();
                // metadata needs to be all scalars
                if(\array_key_exists(Metadata::KEY_PARENT_ID, $metadata) && $metadata[Metadata::KEY_PARENT_ID] instanceof Uuid) {
                    $metadata[Metadata::KEY_PARENT_ID] = $metadata[Metadata::KEY_PARENT_ID]->toString();
                }
                $vector['metadata'] = $metadata;
                
            }

            $vectors[] = $vector;
        }

        $this->client->putVectors([
            'vectorBucketName' => $this->vectorBucketName,
            'indexName' => $this->vectorIndexName,
            'vectors' => $vectors,
        ]);
    }

    /**
     * Queries the vector index for similar documents.
     *
     * Performs an approximate nearest neighbor search query in a vector index using a query vector.
     * By default, it returns the keys of approximate nearest neighbors. You can optionally include
     * the computed distance, vector data, and metadata of each vector in the response.
     *
     * @param Vector $vector Query vector. Must have the same dimension as the vectors in the index.
     * @param array{
     *     max_results?: int<1, max>,
     *     return_distance?: bool,
     *     return_metadata?: bool,
     *     filter?: array<string, mixed>
     * }|array<string, mixed> $options Query options:
     *                - max_results: The number of results to return for each query (default: 10)
     *                - return_distance: Include the computed distance between query vector and each result (default: true)
     *                - return_metadata: Include metadata in the response (default: true)
     *                - filter: Metadata filter to apply during the query. Only vectors matching the filter criteria will be returned.
     *
     * @return iterable<VectorDocument> Iterator of similar documents ordered by similarity
     */
    public function query(Vector $vector, array $options = []): iterable
    {
        $queryParams = [
            'vectorBucketName' => $this->vectorBucketName,
            'indexName' => $this->vectorIndexName,
            'queryVector' => [
                'float32' => $vector->getData(),
            ],
            'topK' => $options['max_results'] ?? self::DEFAULT_MAX_RESULTS,
            'returnDistance' => $options['return_distance'] ?? true,
            'returnMetadata' => $options['return_metadata'] ?? true,
        ];

        // Add metadata filter if provided
        if (isset($options['filter']) && [] !== $options['filter']) {
            $queryParams['filter'] = $options['filter'];
        }

        $result = $this->client->queryVectors($queryParams);

        if ([] === $result['vectors']) {
            return;
        }

        $vectorKeys = \array_map(static fn(array $item): string => $item['key'], $result['vectors']);
        $vectorsResult = $this->client->getVectors([
            'indexName' => $this->vectorIndexName,
            'returnData' => true,
            'returnMetadata' => true,
            'vectorBucketName' => $this->vectorBucketName,
            'keys' => $vectorKeys,
        ])->toArray();

        $vectorsMap = [];
        foreach ($vectorsResult['vectors'] as $vectorItem) {
            $vectorsMap[$vectorItem['key']] = $vectorItem;
        }

        // Yield documents in the same order as query results (ordered by similarity)
        foreach ($result['vectors'] as $item) {
            $vectorKey = $item['key'];
            $vectorInfo = $vectorsMap[$vectorKey] ?? null;

            if (null === $vectorInfo) {
                throw new \RuntimeException(sprintf('Vector with key "%s" not found in GetVectors response.', $vectorKey));
            }

            $vectorData = $vectorInfo['data']['float32'] ?? throw new \RuntimeException('Vector data missing in GetVectors response.');
            $resultVector = new Vector($vectorData);

            $metadata = $item['metadata'] ?? [];

            yield new VectorDocument(
                Uuid::fromString($vectorKey),
                $resultVector,
                new Metadata($metadata)
            );
        }
    }
}

Uh oh!

[Store] Support S3 Vectors as a store #1251

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions