Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions athena-s3vector-connector/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ The connector can be deployed directly using the provided CloudFormation templat
- **AthenaCatalogName**: Name for your Lambda function (lowercase, alphanumeric, hyphens, and underscores only, 1-64 characters)
- **SpillBucket**: S3 bucket name for query result spilling
- **SpillPrefix**: Prefix within SpillBucket (default: athena-spill)
- **S3VectorBucketName**: Name of the S3 Vector bucket to query
- **LambdaTimeout**: Maximum Lambda runtime in seconds (default: 900)
- **LambdaMemory**: Lambda memory in MB (default: 512)
- **DisableSpillEncryption**: Set to 'true' to disable spill encryption (default: false)
Expand Down Expand Up @@ -87,6 +88,7 @@ For subsequent updates after initial deployment:
| AthenaCatalogName | Lambda function name (must match pattern: ^[a-z0-9-_]{1,64}$) | Required |
| SpillBucket | S3 bucket for spilling data | Required |
| SpillPrefix | Prefix within SpillBucket | athena-spill |
| S3VectorBucketName | Name of the S3 Vector bucket to query | Required |
| LambdaTimeout | Maximum Lambda invocation runtime (1-900 seconds) | 900 |
| LambdaMemory | Lambda memory in MB (128-3008) | 1024 |
| DisableSpillEncryption | Disable encryption for spilled data | false |
Expand Down
26 changes: 20 additions & 6 deletions athena-s3vector-connector/athena-s3vector-connector.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ Parameters:
Description: "Lambda memory in MB (min 128 - 3008 max)."
Default: 512
Type: Number
S3VectorBucketName:
Description: "The name of the S3 Vector bucket to query."
Type: String
DisableSpillEncryption:
Description: "WARNING: If set to 'true' encryption for spilled data is disabled."
Default: "false"
Expand All @@ -49,29 +52,40 @@ Resources:
FunctionName: !Sub "${AthenaCatalogName}"
Handler: "com.amazonaws.athena.connectors.s3vector.S3VectorCompositeHandler"
CodeUri: "./target/athena-s3vector-connector-0.1.0.jar"
Description: "A guided example for writing and deploying your own federated Amazon Athena connector for a custom source."
Description: "Federated connector to query vector data stored in S3 Vector buckets."
Runtime: java11
Timeout: !Ref LambdaTimeout
MemorySize: !Ref LambdaMemory
Policies:
# It's common and expected for a single Athena connector to serve queries from multiple workgroups.
# However, if the lambda is restricted to a specific workgroup update the permission to "arn:aws:athena:<region>:<account-id>:workgroup/<workgroup-name>"
- Statement:
- Action:
- athena:GetQueryExecution
Effect: Allow
Resource: '*'
Resource: !Sub 'arn:${AWS::Partition}:athena:${AWS::Region}:${AWS::AccountId}:workgroup/*'
Comment thread
andy-k-improving marked this conversation as resolved.
Version: '2012-10-17'

# S3 Vector read access
# S3 Vector list access (account-wide)
- Statement:
- Sid: AllowApplicationVectorAccess
- Sid: AllowVectorListAccess
Effect: Allow
Action:
- s3vectors:GetIndex
- s3vectors:ListIndexes
Resource: !Sub 'arn:${AWS::Partition}:s3vectors:${AWS::Region}:${AWS::AccountId}:*'
Comment thread
andy-k-improving marked this conversation as resolved.

# S3 Vector data access (scoped to specific bucket and its indexes)
- Statement:
- Sid: AllowVectorDataAccess
Effect: Allow
Action:
- s3vectors:GetIndex
- s3vectors:QueryVectors
- s3vectors:GetVectors
- s3vectors:ListVectors
Resource: '*'
Resource:
- !Sub 'arn:${AWS::Partition}:s3vectors:${AWS::Region}:${AWS::AccountId}:vector-bucket/${S3VectorBucketName}'
- !Sub 'arn:${AWS::Partition}:s3vectors:${AWS::Region}:${AWS::AccountId}:vector-bucket/${S3VectorBucketName}/*'

#S3CrudPolicy allows our connector to spill large responses to S3. You can optionally replace this pre-made policy
#with one that is more restrictive and can only 'put' but not read,delete, or overwrite files.
Expand Down
Loading