[TT-14473] - support for encrypted aws kinesis #876
✅ Check Passed
overview check completed successfully with no issues found.
Details
📊 Summary
- Total Issues: 1
🐛 Issues by Category
📚 Documentation (1)
- ℹ️ AI_RESPONSE:1 - ### 1. Change Impact Analysis
What this PR Accomplishes
This pull request introduces a significant security enhancement to the Tyk Kinesis Pump by adding support for server-side encryption (SSE). Users can now specify an AWS Key Management Service (KMS) key ID in the pump's configuration. The pump will then ensure that the target Kinesis stream is encrypted using this key, protecting data at rest and helping to meet compliance requirements.
Key Technical Changes Introduced
- New Configuration Option: The
KinesisConfstruct inpumps/kinesis.gois extended with aKMSKeyIDfield. This allows users to specify the KMS key via configuration files (kms_key_id) or environment variables (TYK_PMP_PUMPS_KINESIS_META_KMSKEYID). TheREADME.mdhas been updated to document this new setting. - Idempotent Encryption Setup: During initialization (
Initfunction), if aKMSKeyIDis provided, the pump first checks the stream's current encryption status using theDescribeStreamAPI call. - Conditional Encryption Enforcement:
- If the stream is not encrypted, the pump calls
StartStreamEncryptionto enable it with the specified key. - If the stream is already encrypted with the correct key, the pump logs this and continues, avoiding unnecessary API calls.
- If the stream is encrypted with a different key, the pump logs a fatal error and terminates. This "fail-secure" approach prevents running with a critical misconfiguration.
- If the stream is not encrypted, the pump calls
- Secure Logging: The KMS Key ID is partially masked in log outputs to prevent accidental exposure of sensitive information.
- Unit Testing: A new test file,
pumps/kinesis_test.go, has been added to validate the new configuration options and associated logic.
Affected System Components
- Kinesis Pump (
pumps/kinesis.go): The core logic is modified to include the new encryption setup during its initialization phase. - Configuration (
README.md): The configuration schema for the Kinesis pump is updated. Users deploying this pump will need to be aware of the newkms_key_idoption. - IAM Permissions: To use this feature, the IAM role or user associated with the Tyk Pump will require additional permissions:
kinesis:DescribeStreamandkinesis:StartStreamEncryptionfor the target Kinesis stream. Note: This requirement is not yet documented in theREADME.mdand should be added.
2. Architecture Visualization
The following sequence diagram illustrates the updated initialization flow of the Kinesis pump. It highlights the new conditional logic for checking and enabling server-side encryption.
sequenceDiagram
participant P as Tyk Pump
participant KP as KinesisPump
participant AWS_SDK as AWS Kinesis Client
participant Kinesis as AWS Kinesis API
P->>KP: Init(config)
KP->>KP: Parse config, load StreamName & KMSKeyID
alt KMSKeyID is provided
KP->>AWS_SDK: DescribeStream(StreamName)
AWS_SDK->>Kinesis: API Call: DescribeStream
Kinesis-->>AWS_SDK: returns StreamDescription
AWS_SDK-->>KP: returns StreamDescription
alt Stream is NOT encrypted
KP->>AWS_SDK: StartStreamEncryption(StreamName, KMSKeyID)
AWS_SDK->>Kinesis: API Call: StartStreamEncryption
Kinesis-->>AWS_SDK: API Response (Success or Error)
alt API Call Succeeds
AWS_SDK-->>KP: returns success
KP->>KP: Log "Server-side encryption enabled"
else API returns Error
AWS_SDK-->>KP: returns error
KP->>P: Log.Fatalf("Failed to enable encryption")
end
else Stream is encrypted with a different key
KP->>P: Log.Fatal("Encryption enabled with wrong key")
else Stream is already encrypted correctly
KP->>KP: Log "Encryption already enabled"
end
end
KP-->>P: Initialization Complete
Generated by Visor - AI-powered code review
Annotations
Check notice on line 1 in AI_RESPONSE
probelabs / Visor: overview
documentation Issue
### **1. Change Impact Analysis**
#### **What this PR Accomplishes**
This pull request introduces a significant security enhancement to the Tyk Kinesis Pump by adding support for server-side encryption (SSE). Users can now specify an AWS Key Management Service (KMS) key ID in the pump's configuration. The pump will then ensure that the target Kinesis stream is encrypted using this key, protecting data at rest and helping to meet compliance requirements.
#### **Key Technical Changes Introduced**
1. **New Configuration Option:** The `KinesisConf` struct in `pumps/kinesis.go` is extended with a `KMSKeyID` field. This allows users to specify the KMS key via configuration files (`kms_key_id`) or environment variables (`TYK_PMP_PUMPS_KINESIS_META_KMSKEYID`). The `README.md` has been updated to document this new setting.
2. **Idempotent Encryption Setup:** During initialization (`Init` function), if a `KMSKeyID` is provided, the pump first checks the stream's current encryption status using the `DescribeStream` API call.
3. **Conditional Encryption Enforcement:**
* If the stream is not encrypted, the pump calls `StartStreamEncryption` to enable it with the specified key.
* If the stream is already encrypted with the correct key, the pump logs this and continues, avoiding unnecessary API calls.
* If the stream is encrypted with a *different* key, the pump logs a fatal error and terminates. This "fail-secure" approach prevents running with a critical misconfiguration.
4. **Secure Logging:** The KMS Key ID is partially masked in log outputs to prevent accidental exposure of sensitive information.
5. **Unit Testing:** A new test file, `pumps/kinesis_test.go`, has been added to validate the new configuration options and associated logic.
#### **Affected System Components**
* **Kinesis Pump (`pumps/kinesis.go`):** The core logic is modified to include the new encryption setup during its initialization phase.
* **Configuration (`README.md`):** The configuration schema for the Kinesis pump is updated. Users deploying this pump will need to be aware of the new `kms_key_id` option.
* **IAM Permissions:** To use this feature, the IAM role or user associated with the Tyk Pump will require additional permissions: `kinesis:DescribeStream` and `kinesis:StartStreamEncryption` for the target Kinesis stream. **Note:** This requirement is not yet documented in the `README.md` and should be added.
### **2. Architecture Visualization**
The following sequence diagram illustrates the updated initialization flow of the Kinesis pump. It highlights the new conditional logic for checking and enabling server-side encryption.
```mermaid
sequenceDiagram
participant P as Tyk Pump
participant KP as KinesisPump
participant AWS_SDK as AWS Kinesis Client
participant Kinesis as AWS Kinesis API
P->>KP: Init(config)
KP->>KP: Parse config, load StreamName & KMSKeyID
alt KMSKeyID is provided
KP->>AWS_SDK: DescribeStream(StreamName)
AWS_SDK->>Kinesis: API Call: DescribeStream
Kinesis-->>AWS_SDK: returns StreamDescription
AWS_SDK-->>KP: returns StreamDescription
alt Stream is NOT encrypted
KP->>AWS_SDK: StartStreamEncryption(StreamName, KMSKeyID)
AWS_SDK->>Kinesis: API Call: StartStreamEncryption
Kinesis-->>AWS_SDK: API Response (Success or Error)
alt API Call Succeeds
AWS_SDK-->>KP: returns success
KP->>KP: Log "Server-side encryption enabled"
else API returns Error
AWS_SDK-->>KP: returns error
KP->>P: Log.Fatalf("Failed to enable encryption")
end
else Stream is encrypted with a different key
KP->>P: Log.Fatal("Encryption enabled with wrong key")
else Stream is already encrypted correctly
KP->>KP: Log "Encryption already enabled"
end
end
KP-->>P: Initialization Complete
```