Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for BYOVPC for GCP #213

Merged
merged 17 commits into from
Mar 29, 2025
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ steps:
- test_network
- label: test_byoc_aws
key: test_byoc_aws
if: build.tag == null && build.pull_request.labels includes 'byoc-ready'
if: build.tag == null && build.pull_request.labels includes 'ci-ready'
command: make test_byoc_aws
plugins:
- seek-oss/aws-sm#v2.3.2:
Expand All @@ -127,9 +127,10 @@ steps:
depends_on:
- lint
- unit
- test_network
- label: test_byoc_gcp
key: test_byoc_gcp
if: build.tag == null && build.pull_request.labels includes 'byoc-ready'
if: build.tag == null && build.pull_request.labels includes 'ci-ready'
command: make test_byoc_gcp
plugins:
- seek-oss/aws-sm#v2.3.2:
Expand All @@ -146,9 +147,10 @@ steps:
depends_on:
- lint
- unit
- test_network
- label: test_byoc_azure
key: test_byoc_azure
if: build.tag == null && build.pull_request.labels includes 'byoc-ready'
if: build.tag == null && build.pull_request.labels includes 'ci-ready'
command: make test_byoc_azure
plugins:
- seek-oss/aws-sm#v2.3.2:
Expand All @@ -167,6 +169,7 @@ steps:
depends_on:
- lint
- unit
- test_network
- label: Release
key: release
if: build.tag != null
Expand Down
97 changes: 95 additions & 2 deletions docs/resources/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ Required:
Optional:

- `aws` (Attributes) (see [below for nested schema](#nestedatt--customer_managed_resources--aws))
- `gcp` (Attributes) (see [below for nested schema](#nestedatt--customer_managed_resources--gcp))

<a id="nestedatt--customer_managed_resources--aws"></a>
### Nested Schema for `customer_managed_resources.aws`
Expand Down Expand Up @@ -303,6 +304,99 @@ Required:



<a id="nestedatt--customer_managed_resources--gcp"></a>
### Nested Schema for `customer_managed_resources.gcp`

Required:

- `agent_service_account` (Attributes) GCP service account for the agent. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--agent_service_account))
- `connector_service_account` (Attributes) GCP service account for managed connectors. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--connector_service_account))
- `console_service_account` (Attributes) GCP service account for Redpanda Console. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--console_service_account))
- `gke_service_account` (Attributes) GCP service account for GCP Kubernetes Engine (GKE). (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--gke_service_account))
- `redpanda_cluster_service_account` (Attributes) GCP service account for the Redpanda cluster. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--redpanda_cluster_service_account))
- `subnet` (Attributes) GCP subnet where Redpanda cluster is deployed. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--subnet))
- `tiered_storage_bucket` (Attributes) GCP storage bucket for Tiered storage. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--tiered_storage_bucket))

Optional:

- `psc_nat_subnet_name` (String) NAT subnet name if GCP Private Service Connect is enabled.

<a id="nestedatt--customer_managed_resources--gcp--agent_service_account"></a>
### Nested Schema for `customer_managed_resources.gcp.agent_service_account`

Required:

- `email` (String) GCP service account email.


<a id="nestedatt--customer_managed_resources--gcp--connector_service_account"></a>
### Nested Schema for `customer_managed_resources.gcp.connector_service_account`

Required:

- `email` (String) GCP service account email.


<a id="nestedatt--customer_managed_resources--gcp--console_service_account"></a>
### Nested Schema for `customer_managed_resources.gcp.console_service_account`

Required:

- `email` (String) GCP service account email.


<a id="nestedatt--customer_managed_resources--gcp--gke_service_account"></a>
### Nested Schema for `customer_managed_resources.gcp.gke_service_account`

Required:

- `email` (String) GCP service account email.


<a id="nestedatt--customer_managed_resources--gcp--redpanda_cluster_service_account"></a>
### Nested Schema for `customer_managed_resources.gcp.redpanda_cluster_service_account`

Required:

- `email` (String) GCP service account email.


<a id="nestedatt--customer_managed_resources--gcp--subnet"></a>
### Nested Schema for `customer_managed_resources.gcp.subnet`

Required:

- `k8s_master_ipv4_range` (String) Kubernetes Master IPv4 range, e.g. 10.0.0.0/24.
- `name` (String) Subnet name.
- `secondary_ipv4_range_pods` (Attributes) Secondary IPv4 range for pods. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--subnet--secondary_ipv4_range_pods))
- `secondary_ipv4_range_services` (Attributes) Secondary IPv4 range for services. (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--subnet--secondary_ipv4_range_services))

<a id="nestedatt--customer_managed_resources--gcp--subnet--secondary_ipv4_range_pods"></a>
### Nested Schema for `customer_managed_resources.gcp.subnet.secondary_ipv4_range_pods`

Required:

- `name` (String) Secondary IPv4 range name for pods.


<a id="nestedatt--customer_managed_resources--gcp--subnet--secondary_ipv4_range_services"></a>
### Nested Schema for `customer_managed_resources.gcp.subnet.secondary_ipv4_range_services`

Required:

- `name` (String) Secondary IPv4 range name for services.



<a id="nestedatt--customer_managed_resources--gcp--tiered_storage_bucket"></a>
### Nested Schema for `customer_managed_resources.gcp.tiered_storage_bucket`

Required:

- `name` (String) GCP storage bucket name.




<a id="nestedatt--gcp_private_service_connect"></a>
### Nested Schema for `gcp_private_service_connect`
Expand Down Expand Up @@ -1250,7 +1344,6 @@ module "redpanda_byovpc" {
source = "redpanda-data/redpanda-byovpc/aws"

region = var.region
aws_account_id = "879326078624"
public_subnet_cidrs = [
"10.0.16.0/20", "10.0.32.0/20", "10.0.48.0/20"
]
Expand Down Expand Up @@ -1516,6 +1609,6 @@ variable "replication_factor" {
## Import

```shell
terraform import resource.redpanda_cluster.example clusterId
terraform import redpanda_cluster.example clusterId
```

21 changes: 20 additions & 1 deletion docs/resources/network.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Creates a network in the Redpanda Cloud.
Optional:

- `aws` (Attributes) (see [below for nested schema](#nestedatt--customer_managed_resources--aws))
- `gcp` (Attributes) (see [below for nested schema](#nestedatt--customer_managed_resources--gcp))

<a id="nestedatt--customer_managed_resources--aws"></a>
### Nested Schema for `customer_managed_resources.aws`
Expand Down Expand Up @@ -79,6 +80,24 @@ Required:

- `arn` (String) AWS VPC identifier



<a id="nestedatt--customer_managed_resources--gcp"></a>
### Nested Schema for `customer_managed_resources.gcp`

Required:

- `management_bucket` (Attributes) (see [below for nested schema](#nestedatt--customer_managed_resources--gcp--management_bucket))
- `network_name` (String) Name of user-created network where the Redpanda cluster is deployed
- `network_project_id` (String) GCP project ID where the network is created

<a id="nestedatt--customer_managed_resources--gcp--management_bucket"></a>
### Nested Schema for `customer_managed_resources.gcp.management_bucket`

Required:

- `name` (String) GCP storage bucket name for storing the state of Redpanda cluster deployment

## Usage

```terraform
Expand Down Expand Up @@ -117,5 +136,5 @@ variable "cloud_provider" {
## Import

```shell
terraform import resource.redpanda_network.example networkId
terraform import redpanda_network.example networkId
```
2 changes: 1 addition & 1 deletion docs/resources/resource_group.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,5 +145,5 @@ variable "replication_factor" {
## Import

```shell
terraform import resource.redpanda_resource_group.example resourcegroupId
terraform import redpanda_resource_group.example resourcegroupId
```
2 changes: 1 addition & 1 deletion docs/resources/serverless_cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,6 @@ variable "region" {
## Import

```shell
terraform import resource.redpanda_serverless_cluster.example serverlessClusterId
terraform import redpanda_serverless_cluster.example serverlessClusterId
```

2 changes: 1 addition & 1 deletion docs/resources/topic.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ We are not currently able to support topic creation in self hosted clusters. Thi
## Import

```shell
terraform import resource.redpanda_topic.example topicName,clusterId
terraform import redpanda_topic.example topicName,clusterId
```

Where clusterId is the ID of the cluster in Redpanda Cloud
2 changes: 1 addition & 1 deletion docs/resources/user.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ We are not currently able to support user creation in self hosted clusters. This
## Import

```shell
terraform import resource.redpanda_user.example userName,clusterId
terraform import redpanda_user.example userName,clusterId
```

Where clusterId is the ID of the cluster in Redpanda Cloud
1 change: 0 additions & 1 deletion examples/byovpc/aws/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ module "redpanda_byovpc" {
source = "redpanda-data/redpanda-byovpc/aws"

region = var.region
aws_account_id = "879326078624"
public_subnet_cidrs = [
"10.0.16.0/20", "10.0.32.0/20", "10.0.48.0/20"
]
Expand Down
91 changes: 91 additions & 0 deletions examples/byovpc/aws/vpc/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
module "redpanda-byovpc" {
source = "redpanda-data/redpanda-byovpc/aws"
region = var.region

public_subnet_cidrs = [
"10.0.16.0/20", "10.0.32.0/20", "10.0.48.0/20"
]
}

provider "aws" {
region = var.region
}

variable "region" {
type = string
default = "us-east-2"
}

output "management_bucket_arn" {
value = module.redpanda-byovpc.management_bucket_arn
}

output "dynamodb_table_arn" {
value = module.redpanda-byovpc.dynamodb_table_arn
}

output "vpc_arn" {
value = module.redpanda-byovpc.vpc_arn
}

output "private_subnet_arns" {
value = module.redpanda-byovpc.private_subnet_arns
}

output "permissions_boundary_policy_arn" {
value = module.redpanda-byovpc.permissions_boundary_policy_arn
}

output "agent_instance_profile_arn" {
value = module.redpanda-byovpc.agent_instance_profile_arn
}

output "cloud_storage_bucket_arn" {
value = module.redpanda-byovpc.cloud_storage_bucket_arn
}

output "redpanda_agent_security_group_arn" {
value = module.redpanda-byovpc.redpanda_agent_security_group_arn
}

output "connectors_security_group_arn" {
value = module.redpanda-byovpc.connectors_security_group_arn
}

output "redpanda_node_group_security_group_arn" {
value = module.redpanda-byovpc.redpanda_node_group_security_group_arn
}

output "utility_security_group_arn" {
value = module.redpanda-byovpc.utility_security_group_arn
}

output "cluster_security_group_arn" {
value = module.redpanda-byovpc.cluster_security_group_arn
}

output "node_security_group_arn" {
value = module.redpanda-byovpc.node_security_group_arn
}

output "byovpc_rpk_user_policy_arns" {
value = module.redpanda-byovpc.byovpc_rpk_user_policy_arns
}

output "connectors_node_group_instance_profile_arn" {
value = module.redpanda-byovpc.connectors_node_group_instance_profile_arn
}

output "utility_node_group_instance_profile_arn" {
value = module.redpanda-byovpc.utility_node_group_instance_profile_arn
}

output "redpanda_node_group_instance_profile_arn" {
value = module.redpanda-byovpc.redpanda_node_group_instance_profile_arn
}

output "k8s_cluster_role_arn" {
value = module.redpanda-byovpc.k8s_cluster_role_arn
}


Loading