Red Hat OpenShift VPC cluster on IBM Cloud module

Use this module to provision an IBM Cloud Red Hat OpenShift cluster on VPC Gen2. The module either creates the required Cloud Object Storage instance or uses an existing instance. The module also supports optionally passing a key management configuration for secret encryption and boot volume encryption.

Optionally, the module supports advanced security group management for the worker nodes, VPE, and load balancer associated with the cluster. This feature allows you to configure security groups for the cluster's worker nodes, VPE, and load balancer.

By default, the module automatically downloads the required dependant binaries (jq and kubectl from the public internet if they are not detected in th terraform runtime. You can disable this behavior by setting install_required_binaries to false.

Helpful resources

Overview

terraform-ibm-base-ocp-vpc
Submodules
Examples
ℹ️ Ctrl/Cmd+Click or right-click on the Schematics deploy button to open in a new tab.
Deployable Architectures
- Cloud automation for Red Hat OpenShift Container Platform on VPC (Fully configurable)
- Cloud automation for Red Hat OpenShift Container Platform on VPC (QuickStart)
Known issues
Contributing

terraform-ibm-base-ocp-vpc

Usage

module "ocp_base" {
  source               = "terraform-ibm-modules/base-ocp-vpc/ibm"
  version              = "X.Y.Z" # Replace "X.Y.Z" with a release version to lock into a specific release
  cluster_name         = "example-cluster-name"
  resource_group_id    = "xxXXxxXXxXxXXXXxxXxxxXXXXxXXXXX"
  region               = "us-south"
  force_delete_storage = true
  vpc_id               = "79cxxxx-xxxx-xxxx-xxxx-xxxxxXX8667"
  # obtain the below values from the targeted VPC and adjust to the number of zones, subnets, subnet name, cidr_block, id, zone
  vpc_subnets          = {
    zone-1    = [
        {
            cidr_block = "192.168.32.0/22"
            id         = "0717-afc29fbb-0dbe-493a-a5b9-f3c5899cb8b9"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.36.0/22"
            id         = "0727-d65c1eda-9e38-4200-8452-cb8ff5bb3140"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.40.0/22"
            id         = "0737-9a823cd3-16bf-4ba4-a429-9e1fc7db74b8"
            zone       = "us-south-3"
        }
    ]
    zone-2 = [
        {
            cidr_block = "192.168.0.0/22"
            id         = "0717-846b9490-34ae-4a6c-8288-28112dca1ba3"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.4.0/22"
            id         = "0727-ef8db7f6-ffa5-4d8b-a317-4631741a45ee"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.8.0/22"
            id         = "0737-c9a6d871-d95b-4914-abf5-82c22f4161d1"
            zone       = "us-south-3"
        }
    ]
    zone-3 = [
        {
            cidr_block = "192.168.16.0/22"
            id         = "0717-d46e227c-89d4-4b02-9008-d03907a275b6"
            zone       = "us-south-1"
        },
        {
            cidr_block = "192.168.20.0/22"
            id         = "0727-93b1edcb-966c-4517-a7af-6ac63cd93adf"
            zone       = "us-south-2"
        },
        {
            cidr_block = "192.168.24.0/22"
            id         = "0737-807ec4f1-4d84-484e-b2f4-62dd5e431065"
            zone       = "us-south-3"
        }
    ]
  }
  worker_pools         = [
    {
      subnet_prefix    = "default"
      pool_name        = "default"
      machine_type     = "bx2.4x16"
      workers_per_zone = 2
      operating_system = "REDHAT_8_64"
    }
  ]
}

Required IAM access policies

You need the following permissions to run this module.

Account Management
- All Identity and Access Enabled service
  - Viewer platform access
- All Resource Groups service
  - Viewer platform access
IAM Services
- Cloud Object Storage service
  - Editor platform access
  - Manager service access
- Kubernetes service
  - Administrator platform access
  - Manager service access
- VPC Infrastructure service
  - Administrator platform access
  - Manager service access
- IAM Identity Service service
  - User API key creator service access

Optionally, you need the following permissions to attach Access Management tags to resources in this module.

IAM Services
- Tagging service
  - Administrator platform access

Requirements

Name	Version
terraform	>= 1.9.0
external	>=2.3.5, <3.0.0
ibm	>= 1.80.0, < 2.0.0
kubernetes	>= 3.0.0, < 4.0.0
null	>= 3.2.1, < 4.0.0
time	>= 0.9.1, < 1.0.0

Modules

Name	Source	Version
attach_sg_to_api_vpe	terraform-ibm-modules/security-group/ibm	2.8.9
attach_sg_to_lb	terraform-ibm-modules/security-group/ibm	2.8.9
attach_sg_to_master_vpe	terraform-ibm-modules/security-group/ibm	2.8.9
attach_sg_to_registry_vpe	terraform-ibm-modules/security-group/ibm	2.8.9
cbr_rule	terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module	1.35.19
cos_instance	terraform-ibm-modules/cos/ibm	10.14.9
existing_secrets_manager_instance_parser	terraform-ibm-modules/common-utilities/ibm//modules/crn-parser	1.4.2
worker_pools	./modules/worker-pool	n/a

Resources

Name	Type
ibm_container_addons.addons	resource
ibm_container_ingress_instance.instance	resource
ibm_container_vpc_cluster.autoscaling_cluster	resource
ibm_container_vpc_cluster.autoscaling_cluster_with_upgrade	resource
ibm_container_vpc_cluster.cluster	resource
ibm_container_vpc_cluster.cluster_with_upgrade	resource
ibm_iam_authorization_policy.ocp_secrets_manager_iam_auth_policy	resource
ibm_resource_tag.cluster_access_tag	resource
ibm_resource_tag.cos_access_tag	resource
kubernetes_config_map_v1_data.set_autoscaling	resource
null_resource.config_map_status	resource
null_resource.confirm_network_healthy	resource
null_resource.ocp_console_management	resource
terraform_data.install_required_binaries	resource
time_sleep.wait_for_auth_policy	resource
external_external.ocp_addon_versions	data source
ibm_container_addons.existing_addons	data source
ibm_container_cluster_config.cluster_config	data source
ibm_container_cluster_versions.cluster_versions	data source
ibm_iam_auth_token.tokendata	data source
ibm_is_lbs.all_lbs	data source
ibm_is_virtual_endpoint_gateway.api_vpe	data source
ibm_is_virtual_endpoint_gateway.master_vpe	data source
ibm_is_virtual_endpoint_gateway.registry_vpe	data source

Inputs

Name	Description	Type	Default	Required
access_tags	A list of access tags to apply to the resources created by the module, see https://cloud.ibm.com/docs/account?topic=account-access-tags-tutorial for more details	`list(string)`	`[]`	no
additional_lb_security_group_ids	Additional security groups to add to the load balancers associated with the cluster. Ensure that the `number_of_lbs` is set to the number of LBs associated with the cluster. This comes in addition to the IBM maintained security group.	`list(string)`	`[]`	no
additional_vpe_security_group_ids	Additional security groups to add to all existing load balancers. This comes in addition to the IBM maintained security group.	object({ master = optional(list(string), []) registry = optional(list(string), []) api = optional(list(string), []) })	`{}`	no
addons	Map of OCP cluster add-on versions to install (NOTE: The 'vpc-block-csi-driver' add-on is installed by default for VPC clusters and 'ibm-storage-operator' is installed by default in OCP 4.15 and later, however you can explicitly specify it here if you wish to choose a later version than the default one). For full list of all supported add-ons and versions, see https://cloud.ibm.com/docs/containers?topic=containers-supported-cluster-addon-versions	object({ debug-tool = optional(object({ version = optional(string) parameters_json = optional(string) })) image-key-synchronizer = optional(object({ version = optional(string) parameters_json = optional(string) })) openshift-data-foundation = optional(object({ version = optional(string) parameters_json = optional(string) })) vpc-file-csi-driver = optional(object({ version = optional(string) parameters_json = optional(string) })) static-route = optional(object({ version = optional(string) parameters_json = optional(string) })) cluster-autoscaler = optional(object({ version = optional(string) parameters_json = optional(string) })) vpc-block-csi-driver = optional(object({ version = optional(string) parameters_json = optional(string) })) ibm-storage-operator = optional(object({ version = optional(string) parameters_json = optional(string) })) openshift-ai = optional(object({ version = optional(string) parameters_json = optional(string) })) })	`{}`	no
allow_default_worker_pool_replacement	(Advanced users) Set to true to allow the module to recreate a default worker pool. If you wish to make any change to the default worker pool which requires the re-creation of the default pool follow these steps.	`bool`	`false`	no
attach_ibm_managed_security_group	Specify whether to attach the IBM-defined default security group (whose name is kube-) to all worker nodes. Only applicable if `custom_security_group_ids` is set.	`bool`	`true`	no
cbr_rules	The context-based restrictions rule to create. Only one rule is allowed.	list(object({ description = string account_id = string rule_contexts = list(object({ attributes = optional(list(object({ name = string value = string }))) })) enforcement_mode = string tags = optional(list(object({ name = string value = string })), []) operations = optional(list(object({ api_types = list(object({ api_type_id = string })) }))) }))	`[]`	no
cluster_config_endpoint_type	Specify which type of endpoint to use for cluster config access: 'default', 'private', 'vpe', 'link'. A 'default' value uses the default endpoint of the cluster.	`string`	`"default"`	no
cluster_create_timeout	Timeout duration for cluster creation operations. Specify a duration string (e.g., '3h', '45m').	`string`	`"3h"`	no
cluster_delete_timeout	Timeout duration for cluster deletion operations. Specify a duration string (e.g., '2h', '30m').	`string`	`"2h"`	no
cluster_name	The name that is assigned to the provisioned cluster.	`string`	n/a	yes
cluster_ready_when	The cluster is ready based on one of the following:: MasterNodeReady (not recommended), OneWorkerNodeReady, Normal, IngressReady	`string`	`"IngressReady"`	no
cluster_update_timeout	Timeout duration for cluster update operations. Specify a duration string (e.g., '3h', '1h30m').	`string`	`"3h"`	no
cos_name	Name of the COS instance to provision for OpenShift internal registry storage. New instance only provisioned if 'enable_registry_storage' is true and 'use_existing_cos' is false. Default: '<cluster_name>_cos'	`string`	`null`	no
custom_security_group_ids	Security groups to add to all worker nodes. This comes in addition to the IBM maintained security group if `attach_ibm_managed_security_group` is set to true. If this variable is set, the default VPC security group is NOT assigned to the worker nodes.	`list(string)`	`null`	no
disable_outbound_traffic_protection	Whether to allow public outbound access from the cluster workers. This is only applicable for OCP 4.15 and later.	`bool`	`false`	no
disable_public_endpoint	Whether access to the public service endpoint is disabled when the cluster is created. Does not affect existing clusters. You can't disable a public endpoint on an existing cluster, so you can't convert a public cluster to a private cluster. To change a public endpoint to private, create another cluster with this input set to `true`.	`bool`	`false`	no
enable_ocp_console	Flag to specify whether to enable or disable the OpenShift console. If set to `null` the module does not modify the current setting on the cluster. Keep in mind that when this input is set to `true` or `false` on a cluster with private only endpoint enabled, the runtime must be able to access the private endpoint.	`bool`	`null`	no
enable_openshift_version_upgrade	When set to true, allows Terraform to manage major OpenShift version upgrades. This is intended for advanced users who manually control major version upgrades. Defaults to false to avoid unintended drift from IBM-managed patch updates. NOTE: Enabling this on existing clusters requires a one-time terraform state migration. See README for details.	`bool`	`false`	no
enable_registry_storage	Set to `true` to enable IBM Cloud Object Storage for the Red Hat OpenShift internal image registry. Set to `false` only for new cluster deployments in an account that is allowlisted for this feature.	`bool`	`true`	no
enable_secrets_manager_integration	Integrate with IBM Cloud Secrets Manager so you can centrally manage Ingress subdomain certificates and other secrets. Learn more	`bool`	`false`	no
existing_cos_id	The COS id of an already existing COS instance to use for OpenShift internal registry storage. Only required if 'enable_registry_storage' and 'use_existing_cos' are true.	`string`	`null`	no
existing_secrets_manager_instance_crn	CRN of the Secrets Manager instance where Ingress certificate secrets are stored. If 'enable_secrets_manager_integration' is set to true then this value is required.	`string`	`null`	no
force_delete_storage	Flag indicating whether or not to delete attached storage when destroying the cluster - Default: false	`bool`	`false`	no
ignore_worker_pool_size_changes	Enable if using worker autoscaling. Stops Terraform managing worker count	`bool`	`false`	no
install_required_binaries	When set to true, a script will run to check if `kubectl` and `jq` exist on the runtime and if not attempt to download them from the public internet and install them to /tmp. Set to false to skip running this script.	`bool`	`true`	no
kms_config	Use to attach a KMS instance to the cluster. If account_id is not provided, defaults to the account in use.	object({ crk_id = string instance_id = string private_endpoint = optional(bool, true) # defaults to true account_id = optional(string) # To attach KMS instance from another account wait_for_apply = optional(bool, true) # defaults to true so terraform will wait until the KMS is applied to the master, ready and deployed })	`null`	no
manage_all_addons	Instructs Terraform to manage all cluster addons, even if addons were installed outside of the module. If set to 'true' this module destroys any addons that were installed by other sources.	`bool`	`false`	no
number_of_lbs	The number of LBs to associated the `additional_lb_security_group_names` security group with.	`number`	`1`	no
ocp_entitlement	Value that is applied to the entitlements for OCP cluster provisioning	`string`	`null`	no
ocp_version	The version of the OpenShift cluster that should be provisioned (format 4.x). If no value is specified, the current default version is used. You can also specify `default`. This input is used only during initial cluster provisioning and is ignored for updates. To prevent possible destructive changes, update the cluster version outside of Terraform.	`string`	`null`	no
pod_subnet_cidr	Specify a custom subnet CIDR to provide private IP addresses for pods. The subnet must have a CIDR of at least `/23` or larger. Default value is `172.30.0.0/16` when the variable is set to `null`.	`string`	`null`	no
region	The IBM Cloud region where the cluster is provisioned.	`string`	n/a	yes
resource_group_id	The ID of an existing IBM Cloud resource group where the cluster is grouped.	`string`	n/a	yes
secrets_manager_secret_group_id	Secret group ID where Ingress secrets are stored in the Secrets Manager instance.	`string`	`null`	no
service_subnet_cidr	Specify a custom subnet CIDR to provide private IP addresses for services. The subnet must be at least `/24` or larger. Default value is `172.21.0.0/16` when the variable is set to `null`.	`string`	`null`	no
skip_ocp_secrets_manager_iam_auth_policy	To skip creating auth policy that allows OCP cluster 'Manager' role access in the existing Secrets Manager instance for managing ingress certificates.	`bool`	`false`	no
tags	Metadata labels describing this cluster deployment, i.e. test	`list(string)`	`[]`	no
use_existing_cos	Flag indicating whether or not to use an existing COS instance for OpenShift internal registry storage. Only applicable if 'enable_registry_storage' is true	`bool`	`false`	no
verify_worker_network_readiness	By setting this to true, a script runs kubectl commands to verify that all worker nodes can communicate successfully with the master. If the runtime does not have access to the kube cluster to run kubectl commands, set this value to false.	`bool`	`true`	no
vpc_id	ID of the VPC instance where this cluster is provisioned.	`string`	n/a	yes
vpc_subnets	Metadata that describes the VPC's subnets. Obtain this information from the VPC where this cluster is created.	map(list(object({ id = string zone = string cidr_block = string })))	n/a	yes
worker_pools	List of worker pools	list(object({ subnet_prefix = optional(string) vpc_subnets = optional(list(object({ id = string zone = string cidr_block = string }))) pool_name = string machine_type = string workers_per_zone = number resource_group_id = optional(string) operating_system = string labels = optional(map(string)) minSize = optional(number) secondary_storage = optional(string) maxSize = optional(number) enableAutoscaling = optional(bool) boot_volume_encryption_kms_config = optional(object({ crk = string kms_instance_id = string kms_account_id = optional(string) })) additional_security_group_ids = optional(list(string)) }))	n/a	yes
worker_pools_taints	Optional, Map of lists containing node taints by node-pool name	`map(list(object({ key = string, value = string, effect = string })))`	`null`	no

Outputs

Name	Description
api_vpe	Info about the api VPE, if it exists. For more info about schema, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/is_virtual_endpoint_gateway
cluster_crn	CRN of the cluster
cluster_id	ID of the cluster
cluster_name	Name of the cluster
cos_crn	CRN of the COS instance
ingress_hostname	The hostname that was assigned to your Ingress subdomain.
kms_config	KMS configuration details
master_status	The status of the Kubernetes master.
master_url	The URL of the Kubernetes master.
master_vpe	Info about the master, or default, VPE. For more info about schema, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/is_virtual_endpoint_gateway
ocp_version	Openshift Version of the cluster
operating_system	The operating system of the workers in the default worker pool.
private_service_endpoint_url	Private service endpoint URL
public_service_endpoint_url	Public service endpoint URL
region	Region that the cluster is deployed to
registry_vpe	Info about the registry VPE, if it exists. For more info about schema, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/data-sources/is_virtual_endpoint_gateway
resource_group_id	Resource group ID the cluster is deployed in
secrets_manager_integration_config	Information about the Secrets Manager instance that is used to store the Ingress certificates.
vpc_id	ID of the clusters VPC
vpe_url	The virtual private endpoint URL of the Kubernetes cluster.
workerpools	Worker pools created

Known issues

For a list of common known issues see:

Known issues with Red Hat OpenShift on IBM Cloud / IBM Cloud Kubernetes Service when using terraform

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.

Name		Name	Last commit message	Last commit date
Latest commit History 791 Commits
.github		.github
common-dev-assets @ 4f52b6a		common-dev-assets @ 4f52b6a
docs		docs
examples		examples
images		images
kubeconfig		kubeconfig
modules		modules
reference-architectures		reference-architectures
scripts		scripts
solutions		solutions
tests		tests
.catalog-onboard-pipeline.yaml		.catalog-onboard-pipeline.yaml
.gitignore		.gitignore
.gitmodules		.gitmodules
.pre-commit-config.yaml		.pre-commit-config.yaml
.releaserc		.releaserc
.secrets.baseline		.secrets.baseline
.terraformignore		.terraformignore
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
ci		ci
ibm_catalog.json		ibm_catalog.json
main.tf		main.tf
moved.tf		moved.tf
outputs.tf		outputs.tf
renovate.json		renovate.json
variables.tf		variables.tf
version.tf		version.tf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Red Hat OpenShift VPC cluster on IBM Cloud module

Helpful resources

Overview

terraform-ibm-base-ocp-vpc

Usage

Required IAM access policies

Requirements

Modules

Resources

Inputs

Outputs

Known issues

Contributing

About

Uh oh!

Releases 325

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Red Hat OpenShift VPC cluster on IBM Cloud module

Helpful resources

Overview

terraform-ibm-base-ocp-vpc

Usage

Required IAM access policies

Requirements

Modules

Resources

Inputs

Outputs

Known issues

Contributing

About

Topics

Resources

License

Code of conduct

Contributing

Uh oh!

Stars

Watchers

Forks

Releases 325

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages