Skip to content

Commit 370bd59

Browse files
BrianMMcClainBrianMMcClain
committed
Add documentation to refactor Terraform state
1 parent 032cd5f commit 370bd59

File tree

2 files changed

+255
-0
lines changed

2 files changed

+255
-0
lines changed

‎website/data/language-nav-data.json

+4
Original file line numberDiff line numberDiff line change
@@ -1082,6 +1082,10 @@
10821082
"title": "Import Existing Resources",
10831083
"path": "state/import"
10841084
},
1085+
{
1086+
"title": "Refactor state",
1087+
"path": "state/refactor"
1088+
},
10851089
{ "title": "Locking", "path": "state/locking" },
10861090
{ "title": "Workspaces", "path": "state/workspaces" },
10871091
{ "title": "Remote State", "path": "state/remote" },

‎website/docs/language/state/refactor.mdx

+251
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
---
2+
page_title: Refactor Terraform state
3+
description: >-
4+
Learn how to move resources between state files and identify dependencies between resources.
5+
---
6+
7+
# Refactor Terraform state
8+
9+
As your Terraform configuration grows in complexity, you might want to reorganize your resources to improve maintainability and performance. Terraform operations complete faster when you split configuration with large state files.
10+
11+
This topic reviews the steps you must take before you split your Terraform state across multiple configurations. First, you must plan how you will group your resources and identify inter-resource dependencies. Then you must choose the best approach to migrate resources from one state file to another. Even small refactors require that you keep these principles in mind. Keep in mind that refactoring your configuration requires updating both the configuration and the corresponding state files.
12+
13+
## Identify opportunities to refactor
14+
15+
When refactoring your Terraform configuration, identify groups of resources that may benefit from a separate cadence of lifecycle management, or logically-related collections of resources that you can capture in a module instead. Refactoring your Terraform configuration should make it easier to maintain.
16+
17+
The following are some common opportunities for refactoring:
18+
19+
- **Long Terraform applies.** Your configuration has grown over time and become cumbersome to manage. Large, monolithic configuration can cause Terraform plan and apply operations to take a long time to complete and cause unintended changes.
20+
- **Changes to management lifecycles.** Managing frequently updated resources separately from infrequently updated resources can help simplify your operations and reduce the blast radius of unintended changes.
21+
- **Changes to resource ownership.** In some organizations, teams may split the responsibility of maintaining different parts of the architecture. In these cases, it is common for teams to refactor the configuration and state to match their area of ownership and scope.
22+
- **Reusable module opportunity.** You have identified a subsection of your configuration that would make for a good module. If you create the same set of resources in multiple configurations, we recommend grouping those resources into a module and reusing it across your organization.
23+
24+
## Plan your refactored state
25+
26+
You must plan how you will group resources before starting refactoring your configuration.
27+
Consider the following resource properties when deciding how to group resources together:
28+
29+
- **Volatility and rate of change:** By exposing your long-living infrastructure to unnecessary volatility, you introduce more opportunities for accidental changes. For example, you may scale the number of compute resources your configuration deploys several times a day, but your networking configuration may stay static for months. By managing your network resources separately, you reduce the risk of making unintended changes to it.
30+
- **Stateful vs stateless:** By managing stateful resources independently of stateless ones, such as separating databases from compute instances, you limit the blast radius of operations that re-provision resources and help protect against accidental data loss.
31+
- **Access and team responsibility:** By splitting your workspaces by team, you limit the responsibility per workspace and allow teams to maintain distinct areas of ownership. This helps ensure that only users familiar with specific parts of your infrastructure make changes to the related configuration.
32+
33+
To learn about different strategies for grouping resources, refer to the [Workspace best practices](/terraform/cloud-docs/workspaces/best-practices) documentation.
34+
35+
## Identify dependencies
36+
37+
As you migrate resources from one state file to another, you must identify dependencies between resources. For example, you may have compute resources that depend on your network resources. If you migrate those network resources to a new state file, your compute resources no longer be able to reference the network.
38+
39+
We recommend using dynamic references rather than hard-coding information about resources in another configuration. Hard-coding information requires you to manually update the configuration whenever the data changes, which can lead to errors in your configuration or your deployed resources.
40+
41+
You can use the following dynamic approaches to reference resources in another state file:
42+
43+
- If your provider supports it, you can use a resource-specific data source to query your cloud provider. For example, you can use the [`aws_vpc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) data source to look up information about a VPC created in another configuration.
44+
- If you use HCP Terraform or Terraform Enterprise, you can use the [`tfe_outputs`](https://registry.terraform.io/providers/hashicorp/tfe/latest/docs/data-sources/outputs) data source to reference outputs from another workspace.
45+
- If you are using another remote backend, or the local backend, you can use the [`terraform_remote_state`](/terraform/language/state/remote-state-data) data source. To access state in an HCP Terraform workspace, you must explicitly specify which other workspaces have access. Refer to [remote state sharing](/terraform/cloud-docs/workspaces/settings#remote-state-sharing) for more information.
46+
47+
48+
You can use the [`terraform graph`](/terraform/cli/commands/graph) command to help visualize the relationships between the resources in your configuration and identify any dependencies.
49+
50+
## Migrate resources
51+
52+
When you refactor your Terraform configuration, we recommend that you recreate stateless resources in a new Terraform configuration if you can do so without incurring downtime or extra cost.
53+
54+
Stateful resources, such as databases and object stores, are more complicated to migrate. In many cases, you cannot delete and recreate them, or it may be complex and expensive to backup and restore the data. In this case, you can migrate your resources by moving them between state files.
55+
56+
There are two common approaches to migrating resources between two Terraform state files:
57+
58+
- [Remove and import](#remove-and-import): Explicitly remove resources from one Terraform state file, then add it to another. We recommend this approach since the `removed` and `import` blocks help keep a record of the configuration history.
59+
- [Move resources directly to a new state file](#move-resources-directly-between-state-files): Use the `terraform state mv` command to move resources into a different state file. This is considered a legacy command, but is still supported in all versions of Terraform version 1.0 and newer.
60+
61+
### Requirements
62+
63+
[Removing and importing resources](#remove-and-import) requires Terraform version 1.7 or newer.
64+
65+
[Moving resources directly to a new state file](#move-resources-directly-between-state-files) using the Terraform CLI requires Terraform version 1.0 or newer.
66+
67+
### Remove and import
68+
69+
You can use Terraform's configuration-driven `removed` and `import` blocks to move resources between state files without destroying them.
70+
71+
The following example is an initial resource configuration that you will move to a new state file.
72+
73+
```hcl
74+
resource "aws_instance" "example" {
75+
instance_type = "t3.micro"
76+
ami = data.aws_ami.example.id
77+
}
78+
```
79+
80+
In your source configuration, complete the following steps to remove the resources:
81+
82+
1. Retrieve your latest Terraform state and save the output to a file as a backup.
83+
84+
```shell-session
85+
$ terraform state pull > terraform.tfstate.backup
86+
```
87+
88+
1. Review your provider's configuration to check which attribute you must use to import your resource type before you remove it. For example, the import [`aws_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#import) resource uses the `id` attribute to import an EC2 instance into your Terraform state.
89+
90+
```shell-session
91+
$ terrafrom state show aws_instance.example
92+
##...
93+
id = "i-07b510cff5f79af00"
94+
##...
95+
```
96+
97+
1. For each resource you want to migrate, replace the `resource` block with a `removed` block to remove the resource from your state without destroying it. These `removed` blocks can exist anywhere in your configuration, but we recommend your organization standardizes where to declare these blocks to help with future maintenance. One common approach is defining the `removed` block in the file that previously contained the corresponding `resource` block.
98+
99+
```diff
100+
- resource "aws_instance" "example" {
101+
- instance_type = "t3.micro"
102+
- ami = data.aws_ami.example.id
103+
- }
104+
105+
+ removed {
106+
+ from = aws_instance.example
107+
+ lifecycle {
108+
+ destroy = false
109+
+ }
110+
+ }
111+
```
112+
113+
1. Run `terraform plan` to ensure that Terraform will not destroy the resources.
114+
115+
```bash
116+
# aws_instance.example will no longer be managed by Terraform, but will not be destroyed
117+
# (destroy = false is set in the configuration)
118+
. resource "aws_instance" "example" {
119+
id = "i-07b510cff5f79af00"
120+
##...
121+
```
122+
123+
1. Run `terraform apply` to remove the resource from state.
124+
125+
Then, in your destination configuration, complete the following steps to import the resources:
126+
127+
1. Add the resources to the configuration you are migrating the resources to.
128+
1. For each added resource, add an `import` block to add the resource to your state without recreating it. These `import` blocks can exist anywhere in your configuration, but we recommend you standardize on location in your organization to help with future maintenance. One common practice is to define the `import` block in the same file you add the `resource` block to.
129+
130+
```hcl
131+
resource "aws_instance" "example" {
132+
instance_type = "t3.micro"
133+
ami = data.aws_ami.example.id
134+
}
135+
136+
import {
137+
id = "i-07b510cff5f79af00"
138+
to = aws_instance.example
139+
}
140+
```
141+
142+
1. Run `terraform plan` to ensure that Terraform will properly import the resources.
143+
144+
```bash
145+
# aws_instance.example will be imported
146+
resource "aws_instance" "example" {
147+
```
148+
149+
1. Run `terraform apply` to complete the import.
150+
151+
```shell-session
152+
$ terraform apply
153+
154+
##...
155+
156+
Plan: 1 to import, 0 to add, 0 to change, 0 to destroy.
157+
158+
Do you want to perform these actions?
159+
Terraform will perform the actions described above.
160+
Only 'yes' will be accepted to approve.
161+
162+
Enter a value: yes
163+
164+
aws_instance.example: Importing... [id=i-12345678901234567]
165+
aws_instance.example: Import complete [id=i-12345678901234567]
166+
167+
Apply complete! Resources: 1 imported, 0 added, 0 changed, 0 destroyed.
168+
```
169+
170+
After you complete the migration you can optionally remove the `removed` and `import` blocks, or choose to keep them as a record of the resource's lifecycle.
171+
172+
Refer to the following documentation to learn more about the `removed` and `import` blocks:
173+
174+
- [Removing resources](/terraform/language/resources/syntax#removing-resources)
175+
- [Import block reference documentation](/terraform/language/import)
176+
177+
### Move resources directly to a new state file
178+
179+
You can also use the `terraform state mv` command to move resources directly between state files.
180+
181+
<Warning>
182+
183+
The `-state` and `-state-out` flags are legacy options that Terraform maintains for backwards compatibility.
184+
185+
This approach also uses the `terraform state pull` and `terraform state push` commands. While Terraform performs safety checks when you manually update remote state, this method does have some risk of corrupting your remote state.
186+
187+
We recommend that you use the `removed` and `import` blocks documented above for any new migrations.
188+
189+
</Warning>
190+
191+
#### Prepare the state files
192+
193+
If you are using the local state backend, you can move resources directly between two state files. If you are using a remote state backend, such as HCP Terraform or AWS S3, you must first download the state files for the source and destination workspaces.
194+
195+
To download the state files, complete the following steps:
196+
197+
1. In the directory with your source configuration, use the `terraform state pull` command to save the state file locally.
198+
199+
```shell-session
200+
$ terraform state pull > source.tfstate
201+
```
202+
203+
1. In the directory with your destination configuration, use the `terraform state pull` command to save the state file locally.
204+
205+
```shell-session
206+
$ terraform state pull > destination.tfstate
207+
```
208+
209+
#### Move the resources
210+
211+
Next, use the `terraform state mv` command to move the resource from the source state to the destination state.
212+
213+
```shell-session
214+
$ terraform state mv -state source/source.tfstate -state-out destination/destination.tfstate aws_instance.example aws_instance.example
215+
216+
Move "aws_instance.example" to "aws_instance.example"
217+
Successfully moved 1 object(s).
218+
```
219+
220+
The `terraform state mv` command uses the `-state` flag to point to the source state file, and the `-state-out` flag to point to the destination state file. The final two arguments tell Terraform which resource to move out from the source state, and the resource to move it to in the destination state.
221+
222+
Repeat this step for every resource you want to migrate.
223+
224+
#### Push the state files
225+
226+
If you are using a remote backend, you must push the state files to the backend with the `terraform state push` command.
227+
228+
1. In the directory with your source configuration, use the `terraform state push` command to update the remote state.
229+
230+
```shell-session
231+
$ terraform state push source.tfstate
232+
```
233+
234+
1. In the directory with your destination configuration, use the `terraform state push` command to update the remote state.
235+
236+
```shell-session
237+
$ terraform state push destination.tfstate
238+
```
239+
240+
#### Update your configuration and verify the migration
241+
242+
Next, update your configuration to match your state, and then verify that your changes are correct.
243+
244+
1. Remove the resources that you migrated from your source configuration.
245+
1. Run `terraform plan` on your source configuration and ensure that Terraform will not make any changes to your infrastructure.
246+
1. Add the resources that you migrated to your destination configuration.
247+
1. Run `terraform plan` on your destination configuration and ensure that Terraform will not make any changes to your infrastructure.
248+
1. Create pull requests for the repositories that store your source and destination configurations. If your code review process creates a speculative plan for changes to your Terraform configuration, review the results and ensure that Terraform will not make any changes to your infrastructure.
249+
1. Merge the pull requests.
250+
251+
Refer to the [`terraform state mv` command reference documentation](/terraform/cli/commands/state/mv) to learn more.

0 commit comments

Comments
 (0)