Refactoring Terraform state into separate files in GCP GCS
Recently I needed to refactor some Terraform configuration, which had grown into quite a large single state file, into smaller separate directories with their own state files.
This is a little bit tricky but the Terraform CLI tool helps a lot.
The first thing is to get the existing single state file JSON:
terraform state pull | tee terraform.tfstate
This should work for any storage backend.
Alternatively you can use gsutil to copy the Terraform state file from a GCS
bucket:
gsutil cp gs://your-bucket/your-prefix/terraform.tfstate ./terraform.tfstate
Once you’ve got a local copy of the large single Terraform state file, you can start working with the Terraform CLI tool to extract resources out into separate smaller files.
Extracting a single resource looks like this:
terraform state mv \
-state=terraform.tfstate \
-state-out=extracted.tfstate \
'module.entity-smart-commissioning-status["prod"]'{,}
The {,} on the end is a little Bash trick to repeat that argument, e.g.
foobar{,} expands to foobar foobar. This keeps the resource name the same
when moving it into the new state file.
You can run that command for each resource that needs extracting, and for several different extracted Terraform state files if that applies in your case.
Once that is done, you can push the trimmed down original state file back to the storage backend with:
terraform state push terraform.tfstate
Again, there is also the alternative of doing this via gsutil to upload the
file to a GCP bucket without going via Terraform:
gsutil cp ./terraform.tfstate gs://your-bucket/your-prefix/terraform.tfstate
You can use either of the commands to push the new extracted state files to the correct locations from their respective Terraform directories.