CODEWITHSUNDEEP
terraformterraform0.12+
Alex Cohen
Alex Cohen

Reputation: 6206

Terraform: How to automatically resolve removed provider issues from old state files?

I am working on upgrading templates from terraform 0.12.31 to 0.13.7, we need to ensure that we have an automatic system for dealing with deployments that were created under the older version.

An issue I am working through is that I removed the use of all null providers in the move. When I attempt to apply or plan on a state file created on 0.12 when using terraform version 0.13 I recieve the following error:

$ terraform plan --var-file MY_VAR_FILE.json 
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

Error: Provider configuration not present

To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost
its original provider configuration at
provider["registry.terraform.io/-/null"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost,
after which you can remove the provider configuration again.


Error: Provider configuration not present

To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master
its original provider configuration at
provider["registry.terraform.io/-/null"] is required, but it has been removed.
This occurs when a provider configuration is removed while objects created by
that provider still exist in the state. Re-add the provider configuration to
destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master,
after which you can remove the provider configuration again.


Error: Provider configuration not present

To work with
module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config its
original provider configuration at provider["registry.terraform.io/-/null"] is
required, but it has been removed. This occurs when a provider configuration
is removed while objects created by that provider still exist in the state.
Re-add the provider configuration to destroy
module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config, after
which you can remove the provider configuration again.

My manual solution is to run terraform state rm on all the modules listed:

terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config
terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost
terraform state rm module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master

I would like to know how to do this automatically to enable a script to make these changes.

Is there some kind of terraform command I can use to list out these removed modules without the extra test so I can loop through runs of terraform state rm to remove them from the state file?

Or is there some kind of terraform command that can automatically do this in a generic manner like terraform state rm -all-not-present?

Upvotes: 6

Views: 19222

Answers (4)

Jackie
Jackie

Reputation: 25969

for my case, i manually added back a almost blank provider configuration sorted this out:

provider "aws" {
  region = "us-east-1"
}

Upvotes: 0

jtk
jtk

Reputation: 21

You could do something like

for name in $(tf state list | grep ".${PROVIDER_NAME}."); do terraform state rm $name; done

with ${PROVIDER_NAME} null in your case.

This will also remove the resources belonging to the provider name from the remote terraform state.

WARNING: While I added dots . to the start and end of the grepped for string, to only grep for providers and not the name of some resource, I am not 100% sure that this might not delete other resources that have, e.g .null. in their name.

Upvotes: 2

Alex Cohen
Alex Cohen

Reputation: 6206

This gives me a list I can iterate through using terraform state rm $MODULE_NAME:

$ terraform state list | grep 'null_data_source' 
module.gcp_volt_site.module.ce_config.data.null_data_source.cloud_init_master
module.gcp_volt_site.module.ce_config.data.null_data_source.hosts_localhost
module.gcp_volt_site.module.ce_config.data.null_data_source.vpm_config

Upvotes: 5

Jake Nelson
Jake Nelson

Reputation: 2043

There's a few possibilities. Without the source code of the module it's difficult to say so providing that might be helpful.

A couple of suggestions

Cleaning Cache

Remove the .terraform directory (normally in the directory you're running the init, plan, and apply from). The an older version of the module could be cached that still contains the null references.

State Refresh

Using Terraform Refresh you should be able to scan infra and bring state into alignment.

Can be dangerous, not recommended by Hashicorp.

Manual removals

The state rm command like you've suggested could help here and is fairly safe. You have an option of a --dry-run and you point to resources specifically Using terraform state rm like you've suggested to manually remove those resources within the modules in state. Again, you want to check that the module reference isn't pointing to an old version of the module or caching an old one or they will just be recreated.

No there's not a rm --all-missing but if you know it's all the null data sources that will be missing you could use terraform state ls to list all those resources, then iterate over each of them removing them in a loop.

Upvotes: 4

Related Questions