CODEWITHSUNDEEP
amazon-s3terraform
Tom Klino
Tom Klino

Reputation: 2514

Why is terraform ignoring a manually created s3 bucket?

I have found that I have a bucket in my aws account that was not created by terraform.

I wanted to make sure it is stated in my terraform repository, but before I added it, I ran terraform plan and got the following response:

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.

Which is odd because when I run aws s3 ls I see the bucket:

#not real names from obvious reasons
2019-07-15 13:33:16 bucket-from-terrform
2019-07-15 13:19:26 terraform-state-bucket
2019-07-23 17:47:26 bucket-created-manually

More so, when I add a resource with a name matching the previously ignored bucket, terraform tries to add it (even though it already exists)

What is happening? What could be the reason that terraform is ignoring that bucket?

Upvotes: 1

Views: 3816

Answers (3)

Duru Cynthia Udoka
Duru Cynthia Udoka

Reputation: 807

Just adding this piece, in terms of resource or name Collision, If you add a resource in your Terraform code with the same name as the manually created bucket and try to run terraform apply, Terraform will try to create it because it doesn’t know the resource exists (since it's not in the state).

Best to import resource created outside of terraform with terraform import

Upvotes: 0

Martin Atkins
Martin Atkins

Reputation: 74694

Terraform only manages objects that it created or that were explicitly imported into Terraform. This is for a few different reasons:

  • So that you can introduce Terraform into an existing system gradually. You might pilot it by either using it for some greenfield infrastructure or by importing a small part of your infrastructure in order to get experience before going "all-in".

  • So that your system can be decomposed into many separate configurations. Having the entire infrastructure in a single Terraform configuration is not recommended unless your system is relatively small/simple, because it means that any change risks impacting the entire system. Instead, it's common (and recommended) to use several separate configurations, commonly split by rate of change. For example, virtual network topology tends to change infrequently in many systems, whereas the size of a compute cluster may change more frequently in response to changing requirements.

  • So that you can migrate away from Terraform in future without it trying to manage or destroy infrastructure that is now managed by other tools or processes.

If you have an existing object that was created outside of Terraform and you now want to manage it with Terraform, you'll need to import it into Terraform to create the association between the resource address in Terraform and the id of the object in the remote system.

For your S3 example, the import process might go like this:

  • Write a resource "aws_s3_bucket" "example" block in your configuration using the bucket name bucket-created-manually and any other settings that apply to the existing bucket.
  • At the command line, run terraform import 'aws_s3_bucket.example' 'bucket-created-manually'.
  • Run terraform plan to verify that the configuration matches the remote object that was imported. If not, either adjust the configuration to match the remote object or run terraform apply to update the remote object to match the configuration, depending on which one you consider "more correct" in your situation.

Upvotes: 3

Ngenator
Ngenator

Reputation: 11269

Terraform is declarative, you define what resources you want and it will create and manage them for you. It can only manage resources it knows about within the state. To allow terraform to manage resources created manually, you can import them into the state.

For s3 buckets, you'd import via the bucket name https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#import

So if your terraform looks like

resource "aws_s3_bucket" "my-bucket" {
  bucket = "bucket-created-manually"
  acl    = "private"
}

you'd use the command

terraform import aws_s3_bucket.my-bucket bucket-created-manually

and then the next time you run a plan, terraform will know about it and be able to manage it.

Upvotes: 6

Related Questions