AWS project structure using terraform

Question

I am working on a Python and AWS course on coursera. However instead of creating s3 , api gateway and others using boto3 I am using terraform. Till now everything is going fine however I am facing a issue.Below is my lambda directory structure

Every lambda has a different directory structure and I have to cd in each directory to apply changes using terraform apply.

Example

Below is my lambda code in terraform for one of the lambda function.<<validate.tf>>

provider "aws" {
  region = "us-east-2"
}

terraform {
  required_version = "> 0.14"
  required_providers {
    aws = "~> 3.0"
  }
  backend "s3" {
    bucket = "nyeisterraformstatedata2"
    key = "api_gateway/lambda_function/terraform_api_gateway_lambda_validate.tfstate"
    region = "us-east-2"

    dynamodb_table = "terraform-up-and-running-locks-2"
    encrypt = true
  }
}

data "archive_file" "zip_file" {
  type        = "zip"
  source_dir  = "${path.module}/lambda_dependency_and_function"
  output_path = "${path.module}/lambda_dependency_and_function.zip"
}

resource "aws_lambda_function" "get_average_rating_lambda" {
  filename      = "lambda_dependency_and_function.zip"
  function_name = "validate"
  role          = data.aws_iam_role.lambda_role_name.arn
  handler       = "validate.lambda_handler"

  # The filebase64sha256() function is available in Terraform 0.11.12 and later
  # For Terraform 0.11.11 and earlier, use the base64sha256() function and the file() function:
  # source_code_hash = "${base64sha256(file("lambda_function_payload.zip"))}"
  source_code_hash = filebase64sha256(data.archive_file.zip_file.output_path)

  runtime = "python3.8"

  depends_on = [data.archive_file.zip_file]
}

<<variable.tf>>

data "aws_iam_role" "lambda_role_name" {
  name = "common_lambda_role_s3_api_gateway_2"
}

Based on the comment below I created a main.tf with following code

provider "aws" {
  region = "us-east-2"
}


module "test" {
  source = "../validate"

}

but I am trying to import using import statement its giving me an error and I am not able to figure out how to solve it

terraform import module.test.aws_lambda_function.test1 get_average_rating_lambda

Warning: Backend configuration ignored
│ 
│   on ../validate/validate.tf line 10, in terraform:
│   10:   backend "s3" {
│ 
│ Any selected backend applies to the entire configuration, so Terraform expects provider configurations only in the root module.
│ 
│ This is a warning rather than an error because it's sometimes convenient to temporarily call a root module as a child module for testing purposes, but this backend configuration block will have no
│ effect.
╵

Error: resource address "module.test.aws_lambda_function.test1" does not exist in the configuration.

Before importing this resource, please create its configuration in module.test. For example:

resource "aws_lambda_function" "test1" {
  # (resource arguments)
}

So my question is there a way for terraform to tell which all files have change and apply them in one go rather than one by one.Since I am new to terraform too so if anyone think that this is the wrong way to structing the project please do let me know.Thank you

Answer 1

What you could do is create a new directory with a main.tf file and make it a project that contains your whole cloud environment. Each of these existing folders could be imported as a module. If each of your folders is a running terraform project, it can already be imported as a module without changing it.

You would then use the terraform import command to import each of the resources, in a fashion similar to terraform import module.aws_lambda_function my_lambda_id for each lambda and any other managed resources.

Then, instead of a state file for each lambda, you would have a state file for your whole environment. From there, terraform is smart enough to detect the individual changes and update accordingly.