Terraform remote-exec provisioner commands fail to start my Flask app through AWS, but work when run manually over SSH?

Question

I'm setting up a Terraform script to deploy Flask applications on AWS instances behind a load balancer. I've configured everything correctly, including security groups, subnets, and instances. However, when I use Terraform's remote-exec provisioner to install Python packages and start the Flask app, it doesn't seem to work correctly—the load balancer gives 504 errors.

Strangely, if I manually SSH into the instances and run the installation and startup commands (sudo apt update, sudo apt-get install -y python3-pip python3-venv, etc.), everything works fine—the app starts, and the load balancer link becomes functional.

I've ensured that the Python environment and Flask are installed correctly, and paths are set up as expected. What could be causing the remote-exec provisioner to fail in starting the Flask app properly?

Here's what i tried: main.tf:

`variable "cidr" {
  default = "10.0.0.0/16"
}
resource "aws_vpc" "myvpc" {
  cidr_block = var.cidr
}
resource "aws_subnet" "sub1" {

    vpc_id = aws_vpc.myvpc.id
    cidr_block = "10.0.0.0/24"
    availability_zone = "us-east-2a"
    map_public_ip_on_launch = true
}
resource "aws_subnet" "sub2" {

    vpc_id = aws_vpc.myvpc.id
    cidr_block = "10.0.1.0/24"
    availability_zone = "us-east-2b"
    map_public_ip_on_launch = true
}
resource "aws_internet_gateway" "gate" {
  vpc_id = aws_vpc.myvpc.id
}
resource "aws_route_table" "rt" {
  vpc_id = aws_vpc.myvpc.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.gate.id
  } 
}
resource "aws_route_table_association" "rta1" {
  subnet_id = aws_subnet.sub1.id
  route_table_id = aws_route_table.rt.id 
}
resource "aws_route_table_association" "rta2" {
  subnet_id = aws_subnet.sub2.id
  route_table_id = aws_route_table.rt.id 
}
resource "aws_security_group" "sg1" {
  name_prefix = "web-sg-1"
  vpc_id = aws_vpc.myvpc.id
  ingress {
    description      = "HTTP from VPC"
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    
  }
  ingress {
    description      = "SSH"
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
   
  }

  tags = {
    Name = "sg1"
  }
}  
resource "aws_key_pair" "example" {
  key_name   = "example-key"
  public_key = file("~/.ssh/id_rsa.pub")
}

resource "aws_instance" "server1" {
  ami = "ami-09040d770ffe2224f"
  instance_type = "t2.micro"
  key_name = aws_key_pair.example.key_name
  vpc_security_group_ids = [aws_security_group.sg1.id]
  subnet_id = aws_subnet.sub1.id
  connection {
    type        = "ssh"
    user        = "ubuntu"
    private_key = file("~/.ssh/id_rsa") 
    host        = self.public_ip
  }

  # File provisioner to copy a file from local to the remote EC2 instance
  provisioner "file" {
    source      = "app.py"  
    destination = "/home/ubuntu/app.py" 
  }

  provisioner "remote-exec" {
    inline = [
      "echo 'Hello from the remote instance'",
      "sudo apt update -y",
      "sudo apt-get install -y python3-pip python3-venv",
      "python3 -m venv /home/ubuntu/venv",
      ". /home/ubuntu/venv/bin/activate",
      "/home/ubuntu/venv/bin/pip install flask",
      "nohup /home/ubuntu/venv/bin/python /home/ubuntu/app.py &"
    ]
  }
}

resource "aws_instance" "server2" {
  ami = "ami-09040d770ffe2224f"
  instance_type = "t2.micro"
  key_name = aws_key_pair.example.key_name
  vpc_security_group_ids = [aws_security_group.sg1.id]
  subnet_id = aws_subnet.sub2.id
  connection {
    type        = "ssh"
    user        = "ubuntu"
    private_key = file("~/.ssh/id_rsa") 
    host        = self.public_ip
  }

  # File provisioner to copy a file from local to the remote EC2 instance
  provisioner "file" {
    source      = "app.py"  
    destination = "/home/ubuntu/app.py" 
  }

  provisioner "remote-exec" {
    inline = [
      "echo 'Hello from the remote instance'",
      "sudo apt update -y",
      "sudo apt-get install -y python3-pip python3-venv",
      "python3 -m venv /home/ubuntu/venv",
      ". /home/ubuntu/venv/bin/activate",
      "/home/ubuntu/venv/bin/pip install flask",
      "nohup /home/ubuntu/venv/bin/python /home/ubuntu/app.py &"
    ]
  }
}
#creating application load balancer
resource "aws_lb" "myalb" {
  name = "myalb"
  internal = false
  load_balancer_type = "application"

  security_groups = [aws_security_group.sg1.id]
  subnets = [aws_subnet.sub1.id, aws_subnet.sub2.id]

  tags = {
    Name= "web"
  }
}

resource "aws_lb_target_group" "tg" {
  name = "myTG"
  port = 80
  protocol = "HTTP"
  vpc_id = aws_vpc.myvpc.id

  health_check {
    path = "/"
    port = "traffic-port"
  }
  
}
resource "aws_lb_target_group_attachment" "attach1" {
  target_group_arn = aws_lb_target_group.tg.arn
  target_id = aws_instance.server1.id
  port = 80 
}

resource "aws_lb_target_group_attachment" "attach2" {
  target_group_arn = aws_lb_target_group.tg.arn
  target_id = aws_instance.server2.id
  port = 80 
}
resource "aws_lb_listener" "listener" {
  load_balancer_arn = aws_lb.myalb.arn
  port = 80
  protocol = "HTTP"

  default_action {
    target_group_arn = aws_lb_target_group.tg.arn
    type = "forward"
  }
  
}
output "loadbalancerdns" {
  value = aws_lb.myalb.dns_name
}`

app.py

from flask import Flask
from datetime import datetime, timedelta
import random

app = Flask(__name__)

# Store the time when the server starts
start_time = datetime.now()

quotes = [
    "Believe you can and you're halfway there.",
    "The only way to do great work is to love what you do.",
    "Success is not the key to happiness. Happiness is the key to success.",
    "Your time is limited, so don't waste it living someone else's life.",
    "The best way to predict the future is to create it."
]

@app.route("/")
def home():
    current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    random_quote = random.choice(quotes)
    current_time_obj = datetime.now()
    uptime_duration = current_time_obj - start_time
    uptime_str = str(timedelta(seconds=int(uptime_duration.total_seconds())))
    return (f"<h1>Welcome to my Flask app!</h1>"
            f"<p>Current time is: {current_time}</p>"
            f"<p>Motivational Quote: {random_quote}</p>"
            f"<p>Server Uptime: {uptime_str}</p>")

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=80)`

Terraform's remote-exec provisioner should install Python packages and start the Flask app automatically instead of manually executing commands through SSH