How can I make the Docker provider in Terraform wait for an address to become available before attempting to connect to it?

Question

I have the following resource in Terraform:

provider "docker" {
    host = "tcp://${digitalocean_droplet.docker_server.ipv4_address}:2376/"
}

This relies on the value ipv4_address to be known before it can connect to the docker machine. This value is not known until another resource is provisioned:

resource "digitalocean_droplet" "docker_server" {
    image = "docker-18-04"
    name = "docker_server"
    region = "nyc2"
    size = "512mb"
    private_networking = true
    ssh_keys = [
      var.ssh_fingerprint
    ]

    connection {
        user = "root"
        type = "ssh"
        private_key = file(var.pvt_key)
        timeout = "2m"
    }
}

When I run terraform plan, I get the following error:

Error: Error initializing Docker client: unable to parse docker host ``

on docker.tf line 1, in provider "docker": 1: provider "docker" {

It appears that ipv4_address is empty because the docker plugin is trying to connect to the docker machine before it is provisioned. How do I tell it to wait for the machine to be provisioned before trying to connect to it?

---

One thing I tried:

provider "docker" {
    host = "tcp://${digitalocean_droplet.docker_server.ipv4_address}:2376/"
    depends_on = [
        digitalocean_droplet.docker_server.ipv4_address,
    ]
}

When I do that, I get this error:

Error: Reserved argument name in provider block

on docker.tf line 4, in provider "docker": 4: depends_on = [

The provider argument name "depends_on" is reserved for use by Terraform in a future version.

But reading more into depends_on, I don't think that's the solution anyway.

Answer 1

Unfortunately a provider block doesn't support expressions referring to a resource attribute.

This limitation is explained in the provider configuration documentation:

The configuration arguments defined by the provider may be assigned using expressions, which can for example allow them to be parameterized by input variables.

However, since provider configurations must be evaluated in order to perform any resource type action, provider configurations may refer only to values that are known before the configuration is applied.

In particular, avoid referring to attributes exported by other resources unless their values are specified directly in the configuration.

For example, this would work (but not solve your problem):

variable "docker_host" {
  type = string
}

provider "docker" {
  host = "tcp://${var.docker_host}:2376/"
}

But there is a way out.

The solution is made of two steps:

1. split your terraform configuration in two parts (each must reside in its own directory) where the one with the docker provider depends on the one that deploys the droplet. Note that this means that you will have to issue terraform commands separately (you need to apply twice).
2. Establish a unidirectional, read-only "connection" between the two states using a feature called remote state:

Retrieves state data from a Terraform backend. This allows you to use the root-level outputs of one or more Terraform configurations as input data for another configuration.

In you are not already using a "real" remote backend such as S3 + DynamoDB, you can still experiment easily using the local backend as follows.

Directory layout:

├── docker                   <== this performs docker operation
│   ├── main.tf
│   └── terraform.tfstate
└── server                   <== this deploys the droplet
    ├── main.tf
    └── terraform.tfstate

The snippets below are using AWS, but it is trivial to adapt to DO.

File server/main.tf contains something similar to

resource "aws_instance" "server" {     <= equivalent to the Droplet
  ...
}

output "ipv4_address" {
  value = aws_instance.server.public_ip
}

File docker/main.tf contains something similar to

data "terraform_remote_state" "docker_server" {
  backend = "local"

  config = {
    path = "${path.module}/../server/terraform.tfstate"
  }
}

provider "docker" {
  host = "tcp://${data.terraform_remote_state.docker_server.outputs.ipv4_address}:2376/"
}

Finally:

cd server
terraform apply
cd ../docker
terraform apply

Remember: you have to perform also separate terraform destroy, in LIFO order: first destroy docker, then destroy server.