Why does the folder check in Terraform contradict my Python output?

Question

I am a beginner when it comes to Terraform providers which may explain why I have the following problem on my Ubuntu 24.04 machine. First, here is the main.tf file:

terraform {
  required_providers {
    flowtrends = {
      source  = "local/flowtrends"
      version = "1.0.0"
    }
  }
}
 
provider "flowtrends" {}

I executed this command-line statement:

python3 -c "import os; print('Folder exists' if os.path.isdir(os.path.expanduser('~/.terraform.d/plugins')) else 'Folder does not exist')"

Output: "Folder exists"

In addition, I also executed this:

python3 -c "import os; print('Folder exists' if os.path.isdir(os.path.expanduser('~/.terraform.d/plugins2')) else 'Folder does not exist')"

Output: "Folder does not exist"

Apparently, the Python statement checks the path correctly. If a folder really does not exist, this is confirmed. If it exists, it is also confirmed.

However, when typing

terraform init

I got this output:

"... ~/.terraform.d/plugins: no such file or directory..."

So the output of Terraform contradicts the python output. I have no idea why. I am sure the folder exists. I checked that manually and programmatically (python). I do not understand why Terraform denies this. It is blocking me from working with Terraform. Because logically, when a folder exists, it exists. Terraform should use that folder when needed instead of denying the reality. I cannot create a folder that already exists so I cannot fix it.

How can I resolve this discrepancy?

My work is also shown in the attached screenshot:

Update

The .terraformrc file in ~:

provider_installation {
  filesystem_mirror {
    path = "~/.terraform.d/plugins"
  }
}

The version output of Terraform:

Terraform v1.10.3
on linux_amd64

It would be logical to just use absolute path but unfortunately, Terraform gets the idea then to search in the registry instead of locally. Maybe I should find a way to FORCE Terraform to only search locally? How?

Answer 1

The convention of using ~ to represent your home directory is a Unix shell convention rather than a convention of the real virtual filesystem, and so software other than Unix shells will often not support it. That is true for Terraform: it expects you to provide a complete, literal path to your filesystem mirror directory.

For example, if your home directory is /home/daan then you would configure it like this:

provider_installation {
  filesystem_mirror {
    path = "/home/daan/.terraform.d/plugins"
  }
}

---

Unless you are the owner of the "local" organization on GitHub, you should not use that for the source address of your own local provider. Instead, you should either use a GitHub organization you own or you should use a separate hostname you own as your provider source address namespace. Namespaces under registry.terraform.io (the default provider registry) are always mapped to GitHub organization or user names.

For example, if you owned the domain example.com then you could decide that by convention your local Terraform providers belong to the hostname terraform.example.com, and thus specify a source address like this:

terraform {
  required_providers {
    flowtrends = {
      source  = "terraform.example.com/example/flowtrends"
      version = "1.0.0"
    }
  }
}

You can configure Terraform CLI to use your local directory only for plugins that belong to your hostname, which therefore avoids any need to deploy a real Terraform provider registry at that hostname:

provider_installation {
  filesystem_mirror {
    path    = "/home/daan/.terraform.d/plugins"
    include = ["terraform.example.com/*/*"]
  }
  direct {
    exclude = ["terraform.example.com/*/*"]
  }
}

That CLI configuration tells Terraform CLI that it should expect to find any terraform.example.com-sourced providers in /home/daan/.terraform.d/plugins, but all other plugins should be installed from their provider registry as normal.

For terraform.example.com/example/flowtrends you would then create the following directory path to contain your provider's plugin executable and any other files:

/home/daan/.terraform.d/plugins/terraform.example.com/example/flowtrends/1.0.0/linux_amd64

That directory should contain at least an executable file named terraform-provider-flowtrends in order for Terraform to consider it as a valid provider package directory.