My pulumi stack was created with an older plugin version that I can't install on M1 mac

Question

I have a Pulumi project which I haven't touched (deployed etc) for a while.

Now I need to make some changes but I get the "403 HTTP error fetching plugin" described here

The description in the docs makes sense: I have bought a new Apple M1 laptop since last time I worked on it, my stack was built with the digitalocean provider v3.1.1 but I can't install that version on my new laptop.

The docs say that if you have access to an Intel system, set up your project there, update the providers and run pulumi up. The implication is that you can install both old and latest versions of the provider plugin, pulumi up will update the stack using the latest version. After that I should be able to manage the stack from my new laptop using the latest provider version.

I asked on Pulumi Slack about this and confirmed the above.

But it doesn't seem to work for me.

First I started a Docker container with --platform=linux/amd64 to emulate Intel. Inside that I checked out my project, installed pulumi and the old and new provider versions.

In my docker container pulumi plugin ls shows:

NAME          KIND      VERSION  SIZE   INSTALLED  LAST USED
cloudflare    resource  4.7.0    38 MB  n/a        1 day ago
cloudflare    resource  2.8.0    46 MB  n/a        1 day ago
digitalocean  resource  4.14.0   42 MB  n/a        1 day ago
digitalocean  resource  3.1.1    45 MB  n/a        1 day ago

From there I successfully ran pulumi up.

However if I go back to my local shell and try pulumi preview I get:

error: could not load plugin for digitalocean provider 'urn:pulumi:staging::myproject::pulumi:providers:digitalocean::default': no resource plugin 'pulumi-resource-digitalocean' found in the workspace at version v3.1.1 or on your $PATH, install the plugin using pulumi plugin install resource digitalocean v3.1.1

So despite updating the stack Pulumi is still trying to use the old provider version that I can't install.

How do I get around this?

Answer 1

You also need to update your pulumi program's dependencies.

Simply installing the new plugin isn't enough.

If you do pulumi stack export you'll see a JSON file with all your resources. Those resources have a provider attached to them with a specific version of the plugin. As an example:

{
  "version": 3,
  "deployment": {
    "manifest": {
      "time": "2022-06-23T12:03:30.071863-07:00",
      "magic": "eccb7d9cc1cab43d7465783c52b0648063d5e7228dd3bb2fc7600583a8bca5d5",
      "version": "v3.34.1"
    },
    "secrets_providers": {
      "type": "service",
      "state": {
        "url": "https://api.pulumi.com",
        "owner": "jaxxstorm",
        "project": "s3_event_bridge",
        "stack": "dev"
      }
    },
    "resources": [
      {
        "urn": "urn:pulumi:dev::s3_event_bridge::pulumi:pulumi:Stack::s3_event_bridge-dev",
        "custom": false,
        "type": "pulumi:pulumi:Stack",
        "outputs": {
          "bucketName": "test001-c86ab36"
        },
        "sequenceNumber": 1
      },
      {
        "urn": "urn:pulumi:dev::s3_event_bridge::pulumi:providers:aws::default_5_9_1",
        "custom": true,
        "id": "484b75ed-d5cd-4ee1-96d1-b3f641236ab6",
        "type": "pulumi:providers:aws",
        "inputs": {
          "region": "us-west-2",
          "version": "5.9.1"
        },
        "outputs": {
          "region": "us-west-2",
          "version": "5.9.1"
        },
        "sequenceNumber": 1
      },
      {
        "urn": "urn:pulumi:dev::s3_event_bridge::aws:s3/bucket:Bucket::test001",
        "custom": true,
        "id": "test001-c86ab36",
        "type": "aws:s3/bucket:Bucket",
        "inputs": {
          "__defaults": [
            "bucket",
            "forceDestroy"
          ],
          "acl": "private",
          "bucket": "test001-c86ab36",
          "forceDestroy": false,
          "tags": {
            "Environment": "Dev",
            "Name": "My bucket",
            "__defaults": []
          }
        },
        "outputs": {
          "accelerationStatus": "",
          "acl": "private",
          "arn": "arn:aws:s3:::test001-c86ab36",
          "bucket": "test001-c86ab36",
          "bucketDomainName": "test001-c86ab36.s3.amazonaws.com",
          "bucketRegionalDomainName": "test001-c86ab36.s3.us-west-2.amazonaws.com",
          "corsRules": [],
          "forceDestroy": false,
          "grants": [],
          "hostedZoneId": "Z3BJ6K6RIION7M",
          "id": "test001-c86ab36",
          "lifecycleRules": [],
          "loggings": [],
          "objectLockConfiguration": null,
          "region": "us-west-2",
          "replicationConfiguration": null,
          "requestPayer": "BucketOwner",
          "serverSideEncryptionConfiguration": null,
          "tags": {
            "Environment": "Dev",
            "Name": "My bucket"
          },
          "tagsAll": {
            "Environment": "Dev",
            "Name": "My bucket"
          },
          "versioning": {
            "enabled": false,
            "mfaDelete": false
          },
          "website": null
        },
        "parent": "urn:pulumi:dev::s3_event_bridge::pulumi:pulumi:Stack::s3_event_bridge-dev",
        "provider": "urn:pulumi:dev::s3_event_bridge::pulumi:providers:aws::default_5_9_1::484b75ed-d5cd-4ee1-96d1-b3f641236ab6",
        "propertyDependencies": {
          "acl": null,
          "tags": null
        },
        "sequenceNumber": 1
      },
      {
        "urn": "urn:pulumi:dev::s3_event_bridge::aws:s3/bucketNotification:BucketNotification::bucketNotification",
        "custom": true,
        "id": "test001-c86ab36",
        "type": "aws:s3/bucketNotification:BucketNotification",
        "inputs": {
          "__defaults": [],
          "bucket": "test001-c86ab36",
          "eventbridge": false
        },
        "outputs": {
          "bucket": "test001-c86ab36",
          "eventbridge": false,
          "id": "test001-c86ab36",
          "lambdaFunctions": [],
          "queues": [],
          "topics": []
        },
        "parent": "urn:pulumi:dev::s3_event_bridge::pulumi:pulumi:Stack::s3_event_bridge-dev",
        "dependencies": [
          "urn:pulumi:dev::s3_event_bridge::aws:s3/bucket:Bucket::test001"
        ],
        "provider": "urn:pulumi:dev::s3_event_bridge::pulumi:providers:aws::default_5_9_1::484b75ed-d5cd-4ee1-96d1-b3f641236ab6",
        "propertyDependencies": {
          "bucket": [
            "urn:pulumi:dev::s3_event_bridge::aws:s3/bucket:Bucket::test001"
          ],
          "eventbridge": null
        },
        "sequenceNumber": 1
      }
    ]
  }
}

If you look at my BucketNotification resource, you can see a provider field which has a version in it for the AWS provider I've used:

"provider": "urn:pulumi:dev::s3_event_bridge::pulumi:providers:aws::default_5_9_1::484b75ed-d5cd-4ee1-96d1-b3f641236ab6"

Which in this case is 5.9.1

So, in order to fix this problem, you need to update your resources to have a new version of the provider.

To do this, it depends on the language you're using with Pulumi.

If you're using TypeScript or JavaScript, update your @pulumi/digitalocean dependency in your package.json

If you're using Python, update pulumi_digitalocean in your requirements.txt

Make sure you update with your package manager with npm update or pip3 upgrade

The same applies if you're using DotNet, Go and Java.

then you need to run a successful pulumi up. Pulumi will update the provider version associated with each resource as you saw above, you can verify this by again doing a pulumi stack export

From here, you should be able to successfully use your M1 mac without the legacy plugins.