CODEWITHSUNDEEP
google-cloud-platformterraformgoogle-cloud-pubsubterraform-provider-gcpgoogle-cloud-pubsublite
Sanjay M. P.
Sanjay M. P.

Reputation: 1007

Pub Sub Lite topics with Peak Capacity Throughput option

We are using Pub Sub lite instances along with reservations, we want to deploy it via Terraform, on UI while creating a Pub Sub Lite we get an option to specify Peak Publish Throughput (MiB/s) and Peak Subscribe Throughput (MiB/s) which is not available in the resource "google_pubsub_lite_topic" as per this doc https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/pubsub_lite_topic.

pubsublite

resource "google_pubsub_lite_reservation" "pubsub_lite_reservation" {
  name = var.lite_reservation_name
  project = var.project
  region  = var.region
  throughput_capacity = var.throughput_capacity
}

resource "google_pubsub_lite_topic" "pubsub_lite_topic" {
  name    = var.topic_name
  project = var.project
  region  = var.region
  zone    = var.zone
  partition_config {
    count = var.partitions_count
    capacity {
      publish_mib_per_sec   = var.publish_mib_per_sec
      subscribe_mib_per_sec = var.subscribe_mib_per_sec
    }
  }

  retention_config {
    per_partition_bytes = var.per_partition_bytes
    period              = var.period
  }

  reservation_config {
    throughput_reservation = google_pubsub_lite_reservation.pubsub_lite_reservation.name
  }

}

Currently use the above TF script to create pub sub lite instance, the problem here is we are mentioning the throughput capacity instead of setting the peak throughput capacity, and capacity block is a required field. Please help if there is any workaround to it ? we want topic to set throughput dynamically but with peak limit to the throughput, as we are setting a fix value to the lite reservation.

Upvotes: 0

Views: 346

Answers (1)

Hitobat
Hitobat

Reputation: 3037

If you check the bottom of your Google Cloud console screenshot, you can see it suggests to have 4 partitions with 4MiB/s publish and subscribe throughput.

Therefore your Terraform partition_config should match this. Count should be 4 for the 4 partitions, with capacity of 4MiB/s publish and 4MiB/s subscribe for each partition.

The "peak throughput" in web UI is just for convenience to help you choose some numbers here. The actual underlying PubSub Lite API doesn't actually have this field, which is why there is no Terraform setting either. You will notice the sample docs require a per-partiton setting just like Terraform.

eg. https://cloud.google.com/pubsub/lite/docs/samples/pubsublite-create-topic

I think the only other alternative would be to create a reservation attached to your topic with enough throughput units for desired capacity. And then completely omit capacity block in Terraform and let the reservation decide.

Upvotes: 0

Related Questions