CODEWITHSUNDEEP
pythongoogle-cloud-platformgoogle-cloud-pubsub
intrigued_66
intrigued_66

Reputation: 17268

Googlr PubSub Request messages even if acknowledged?

I am receiving messages from a subscription I created using:

subscriber = pubsub_v1.SubscriberClient(credentials=credentials)
subscriber.create_subscription(name=subscription_name,topic=topic_name, ack_deadline_seconds=60)

I am receiving using this:

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(project_id, subscription_id)
future = subscriber.subscribe(subscription_path, callback=callback())

with subscriber:
    try:
        future.result()
    except TimeoutError:
        future.cancel()

def callback(msg):
    my_logic(msg)
    msg.ack()

Although I acknowledge at the end of the callback I missed the first ~600 messages from the Publisher (the subscription existed). I don't think they were received but due to minimised logging I cannot confirm it. Anyway, I need to recover these messages.

I read this page:

https://cloud.google.com/pubsub/docs/replay-overview

And unfortunately my subscription did not set retain_acked_messages=true.

I have two questions:

  1. What is the simplest way to retrieve these missing messages? I know the time period they would have occurred. Is it possible to instruct the service to replay them using timestamps?

  2. Are there any changes to my subscription I should make for future problems? Should I create a snapshot on startup?

Upvotes: 1

Views: 470

Answers (1)

guillaume blaquiere
guillaume blaquiere

Reputation: 76000

If you don't retain the acked message, you can't seek your PubSub subscription and replay the past ack messages.

A snapshot is great, especially for tests (replayability) or before releasing a new version in production (rollback in case of errors).

IMO, the retain_acked_messages is important to activate in your case. Snapshots are less useful when you have this param if your use case is to replay acked messages.

Upvotes: 1

Related Questions