Retries during secret manager access in spring boot

Question

Context: We use secret manager in our spring boot production app. From time to time, we get DEADLINE_EXCEEDED errors when accessing a secret via secretManagerTemplate.getSecretString("name"). This is usually a short time problem and if we retry, the issue is gone.

Need We'd like to be able to avoid these errors with a simple declarative approach (without much boilerplate). Is there a way to achieve this without building our own retry mechanism?

Extra info:

• our current version:

    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>spring-cloud-gcp-starter-secretmanager</artifactId>
      <version>2.0.2</version>
      <scope>compile</scope>
    </dependency>

• the exception stack trace:

io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 59.999980428s. [buffered_nanos=1736630, remote_addr=secretmanager.googleapis.com/172.217.23.106:443]

at io.grpc.Status.asRuntimeException ( io/grpc/Status.java:535 )
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose ( io/grpc.stub/ClientCalls.java:533 )
at io.grpc.internal.ClientCallImpl.closeObserver ( io/grpc.internal/ClientCallImpl.java:553 )
at io.grpc.internal.ClientCallImpl.access$300 ( io/grpc.internal/ClientCallImpl.java:68 )
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal ( io/grpc.internal/ClientCallImpl.java:739 )
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext ( io/grpc.internal/ClientCallImpl.java:718 )
at io.grpc.internal.ContextRunnable.run ( io/grpc.internal/ContextRunnable.java:37 )
at io.grpc.internal.SerializingExecutor.run ( io/grpc.internal/SerializingExecutor.java:123 )
at java.util.concurrent.Executors$RunnableAdapter.call ( java/util.concurrent/Executors.java:515 )
at java.util.concurrent.FutureTask.run ( java/util.concurrent/FutureTask.java:264 )
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run ( java/util.concurrent/ScheduledThreadPoolExecutor.java:304 )
at java.util.concurrent.ThreadPoolExecutor.runWorker ( java/util.concurrent/ThreadPoolExecutor.java:1128 )
at java.util.concurrent.ThreadPoolExecutor$Worker.run ( java/util.concurrent/ThreadPoolExecutor.java:628 )
at java.lang.Thread.run ( java/lang/Thread.java:829 )

• we already opened an issue on GitHub which can be found here