CODEWITHSUNDEEP
javaspringhikaricp
akop
akop

Reputation: 7845

Shutown Spring when DB is lost

Use case

Our spring-boot-backend (v 2.3.1) using a postgres-database with HikariCP as connection pool.
The application is online and a admin accidentally kills the database.

Expected behavior

Spring notice that the connection to DB was lost. The application will gracefully shutdown.

Actual behavior

Spring is still running. All methods, which needs the db, ending up in exceptions.
To recover a complete good state, we need manually to restart spring.

Notes
We have a lot of async workers and they cannot recover correctly, when the database is going back online.
And docker/kubernetes will notice when the application shutdown and can automatically restart it.

Question

How can I reach the expected behavior?
Sadly I found nothing similar in the web.

Upvotes: 1

Views: 1298

Answers (3)

Lunatic
Lunatic

Reputation: 1906

You can disable the actuator default provided health indicators in your property files and replace your custom DatasourceHealthIndicator by register it as bean.

@Bean
public DataSourceHealthIndicator dataSourceHealthIndicator(){
   return new DataSourceHealthIndicator(dataSource, "SELECT 1");
}

@Component
@RequiredArgsConstructor
public class CustomHealth implements HealthIndicator {

   private final DataSourceHealthIndicator healthIndicator;

   @Override
   public Health health() {
      // ...
      return Health.status(healthIndicator.health().getStatus()).build();
   }
}

And set your properties like this.

application.yaml

management:
   health:
      db:
         enabled: false

application.properties

management.health.db.enabled: false

Upvotes: 1

akop
akop

Reputation: 7845

With the hints from @Zubair I build a new small solution.
Iam using the Spring-Actuator-package, because they have some ready classes for this use-case.

All what I need was a HealthIndicator-Bean, like this

@Bean
public HealthIndicator dataSourceHealthIndicator(final DataSource dataSource) {
  return new DataSourceHealthIndicator(dataSource, "SELECT 1;");
}

and scheduled watcher (because HikariCP nor the HealthIndicators has any events).

@Scheduled(fixedRate = 10000L)
public void checkDBHealth() {
  final Status status = this.dataSourceHealthIndicator.health().getStatus();
  if (!Status.UP.equals(status)) {
    log.error("DATABASE IS OFFLINE! SHUTTING DOWN!");
    System.exit(1);
  }
}

I hope that will be useful for others.

Edit

I had to change the config of HikariCP. Otherwise the health-checker waiting almost endless for database-connection.

@Bean
public DataSource dataSource() {
  final HikariConfig config = new HikariConfig();
  // default settings with user, password, jdbc, ...
  config.setInitializationFailTimeout(1000);
  config.setConnectionTimeout(1000);
  config.setValidationTimeout(1000);

  return new HikariDataSource(config);
}

Upvotes: 3

Zubair
Zubair

Reputation: 6203

If its spring 2.0, you may call the shut-down actuator from some monitoring service.

Upvotes: 1

Related Questions