Cannot access SQL azure

Question

Just had a bizarre issue with SQL Azure, and it's happened in a small phase just before full go live with some users doing some data entry.

"Database 'dbname' on server 'xxx' is not currently available. Please rety the connection later. If the problem persists, contact customer support."

When I tried to connect via SQL Azure database website I got: "Firewall check failed. Resource ID : 1. The request minimum guarantee is 0, maximum limit is 180 and the current usage for the database is 0. However, the server is currently too busy to support request greater than 0 for this database."

Looking at the databases section of the Azure Management website the site reported it couldn't access the DB, but I didn't capture the exact error message unfortunately.

Bizarrely, a couple of my users were still able to login to our system website that access the DB, and view and save data. Eventually they lost connection too however.

After an hour or so, the databases came back to life and we could fully access them again.

I have looked at the servers master db event table using queries from here and there was a couple of connection failures but nothing interesting. No throttling or deadlocks, a couple of failed connections that said "Client may have timed out when establishing connection. Try increasing the connection timeout." in the description

Any ideas where else to look?

Business users have had a massive drop in confidence because of this.

Answer 1

What your describing normally occurs because of :

1) SQL Connection limit being hit. Assuming you don't see this often you unlikely to be the cause. But worth checking putting a limit on your connection pool can help.

2)You neighbours being extremely noisy and thus the node re-adjusts.

3) Hardware failure and Microsoft bringing your database back online in a different node. This can take some time.

Normally I have seen this when Microsoft have throttled or had problems with a box and had to recover everyone over. Because you are on a shared system you have to keep in mind that they are recovering everyone else also in that node also and thus sometimes this takes time.

The best bet if you are worried and need to get a resolution for the business is to open a support ticket with MS and give them the time and error message you saw this. They will investigate and generally they have really good back end telemetry that will point to a reason. This will allow you to give the business a resolution and then you can make a call on future plans and contingencies. You have to keep in mind though that SQL Azure is shared system and transient errors can happen, you might need to design more failover into your designs.