How to detect malicious activty in Windows Azure - way to see incoming HTTP request on Azure instances?

Question

I had a very stable application hosted on Azure + SQL Azure since 18 months.

Since monday i get random long period of deadlocking in SQL Azure which just prevent any update in the database for hours.

Then it goes. Then it comes back.

This is not related to instances as the "reading" parts of the websites are OK.

• I do use WITH (NOLOCK) on each reader request as I consistency is not critical in my case. So datareader in azure does works but each update throw a SQL Timeout connection.

• I do use Retry Logic for Transient Failures in Windows Azure SQL.

• A centralized database serve a few websites, trafic is around 25K visits day combined, I never had a problem since 18 months, current trafic has not jumped from last week when everything was ok.

I got no clue of what might be happening. And for the first time i find myself trapped in the cloud not having a concrete way to control anything.

Maybe an attack. So how do I detect that in Azure ? Is there a way to see incoming HTTP request on Azure instances ?

Answer 1

If you don't mind using third-party services & products and need active monitoring of either your current requests/sec or load on the SQL Azure database (active queries, blocking queries,open connections, etc) - check out AzureWatch @ http://www.paraleap.com - it's a monitoring service dedicated to Azure and will alert you within seconds of any issues with your Roles or SQL Azure db.

Answer 2

While analysing requests is one place to start, the other place is to understand exactly what is going on in your database. That's where you are picking up the issues, right? If you can find the rogue query, you should be in a better place to trace the source. Unfortunately Windows Azure SQL Database doesn't give you the full options available to SQL Server (such as profiling), but there is some help. Look at the system views — sys.dm_exec_query_stats and sys.dm_tran_active_transactions might be good places to start. Go through every single view of the understand exactly what each is showing. This will give you a better idea of what is happening in the database than you have now, which is good info regardless of your specific problem.

If you can do a redeployment, absolutely try Newrelic, which is an agent that hooks itself in to your app as a profiler and gives bucketloads of useful information. It is worthwhile if you have a serious problem, even if you just sign up for the trial period while you sort the problem out. Beware, Newrelic is a bit fiddly to get working, and doesn't seem to support .NET 4.5 target platform on Azure.

Answer 3

You could take a look at the CPU activity in the management portal, although limited it may allow you to glean some information. Is your web-role collecting server logs? If so you could use a tool like Cerebrata Diagnostics Manager to visualise the log files.

Answer 4

A bit of quick Googling should point you to articles that describe how to export your IIS logs from Azure. Then you would be able to parse them just as if you were running IIS locally.

However, if you are having these issues and they started suddenly without you making a change, you may wish to just reach out to Microsoft support.