What is making our site "spin" in production?

Question

We just deployed a site into production. It works fine on my machine (running through Visual Studio 2015) and in test. However, the users report at some point during the day it "spins" (the circle in the tab of the Chrome browser continues to indicate it is "working") and that they can't do anything. I can't duplicate this. I suspect that it's because they're having 5+ users on at one time. Recycling the app pool fixes the issue. How do I tell what's going on and how do I fix it?

We have a asp.net MVC site using Telerik Kendo controls and Telerik Open Access to connect to a SQL Express database. The site is hosted Windows Server 2012R2 at the client site and the database is on the same server (client's decision, not mine; can't be changed). It's configured in IIS version 8.5. I've tried to match the IIS settings to be similar to our testing environment. There's no rhyme or reason to when the "spinning" happens; it doesn't occur with a certain number of users or a certain time of day. Our IT guy has set the app pool to recycle in the early morning before they start work. My boss thinks that the app pool is "filling up". Is there any way to increase the size of the app pool (not even sure if that's the right terminology)?

I'd appreciate any advice on how to figure out the problem and how to fix it. Thank you!

Answer 1

Check your website server's log file.

Look for a request that caused your server to respond slowly. Determine the approximate time that 'the users report at some point during the day it "spins"', which is when the web server began to respond slowly to browser requests.

Your web server's slowness issues probably began just after one or more ill-handled requests were processed.

To find where your website puts its log files, go into IIS and open the Logging icon.

The directory path that holds the logging files is defined there as well as the frequency that new log files are generated. (These images came from IIS 7.5.)

Answer 2

My guess is that there's a number of different things going on.

First, SQL Express is not intended to run a production website. It doesn't include many optimizations that happen by default or can be enabled/configured with a full SQL Server instance.

Second, both SQL Server and IIS are resource hogs. In doing what they do, they consume large amounts of RAM and CPU cycles. The recommendation is that they should be the only app running on any server they are installed on, which means they should be on two separate servers. Otherwise, they will continually be fighting each other for system resources.

Third, the App Pool will recycle by default after a period time and can recycle on its own for any number of other reasons throughout the course of a day. When this occurs, some amount of reinitialization of the application must occur, which can take some period of time before requests will be served. This can be minimized by precompiling your web application during deployment.

Fourth, the App Pool is really an abstract concept. It represents one or more processes that serve up your website. By default, each of those processes has a thread pool of 1000, often known in web server lingo as "max requests" because 1 thread generally equals one request. Theoretically, the larger this number is, the more simultaneous requests you can serve before requests must be queued. However, each thread has an overhead, so you can easily become resource-constrained or actually degrade performance by raising it too high. Remember that how many simultaneous operations the CPU can handle is a fixed entity. Even if you had enough RAM to handle some ridiculous number of threads, if they're queuing up for CPU time, then you get no benefit. Generally, it's preferable to go multiprocess rather than increasing the thread pool, as each process can utilize a separate core of the CPU. This is achieved my enabling web workers on the App Pool. Each worker corresponds to a separate process and another pool of 1000 threads. However, you should not have more workers than CPU cores, as each core can only handle operations for one process at a time. Also, bear in mind that each process is essentially its own little version of your website. Just as you would if you need to shared things like session data between diffent websites, you would also need to handle your web workers in a similar way. In other words, you should set a machine key in your web.config and employ decentralized storage for things like session data and caching. If you use in proc sessions or something like MemoryCache, all that will be confined to a single worker process.

Fifth, it could boil down to general inefficiency. Even if you have a slow database, if you can reduce the amount of queries it needs to handle by querying more efficiently/caching results when possible, you might still be able to get by. Using things like output cache, as well, can reduce load on the web server. Look for areas of the site that perform slower, in general, or which you know do complex tasks, and find ways to optimize those areas through refactoring code, combining queries, judicious caching, etc.