Show actual progress of code behind process on ASP.NET page

Question

I titled this with "Actual Progress" because everything I've found regarding the <asp:UpdateProgress> doesn't actually show progress, it just shows that the user needs to wait. I'm very new to any web dev so if I'm missing something obvious on this please be sure to dope-slap me with an obvious answer.

In a WinForms app, I'm trying to show progress while code behind is zipping files. I'm using System.IO.Compression and streaming to "copy" the files to the zip file so I have the number of bytes with which to update a progress bar of some sort. The creation of the zip file works fine.

A relatively simple technique seemed to be setting a __doPostBack interval on an update panel, then having the _Load of the UpdatePanel read a session variable which is set by zipping process which is running on another thread. The UpdatePanel's _Load is firing at the correct interval but the session variable is always blank in the _Load event.

I created a very simple form with a label inside an update panel to try and get something working. Any suggestions of a better way to show actual progress would be welcome.

XML:

<%@ Page Language="vb" MasterPageFile="~/MstrPg.Master" 
    AutoEventWireup="false" CodeBehind="TestForm.aspx.vb" 
    Inherits="TekntypeUpDownLoad.TestForm" %>

<asp:Content ID="CntntTest" runat="server" ContentPlaceHolderID="MainPg">
    <script type="text/javascript">
        window.onload = function () {
            setInterval("__doPostBack('<%=UpdatePanel1.ClientID%>', '');", 1000);
        }
    </script>
    <fieldset style="width:400px" class="transParent">
        <legend>Progress bar example</legend>
        <asp:UpdatePanel ID="UpdatePanel1" runat="server" >
            <ContentTemplate>
                <asp:Label ID="lblStatus" runat="server" Text="" ClientIDMode="Static"></asp:Label>
            </ContentTemplate>
        </asp:UpdatePanel>
        <asp:Button ID="btnSubmit" runat="server" Text="Submit" onclick="btnSubmit_Click" />
    </fieldset>
</asp:Content>

Code behind:

Protected Sub btnSubmit_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btnSubmit.Click
    'System.Threading.Thread.Sleep(4000)
    'lblStatus.Text = "Processing Completed"

    Dim ZipUpdThrd As New Threading.Thread(
        Sub()
            Session("CntrValue") = "Starting."
            For i = 1 To 5
                System.Threading.Thread.Sleep(1000) 'cause a pause
                'ScriptManager.RegisterStartupScript(Me, Me.GetType, "UpdProg" & i, "UpdProgress('lblStatus', '" & i & "');", True)
                Session("CntrValue") = i.ToString
                Debug.Print("i: " & i & " -- Session: " & Session("CntrValue"))
            Next
            Debug.Print("end ofloop")
            Me.lblStatus.Text = "Complete."
        End Sub
            )
    ZipUpdThrd.Start()

    'ScriptManager.RegisterStartupScript(Me, Me.GetType, "UpdProg", "UpdProgress('lblStatus', 'Complete.');", True)
End Sub

Private Sub UpdatePanel1_Load(sender As Object, e As EventArgs) Handles UpdatePanel1.Load
    Debug.Print("panel load-" & Now & ": " & Session("CntrValue"))
    If Session("CntrValue") IsNot Nothing Then
        Me.lblStatus.Text = Session("CntrValue")
    Else
        Me.lblStatus.Text = "Session is nothing: " & Now
    End If
End Sub

Answer 1

Well, keep in mind the browser + server, and how they interact with each other. In effect, you can almost think of this like sending an email. When you send an email to someone, you don't see their changes until they are finished typing and they send the email back. The browser and server work much the same.

When you post-back the browser, a whole copy is sent up to the server. Then the page class (instance) is created, code behind runs. And if you set a text box, even in a loop, or even inject script (register script), all of this is occurring on a COPY of the browser, and one that is now up on the server. The end user just sees the browser spinner. In effect, the server is modifying the email message, and you not see ANY changes until that code is 100% finished, and then message (copy of the browser returns back to the client side).

So, as you modify a text box, inject script etc., your code is NEVER directly interacting with the end user, but is modifying a copy of the browser page up on the server. During this so called "round trip", that page up on the server is being modified by your code, but the end user see's no changes. (all they see is the browser wait spinner icon).

When your code (and all of your code behind) for that page is done, then the WHOLE copy of the browser is now sent back to the end user. At this point, the page class and code behind are disposed of, and removed from memory. So, between each post-back (often called a round trip), then all the code and variables for that page class are lost and go out of scope.

The browser renders the page, displays it, loads the JavaScript engine, and THEN the JavaScript code starts running (again, a critical concept to grasp). With post-backs, then JavaScript on the browser (client side) does NOT keep running, nor does its variables remain intact either!!! - they don't persist after a round trip.

So, if you write some JavaScript to do a _DoPostBack, then the whole page is sent up to the server again, the page class is RECREATED from scratch (for each post back), and then once again, the code behind runs.

You don't have a direct connection to the user's browser like you do on a desktop to some memory-mapped screen/monitor.

And your code goes out of scope ONCE the code behind is done making changes to the browser. Much of how this works is the web server after sending your page back to the client THEN blows out, disposes of that web page, since now the web server is waiting for any user - not just you to do a post-back. So, you don’t have one computer like you do with desktop software, but have one web server, and that ONE program system has to service all users, not just you. In effect, it would be like multiple people using Word on one computer. When the next user comes along, they would have to re-load their Word document, and when done they have to close it, so the next user can load their Word document, or in this case load the web page. A grasp of this post-back model is critical for grasping how web-based software works.

You thus cannot from server code (in general) push out, or have the server modify parts of the web page in real time. However, you could consider writing some JavaScript and setup what is called a "web socket". This is a bit of work; hence existing libraries exist that will do most of the moving parts and wiring up of such a setup. This technology is thus used say for a chat page in which 2 (or even more) users can all type on the page at the same time, and everyone sees what everyone else is typing. However, adopting such a stack of software and setup for your needs is a bit overkill, but is a way to have server-side code "push" out information to the browser in place of the post back model (the so-called round trip).

So, the idea to have at some interval the browser "talk to" or "ask" the server how much of the long running process is completed is the right idea.

In fact, in place of writing JavaScript (with timer code), you can drop in a ready-made control called a timer (which does much the same thing as writing some JavaScript with post-backs. However, MUCH better would be to write AJAX calls from JavaScript, since then we are avoiding the post-back model 100%.

There is also a magic trick control in webforms called an update panel. What the update panel does is in effect "wire up" a ajax like part of the page in which then you don't suffer a whole page post-back (but, if you trigger a post-back from JavaScript, then the whole page is posted back - we don't want that).

So, let's start that thread. Since the thread is 100% different then the current running code on the given page (which as I pointed out is blown out of memory and disposed EACH time for each post back, and that code behind DOES NOT exist in memory anymore, as the web server is now read and waiting for the NEXT user and post-back (or your post-back). So, keep in mind this so called "state-less" operation of web-based software. This also means to keep values for code between each post-back, you need to use session().

So, then say this markup:

        <asp:ScriptManager ID="ScriptManager1" runat="server"></asp:ScriptManager>

        <asp:Button ID="cmdStart" runat="server" Text="Start"
            OnClick="cmdStart_Click" />

        <asp:UpdatePanel ID="UpdatePanel1" runat="server">
            <ContentTemplate>
                Current Progress:
                <asp:Label ID="Label1" runat="server" Text=""></asp:Label>

                <asp:Timer ID="Timer1" runat="server"
                    Enabled="False"
                    Interval="1000"
                    OnTick="Timer1_Tick">
                </asp:Timer>

            </ContentTemplate>
        </asp:UpdatePanel>

Note close in above, any button or even the timer placed inside of the update panel means ONLY that part of the page is posted back to the server. Keep in mind this is STILL A ROUND TRIP!!! The only new part is that only that part of the page will be updated. So, in place of a button (to post-back the update panel content), I used a timer control, and hence note how it is placed inside of the update panel.

In effect then, we are going to post-back the update panel every second, and then code behind will run, and then the update panel is returned to the browser. Do keep in mind, that this is STILL a post-back, and the page load event fires first, and then say your button code, or timer event code then runs each time.

So, now our code behind:

Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load


End Sub

Protected Sub cmdStart_Click(sender As Object, e As EventArgs)

    Session("Progress") = 0

    Dim MyThread As New Threading.Thread(
        Sub()

            For i = 1 To 20
                Session("Progress") = i
                System.Threading.Thread.Sleep(1000)
            Next

        End Sub)

    MyThread.Start()
    Timer1.Enabled = True

End Sub

Protected Sub Timer1_Tick(sender As Object, e As EventArgs)

    Dim iStep As Integer = Session("Progress")

    Label1.Text = iStep

    If iStep >= 20 Then
        Timer1.Enabled = False
    End If

End Sub

Note how our timer event code looks at session(), which is about the ONLY practical way to have code behind "communicate" with the thread we started. In fact, the timer interval can be 2 seconds or whatever, since the thread is running as separate process. Since the thread is a separate process, and since as I explained the code behind will go out of scope EACH round trip? Then not only does the code behind need a way to look at and find out the status of the separate thread, that separate thread can NOT modify the web page, since it not even existing and up on the server any more!

So, your zipping code, or file copy code, or whatever will have to update some value that the web page can get/grab/see to update our content in the browser page. Probably a great idea to consider a jQuery UI progress bar.

Thus, running above, we see this:

So, use a timer control.

Of course, as your skill set increases, then one would probably be better to use AJAX, and create a web method for the given page. That way, no post-backs or round trips to the code behind would be required for above.

I also suggest you try running this page without the update panel (just remove that part of the markup - everything else should continue to work, as you want to see the page in operation without the update panel. So, remove this markup:

        <asp:UpdatePanel ID="UpdatePanel1" runat="server">
            <ContentTemplate>

and the closing part:

            </ContentTemplate>
        </asp:UpdatePanel>

Now try running the sample code again. This will show what is really going on here, since the update panel is sort of a "fake" AJAX like call in which only part of the browser is being updated. However, keep in mind that the update panel is still a "standard" round trip, and a post-back is occurring. And for any button or timer or whatever code inside of the update panel that does the "hidden" post-back? Your page load event still fires each and every time (like with a regular full-page post back), and THEN your button or timer or whatever code stub runs (again, just like it does with a regular full page post-back).

Edit: Using session in a separate processor thread

So, as noted, it seems our new separate thread is confused and not able to use session. In my test code, I'm using "in-proc" session, and working on my dev computer (using the automatic launched IIS by using F5 to run + debug, and while it worked, it was a poor example on my part).

However, if we stop and think about this, as I stated, once the code behind is complete, then the current code page, the page class etc. goes out of scope. (kind of much like calling a sub routine - when we exit that sub, then all the variables and values go out of scope - including the session()).

I mean, if we are to start a new processor thread, and 10 people post a page to the server, then what session() are we using? So, we need to BE SURE that this new separate processor thread is using the correct session(), and we can't assume nor simply use session(), since which user does session() belong to? (answer: we don't know, or better stated that separate running bit of code has no idea who's session() to use, and what is the scope here?).

Since the current page class (with the code behind) is 100% disposed of when our code runs (and the web page, like our "email" message is then sent back to the client). Now the separate code thread keeps running, but what session() is it to use? We are now gambling here.

So, I suggest we move the processing code out of the current page class. Let's add (create) a standard code module, and place our processing code in that code module. So, it is a "good idea" to move that code out of the current page class (the code behind for that page).

Hence, let's add a code module:

I already have a few code modules of "general code" that many pages will call, so it defaulted to Module5.vb - not really matters much.

So, now, let's place our processing code in that module.

We have this code:

Module Module5


    Public Sub MyProcess(MyParms() As Object)


        Dim MySession As HttpSessionState = MyParms(0)
        For i = 1 To 5

            MySession("Progress") = i
            System.Threading.Thread.Sleep(1000)

        Next


    End Sub


End Module

Note that I passed an array of objects, since when starting a thread, you are only allowed ONE parameter, but by using an array of objects, we in fact can still pass several (many) values to the processor routine.

Hence, our code behind for the page now needs to pass the current session, so the code knows what session() to work with.

Hence this code:

Protected Sub cmdStart_Click(sender As Object, e As EventArgs)

    Session("Progress") = 0

    Dim MyThread As New Thread(New ParameterizedThreadStart(AddressOf MyProcess))

    ' threads only allow one param,
    ' increase array to pass more parameters
    Dim MyP(0) As Object

    MyP(0) = HttpContext.Current.Session
    MyThread.Start(MyP)
    Timer1.Enabled = True

End Sub

So, note how we pass the user's current session, and thus the new separate processing routine will use the correct session(), since you have many users on the site, and thus the external code running outside of the current web page will work with, and use the correct session() for the current user.

Give the above a try, it should work. We also should perhaps post a working example using JavaScript "web method" calls, and thus we would not requite the timer control + the update panel, which as noted still is a post-back and round trip. However, the update panel should suffice for this example. So, we could adopt JavaScript and eliminate the need for page post backs here.

Edit: JavaScript example

As noted, we can dump the post-back model, adopt JavaScript, and do quite much the same thing without a update panel. The code is rather similar, and I assume we have jQuery installed to make this easier.

So, say right below the above markup, we can have this markup:

        <asp:Button ID="cmdProcess2" runat="server" Text="JavaScipt (AJAX) Example"
            OnClick="cmdProcess2_Click"
            />
        <br />
        <asp:Label ID="lblProcess2" runat="server" Text=""
            ClientIDMode="Static">
        </asp:Label>

        <script>

            var MyTimer
            function pollstart() {

                // start a timer to call client side routine
                // over and over
                MyTimer = setInterval(mypolling, 1000)
            }

            function mypolling() {

                var lblMsg = $('#lblProcess2')

                $.ajax({
                    type: "POST",
                    url: "ProgressSteps.aspx/GetStatus",
                    data: "{}",
                    contentType: "application/json; charset=utf-8",
                    dataType: "json",
                    success: function (rData) {
                        console.log("running = " + rData.d)
                        lblMsg.html(rData.d)
                        if (rData.d >= 5) {
                            // we are done, stop the pooling (timer)
                            clearTimeout(MyTimer)
                        }
                    },
                    failure: function (rData) {
                        alert("error " + rData.d);
                    }
                });

            }

        </script>

Our code behind:

<WebMethod(EnableSession:=True)>
Public Shared Function GetStatus() As Integer

    Dim iStep As Integer = HttpContext.Current.Session("Progress")

    Return iStep

End Function


Protected Sub cmdProcess2_Click(sender As Object, e As EventArgs)

    Session("Progress") = 0

    Dim MyThread As New Thread(New ParameterizedThreadStart(AddressOf MyProcess))

    ' threads only allow one param,
    ' increase array to pass more parameters
    Dim MyP(0) As Object

    MyP(0) = HttpContext.Current.Session
    MyThread.Start(MyP)

    ' now start a timer on the client side page

    Dim sJava As String = "pollstart();"

    ScriptManager.RegisterStartupScript(Page, Page.GetType, "myjavakey", sJava, True)


End Sub

has the button to start the task, and that also calls the JavaScript routine to start the browser code. Remember, as pointed out, the RegisterStartUp script is no different then modifying a text box or whatever, and that script does not until such time the WHOLE browser page is returned to the client. So, script injecting, change a text box, or whatever? User see's none of these changes until the fresh copy of the browser is sent back to the client (like our email response idea).

However, now in place of post-backs, we call a web method, and thus no post-backs are used.

The effect is really the same, but we traded the UpDate panel, and the timer control for some JavaScript code that achieves quite much the same goal. And this approach as I noted will change ZERO in regards of the Session() issue we are experiencing.

My best guess is that the session object is going out of scope, and thus the second separate process we are running loses it's reference to Session().

So, this works for in-proc Session() on my dev computer:

However, I think we have to find some other means to communicate from that separate process to the current user, as it does look like Session() is going out of scope.

Answer 2

Wow, Albert, this is above and beyond, can't thank you enough for taking the time. The email analogy might be a common thing in ASP.NET circles but it's bloody brilliant, best description I've seen, thanks for that too.

After some trouble I decided to try your code verbatim and from my test page removed the master page dependency and pasted in your code, the page shows the zero but it never changes because the session variable in the tick event is always zero. :o/

Details of my overall setup:

The first page of my app is a login page. Currently it simply redirects to my test page using Response.Redirect("TestForm.aspx", True) in it's _LoadComplete event.

Early on with this project Session variables weren't working in VisualStudio, I'd set it one place but it wasn't available in another, though they worked when I published to the webserver. After turning on the "ASP.NET State Service" and updating my web.config with mode="StateServer" I haven't had a problem since.

But, something isn't right, the result on the page shows the zero, but it doesn't change and the Session variable is always zero in the _Tick event. The code below is your code with only the two Debug.Print lines added and the interval of the loop reduced from 20 to 5. When I run this code I'm getting this result in the Immediate window, with the "Exception thrown:" line being the redirection to the test page, of course:

Exception thrown: 'System.Threading.ThreadAbortException' in mscorlib.dll
Session var in the loop: 1
iStep in Timer1_Tick: 0
Session var in the loop: 2
iStep in Timer1_Tick: 0
Session var in the loop: 3
iStep in Timer1_Tick: 0
Session var in the loop: 4
iStep in Timer1_Tick: 0
Session var in the loop: 5
iStep in Timer1_Tick: 0
iStep in Timer1_Tick: 0
iStep in Timer1_Tick: 0
.
.
.

What am I missing?!

Thanks again

XML:

<%@ Page Language="vb" AutoEventWireup="false" CodeBehind="TestForm.aspx.vb" Inherits="TekntypeUpDownLoad.TestForm" %>

<html xmlns="http://www.w3.org/1999/xhtml">
    <body>
        <form id="TstForm" runat="server">

            <asp:ScriptManager ID="ScriptManager1" runat="server"></asp:ScriptManager>

            <asp:Button ID="cmdStart" runat="server" Text="Start"
                OnClick="cmdStart_Click" />

            <asp:UpdatePanel ID="UpdatePanel1" runat="server">
                <ContentTemplate>
                    Current Progress:
                    <asp:Label ID="Label1" runat="server" Text=""></asp:Label>

                    <asp:Timer ID="Timer1" runat="server"
                        Enabled="False"
                        Interval="1000"
                        OnTick="Timer1_Tick">
                    </asp:Timer>

                </ContentTemplate>
            </asp:UpdatePanel>
        </form>
    </body>
</html>

VB:

Public Class TestForm
    Inherits System.Web.UI.Page

    Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

    End Sub

    Protected Sub cmdStart_Click(sender As Object, e As EventArgs)

        Session("Progress") = 0

        Dim MyThread As New Threading.Thread(
        Sub()

            For i = 1 To 5
                Session("Progress") = i
                System.Threading.Thread.Sleep(1000)
                Debug.Print("Session var in the loop: " & Session("Progress"))
            Next

        End Sub)

        MyThread.Start()
        Timer1.Enabled = True

    End Sub

    Protected Sub Timer1_Tick(sender As Object, e As EventArgs)

        Dim iStep As Integer = Session("Progress")

        Debug.Print("iStep in Timer1_Tick: " & iStep)

        Label1.Text = iStep

        If iStep >= 5 Then
            Timer1.Enabled = False
        End If

    End Sub
End Class

Edit 1

Taking this out of my web.config made both the most recent iteration (with the separate module) and the previous iteration work, both in Visual Studio and published to my web server:

<sessionState 
    mode="StateServer"
    stateConnectionString="tcpip=loopback:42424"
    cookieless="false"
    timeout="20" />

I looked up <sessionState> again but didn't find anything I thought might be helpful. Maybe this is a clue to what's going on?: After creating the separate module but before removing the <sessionState> I was getting a 1 in the label instead of the original zero, but then it got stuck at that.

I'd been using <sessionState> because I'd been having trouble in Visual Studio with the session variables not staying from one page to the next, thought it was OK when I published to the web. That now seems to have resolved itself, though I couldn't say why, possibly with the recent update of VisualStudio.

Is there any advantage to using <sessionState>?

It sounds to me like your suggestion of using JavaScript "web method" calls would be the way to go and I tried to look up what you were referring to but I wasn't able to discern anything I could try. Can you point me to an example someplace I might learn more?

Thanks again