CODEWITHSUNDEEP
excelscalahttpgethttpurlconnection
ztaylor54
ztaylor54

Reputation: 406

HTTP GET responds with empty body, downloads excel document in IE

I am trying to figure out what is happening on this page when I download an excel report. It only works for Internet Explorer, and when I attempt to download the report in Scala using ScalaJ, I the input stream for the response body is null. Can someone help me understand what's going on here?

The plaintext request:

GET /oca_ReportViewer.aspx?ReportName=District_and_Statutory_County_Court/DSC_Civil_Family_Activity_Detail_N.rpt&ddlFromMonth=9&ddlFromYear=2010&txtFromMonthField=@FromMonth&txtFromYearField=@FromYear&ddlToMonth=10&ddlToYear=2010&txtToMonthField=@ToMonth&txtToYearField=@ToYear&ddlCountyPostBack=0&txtCountyPostBackField=@CountyID&chkAggregateMonthlyReport=0&export=1625 HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Referer: http://card.txcourts.gov/ReportSelection.aspx
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: card.txcourts.gov
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: ASP.NET_SessionId=j0tgci45qj3t1uqygrvkqz55

The code so far:

import java.io.{BufferedOutputStream, FileOutputStream, FileWriter, InputStream}
import scalaj.http._

object Downloader extends App {

    val url = "http://card.txcourts.gov/oca_ReportViewer.aspx"
    val keys: Map[String, String] = Map (
        ("ReportName", "District_and_Statutory_County_Court/DSC_Civil_Family_Activity_Detail_N.rpt"),
        ("ddlFromMonth", "9"),
        ("ddlFromYear", "2010"),
        ("txtFromMonthField", "@FromMonth"),
        ("txtFromYearField", "@FromYear"),
        ("ddlToMonth", "10"),
        ("ddlToYear", "2010"),
        ("txtToMonthField", "@ToMonth"),
        ("txtToYearField", "@ToYear"),
        ("ddlCountyPostBack", "0"),
        ("txtCountyPostBackField", "@CountyID"),
        ("chkAggregateMonthlyReport", "0"),
        ("export", "1625")
    )

    //println(keys)

    val heads: Map[String, String] = Map (
        ("Accept", "text/html, application/xhtml+xml, image/jxr, */*"),
        ("Accept-Encoding", "gzip, deflate"),
        ("Accept-Language", "en-US"),
        ("Cache-Control", "no-cache"),
        ("Connection", "Keep-Alive"),
        ("Cookie", "ASP.NET_SessionId=j0tgci45qj3t1uqygrvkqz55"),
        ("Host", "card.txcourts.gov"),
        ("Referer", "http://card.txcourts.gov/ReportSelection.aspx"),
        ("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko")
    )

    def parse(status: Int, headers: Map[String, IndexedSeq[String]], inputStream: InputStream): Unit = {
        val output = new BufferedOutputStream(new FileOutputStream("test.xls"))
        val bytes = new Array[Byte](1024) //1024 bytes - Buffer size
        //println(status)
        //println(headers)
        Stream
            .continually (inputStream.read(bytes))
            .takeWhile (-1 !=)
            .foreach (read=>output.write(bytes,0,read))

        output.close()
    }

    val response: HttpResponse[Unit] = Http(url).params(keys).headers(heads).charset("US-ASCII").timeout(1000, 60000).exec[Unit](parse)
    println(response.isSuccess)
}

If anyone can help me, I would greatly appreciate it! I just need to have the response saved to a .xls file. And if you have time, I also can't seem to find where the browser is picking up the session id, so that would be really helpful to find out as well.

EDIT:

I can see that the excel data is actually being sent, check out this stream follow using wireshark:

stream

Thanks!

Upvotes: 3

Views: 398

Answers (1)

ztaylor54
ztaylor54

Reputation: 406

Ok, not really sure why my stream was going null, but I changed the code to just close the output stream when it hit the exception. I know this is silly, but it worked and the file is downloaded completely. If anyone else knows a better explanation, I'd appreciate it!

Here's the new parse function:

def parse(status: Int, headers: Map[String, IndexedSeq[String]], inputStream: InputStream): Unit = {
    val output = new BufferedOutputStream(new FileOutputStream(filename))
    try {
      Iterator
        .continually (inputStream.read)
        .takeWhile (-1 !=)
        .foreach (output.write)
    }
    catch {
      case _: Throwable => output.close()
    }
    finally{
      output.close()
    }
  }

Upvotes: 1

Related Questions