CODEWITHSUNDEEP
pythoncsvsolrflaskxls
Josh Usre
Josh Usre

Reputation: 683

Force Output from SOLR Query to Download as CSV

I'm using SOLR with a Flask (python) wrapper that makes queries via localhost / HTTP GET. Flask then passes the result back to the user on the net. I'd like to be able to offer users the ability to download their query as a .csv file (instead of view it in the browser and having to do it manually). I'd also like to be able convert the csv to an .xls format and allow that as another file download. Is there a way to have flask cache the file and offer it as a whole to download, with either extension (.csv or .xls)?

Thanks for your time.

Upvotes: 0

Views: 1153

Answers (2)

Josh Usre
Josh Usre

Reputation: 683

This is a hacky, but works:

Once we know the user's query, we hit SOLR to get the numFound. (I couldn't find a way just to get a count, if there's a simpler way, let me know?)

Next, since I know the header and results will always match, I just add the hardcoded header. (I'll add a way to get it from solr pythonically at a future date.)

Then we change the user query from json to csv and iterate over the query results, after calculating the page count based on the rows specified for each page in the return, and build final_csv to return.

Finally, we send final_csv back to the user with Response and the appropriate headers so that it pops up as a file download.

    final_csv = """"""

    header = "service_name_and_version,_version_,run_name,baseline_value,added_on,site_id,metric_type,baseline_run_id,service_name,tag_name,url,service_version,change_in_value,id,run_id,run_value,page_id,cryptic_tag_name"

    nice_header = header.split(",")

    final_csv += str(nice_header)[1:-1]

    r = requests.get(solr_query)

    solr_return = r.json()

    length = solr_return["response"]["numFound"]
    #(n + d // 2) // d

    print length

    pages = ((length + return_count // 2) // return_count) + 1

    #print pages

    counter = 0
    cursor = 0


    csv_query = solr_query.replace("&wt=json", "&wt=csv&csv.header=false")

    while counter < pages:

        iter_query = csv_query + "&rows=" + str(return_count) + "&start=" + str(cursor)

        r = requests.get(iter_query)

        solr_return = r.text

        solr_docs = solr_return.split("\n")
        print solr_docs
        print len(solr_docs)

        for doc in solr_docs[:-1]:

            row = str(doc)

            split_row = row.split(",")

            final_csv += "\n" + str(split_row)[1:-1]


        counter += 1
        cursor += return_count

    response = make_response(final_csv)

    response.headers['Content-type'] = 'text/csv'

    response.headers["Content-Disposition"] = "attachment; filename={0}.csv".format(csv_name)

    return response

Upvotes: 0

godfrey.obinchu
godfrey.obinchu

Reputation: 111

Take a look at this: http://wiki.apache.org/solr/CSVResponseWriter

Solr doesnt support xls, but it does CSV. You may need to get flask to download the CSV output response

Upvotes: 3

Related Questions