How to get data from MySQL DB as CSV file with Java

Question

I'm using following Java code for getting data from MySQL table as CSV file.

Using this command I can get data from MySQL as CSV file:

mysql -uuname -ppassword BR_RNCM -e'select * from Inward where inwarddate between "2012-08-21 00:00:00" and "2012-08-21 23:59:59"'>/home/praveen/downloadfile.csv

How can I optimize my code\algorithm?

My code is:

package com.mypackage.CAF.APController;

import com.rajsoft.CAF.util.DB2Connection;
import java.io.FileWriter;
import java.io.IOException;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class InwardDataDownload extends HttpServlet {

    /**
     * Processes requests for both HTTP
     * <code>GET</code> and
     * <code>POST</code> methods.
     *
     * @param request servlet request
     * @param response servlet response
     * @throws ServletException if a servlet-specific error occurs
     * @throws IOException if an I/O error occurs
     */
    protected void processRequest(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        response.setContentType("text/html;charset=UTF-8");
        PrintWriter out = response.getWriter();

        Connection con = null;
        ResultSet rs = null;
        Statement st = null;

        String date1 = null;
        String date2 = null;

        date1 = request.getParameter("fromdate");
        date2 = request.getParameter("todate");

        System.out.println("Date1 : " + date1 + "Date2 : " + date2);

        try {
            response.setContentType("text/csv");

            out.write("CAF");
            out.write(',');
            out.write("MDN");
            out.write(',');
            out.write("CRT CODE");
            out.write(',');
            out.write("DISTRIBUTOR NAME");
            out.write(',');
            out.write("CLUSTER");
            out.write(',');
            out.write("MNP");
            out.write(',');
            out.write("LOT NO");
            out.write(',');
            out.write("INDEX NO");
            out.write(',');
            out.write("BOX NO");
            out.write(',');
            out.write("USER");
            out.write(',');
            out.write("INWARD DATE");
            out.write('\n');

            query = "select * from Inward where inwarddate between '" + date1 + "' and '" + date2 + "';";
            con = (Connection) new DB2Connection().getDatabaseConnection();
            st = con.createStatement();
            rs = st.executeQuery(query);
            while (rs.next()) {
                out.write(rs.getString(1));
                out.write(',');
                out.write(rs.getString(2));
                out.write(',');
                out.write(rs.getString(3));
                out.write(',');
                out.write(rs.getString(4));
                out.write(',');
                out.write(rs.getString(5));
                out.write(',');
                out.write(rs.getString(6));
                out.write(',');
                out.write(rs.getString(7));
                out.write(',');
                out.write(rs.getString(8));
                out.write(',');
                out.write(rs.getString(9));
                out.write(',');
                out.write(rs.getString(10));
                out.write(',');
                out.write(rs.getString(11));
                out.write('\n');
            }

            response.setContentType("application/download");
            response.setHeader("Content-disposition", "attachment; filename =InwardData.csv");
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            out.close();
        }
    }
}

Answer 1

There's no way for you to eliminate that while loop and that's a large chunk of your execution time. That loop basically makes your process a O(n) which is pretty good. Since you don't have any other loops to eliminate, you're already in a pretty good position.

PrintWriters buffer to memory before flushing to disk so you're not taking a performance hit by waiting for the write to disk to complete. Although, your code doesn't mention anything about flushing so you may run into out of memory issues if you're working with a really large dataset. Maybe do an out.flush() every thousand records (depending on what works best).

Answer 2

You can use rs.getMetadata() to get a ResultSetMetaData object. That will tell you how many columns there are, and what the column names are.

With that info you can output the header using a loop.

Now that you know the number of columns, you can also output each row using a loop.