How to parse a text file and create a database record from it

Question

I'm trying to basically make a simple Test Generator. I want a button to parse a text file and add the records to my database. The questions and answers are in a text file. I have been searching the net for examples but I can't find one that matches my situation.

The text file has header information that I want to ignore up until the line that starts with "~ End of Syllabus". I want "~ End of Syllabus" to indicate the beginning of the questions. A couple of lines after that look for a line with a "(" in the seventh character position. I want that to indicate the Question Number line. The Question Number line is unique in that the "(" is in the seventh character position. I want to use that as an indicator to mark the start of a new question. In the Question Number line, the first three characters together "T1A" are the Question Group. The last part of the T1A*01* is the question number within that group.

So, as you can see I will also need to get the actual question text line and the answer lines as well. Also typically after the four Answer lines is the Question Terminator indicated by "~~". I don't know how I would be able to do this for all the questions in the text file. Do I keep adding them to an array String? How would I access this information from the file and add it to a database. This is very confusing for me and the way I feel I could learn how this works is by seeing an example that covers my situation. Here is a link to the text file I'm talking about:http://pastebin.com/3U3uwLHN

Code:

public static void main(String args[]) {

    String endOfSyllabus = "~ End of Syllabus";
    Path objPath = Paths.get("2014HamTechnician.txt");
    String[]  restOfTextFile = null;

    if (Files.exists(objPath)){

        File objFile = objPath.toFile();
        try(BufferedReader in = new BufferedReader(
                new FileReader(objFile))){

            String line = in.readLine();
            List<String> linesFile = new LinkedList<>();

            while(line != null){
                linesFile.add(line);  
                line = in.readLine();
            }

            System.out.println(linesFile);
        }
        catch(IOException e){
            System.out.println(e);
        }
    }
    else{
        System.out.println(
                objPath.toAbsolutePath() + " doesn't exist");
    }

    /* Create and display the form */
    java.awt.EventQueue.invokeLater(new Runnable() {
        public void run() {
            new A19015_Form().setVisible(true);
        }
    });
}

Answer 1

Reading a text file in Java is straight forward (and there are sure to be other, more creative/efficient ways to do this):

try (BufferedReader reader = new BufferedReader(new FileReader(path))) { //try with resources needs JDK 7 

int lineNum = 0; 
String readLine;
while ((readLine = reader.readLine()) != null) { //read until end of stream

Skipping an arbitrary amount of lines can be accomplished like this:

   if (lineNum == 0) {
       lineNum++;
       continue;
   }

Your real problem is the text to split on. Had you been using CSV you could use String[] nextLine = readLine.split("\t"); to split each line into its respective cells based on tab separation. But your not, so you'll be stuck with reading each line, and than find something to split on.

It seems like you're in control of the text file format. If you are, go to an easier to consume format such as CSV, otherwise you're going to be designing a custom parser for your format.

A bonus to using CSV is it can mirror a database very effectivly. I.e. your CSV header column = database column.

As far as databases go, using JDBC is easy enough, just make sure you use prepared statements to insert your data to prevent against SQL injection:

     public Connection connectToDatabase(){
          String url = "jdbc:postgresql://url";
          return DriverManager.getConnection(url);    
     }

     Connection conn = connectToDatabase();
     PreparedStatement pstInsert = conn.prepareStatement(cInsert);
     pstInsert.setTimestamp(1, fromTS1);
     pstInsert.setString(2, nextLine[1]);
     pstInsert.execute();
     pstInsert.close();
     conn.close();

--Edit--

I didn't see your pastebin earlier on. It doesn't appear that you're in charge of the file format, so you're going to need to split on spaces ( each word ) and rely on regular expressions to determine if this is a question or not. Fortunately it seems the file is fairly consistent so you should be able to do this without too much problem.

--Edit 2--

As a possible solution you can try this untested code:

try{
        BufferedReader reader = new BufferedReader(new FileReader("file.txt")); //try with resources needs JDK 7

        boolean doRegex = false;
        String readLine;
        while ((readLine = reader.readLine()) != null) { //read until end of stream
            if(readLine.startsWith("~~ End of Syllabus")){                  
                doRegex = true;
                continue;   //immediately goto the next iteration
            }
            if(doRegex){
                String[] line = readLine.split(" "); //split on spaces
                if(line[0].matches("your regex here")){
                      //answer should be line[1]
                      //do logic with your answer here
                }                   
            }
        }
    } catch (IOException e) {
        e.printStackTrace();  //To change body of catch statement use File | Settings | File Templates.
    }