CODEWITHSUNDEEP
javajsoup
user2817587
user2817587

Reputation: 23

How do I insert scanned urls into url table?

I am writing a web crawler and I am having some trouble. Here is some pseudo code of what I am trying to do:

for every url in url-list {
    urlid = NextURLID;

    Insert urlid and url to their respective columns URL table

    NextURLID++;
}

Here is what I have so far:

void startCrawl() {
        int NextURLID = 0;
        int NextURLIDScanned = 0;

        try
        {
            openConnection(); //Open the database
        }
        catch (SQLException | IOException e)
        {
            e.printStackTrace();
        }

        String url = "http://jsoup.org";
        print("Fetching %s...", url);

        Document doc = Jsoup.connect(url).get();
        Elements links = doc.getElementsByTag("a");

        for (Element link : links) {
            urlID = NextURLID;

            //Code to insert (urlid, url)  to the URL table

            NextURLID++;

        }
}

As you can see I do not have the code to insert the url into the table yet. I would think it would be something like this: 

stat.executeUpdate("INSERT INTO urls VALUES ('"+urlID+"','"+url+"')");

But how do I overwrite the urlID and url variables with the new urls each loop iteration? Thanks

Upvotes: 0

Views: 55

Answers (1)

Stephan
Stephan

Reputation: 43033

In your case a PreparementStatement is more appropriate:

String insertURL = "INSERT INTO urls(urlID, url) VALUES (?, ?)";
PreparedStatement ps = dbConnection.prepareStatement(insertURL);

for (Element link : links) {
     ps.setInt(1, NextURLID);
     ps.setInt(2, link.attr("abs:href"));  
     ps.executeUpdate();

     NextURLID++;
}

// ...

Upvotes: 1

Related Questions