Jeffrey Holmes
Jeffrey Holmes

Reputation: 367

Java * Multithreading with MySQL

This is my first post so please let me know if there is anything I should add to the post if needed.

I have a program that grabs urls from my MySQL database and sends them to a class that will download the images to my desktop.

I have 80,000 images I need to get, and I well I want to utilize multithreading to make it quicker.

I have gotten somewhere but I am now stuck and have spent hours researching as I am brand new to multithreading.

The problem is when the program runs, it goes through a loop processing the first 5 entries over and over again. This happens to be the ExecutorFixedPoolSize.

Can someone help me please.

My Code is as follows.

    public static void main(String[] args) {

        try{

            countAllAltPicsNotSaved();
            System.out.println("Not saved: " + totalAltPicsNotSaved);

            Class.forName("com.mysql.jdbc.Driver");
            Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
            Statement s = conn.createStatement ();
            s.executeQuery ("SELECT DISTINCT id from productsaltimages WHERE saved != 1");
            ResultSet rs = s.getResultSet();
            int saveCounter = 0, altImageCount = 0;

            List<Thread> threads = new ArrayList<Thread>();



            while (rs.next()) { //Get ID
                altImageCount = 0; //Product Alt Image Counter
                String id = rs.getString("id");  //Product Table ID

                Statement news = conn.createStatement (); //New Conn for get Alt Images From ID
                news.executeQuery ("SELECT url from productsaltimages WHERE id ='"+id+"' AND saved != 5"); //Gets query from mySQL

                ResultSet newrs = news.getResultSet(); //Resultset for AltImges for ID

                ExecutorService executor = Executors.newFixedThreadPool(5);
                Runnable task = null;

                while (newrs.next()){ //Get Images
                    altImageCount++; //Increment ID Alt Image Counter
                    String url = newrs.getString("url");
                    task = new DownloadImage(url, id, altImageCount);
                    executor.execute(task); 

                }
            }

        } catch (Exception e){
        }

    }

    public static void countAllAltPicsNotSaved() throws Exception {
        try{
            totalAltPicsNotSaved=0;
            Class.forName("com.mysql.jdbc.Driver");
            Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
            Statement s = conn.createStatement ();
            boolean sqlExecuteok = false;
            s.executeQuery ("SELECT * from productsaltimages WHERE saved = '0'");
            ResultSet rs = s.getResultSet ();
            while (rs.next()) {
                totalAltPicsNotSaved++;
            }
            rs.close();
            s.close();
            conn.close();
        } catch (Exception e){
        }
    }


For my DownloadImage class code is: 

    public class DownloadImage implements Runnable
{
   String url, id, threadName;
   private int altImageCount = 0;
   private static String userName = "owner"; 
    private static String password = "hello123";
    private static String dbUrl = "jdbc:mysql://localhost/";
    private static String dbName = "shop"; 
    private static String dbClass = "com.mysql.jdbc.Driver";

   public DownloadImage(String url, String id, int altImageCount)
   {
      this.url = url;
      this.id = id;
      this.altImageCount = altImageCount;
   }
   public void run()
   {
      while(true)
      {

        File file=new File("D:\\FBShop_AltImages\\" + id + "\\");
        boolean exists = file.exists();
        if (!exists) {  
            // Create multiple directories
            boolean success = (new File("D:\\FBShop_AltImages\\" + id + "\\")).mkdirs();
        }

        String newFilename = "D:\\FBShop_AltImages\\" + id + "\\" + id + "_" + altImageCount + ".jpg";

        try {

            try {
                BufferedImage image = null;
                URL imgurl = new URL(url);
                URLConnection con = imgurl.openConnection();
                con.setConnectTimeout(50 * 10000);
                con.setReadTimeout(50 * 10000);
                InputStream in = con.getInputStream();
                image = ImageIO.read(in);
                ImageIO.write(image, "jpg", new File(newFilename));

            } catch (Exception e) {
                System.out.println("Error getting image: " + url);
            }  

            try {
                //UpdateTable
                Class.forName("com.mysql.jdbc.Driver");
                String updatesql = "UPDATE productsaltimages SET saved = '1' WHERE id = '"+id+"'";

                Connection conn =DriverManager.getConnection(dbUrl + dbName, userName, password);
                Statement ups = conn.createStatement ();
                int val = ups.executeUpdate(updatesql);
                System.out.println("Task Complete: " + url);
                try {
                    Thread.sleep(5000);
                } catch (Exception e) {
                }
            } catch (Exception e) {
            }
        } catch (Exception e) {
        }
         //System.out.println("Thread Finished: " + threadName);
      }
   }
}

Upvotes: 1

Views: 732

Answers (1)

Eugene
Eugene

Reputation: 120848

 while(true){
    //lots of code inside you Runnable

    Thread.sleep(5000);
    //After the Thread will sleep it will restart it's work again, and again..
 }

When will this end? Remove the while true and you should be good.

Upvotes: 2

Related Questions