CODEWITHSUNDEEP
javaweb-scraping
Skaros Ilias
Skaros Ilias

Reputation: 1068

download a file using Java, no api, strange error

I am trying to download a file from this site, by clicking on the Excel icon. By right clicking on the icon I got the link which I pasted to my java program as such:

public static void main(String[] args){

    BufferedReader br;
    String thisLine="";
    String file="";
    try {
      // connect and download the file
      ReadableByteChannel rbc;
      file="test.xls";
      URL website = new URL("http://www.anaptyxi.gov.gr/DesktopModules" +
                "/AVMap.ErgaReports_v2/SearchHandler.ashx?lang=el-GR" +
                "&pageMode=3&searchValue=&searchField=&dateFrom=&dateTo=" +
                "&perioxesMode=2&selectedPerioxes[]=01_36_514&ergaType[]=1" +
                "&ergaType[]=2&ergaType[]=3&enisx=&kad=&company=&includePollaplhs=1" +
                "&export=xls");
      rbc = Channels.newChannel(website.openStream());

      FileOutputStream xls = new FileOutputStream(file);  
      xls.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);             
      xls.close();               
    } catch (FileNotFoundException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }   
}

This creates an Excel file, but it only contains the string:

Error executing the request. Try limiting your criteria. Any ideas?

Upvotes: 1

Views: 300

Answers (3)

Daniel Sperry
Daniel Sperry

Reputation: 4491

This is how it should look like if using java 8.

import java.io.IOException;
import java.net.URL;
import java.nio.file.FileSystems;
import java.nio.file.Files;

public class Main {
    public static void main(String[] args) {

        try {
            // connect and download the file
            URL website = new URL("http://www.anaptyxi.gov.gr/DesktopModules" +
                    "/AVMap.ErgaReports_v2/SearchHandler.ashx?lang=el-GR" +
                    "&pageMode=3&searchValue=&searchField=&dateFrom=&dateTo=" +
                    "&perioxesMode=2&selectedPerioxes[]=01_36_514&ergaType[]=1" +
                    "&ergaType[]=2&ergaType[]=3&enisx=&kad=&company=&includePollaplhs=1" +
                    "&export=xls");
            URLConnection con = website.openConnection();
            con.addRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
            con.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36");

            Files.copy(con.getInputStream(), FileSystems.getDefault().getPath("test.html"));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Upvotes: 0

Daniel Sperry
Daniel Sperry

Reputation: 4491

The site is probably checking the User-Agent header and not responding because you're using java.

This should fix it:

   public static void main(String[] args){

   BufferedReader br;
   String thisLine="";
   String file="";
   try {
       //   connect and download the file
      ReadableByteChannel rbc;
      file="test.xls";
      // connect and download the file
      URL website = new URL("http://www.anaptyxi.gov.gr/DesktopModules" +
                        "/AVMap.ErgaReports_v2/SearchHandler.ashx?lang=el-GR" +
                        "&pageMode=3&searchValue=&searchField=&dateFrom=&dateTo=" +
                        "&perioxesMode=2&selectedPerioxes[]=01_36_514&ergaType[]=1" +
                        "&ergaType[]=2&ergaType[]=3&enisx=&kad=&company=&includePollaplhs=1" +
                        "&export=xls");

      // Adding request headers to mimic the browser
      URLConnection con = website.openConnection();
      con.addRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
      con.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36");
      rbc = Channels.newChannel(con.getInputStream());  // !!!


      FileOutputStream xls = new FileOutputStream(file);  
      xls.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);             
      xls.close();               
    } catch (FileNotFoundException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

Upvotes: 1

Scott Sosna
Scott Sosna

Reputation: 1413

There's something wrong with your URL. Usually when making an HTTP-type request, you need to check the response code, if it's not 200 something bad happened. Besides a possible bad response code, looks like the service is returning an error also in the body instead of the expected data, perhaps that the query requested is going to return too much data, hence the "try limiting the criteria."

Read up on SLA defined with the service you're calling, it'll likely say something about how much data can be returned in any one call

Upvotes: 0

Related Questions