Reputation: 31
I'm using http client to get data:
public static String getHttpResponse(String url) {
//LOGGER.info("Download page context from URL " + url);
String httpClientResponse = null;
try {
URI uri = new URIBuilder(url).build();
HttpResponse response;
HttpHost target = new HttpHost(uri.getHost());
HttpGet request = new HttpGet(uri);
//request.setConfig(config);
request.addHeader(new BasicHeader("User-Agent", "Mozilla/5.0"));
request.addHeader(new BasicHeader("Content-Type", "text/html"));
request.addHeader("Accept-Ranges", "bytes=100-1500");
org.apache.http.client.HttpClient
client = HttpClients.custom().build();
response = client.execute(target, request);
//LOGGER.info("Status Line for URL {} is {}", uri.getHost() + File.separator + uri.getPath(), response.getStatusLine());
InputStream inputStream = response.getEntity().getContent();
if (inputStream == null || response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {
/*LOGGER.error("Non-success response while downloading image. Response {}", response.getStatusLine());
LOGGER.error("Error while download data from url {}", url);*/
} else {
httpClientResponse = IOUtils.toString(inputStream, CharEncoding.UTF_8);
}
} catch (Exception e) {
System.out.println("Error while download content from URL");
}
return httpClientResponse;
}
Also: Can we do this using Jsoup?
Thanks.
Upvotes: 0
Views: 5009
Reputation: 2246
Replace:
request.addHeader("Accept-Ranges", "bytes=100-1500");
with:
request.addHeader("Range", "bytes=100-1500");
The Accept-Ranges
header is part of server response, which indicates that the server accepts partial requests.
In your request you should use Range
header, which indicates which part of document server should return.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Ranges https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range
Upvotes: 1