Reputation: 32073
I have a file online with information about some Minecraft blocks. When I first made this test file, I gave it three rows and a header expiration date of next Sunday (whenever that may be). My Java app fetched this no problem!
However, now I have inserted three more rows into this small database and changed the expiration date to last week, but my Java app still displays the original 3! When I visit the page in a browser, it gives me the full, current table. How come the Java app is still only fetching the old version?
The key code:
InputStream in;
URLConnection urlc = url.openConnection(); // url is a valid java.net.URL object
urlc.setAllowUserInteraction(false);
urlc.setDoInput(true);
urlc.setDoOutput(false);
urlc.setRequestProperty("User-Agent", "BHMI/3.0.0 (+http://prog.BHStudios.org/BHMI) Java/" + System.getProperty("java.version") + "(" + System.getProperty("java.vm.name") + ")"); // GoDaddy blocks Java clients, so we must have a custom user agent string
urlc.setDefaultUseCaches(false);
urlc.setUseCaches(false);
urlc.connect();
System.out.println("Connection successful! Database expires " + new Date(urlc.getExpiration()));
in = urlc.getInputStream();
int data;
StringBuilder sb = new StringBuilder();
while ((data = in.read()) != -1)
sb.append((char) data);
System.out.println("RAW DATA:\r\n"+sb);
Sample output:
Connection successful! Database expires Tue Nov 26 00:09:05 EST 2013
RAW DATA:
minecraft:air,Air,0,0,,
minecraft:stone,Stone,1,0,2,
minecraft:grass,Grass,2,0,,
I cleared the Java network cache through Windows control panel, and all caches and temporary files on my local machine with CCleaner, but this still happens. Heck, it happens across machines, so it can't be that. I've cleared all edge caches from my server, so it also can't be that. I've even tried downloading the file after telling my browser to use my Java app's User-Agent string, and it fetched all 5 lines.
From my Java app:
GET /http/bhstudios/v2/prog/bhmi/database/get HTTP/1.1
User-Agent: BHMI/3.0.0 (+http://prog.BHStudios.org/BHMI) Java/1.7.0_45(Java HotSpot(TM) 64-Bit Server VM)
Cache-Control: no-cache, must-revalidate, max-age=0, no-store
Pragma: no-cache
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Connection: close
Host: BHStudios.org
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
From Chrome, spoofing the same User-Agent string:
GET /prog/bhmi/database/get/ HTTP/1.1
Host: prog.bhstudios.org
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
User-Agent: BHMI/3.0.0 (+http://prog.BHStudios.org/BHMI) Java/1.7.0_45(Java HotSpot(TM) 64-Bit Server VM)
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Cookie: __cfduid=dc9d0394ed55ebb1214fcbb5fc825626b1385426208553; visitorId=5293ed2b758cb1b5620000b0
From my Java app:
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Tue, 26 Nov 2013 02:17:39 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d4432e3d81cf9e5b9393f2cca483e4b2d1385432256651; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.bhstudios.org; HttpOnly
X-Powered-By: ASP.NET
X-UA-Compatible: chrome=IE8
CF-RAY: d33155416660862
Note that suspicious cookie expiration expires=Mon, 23-Dec-2019 23:50:00 GMT
. Could this be the cause?
I also note that, when fetching from Chrome and using the same User-Agent string as my app, the header is:
HTTP/1.1 200 OK
Server: cloudflare-nginx
Date: Wed, 27 Nov 2013 17:30:01 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-cache, must-revalidate, max-age=0, no-store
Pragma: no-cache
Expires: Mon, 18 Nov 2013 10:30:01 America/Phoenix
Content-Description: File Transfer
Content-Disposition: attachment; filename=BHMI_Items_Vanilla_172.csv
Content-Transfer-Encoding: base64
X-Powered-By: ASP.NET
X-UA-Compatible: chrome=IE8
CF-RAY: d408b3c56320098
Content-Encoding: gzip
which is the intended header, with an expiration date of last week.
Upvotes: 0
Views: 207
Reputation: 32073
... I was requesting the wrong file. Sorry for wasting your time >.<
As the header shows, I was addressing an old URL scheme, /http/bhstudios/v2/prog/bhmi/database/get
, when I wanted /prog/bhmi/database/get
Upvotes: 0
Reputation: 10707
You have to put in your request header info that you are looking for data that are not cached:
urlc.setRequestProperty("Cache-Control","no-cache, must-revalidate"); //HTTP 1.1
urlc.setRequestProperty("Pragma","no-cache"); //HTTP 1.0
Upvotes: 2