Reputation: 85765
I have some urls in the database. The problem is there urls are urls that redirect to what I want.
I have something like this
http://www.mytestsite.com/test/test/?myphoto=true
now if I go to this site it would do a redirect to the photo so the url would end up being
http://www.mytestsite.com/test/myphoto.jpg
Is it possible to somehow scrape(download) through C# and then have it redirect and get the real url so I can download the image?
Upvotes: 3
Views: 8193
Reputation: 30443
I had issues trying to get HttpWebRequest
to always fully redirect when using it with SharePoint external URLs; I simply couldn't get it to work.
After a lot of faffing about I discovered that this can be done with WebClient
too and that proved more reliable for me.
To get that to work with WebClient
you seem to have to create a class that derives from WebClient
so that you can manually force AllowAutoRedirect
to true.
I wrote up a bit more on this in this answer, which borrows its code from this question.
The key code was:
class CustomWebclient: WebClient { [System.Security.SecuritySafeCritical] public CustomWebclient(): base() { } public CookieContainer cookieContainer = new CookieContainer(); protected override WebRequest GetWebRequest(Uri myAddress) { WebRequest request = base.GetWebRequest(myAddress); if (request is HttpWebRequest) { (request as HttpWebRequest).CookieContainer = cookieContainer; (request as HttpWebRequest).AllowAutoRedirect = true; } return request; } }
Upvotes: 3
Reputation: 11201
I think you are after the HttpWebRequest.AllowAutoRedirect Property. The property gets or sets a value that indicates whether the request should follow redirection responses.
Example taken from MSDN
HttpWebRequest myHttpWebRequest=(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
myHttpWebRequest.MaximumAutomaticRedirections=1;
myHttpWebRequest.AllowAutoRedirect=true;
HttpWebResponse myHttpWebResponse=(HttpWebResponse)myHttpWebRequest.GetResponse();
Upvotes: 7