CODEWITHSUNDEEP
nutchnutch2
Ravi Kiran
Ravi Kiran

Reputation: 65

I had some questions on db_redir_temp

I had injected some urls to crawl that is one round and I found some urls as db_redir_temp. {"url":"http://www.universityhealth.org","pst":"temp_moved(13), lastModified=0: https://www.universityhealth.org/"} {"url":"http://silvercappartners.com","pst":"temp_moved(13), lastModified=0: http://silvercappartners.com/index.html"}

may i know that the http://www.universityhealth.org is pointing to same url why it is showed db_redir_temp. This url is pointing to http://silvercappartners.com to this url http://silvercappartners.com/index.html should I consider the pst column will give the redirected url page.

Upvotes: 0

Views: 44

Answers (1)

Sebastian Nagel
Sebastian Nagel

Reputation: 2239

The two URLs

http://www.universityhealth.org
https://www.universityhealth.org/

differ in one important point, the protocol (or scheme) - http vs. https. These are not always equivalent, eg. a web server may not support https. The other point (the trailing /) is irrelevant, the HTTP request for both the empty path and the server root path is GET / HTTP/1.1 (maybe using a different protocol version).

But true reason is simply that the server responded with HTTP/1.1 302 Found which is a redirect, see HTTP 302.

The "pst" or "protocol status" metadata field may include a message. For redirects it contains the redirect target.

Upvotes: 1

Related Questions