CODEWITHSUNDEEP
xmlsitemap
gnarkill619
gnarkill619

Reputation: 23

XML Sitemap Generator for URL with 1.5 million pages?

I am trying to create an XML Sitemap for Bloomberg Businessweek, specifically all pages that begin with:

http://investing.businessweek.com/research/stocks/private/snapshot.asp?privcapId=

It's my understanding that there are approximately 1.5 million pages that begin with this prefix. Is it even possible to create an XML Sitemap for this many pages?

Upvotes: 2

Views: 1975

Answers (2)

Maviq Software
Maviq Software

Reputation: 11

Yes. You can do it using xml generator.

Upvotes: 0

Christian Kuetbach
Christian Kuetbach

Reputation: 16060

Yes it is possible. It is just a question of writing a crawler (if you don't have access to the Database with the content) and the generatiion of a xml file. (Don't use an DOM based XML api, because of the memory consumption for large files).

But what do you want to do with a File of min 100 MB size [1].

I don't think that this file would be helpful at all.

1) explaination:

1.500.000 pages * (length of URL + 100 Bytes XML overhead)

Upvotes: 1

Related Questions