CODEWITHSUNDEEP
performanceapachehttp-redirecturl-rewritingmax
Randy
Randy

Reputation:

Apache Redirects/Rewrite Maximum

I have a migration project from a legacy system to a new system. The move to the new system will create new unique id's for the objects being migrated; however, my users and search indexes will have the URLs with the old ids. I would like to set up an apache redirect or rewrite to handle this but am concerned about performance with that large number of objects (I expect to have approximatelty 500K old id to new id mappings).

Has anyone implemented this on this scale? Or knows if apache can stand to this big a redirect mapping?

Upvotes: 2

Views: 4103

Answers (4)

jeffatrackaid
jeffatrackaid

Reputation: 346

I see this is an old topic but did you every find a a solution?

I have a case where the developers are using htaccess to redirect more than 30,000 URLs using RedirectMatch in a .htaccess file.

I am concerned about performance and management errors given the size of this file.

What I recommended is that since all of the old urls have:

/sub/####

That they move this to the database and create

/sub/index.php

Redirect all requests for:

www.domain.com/sub/###

to

www.domain.com/sub/index.php

Then have index.php send the redirect since the new URLs and old ids can be looked up in the database.

This way only HTTP requests for the old URLs are hitting re-write processes instead of every single HTTP request.

Upvotes: 1

Gergely Zsamboki
Gergely Zsamboki

Reputation: 313

I had the very same question recently. As I found no practical answer, we implemented an htaccess 6 rules of which 3 had 200,000 conditions.
That means an htaccess file with the size of 150 MB. It was actually fine for half a day, when noone was using this particular website, even though page load times were in the seconds. However next day, our whole server got hammered, with loads well above 400. (machine is 8 cores, 16 GB RAM, SAS RAID5, so no problem with resources usually)

I suggest if you need to implement anything like this. Design your rules, so they don't need conditions, and put them in a dbm rewrite map. this easily solved the performance issues for us.

http://httpd.apache.org/docs/current/rewrite/rewritemap.html#dbm

Upvotes: 2

Gumbo
Gumbo

Reputation: 655489

If you have a fixed set of mappings, you should give a mod_rewrite rewrite map of the type Hash File a try.

Upvotes: 2

skaffman
skaffman

Reputation: 403551

Can you phrase the rewrites using a smaller number of rules? Is there a pattern which links the old URLs to the new ones?

If not, I'd be concerned about Apache with 500K+ rewrite mappings, that's just way past its comfort zone. Still, it might surprise you.

It sounds to me like you need to write a database-backed application just to handle the redirects, with the mapping itself stored in the database. That would scale much better.

Upvotes: 1

Related Questions