CODEWITHSUNDEEP
seorobots.txt
Jason McCreary
Jason McCreary

Reputation: 72971

robots.txt disallow path containing a URL parameter regardless of order

The Problem

I am trying to block a path that contains a certain URL parameter with robots.txt. I want to block the path regardless of where this URL parameter appears in the query string.

What I have tried

After reading several answers, I have tried:

Disallow /*?param=

Also:

Disallow /*?param=*

These will only block the path if param is the first URL parameter. But not if it appears later in the URL.

I also tried:

Disallow /*?*param=*

While this works, it also blocks any path that has a URL parameter with suffix of param and as such is not an acceptable solution.

The Question

How can I block a path that contains a specific URL parameter regardless of where it appears in the query string?

Upvotes: 1

Views: 1292

Answers (1)

plasticinsect
plasticinsect

Reputation: 1752

If you want to block:

/path/file?param=value
/path/file?a=b&param=value

but you do not want to block:

/path/file?otherparam=value
/path/file?a=b&otherparam=value

You need to use two disallow lines, like this:

User-agent: *
Disallow: /*?param=
Disallow: /*&param=

There is no way to do this reliably with a single line.

Upvotes: 3

Related Questions