How to block a certain type of urls on robots.txt or .htaccess?

Member

by jasen , in category: Third Party Scripts , a month ago

How to block a certain type of urls on robots.txt or .htaccess?

Facebook Twitter LinkedIn Telegram Whatsapp

1 answer

Member

by darion , a month ago

@jasen 

To block a certain type of URLs on robots.txt, you can use the "Disallow" directive followed by the URL pattern you want to block. For example, to block all URLs that contain the word "example", you can add the following line to your robots.txt file:

1
Disallow: /example/


To block a certain type of URLs in .htaccess, you can use the "RewriteRule" directive to redirect or block specific URLs. For example, to block all URLs that contain the word "example", you can add the following lines to your .htaccess file:

1
2
RewriteEngine On
RewriteRule ^(.*)example(.*)$ - [F]


This will return a 403 Forbidden error for any URLs that contain the word "example". Make sure to test these changes thoroughly to ensure they are working as expected.