Restrict access to x pages by IP address to prevent abuse and harvesting

view full story

http://www.linuxquestions.org – We have a Linux (GNU/Linux 3.6.6) web server with hundreds of sites and some of these sites have hundreds of millions of html pages. As you can imagine we have occasional abuse problems and harvesting problems. So we are looking for a way to restrict access to some of these domains (web sites) by limiting the access to about 500 pages per 24 hour period per IP address. This will ensure that the people consulting the sites are consulting them for valid reasons and are not harvesting the data. Can someone please direct me to a document or html page that explains how to restrict access to a we (HowTos)