Spider Trap at version 1.0

Johannes Beus
Spider Trap, a project by Thomas Zeithaml – many will know him from forums where he goes by jr-ewing – has now arrived at version 1.0. Spider Trap recognizes crawlers who are not complying with the robots.txt and will automatically block access for those IP-addresses. In this first “full” version many improvements and new features were implemented and it is worthy of a look.
Johannes Beus - on Mon (01/22/2007) at 13:46 PM

Add Comment

more
This posting is older than 30 days and therefore closed for new comments.