[REQ] PHP Web Crawler

[donjoe1]

PHP Web Crawler is a software that searches for
links in the web. It stores the links and some extra data in a database
and shows them as HTML output.


Features:

• The crawler can be run as multiple instances
• It can be run by a cron job.
• Crawl results are saved in a MySQL database. It generates the table "urls" to store the crawls.
• For each url it saves the url of source, the url of the destiny and the anchor text.
  - Validates the urls via a regular expression. It
  avoids the links to static data into the site. Including the unnecessary
  media files. Despite this I can't ensure that the crawler avoids all
  the media files. That be more complex to validate.

http://www.binpress.com/app/php-web-crawler/113

thanks