09-14-2011, 03:19 AM
This is a very large file.
I recommend that you divide it into smaller groups and check for duplicates using this free Scrapebox program.
You can get it here.
Code:
http://hotfile.com/dl/129676299/33c2916/2381167_unique_edu__gov_urls.txt.zip.html
I recommend that you divide it into smaller groups and check for duplicates using this free Scrapebox program.
You can get it here.
Code:
http://www.scrapebox.com/free-dupe-remove