Best Blackhat Forum

Full Version: External Link Spidering
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
....so I'm crawling pages searching for external links. Mostly to file sharing websites. I'm coming across one that is more and more common now:

wcrypt.com

They load the file sharing sites URL within a frame. Currently, I right click the page to view frame source, but that's manual and takes forever when you have 10,000 links to follow.

Any idea of a way to automate this a bit faster?

Thanks so very much!

Alex A.
You need to write code to get url in the frame and you can get the frame source url code
That's why I use Screaming Frog SEO spider for it. I'm not really a programmer. :) I was hoping there was a way within that program, or an alternative......
Reference URL's