07-01-2013, 09:21 AM
Hi guys.
I need to download all files from a website. The problem is that they
use several GET request to different PHP scripts to get them.
First, it sends something like this to get to the file's download page: GET /download/315532/0 HTTP/1.1
Then, on the send page, you need to click on download again and you send
this: GET /download.php?file=FILE_NAME.ZIP&646965504532567d59
54262e5a=1 HTTP/1.1
Lastly, on the third page, you send this via GET: DOMAIN-NAME/download.php?file=FILE_NAME&646965504532567d595426 2e5a=1
The question is which software can crawl to the whole site and download all .zip/.rar files?
I've tried several web spiders, none seem to work.
I need to download all files from a website. The problem is that they
use several GET request to different PHP scripts to get them.
First, it sends something like this to get to the file's download page: GET /download/315532/0 HTTP/1.1
Then, on the send page, you need to click on download again and you send
this: GET /download.php?file=FILE_NAME.ZIP&646965504532567d59
54262e5a=1 HTTP/1.1
Lastly, on the third page, you send this via GET: DOMAIN-NAME/download.php?file=FILE_NAME&646965504532567d595426 2e5a=1
The question is which software can crawl to the whole site and download all .zip/.rar files?
I've tried several web spiders, none seem to work.