5

How do I get wget to download a cgi file behind robots.txt?

view full story
linux-howto

http://unix.stackexchange.com – As an example - http://aok.heavengames.com/cgi-bin/aokcgi/display.cgi?action=t&fn=22. I found a way to get through the robots.txt restrictions, but even then, it just downloads a binary file that's unreadable by anything. (HowTos)