If you "Data Mine" using tools such as Web-Harvester or others that are Java Based, it is difficult to get in the front door of many sites because your user-agent string says something like this:
Java/1.6.0-rc
This is immediately detected and you are refused. Unfortuantly if you attempt to...