Script to extract contact info...

[nvanprooyen]

Basically, I want to feed in a set domains and crawl those for email patterns and extract. If that's not possible, I'm probably going to try and get something written that will get around captchas on who is records. Anyone know of anything? Or a better way to go about this?

 

[BlueMendicant]

Combine the information found at How to Find or Validate an Email Address and Python A Simple Web Crawler

Start it on some directory like dmoz.

 

[Deliguy]

man i swear i had a post on this but i guess i never hit publish

In the gist:
go to a free script site like hotscripts and grab a simple whois lookup script
grab a list of nic servers and throw them in.
In the portion of the script that prints out the contact info grab the variable that holds the email address and write that out to a flat file or your db (whatever you're using).
Remove the variable that pulls the domain to lookup from the form and replace it with an array of all the domains you want to pull.
Then put the whole script in a while loop to go through the array.

it's easier than it sounds and better than starting from scratch.

 

[dogfighter]

There are tons of pre-built programs and scripts like this out there, unless you're absolutely married to the idea of writing your own.

 

[nvanprooyen]

man i swear i had a post on this but i guess i never hit publish

In the gist:
go to a free script site like hotscripts and grab a simple whois lookup script
grab a list of nic servers and throw them in.
In the portion of the script that prints out the contact info grab the variable that holds the email address and write that out to a flat file or your db (whatever you're using).
Remove the variable that pulls the domain to lookup from the form and replace it with an array of all the domains you want to pull.
Then put the whole script in a while loop to go through the array.

it's easier than it sounds and better than starting from scratch.

That makes sense conceptually. I'll see if I can pull it off. Thanks Eli.

 

[nvanprooyen]

There are tons of pre-built programs and scripts like this out there, unless you're absolutely married to the idea of writing your own.

I'm not married to the idea of writing it myself. I just need a solution. I have a bunch of domains (thousands) I need contact info for and am trying to find a way to feed that array in and do mass whois lookups on it and exract the email. What Eli said above sounds like it will work. If you know of something that can do this out of the box, I'm all ears.