I'm looking for someone to create a script/program/whatever that will crawl through myspace's high school database collecting all the ID's that meet a certain criteria.
I would want the script to search by state and then 'crawl' through all of the cities and highschools within those cities. The criteria I would need to choose from is age and gender. It would then save all the IDs to a file in the following form:
<id1>
<id2>
<id3>
The program should also be able to pause and resume from where it left off. Or if you could make it run from a webserver and save it to a database that would work too.
I would like the files to be seperated by state and city: i.e. I could select all IDs from Wisconsin, as well as specific as Madison, Wisconsin (city, state). This would be in addition to displaying all IDs. The option to save them in a text file in the format shown above would be needed.
I realize that these thoughts are probably scrambled and you will have plenty of questions for me so feel free to ask away- I will most likely respond back within a few minutes.
Thanks,
-Chris
P.S. There is a chance that I will need updates done to this crawler as myspace updates their system. These will be paid updates.
I would want the script to search by state and then 'crawl' through all of the cities and highschools within those cities. The criteria I would need to choose from is age and gender. It would then save all the IDs to a file in the following form:
<id1>
<id2>
<id3>
The program should also be able to pause and resume from where it left off. Or if you could make it run from a webserver and save it to a database that would work too.
I would like the files to be seperated by state and city: i.e. I could select all IDs from Wisconsin, as well as specific as Madison, Wisconsin (city, state). This would be in addition to displaying all IDs. The option to save them in a text file in the format shown above would be needed.
I realize that these thoughts are probably scrambled and you will have plenty of questions for me so feel free to ask away- I will most likely respond back within a few minutes.
Thanks,
-Chris
P.S. There is a chance that I will need updates done to this crawler as myspace updates their system. These will be paid updates.