Scraping Craigslist Results

R

Romuloux

New member
Mar 29, 2012
15
0
0
I recently built a CL scraper for my own personal use and as a first Ruby on Rails learning project.

I would like to let a few friends and family use it but I am not sure what sets off their scraping detection. I don't have any kind of IP anonymity functions built into right now because it was just a person project.

Is this allowed?

I have been reading up on the 3Taps case and it seems to me that they can be barred from accessing Craigslist directly and scraping the data.

3Taps said they were using things like Google cache to scrape the data secondhand so they technically weren't accessing CL servers.

What do you guys think? Can they really bar you from accessing publicly available data?
 


Are you serious?

YES they can ban you from accessing THEIR site.

If you built a scraper that scrapes craigslist and WORKS and you haven't run into issues then your scraper must be VERY slow or you haven't even tested it "in the wild" yet.
 
Ok, probably worded it badly. What I mean is, do you think using a proxy service to circumvent an IP ban is actually illegal in the US? Or did I miss the point of what they are being taken to court over.

Also, like I said, the scraper is a hobby for personal use that doesn't need to make very many requests, certainly not more than a human could make in the same amount of time.
 
Doing something illegal, and then worrying about HOW you do it illegally seems kind of pointless does it not?
 
I've been doing it since 2008. they finally stepped it up. put yourself in their admin's shoes and assume he/she isnt retarded. Then, don't be retarded, so you can outthink their admin. Thats how you fix it.
 
You must log in or register to reply here.
Top Bottom
Back