Scraping sites

H

hmmburger

New member
Jun 3, 2008
75
1
0
Hey guys.

What would be the best way to scrape product sites for data? I'm not a coder at all, so perhaps something like uBot would be ideal for me?

I'm talking about product sites where you can pull the product name, image and other data from. So take Google Shopping, you can scrape the product name, image, description and any other data provided?

Have had a long look through the uBot thread and sounds like an ideal choice but figured I'd see if there's any other solutions...

TIA.
 


Thanks for the link bud but not really where I intend to go with this.

What I need is the data in a nice easy excel sheet that's all. Don't need to be fucking about with databases etc. Although I will be utilizing databases just not in that manner.

Just a case of going on the page, finding each product on there, pulling each parameter of data I specify and exporting in a nicely organized .csv file.
 
Regardless of whether or not you are dealing with a relational database or a .csv file, you still follow the same general practice of scraping the data.

All you change is how you store it. Instead of pushing the data into a relational database, you'll be formatting it and then writing it to a file.
 
Rexibit said:
Regardless of whether or not you are dealing with a relational database or a .csv file, you still follow the same general practice of scraping the data.

All you change is how you store it. Instead of pushing the data into a relational database, you'll be formatting it and then writing it to a file.
Click to expand...

I may be missing the point here in which case I apologize dude, lack of sleep 'n that.

That thread talks about some of the ideas / uses and some links for DB's out there.

In terms of getting the data, that thread talks about coding and using PHP or whatever language to get it done?

For a code n00b like myself something like uBot could do the trick?
 
Yep ubot can do that but be warned that right now it can only run with 1 thread so if you're looking to scrape more than 10,000 pages you're better off with one of the commercial scrapers or paying somebody else to do it for you.
 
Appreciate the heads up. Seeing as there's no immediate rush for what I need it may be worthwhile learning it myself, especially if it's more simpler than I imagine it to be.

Might also look to buy the book referenced in that thread (http://www.wickedfire.com/traffic-content/101961-database-sites.html#post991265) and perhaps I will be able to do it myself.

Actually I'm glad you've pointed me in this direction. It really would be a huge advantage to my IM shit if I could code heh.

Thx guys.
 
You must log in or register to reply here.
Top Bottom
Back