Price Scraping Script - Any recommendations?

[MikevanderHeijd]

Top of the day brethren!

Basically what I'm looking for is a script (or WP plugin) that can scrape product prices of sites that I enter.

As you can tell, I am kinda new to all of this scraping/automating process of websites.

Any advice would be greatly appreciated

 

[hehejo]

Sounds like you need someone to code you a custom solution unless you want to learn coding yourself.

 

[pinchyfingers]

I would start here: lxml.html and here: Requests: HTTP for Humans — Requests 0.14.1 documentation

If you're willing to put in a small amount of work, you can piece together a simple scraper pretty easily.

 

[computergames]

I doubt you wil find that as a WP plugin...

 

[UNIXy]

Unless all these websites display prices in a uniform way with some valid structure (think XML schema) it won't be possible to get these prices reliably.

Regards

 

[Mahzkrieg]

Bro, at one point you've gotta stop thinking about the world in terms of Wordpress Plugins and start investing in yourself.

require "open-uri"
require "nokogiri"

keywords = "tickle me elmo"
url = "http://www.amazon.com/s/keywords=#{URI.escape(keywords)}"

doc = Nokogiri::HTML(open(url))
puts doc.at("#result_0").to_s[/\$\d+\.?\d{0,2}/]

This Ruby script launches an Amazon search for arbitrary keywords and displays the first "$<digits>.<digit><digit>" match that it finds in the first result.

 

[MikevanderHeijd]

Thanks for the input guys, @Mahzkrieg sounds easy enough