Accurately Scraping Google Serps for Local SEO Terms?

dchuk

dchuk

Senior Botter
Oct 30, 2008
7,044
224
0
San Diego,CA
serpiq.com
I'm going to need to start tracking thousands of keywords automatically to monitor when they hit the front page. All of the keywords are local SEO terms ( location + niche ).

Now I'm nowhere near an expert on this, but I know that Google has been pushing to localize everything for people. So different datacenters can return different results based on locations.

I'm planning to do this scraping through proxies to be safe (all US based).

Are there any techniques to use to ensure that I'm getting accurate local results? Can I target certain datacenters based on the local keyword location?

Is this even a concern?
 


You'll need to use a proxy that is in your local area most likely. You can't really tell google where you are, they go by your IP.
 
A few more notes on this. If you don't want to use the preview tool, you can get the same functionality by adding a few parameters to the query string. So for example, if you wanted to see the AdWords placements for "camping tents" in Chicago, the query string would look like this:

camping tents - Google Search

Additional Query String Parameters:

?adtest=on
&gr=<Country/State Code>
&gcs=<City Name>
 
  • Like
Reactions: uplinked
If you want to get the stuff that fires in the local blended maps, you can do that too as a separate process. The query below would yield painting contractors in Chicago:

hxxp://maps.google.com/maps?hl=en&num=100&um=1&ie=UTF-8&q=painting+contractors&fb=1&gl=us&hq=painting+contractors&hnear=Chicago,+IL&view=text&ei=FO9US--QPMmUtgfKqbQB&sa=X&oi=local_group&ct=more-results&resnum=4&ved=0CDMQtwMwAw
 
  • Like
Reactions: Grindstone and seocracy
nvanprooyen said:
A few more notes on this. If you don't want to use the preview tool, you can get the same functionality by adding a few parameters to the query string. So for example, if you wanted to see the AdWords placements for "camping tents" in Chicago, the query string would look like this:

camping tents - Google Search

Additional Query String Parameters:

?adtest=on
&gr=<Country/State Code>
&gcs=<City Name>
Click to expand...

This looks really promising. I'm looking to do this for organic SEO results, not PPC stuff, so I'm wondering how reliable that adwords preview tool is?

Also, assuming I start passing in location codes like that, do you think the IP would still contribute to the results? Or is this preview tool designed to avoid the IP since it's meant for geotargeting anyways?
 
In playing with it for just a few seconds, it looks like using "gll=" allows you to pass in lat/long, which is probably what I would go with because I'm already geocoding/storing that info. No need for figuring out all of the codes.

The question still stands though, how accurate are the organic serps on those preview pages? Can I trust them?
 
Just an update to this thread, I'm hoping to begin coding this tool this weekend, potentially next weekend, so as I start digging in I'll drop some code and results here for anyone who's interested
 
painting contractor

nvanprooyen said:
If you want to get the stuff that fires in the local blended maps, you can do that too as a separate process. The query below would yield painting contractors in Chicago:

hxxp://maps.google.com/maps?hl=en&num=100&um=1&ie=UTF-8&q=painting+contractors&fb=1&gl=us&hq=painting+contractors&hnear=Chicago,+IL&view=text&ei=FO9US--QPMmUtgfKqbQB&sa=X&oi=local_group&ct=more-results&resnum=4&ved=0CDMQtwMwAw
Click to expand...

nvanprooyen, I am a local painting contractor in massachusetts. I would like to learn more about what you explained here about showing up in the local maps. I am relatively new to internet marketing but have done very minor dabbling in AM
 
dchuk said:
Just an update to this thread, I'm hoping to begin coding this tool this weekend, potentially next weekend, so as I start digging in I'll drop some code and results here for anyone who's interested
Click to expand...
Definitely interested. Will either chat w/ you on Google or here. Either way. I have some interesting ideas around local data mining.
 
this is a project I definitely need to get back to. Based on what nvan said, you can modify that ranking tracker ruby script i posted a while back and can probably have a solid start.

I'll get a better solution up soon.

Fucking time, never enough of it
 
I was looking for this thread the other day. Better bookmark it before I lose it again.
 
You must log in or register to reply here.
Top Bottom