I'm Overpaying for My Server

cardine

cardine

...
Jan 9, 2008
3,582
77
0
wordai.com
Recently I got a 8 Core (Dual Quad Core) Harpertown server with 8GB of RAM to run some of my iMacros stuff. I figured that the server would be plenty powerful to run everything. However, running 15 iMacros browses concurrently atm (each with 8 tabs open at the same time... and all of these tabs are resource intensive flash sites) I'm running into performance issues. My total CPU usage is only ~25%. However one of my processors (the first one) is running at 100% usage, while all of the other processors are only running at ~5% each.

This makes me feel like I'm not getting nearly enough out of my server, and that iMacros is just running on the 1 processor... which means I'm wasting my money.

Am I correct in thinking this? Is there anything I can do to try to split up the load between the different cores? Since I'm running 15 iMacros browsers, is there anyway I can get maybe 2 of them running on each core or something like that? Also if the main resource hog is the flash on all of the sites iMacros is browsing, is there anyway to get the server to split that up between all 8 cores?

If not, would I just be better of getting 3-4 smaller servers and run iMacros on all of them at the same time?
 


Split them up with virtual machines,
it will manage the cores in a much more efficient manner.
 
I currently have 5 different Windows users logged in at the same time (I'm running Windows 2008), each of which is running 3 instances of iMacros.

Would this count as a virtual machine? Or would each virtual machine have Windows separately installed on it (like a VPS)? Is there an affordable (and easy to install/setup) Windows option that can let me run 5 different virtual machines?
 
There are better ways to scrape (I'm assuming you're scraping) using a windows server than using iMacros.

I've actually scraped a ton of records using command line tools from a 1-gig laptop over a Verizon air card. Oh sure it took a long time but I wanted to fly a little below the radar and not tax their server too much either.
 
cardine... i dunno it will work or not, but have you try to set different affinity for each process? (open task manager -> process list -> right click the process -> set affinity)
 
FortressDewey said:
VMware is a good software and will definitely help. Free VMware ESXi: Bare Metal Hypervisor is where to get it, and best of all, it's free.
Click to expand...
eliquid said:
still gotta pay licenses for the extra windows OS right? unless he running linux in the VMware?
Click to expand...
That is my concern as well. I'm paying $30/mo for Windows, and I don't want to be paying $150/mo just in licensing fees. Would I only need 1 Windows license to do this?

LotsOfZeros said:
There are better ways to scrape (I'm assuming you're scraping) using a windows server than using iMacros.

I've actually scraped a ton of records using command line tools from a 1-gig laptop over a Verizon air card. Oh sure it took a long time but I wanted to fly a little below the radar and not tax their server too much either.
Click to expand...
I'm not doing scraping. Beyond that I can't say anything.

gondreus said:
cardine... i dunno it will work or not, but have you try to set different affinity for each process? (open task manager -> process list -> right click the process -> set affinity)
Click to expand...
I never thought to do that, and it seems like the easiest way. Would this work? Also if flash is causing most of the CPU usage, would changing an iMacros instance to run on a different CPU also make the flash run on that same CPU? Or will the flash stay on CPU 1? Also is there a command line way to do this?
 
If you have Windows 08 Datacenter you can do as many windows instances as you want. I recently put together a quote and had to do a lot of research on it.

Perhaps do a single xeon chip with hyper v 5405,20, 5520 etc 4 gig ram (+or-).

Unfortunately, Windows screws ya on their licensing. Datacenter's don't make any money off of it, and when I mean any, I mean nada.
 
subigo said:
Sounds like you're running programs that are single-threaded and can't take advantage of additional cores.
Click to expand...

If a program is single-threaded if I were to change its affinity would it be able to run on one of the other 7 cores? Since I have 15 iMacros instances running, even though they are all individually single threaded, could I make 2 of them have an affinity for each of the 8 cores and would that evenly split up the load that way?

2 iMacros Browsers with Affinity for 1st Core
2 iMacros Browsers with Affinity for 2nd Core
etc.

Would that work?
 
cardine said:
If a program is single-threaded if I were to change its affinity would it be able to run on one of the other 7 cores? Since I have 15 iMacros instances running, even though they are all individually single threaded, could I make 2 of them have an affinity for each of the 8 cores and would that evenly split up the load that way?

2 iMacros Browsers with Affinity for 1st Core
2 iMacros Browsers with Affinity for 2nd Core
etc.

Would that work?
Click to expand...

You should be able to... Just open the task manager and click on "Set Affinity" (right click on the program) and choose the core. At least that's what I think you do in Windows.
 
subigo said:
You should be able to... Just open the task manager and click on "Set Affinity" (right click on the program) and choose the core. At least that's what I think you do in Windows.
Click to expand...
That should certainly work just fine!
 
He mentioned Flash, so I'm guessing social games, videos, or gambling.

Find a real hacker that'll let you get under the hood of the Flash so you don't need all that hardware.
 
I tried setting the affinity. It works at first, but after ~30 seconds iMacros reverts back to using only the 1st core. I see my CPU usage climb from 20% to 95%, and then after 30 seconds drop back down to 20% (since its all gone back to the first core). I've sent an email to iMacros asking why this could be. I could write a bot that automatically changes the affinity every 15 seconds (I'm trying to find a dos command to do that, and just have it run the command every 15 seconds), but this seems like a very inefficient solution.

@Supergeek: I'm talking to one right now, and I may rewrite the bot with him over the next week or so (so that I won't need iMacros). This is making me a lot of money right now, so most hackers I'd talk to would just copy what I'm doing. I wouldn't feel comfortable handing over the idea to somebody who I haven't met in person.
 
What do you mean by imacro browser. You mean firefox running imacros?

Firefox runs all it's windows and tabs under a single process AFAIK. So you changing the affinity doesn't matter because it's still the same process.
 
subigo said:
How can I permanently set the processor affinity for an application on Multi-CPU machines on a Windows machine? - 1-3YQG92

^might work.
Click to expand...

I just tried doing that and it still automatically resets. There has to be something programmed into iMacros that forces it to automatically reset. I might play around with the macro (as it keeps on happening at the same spot in the code). Otherwise I'll probably have to wait to hear back from iMacros support or write a dos command that changes the affinity every 15 seconds.

kingfish said:
What do you mean by imacro browser. You mean firefox running imacros?

Firefox runs all it's windows and tabs under a single process AFAIK. So you changing the affinity doesn't matter because it's still the same process.
Click to expand...
No, I'm using the actual iMacros browser that they provide. Since I'm doing flash (which requires mouse clicks), the Firefox version doesn't work. I might look into running the IE iMacros browser and seeing if that allows me to change the affinity. Of course that would require me to completely rewrite the bot I wrote that manages all of these iMacros browsers.
 
I would go with the posters that said hire some real knowledgeable help and get past having to use imacros for it. If your that worried about the person stealing your idea though just have them whacked when they are done.
 
You must log in or register to reply here.
Top Bottom