Trying to find the bloat.

Status
Not open for further replies.
kblessinggr

kblessinggr

PedoBeard
Sep 15, 2008
5,725
80
0
G.R., Michigan
www.kbeezie.com
I have about 20 domains on my hosting (a couple by other people). It seems lately that I'm actually eating up well over 200mb of ram from the hosting. And for the life of me I don't know from where.

All I have is this graph of the last 24 hours.

usage.png


Probably the busiest site I have (as per google analytics) is my ipodtouch wordpress site, which around 200 unique visits a day. But I doubt very much, that even that could run that much memory up.

So I'm wondering if there's something I can do to help identify which domain is taking (or was) up so much ram. I already sent in a request to get more details (that rendered the graph) but the email has yet to be responded to.
 


have you been keeping an eye on your apache's server-status page? also, if you're on a box with limited ram, create a cron job that restarts apache regularly, or tweak your apache configuration to have less idle servers, or perhaps lower the timeout for child processes.
 
ps aux | less shows this...

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.4 0.1 1944 648 ? Ss 01:00 2:42 init [2]
dhapache 1847 0.0 2.7 47384 15160 ? S 09:36 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 1850 0.0 2.7 47404 15184 ? S 09:36 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 1851 0.0 2.7 47380 14936 ? S 09:36 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 3541 0.0 2.7 47380 14928 ? S 09:44 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 3543 0.0 2.7 47388 14964 ? S 09:44 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 3544 0.0 2.7 47472 15004 ? S 09:44 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
postfix 17449 0.0 0.2 3052 1176 ? S 10:35 0:00 pickup -l -t fifo -u -c
kblessi 17488 0.0 0.2 2840 1396 ? S 10:36 0:00 proftpd: kblessing - 71.205.12.252: IDLE
mr_chang 17597 0.0 0.2 2840 1420 ? S 10:36 0:00 proftpd: mr_chang - 71.205.12.252: IDLE
postfix 22546 0.0 0.2 3232 1392 ? S 11:00 0:00 trivial-rewrite -n rewrite -t unix -u -c
dhapache 23079 0.0 2.1 46724 11920 ? S 01:59 0:00 /dh/apache2/apache2-ps6446/fcgi-pm -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 23080 0.0 2.2 47044 12156 ? S 01:59 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
root 23149 0.0 0.3 6084 1808 ? Ss 11:03 0:00 sshd: kblessing [priv]
kblessi 23158 0.0 0.1 6068 1088 ? S 11:03 0:00 sshd: kblessing@pts/0
kblessi 23159 0.4 0.2 2640 1424 pts/0 Ss 11:03 0:00 -bash
kblessi 23167 0.0 0.1 2564 852 pts/0 R+ 11:03 0:00 ps aux
kblessi 23168 0.0 0.0 2640 488 pts/0 D+ 11:03 0:00 -bash
dhapache 24285 0.0 2.7 47512 15032 ? S 08:57 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 24286 0.0 2.7 47488 14968 ? S 08:57 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 27073 0.0 2.7 47836 15284 ? S 09:08 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 27093 0.0 2.7 47488 14992 ? S 09:08 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 27094 0.0 2.7 47480 15024 ? S 09:08 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29059 0.0 2.7 47480 15040 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29063 0.0 2.7 47388 14944 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29064 0.0 2.7 47496 14960 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29065 0.0 2.7 47512 15292 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29070 0.0 2.7 47840 15220 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29071 0.0 2.7 47464 15008 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29072 0.0 2.7 47420 14976 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
dhapache 29073 0.0 2.7 47480 14996 ? S 09:14 0:00 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
root 32017 0.0 0.2 2856 1536 ? Ss 01:00 0:00 /sbin/syslog-ng -p /var/run/syslog-ng.pid
root 32042 0.0 3.2 47044 17564 ? Ss 01:00 0:01 /dh/apache2/apache2-ps6446/apache2-ps6446-httpd -DModSSL -d /dh/apache2/apache2-ps6446
root 32071 0.0 0.0 1616 476 ? Ss 01:00 0:00 /usr/sbin/inetd
root 32191 0.0 0.2 3044 1152 ? Ss 01:00 0:00 /usr/lib/postfix/master
postfix 32195 0.0 0.2 3084 1208 ? S 01:00 0:00 qmgr -l -t fifo -u -c
root 32201 0.0 0.1 3508 904 ? Ss 01:00 0:00 /usr/sbin/sshd
nobody 32211 0.0 0.1 2616 784 ? Ss 01:00 0:00 proftpd: (accepting connections)
daemon 32214 0.0 0.0 1768 388 ? Ss 01:00 0:00 /usr/sbin/atd
root 32217 0.0 0.1 1824 752 ? Ss 01:00 0:00 /usr/sbin/cron
root 32227 0.0 0.0 1572 448 ? Ss 01:00 0:00 /usr/bin/ud -s
Click to expand...

At the time of running that command dreamhost's PS graph claims I'm using lil over 300mb of rams.
 
move everything off of dreamhost right now... don't wait, do it right fucking now while you still can...

the dreamhost PS (Piece of Shit) is completely worthless... I actually think they designed it to be so bad just to shed customers who use anything more than zero resources... if you keep using them you will have nothing but problems...
 
PHP runs in the same process as Apache, and with the forking model, you end up with a lot of memory used. I count roughly 20 httpd processes in your process list, each with a RSS of about 15MB, so there's 300 MB right there.

Restarting apache only relieves the problem temporarily. Eventually each slot is going to serve a PHP request, and will carry the bloat with it for the next request, even if it's a static file.

Fastcgi is a good way of getting around it. You pre-fork a number of PHP processes, say 5, and then configure httpd to send PHP requests over a socket instead of doing it as a PHP module. It's a bit of work to configure on Apache though. You can also try out the other MPMs that run threaded or event based.

I've been very happy with nginx on VPSes lately using fastcgi. Nginx is event based, so it's one (well, 2, but only one does the work) processes to handle thousands of simultaneous connections. It has most of the Apache features, you just have to work a bit to change .htaccess configuration to nginx-ese.

If you're looking to move VPS providers, I can't speak highly enough about Linode. I've moved my stuff to one, and have also moved several Wordpress and Rails sites over on behalf of other people.
 
SeanW said:
PHP runs in the same process as Apache, and with the forking model, you end up with a lot of memory used. I count roughly 20 httpd processes in your process list, each with a RSS of about 15MB, so there's 300 MB right there.
Click to expand...

Actually its supposed to show up as php.cgi since dreamhost doesn't allow mod_php by default. (to avoid hiding heavy tasks behind apache).

Anyways doesn't really matter much, soon as I figure out how to configure the VPS at the new hosting (I started out with webmin, but kinda gave up and requesting an upgrade to cpanel, despite the minor resource increase it'd cause).
 
If bots aren't calculated in your "hits per day" it could be a spider. I remember hearing a few months ago about how the [FAIL] Cuil bot tried to over-spider everything, even ignoring robots.txt, and crashed sites much in the manner of digg.

Just an out of the box possibility.
 
trigatch4 said:
If bots aren't calculated in your "hits per day" it could be a spider. I remember hearing a few months ago about how the [FAIL] Cuil bot tried to over-spider everything, even ignoring robots.txt, and crashed sites much in the manner of digg.

Just an out of the box possibility.
Click to expand...

I was looking at access logs at that possibility, but didn't find anything, especially since bandwidth didn't change much during the peak.

But that kind of thing happened to a client I was working with (not on my hosting but on hostgator), they made the site live. And apparently along came googlebot, decided it liked it's video section which was a a transparent loading of videos based on tags from youtube. Well googlebot decided it was going to inspect the tag links under each video which naturally loads the client's site up again and shows videos for that tag. Well videfeeder also creates a folder of caches (txt files). Within 24 hours, the site was crashing because there was no harddrive space left for the smarty template engine and other tasks to compile/create files.

And speaking of GoogleBot, at one point of time, Dreamhost was actually telling people to block GoogleBot in their robot.txt all together.
 
Status
Not open for further replies.
Top Bottom