There's some nice information on linode's forums about how people scale their linux distributions. I forget the post, but the guy who's point I really liked was using their lowest plan (which has about 250MB of RAM if I remember correctly, since I have long since moved from their service). He was getting about 24k visitors a day to his site and the main thing that caused him any grief were dynamic pages on the site. That's what ate up the CPU and RAM.
So, he let Apache run the dynamic calls and had the config let another web server deal with static pages that was more efficient for this. I think he said he went to about 36-40k+ visitors a day prior to upgrading to another plan using this method.
A lot of people use dedicated servers when they'd be just fine with a VPS, and a lot of people use VPS' when they could get by just fine on shared hosting. It really comes down to the configuration of your box and distribution.
If you compress your databases, put indexes on commonly called fields in database tables, compress your large files and images, compress your javascripts and CSS files, then you'd have no problem sitting on a shared host account. I sat on one and used my .htaccess file to make directories appear as their own site so I could have 12 sites running on a plan that cost me $60 a year. I think each site was averaging 500-1200 visitors a day depending on the time of year. I only swapped off there because I needed to buy a VPS to do some other things.