I use and love FireFox, but the majority of people out there use IE 4, 5 and 6. Sadly if you want to appeal to your widest market possible you can't discount IE as that is hte browser most people will use to look at your web site.
IE 7 will take a little while to get out there. It will be packaged with the Windows OS and over time more and more people will use it. The most popular browser will still be IE 6 for a long time to come. Popular in terms of actual use not in real popularity.
Most people - not generally the type of people that visit these sort of boards, see a web browser on their computer and that's it. They never bother to change or look for alternatives. Why would they? They just want to look at a web page, do what they have to do and move on. If the page doesn;t look right they wont blame the browser, rather they will blame the web site and probably never go back to it.