My inbox has been a disaster area for years. Even with 500+ lines of procmail filtering, and archiving quite a lot of stuff, it was approaching 12000 messages and there was no realistic prospect of me ever getting round to replying to most of that. On top of this, both work and personal mail came into the same inbox, so if unarchived work mail for the day caused some personal mail to scroll off the top, I probably wouldn't get around to the personal mail. The whole thing came up in my performance review at work, and after a friend poked me about not having replied to "hey, let's catch up"-type mails for ages, I finally pulled my finger out and decided to try Inbox Zero.
Six days later, I have: three empty inboxes (work, Debian, and personal); one mailbox containing what used to be in my inbox, from which I've archived about 2500 mails elsewhere; one to-do (or "to reply to") mailbox with five items in it; and some happy people with prompt replies from me. Moving my old inbox aside would achieve nothing if I weren't dealing with new things coming in or making progress with its former contents, so this seems like a pretty good result so far.
Thing I haven't got right so far: I'm now checking my e-mail too often because the inbox is so shiny and clean that I can't quite get over it. :-) I'm spending next week in London, so that should forcibly help me out of that habit.
Thing I don't like about Inbox Zero (but fortunately is a non-essential component): the recommendation to use very few archive folders. Aside from not especially liking the idea of relinquishing all my mail to Gmail, I find that I'm much more likely to be able reliably to identify the topic identifying its folder (and then maybe to pare that down by sender or something) than I am to be able to identify a halfway-useful set of search terms. If I went back and labelled everything to match my topic-based folders then I might be able to use a single archive folder, but I don't really see the value in bothering with that. I don't often find myself spending much time searching archive folders, and when I do it's usually when I'm running statistics on things like bug folders. I'll ignore this bit unless and until it becomes a problem.
Edit: Which all means "if you've sent me personal mail expecting a reply and I haven't replied to it yet, I probably won't, so please resend it".