When Is It Ok Not To Leave A Tip?

I consider myself a good tipper.  I see all the crap that waiters and waitresses put up with and I understand.  But is it ever ok not to leave a tip?

Last week we decided to treat ourselves and hit up one of our favorite restaurants.  We go to this place all the time and have never had a problem.  Well, I guess there is a first time for everything.

We got seated and waited about 10 minutes before even getting acknowledged by our waitress.  Then she approached us and said, "What do you want."  In a really ugly tone.  We went ahead and placed our entire order to speed things up.  Then we waited more.

After about 20 minutes I asked another waiter to find our waitress.  When she finally came over I asked for our drinks and she rolled her eyes and left our table.  She never came back.  Turns out she never even placed our orders.

The manager hunted her down and made her return to help us.  At this point I was ready to just leave.  She came with our drinks and basically threw them at us.  That was it for me.  I wanted to walk out.

But my hubby told me to calm down and he asked for the check.  We paid for our drinks-we never got our food-and he threw down a few dollars for a tip.  I couldn't believe he did that, after sitting in this restaurant for over an hour and not even getting a drink.

He explained to me that she still worked for us and we couldn't not leave a tip.  So am I wrong to not want to leave a tip in this situation?  We technically didn't receive any services.  They just wasted our time.