From feeling unwell to simply not wanting to go, here’s how to know when saying no is reasonable and when it may hurt someone.