‘I've never dismissed a housemate's feelings, in all honesty,’ controversial contestant said