Local residents said the decision ‘doesn’t change the fact that a fortress doesn’t belong in our community’.