Irish pop manager says taking part in the show was a ‘big mistake’ as he suffered from serious anxiety and ‘had to get the medics’