The 43-year-old journalist opens up about his ‘horrible’ experience on the new series