Fans have been concerned about the comedian since his dramatic exit from the BBC game show