"Going from Wiles to Miller is like trying to get through a dinner party by switching from wine to meth," Colin Jost jokes