The Queen delivered a speech at her book festival after Earl Spencer doubled down on claims in his upcoming memoir.