Some 54 special advisers were working directly for the PM as of the end of March, on salaries of up to £160,000.