Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data