When web scraping, one of the problems you may come across is being blocked from a site as you have tried to access it too many times in a short period of time.
One tip I found out during our web scraping sessions was about using the Throttle tool.
You can find the tool in the “Developer” tab or if it is easier, you can search for it in the search bar.
Drag the tool in after your data set, but before your Download tool.
The configuration of the Throttle tool is simple, just state how many URLs should be downloaded per minute.
The Throttle tool allows you to slow down the process of your download tool. Only retrieving the amount of data you have specified within the throttle configuration, you can now specify the records per minute retrieved based on particular websites request limit.