Top 7 Web Scraping Tips

Used well, web scraping is a lifesaver. It allows you to collect the data you need, parse it in a reusable way. We know how important it is for you to have accurate and up-to-date data. Since we’ve started scraping the web, we’ve learned a lot of dos and dont. In this article, we share […]

Top 5 Best Proxy Providers for Web Scraping

Extracting content from a website is never easy. The proxy plays an important role in web scraping. That’s why you need to choose which proxies you want to use and which proxies you need, dedicated or shared proxy. Here is a quick selection of the best proxy providers with some advice to help you choose […]

How to scrape Infinite Scroll Web Pages?

Some pages are harder to scrape than other. In the case of infinite scrolling pages, scraping them will result in incomplete data collection. Infinite scrolling can be a way to prevent web scraping. You can read about other anti scraping methods in this article. What are Infinite Scroll Pages and what makes them harder to […]

What are the methods used against web scraping?

Even though web scraping is commonly used across most industries, most websites do not appreciate it and new anti-scraping methods are being developed regularly. The main reason is that aggressive web scraping can slow down the website for regular users, and in the worst-case result in a denial of service. To prevent you from scraping […]

NEW FEATURE – Live Test

For those who are are not web developpers we have now a new feature allowing you to test ScrapingBot without any coding. Simply copy and paste the product page URL that you want to scrape and see the results you can get straight away. The Scraping bot NEW FEATURE To access this feature you must […]

How to scrape a website without getting blocked?

When you want to collect and analyze data, let it be for price comparison, statistics or to see a general evolution, scraping is a great and essential time saver. However, many websites do not appreciate to be heavily scraped, some of them don�t allow it at all, especially in the retail industry sector. There are […]

An even better scraping bot is born�!

After running a couple of e-commerce websites and trying out most of the tools available to scrape data, we decided to create our own as we were not satisfied of the performance of the APIs we tried. ScrapingBot is aimed at developers who cannot dedicate as much time as we did on developing it ! […]