Case Study: Web Scraping & Data Extraction for Ecommerce
After 9 years in business, our client database has expanded greatly and so is the variety of businesses we help across the world. Besides the sites (over 70K), Price2Spy is monitoring for our clients (over 650) from different industries (furniture, music equipment, nursery, pet supplies, sports, foods & drinks and so on), it also helps them to get their desired data through web scraping / web crawling.
Over the years, Price2Spy team has mastered a web scraping process that could help your business gather any valuable data in bulk. To provide you some of the best practices and samples from Price2Spy’s technical capabilities, we thought that the following web scraping case study might be of interest to you.
Click here to see more web scraping case studies from Price2Spy.
Price2Spy harvests and prepares raw data with the help of web scraping for the leading household products brand, worldwide
This client is one of the world’s most famous manufacturer of household products. They are working with hundreds of retailers worldwide, also they are monitoring several hundreds of websites including their’s competitors. Most importantly, they keep track of product reviews closely, because, they want to determine consumer satisfaction.
When this client approached us a couple of years ago, their major problems were:
- Capturing product reviews from retail websites, in order to determine consumer satisfaction towards their products
- Detecting resellers who are not in accordance with strict MAP policy they contracted
One of our first tasks was to perform web scraping on the sites that the client provided, in order to capture product reviews.
This task may sound simple, but it wasn’t, there were dozens of pages to be scraped and not all of them were unique, there were technically very complex and bot-aware websites (like Amazon).
The task was tough, but we managed to do it in a short period of time – thanks to our experience in web scraping and crawling technologies, as well as our distributed server technology, which enabled us to perform web scraping at a very fast rate, without causing suspicion on the target websites.
Anyway, at the end of web scraping process, we provided the results in Excel files to our client. The client was very satisfied with the raw data that they got from us, and they decided to automate the web scraping procedure. We introduced repetitive web scraping as a part of their organization and still keep doing it on a monthly basis.
Moreover, Price2Spy’s data entry team performed product matching on the retailer and competitor sites for the products supplied by the client and entered them into the account. Today all those products price/availability data is checked in every 3 hrs (8 times per day), so the client is able to monitor and enforce MAP policy strictly.
