How to Do Price Difference Monitoring Across Different Amazon Sellers?

Why Extract Pricing from an Offer Listing Page?

• Analyze delivery channels, track your product selling, and find any damage to your brand
• Know about customers’ behavior to align pricing and products toward the highest profits
• Know the share standings and feasibility approach for buying a box
• To efficiently monitor competition, get insights about selling price and product movement.

What Data Are We Scraping?

Here is the listing of product information that we are going to scrape:

• Seller’s Name
• Pricing
• Product Condition
• Product Shipping
• Delivery Options
• Seller Ratings
• Positive Percentage

Here is the screenshot of a few data fields we would be scraping:

Build A Web Scraper

You require a computer having Python 3 & PIP installed on it. This code won’t work on Python 2.7.

Most UNIX OS, including Mac OS or Linux, come having pre-installed Python. Although not all the Linux OS ship with defaulting Python 3.

Let’s observe a Python version. Open a terminal (with Mac OS or Linux) or Windows (Command Prompt).

And then press enter. If the result looks like Python 3.x.x, you get Python 3 installed. If it shows Python 2.x.x, then you have Python 2. In case you don’t get Python 3, then install that first. You probably don’t get Python installed if it prints the error.

Installing Pip and Python 3

Here is the guide of installing Python 3 with Linux – http://docs.python-guide.org/en/latest/starting/install3/linux/

For Mac Users, you can follow a guide – http://docs.python-guide.org/en/latest/starting/install3/osx/

Installing Packages

Python Requests, for making requests and downloading HTML content of pages (http://docs.python-requests.org/en/master/user/install/).

Use Python LXML for parsing HTML’s Tree Structure with Xpaths (Find how to make installation here at http://lxml.de/installation.html)

UnicodeCSV to handle Unicode characters within output files. Install that with pip installing unicodecsv.

IPython for checking Xpaths. You may install that with pip install ipython

Construct a URL

Recognize the products’ ASIN. With the product given – Pockit Lightweight Stroller (https://www.amazon.com/GB-616230013-Pockit-Lightweight-Stroller/dp/B01DQ2B8UY/ref=olp_product_details?_encoding=UTF8&me=), its ASIN includes B01DQ2B8UY

Make a URL to find data of different sellers. Open a browser and visit a product Offers Listing page on Amazon. Its URL looks like that: https://www.amazon.com/gp/offer-listing/B01DQ2B8UY/ For getting a product offer listing, you just have to replace an ASIN at end of a link. It is a URL if you don’t use any filters. Then, Click on a checkbox for main suitability.

The URL alters to that:

https://www.amazon.com/gp/offer-listing/B01DQ2B8UY/ref=olp_f_primeEligible?ie=UTF8&f_all=true&f_primeEligible=true

The next step to follow is to build a web scraper, which extracts data from all offer listings – Price, Delivery, Condition, and Seller details as per every applied filter.

Applied Filters:

Every argument gets pass through a general line and it is gained and saved. We want arguments for product condition, ASIN, and shipping. A GIF given below indicates how to find a URL depending on a filter applied:

For every applied filter, we need to make a URL. We will make a filter for URL mapping.

The constructed URL will have an ASIN, shipping, and product condition that would look like:

https://www.amazon.com/gp/offer-listing/B01DQ2B8UY/ref=&f_new=true&f_primeEligible=true

Find an XPath

XPaths can be utilized to tell a script where every field we want is available in HTML. The XPath gives you a location of the element like a list does for the books. We’ll get XPaths for all the fields needed and put it into the scraper. After scraping the data, we’ll save that in the CSV file.

Now it’s time to check XPaths with IPython:

This request has been succeeded.

Let’s go through XPath of every listing:

The GIF indicates where we’ll get the data available in the HTML format.

So, there are 10 listings of sellers for the product. Let’s observe the pricing for initial listings:

Get the Necessary Data:

Now check some data fields needed in the scraper to observe if the data needed is correct:

Prime Suitability:

Conditions:

Shipping:

Delivery:

Seller’s Name:

Repeating the listing to get sellers’ data:

The Code

Run a Code

Assume that a script is named amazon_seller.py. In case, you type a script name within terminal or command prompt together with a -h

Here are a few examples about scraping the seller listing with an ASIN ‘B01DQ2B8UY’:

python amazon_seller.py B01DQ2B8UY “all” “all”

To get a seller listing under a condition ‘new’ and ‘prime’:

Let us understand in the below comments about how the scraper has worked.

For more information, contact Actowiz Solutions now!

You can also contact us for all your mobile app scraping and web scraping services requirements.