Building an E-Commerce Price Comparison Application with Selenium and Tkinter – A Universal Approach

Inspiration

In today's world of online shopping, finding the best deal for a product can be time-consuming and overwhelming. With countless options available, getting lost in a sea of products, each with different prices, ratings, and reviews, is easy. This process becomes even more daunting when looking for the perfect combination of quality, affordability, and seller reliability.

Imagine you're in the market for a new headset. You spend hours researching various brands, comparing specifications, and weighing the pros and cons until you finally settle on the ideal product. However, the challenge continues. Now, you need to determine which online store offers the best price for the headset. Some stores might be offering a significant discount, while others have higher prices but boast excellent seller ratings. The many options leave you perplexed and unsure about making a simple purchase.

Wouldn't it be great to automate this process and find the optimal purchase without the hassle? As our reliance on e-commerce continues to grow, having a tool that automates price comparisons based on ratings, number of ratings, and price would be a valuable asset. In this blog, we will guide you through building a web scraper using Python, specifically the Selenium library and Tkinter framework, to create an efficient and user-friendly e-commerce price comparison application.

To-Do List

This project will focus on creating a price comparison application specifically designed for Amazon and eBay, two of the most prominent e-commerce platforms. The goal is to streamline finding the best deals by utilizing the BeautifulSoup and Selenium libraries in Python.

By the end of this article, we will achieve the following objectives:

Data Scraping: We will leverage the power of BeautifulSoup and Selenium to scrape relevant data from Amazon and eBay. This includes product information such as name, price, ratings, and number of ratings.

Sorting the Dataset: Once we have gathered the data, we will sort it based on the number of ratings and price. This will help us identify the top 10 potential purchases that balance popularity and affordability.

User-Friendly GUI: We will create a graphical user interface (GUI) using the Tkinter library to bring it all together. The GUI will provide an intuitive and user-friendly interface for inputting search queries and displaying organized results.

By automating comparing prices, ratings, and the number of ratings, we aim to save time and money. With Python as our tool of choice, we can efficiently filter out unnecessary deals, allowing you to make informed purchasing decisions.

Our Scrapers

To begin our analysis, we will develop two parsers, one for each e-commerce website (Amazon and eBay). The first step is to prompt the user for input, which includes a search term and the desired page range to scrape. For instance, if the user searches for "sea salt" and specifies a page range of 5, our script will iteratively visit the Amazon/eBay web pages and scrape sea salt information until the end of page 5.

Let's dive into the code. We need to convert the syntax of the search term to match the URL structure. We can construct a base URL using the respective website domains, followed by the formatted search term and the page number we want to scrape. We can direct our scraper to this specific URL and extract the desired information by utilizing the Selenium API.

url = https://www.ebay.com/sch/i.html?&_nkw=" + search_term + '&_sacat=0&_pgn={}'

Each item on a webpage is structured using HTML attributes. We can extract the desired information by identifying the relevant HTML tags associated with the items. The BeautifulSoup API allows us to access the HTML source code of the webpage and locate these tags. It is often a trial-and-error process to determine the appropriate tags to target. You can begin by printing out the HTML code returned by your tag-based query and iteratively modify it until you successfully identify the desired item.

In our implementation, we default to using Chrome as the browser. However, you can choose any browser by changing the web driver type in Selenium. Now, let's address the following challenge: iterating through different pages. The approach is straightforward. In the code snippet below, we have used curly braces at the end of the URL to format it with the page number. We can iterate through the pages by setting up a for-loop and formatting the URL with the corresponding page number in each iteration. This allows us to scrape product information from each page.

We assume we are scraping eBay to extract product information in the example. The code demonstrates how to iterate through the desired page range and dynamically generate URLs using the formatted search term and the page number. We then use BeautifulSoup to parse the HTML source code and locate the HTML tags that contain the desired product information. By employing try-except statements, we handle cases where the information is unavailable and ensure consistency in the parsed data. Finally, we store the extracted information in a list of dictionaries (products).

The same logic applies when creating the parser for Amazon. If you want to review the code, you can find it in the GitHub repository. Once both parsers are implemented, we need an external function that takes the resulting datasets and performs preprocessing. In the GitHub repository, we have included a function called 'main' to handle this. Within this function, we convert string variables to numeric format to enable sorting based on the number of ratings for each product. This metric is the one we chose to filter and identify the top 10 items in a search query. However, you can easily modify this metric based on your preference.

Making a GUI

Now, let's transform the parser into a tool that others can use by creating a graphical user interface (GUI) using the Tkinter library in Python. Tkinter provides a convenient way to create a user-friendly interface for our web scraper.

To build the GUI, we utilize several modules from the Tkinter library, including:

• tk.Button: Creates buttons that trigger functions when clicked.
• tk.Canvas: Used to set the dimensions of the application window.
• tk.Entry: Creates text boxes where users can input data.
• tk.Frame: Creates rectangular areas where we can place widgets.

When designing the layout of our widgets, we exclusively use the "place" method provided by Tkinter. This method allows us to specify each widget's exact x and y coordinates, making it straightforward to arrange them. Tkinter also provides two other layout methods, "pack" and "grid," which you can explore if interested.

In our application, we create six frames. One frame contains the search bar where users can enter their desired product. Two frames serve as the background for the canvas. Another frame houses two sub-frames that display the datasets obtained from the scraping script. We also include two entry toolbars, one for the product name and the other for the maximum page number to scrape. Finally, we add a "Compare" button that triggers the "main" function discussed earlier, which returns the sorted data frames for both websites.

With the GUI, users can input their desired product and specify the number of pages to scrape. When they click the "Compare" button, the scraping script runs, and the top 10 queries from eBay and Amazon are displayed in the respective frames.

Based on the sorted data frames, we can observe that the top 10 choices for the product on eBay and Amazon have similar prices. Therefore, purchasing from either website would suffice unless you want to save around $50 by purchasing a refurbished product.

Now that you understand how this tool works, you can build your parser and incorporate different websites for comparison. Remember that the HTML tags you query and the website URLs may vary slightly for different websites. To do this, carefully inspect the website attributes and experiment with different HTML queries until you obtain the desired information.

As demonstrated, this tool can be highly beneficial in comparing product deals across multiple websites, saving you valuable time. Instead of manually searching through numerous tabs and scrolling through countless products, you have the best purchase options.

If you have any questions or suggestions, please don't hesitate to reach out. Thank you for reading!

Please don't hesitate to contact us for all your mobile app scraping, instant data scraper and web scraping service needs. We offer professional and reliable services to cater to your specific requirements. Whether you need data extraction from mobile apps or websites, our team is here to assist you.

Feel free to reach out to us for a free quote. We will gladly discuss your project and provide a customized solution for your scraping needs. Our expertise in scraping technologies and techniques ensures accurate and efficient data retrieval.

Take advantage of the opportunity to automate your data collection processes and gain valuable insights. Contact us today for top-notch scraping services tailored to your business needs.