In this blog, we will cover how to extract ASINs from the product list, where every product has many variants that require scraping.
It can be intimidating; however, never fear - through a few clever methods, you can enormously simplify a problem and find the required data you want for a business.
Why is It Not an Easy Job to Do?
The initial idea is to visit the list pages, extract links to every page, and use these links for visiting product pages to extract all variants' ASINs.
Regrettably, it rapidly becomes superficial that it won't work. These are two fundamental problems available themselves:
Every product can get its unique group of variants. How could we tell bots to choose them without hardcoding thousands of clicks?
Clicking every variant could often change the page layouts. It means a sole scraper won't be enough - we'd have to find a way to switch the data scraper set up dynamically.
Reproduce this with several products we need to cope with, and automation will become unmanageable. You'd applied more time to try and build it than saving by running that. Therefore, is there a unique way?
How to Utilize Amazon's Coding for Our Benefits?
While using Actowiz Solutions, you can have great mileage by observing how a website is created and using that to your benefit.
While doing an Amazon search, it's easy to observe that product variants have their search listing.
It is helpful as now this looks like we could avoid a problem about finding how to cycle every variant; Amazon's developer team has already given this code. We need to type a product name in a search bar. All the alternates would appear in a search listing.
Although we still have another problem choosing which data scraper we want for any particular page. Could we solve that too?
The answer here is yes! Let's understand the anatomy of an Amazon's page URL:
Can you see some numbers highlighted in the color green? They are the ASINs we're searching for! Therefore, if we could make the bot, which grabs URLs and scrapes the ASINs, that's it!
There's the last factor to consider, sponsored content. Amazon augments this associated content with the top search pages; however, it creates problems if you need specific products. Luckily, Actowiz Solutions offers tools to cope with that; we can make a selection that excludes the promoted products.
So, we have got all the required pieces, so let's create a bot!
How to Create a Bot to Extract Amazon ASIN Data?
1. Prepare a Google Sheet
Prepare a Google Sheet that has two tabs. Add the product names you need to extract ASINs in column A in an account named "Search." We'll give them names like "Search" & "Results."
2. Make a New Automation
Click on the "+ New Automation" tab to create new automation!
3. Start with a Blank
We're creating this from scratch by adding our steps.
4. Read Information from the Google Sheet
Add the "Read Information from the Google Sheet" step and choose a tab named "Search."
5. Relate with Amazon Webpage
Add the "Interact with the page interface" step. The step has all sub-steps needed to relate to the Amazon webpage.
6. Visit the URL
Set an Amazon search page in the sub-step "Go to URL."
7. Enter the Text in Amazon Search Field
Add the "Enter Text" substep. Then click "Select" and choose a search bar's input fields. After that, click on "Insert Data" and add google-sheet-data, and from the popup, choose the column having product names.
8. Trigger Your Searches on Amazon
Add the sub-step "Click Element." This step will click the search button and update search results.
That's the initial part of a bot done - merely some more steps for adding. Now, you can do the test run!
9. Adding One Step to Extract Amazon Data
Add the "Get data from the webpage" step. Initially, let's extract product titles.
Choose a title (ignore all sponsored content) and choose a second title for creating a repeated selection. Add one new column, and click a dropdown to select a "Link" data. Then choose titles again and grab links to product pages.
Set Max Results in settings to 10 for the initial few runs while testing a bot - we could turn that off later while we're happy and everything works fine.
10. Write Data in the Google Sheet
Add "Write data to the Google Sheet" and set "Sheet name" into "Results." A "Data" dropdown needs to be set correctly for using interact-data variables from the 'Interact' step, and double-check if it looks OK.
11. Read Extracted Data from Google Sheet
Next, add one more "Read data from the Google Sheet." Here, choose a sheet name "Results."
12. Scrape ASIN from URL Part 1
Add the new step named "Split by the character." With the "Data" field, choose the "google-sheet-data__1" step – which is our second step of "Read Data from the Google Sheet" - and select column B, one with a URL.
With the field named "Character," add "dp/" (without any quotation marks!)
13. Scrape ASIN from URL Part 2
Add another "Split by character" and set "Character" to "/" (with no quotation marks). With "Data," choose "split-by-character" - what we need here is passing the results of the initial split into a second split, binding them together. When a preview comes, choose column A.
14. Write Data to a Google Sheet
We're nearly done! Let's add the last "Write data to the Google Sheet" step.
Get a sheet you created earlier with a "Spreadsheet URL" field. Set "Sheet name" with "Results," and ensure "Data" is charged with "split-by-character__1", a data variable for the last step, the "Split by character" action.
Closure "Clear data before writing | Add to existing data" and shoe "Add to existing data" and set a beginning cell option at "C1".
15. The Bot is All Set to Extract Amazon
Hurrah! we're done! You have created a bot to scrape ASINs from the Amazon website without any lines of code. Just go ahead and run your bot!
Conclusion
A few websites don't make it easy to extract data; however, by analyzing a website's structure and perceiving its behavior, we could often exercise how to remove the data we need.
With an easy and simplified approach, you can quickly build things. A minor saving in complication can result in much bigger returns than expected! This is worth taking some time to think about the most acceptable way of solving your problems before getting in.
Contact us for more information about your Amazon ASIN data scraping requirements!
You can also reach us for your mobile app scraping and web scraping services requirements.
