Start Your Project with Us

Whatever your project size is, we will handle it well with all the standards fulfilled! We are here to give 100% satisfaction.

  • Any feature, you ask, we develop
  • 24x7 support worldwide
  • Real-time performance dashboard
  • Complete transparency
  • Dedicated account manager
  • Customized solutions to fulfill data scraping goals
How-to-Scrape-Google-Events-Results-Data-with-Nodejs.jpg

What data will be extracted?

What-data-will-be-extracted.jpg

Complete Code

In case, you don’t want any explanation, just look at a complete code example given in online IDE

Complete-Code.jpg Complete-Code-2.jpg Complete-Code-3.jpg

Research

First, we have to make a Node.js* project as well as add npm packages to parse parts of a HTML markup, as well as axios to make the request for a website.

To make this in a directory with project, open a command line to enter:

Research.jpg

And after that:

And-after-that.jpg

*In case, you don’t get Node.js installed, it’s easy to download from nodejs.org as well as follow an installation documentation.

Procedure

Initially, we have to scrape data from an HTML elements. The procedure of having the correct CSS selectors is very easy through SelectorGadget Chrome extension that help us take CSS selectors by clicking on a desired element in a browser. Though, it doesn’t always work perfectly, particularly when a website is weightily utilized by JavaScript.

The Gif given here shows an approach of choosing various parts of results.

Procedure.jpg

Code Clarification

State constants from axios and cheerio libraries:

Code-Clarification.jpg

After that, we write about what we need to search with request options: HTTP headers having User-Agent that is used for acting as the "real" user visiting, and the required parameters to make a request:

Code-Clarification-2.jpg Code-Clarification-3.jpg

Note: Default axios request’s user-agent is axios/ therefore websites know that it's the script which sends the request and may block it. Observe what's the user-agent.

After that, we write the function, which makes a request as well as returns the required data from a page. We established the reply from axios request, which has a data key, which we de-structure and parse that using cheerio:

After-that,-we-write-the-function.jpg

After that, we check in case no “events” results on a page, we revert null. We do it to stop the scraper while there are no pages left:

After-that,-we-check-in-case-no-events.jpg

After that, we have to find images data from a script tags, as when a page loads for thumbnails as well as images utilize placeholders having resolution 1px x 1px with the real images and thumbnails are set with JavaScript in a browser.

Primary, we outline imagesPattern, then use spread syntax to create an array from the iterable iterators of matches, established from matchAll technique.

After that, we take results and create objects with the image url and id. To offer a valid url we have to remove "\x" chars (with replaceAll technique), decode that (with decodeURIComponent technique) and make from the objects images aray:

After-that,-we-take-results-and-create.jpg

After that, we have to get various parts of a page with next methods:

After-that,-we-have-to-get-various-parts-of-a-page-with-next-methods.jpg After-that,-we-have-to-get-various-parts-of-a-page-with-next-methods-2.jpg

After that, we write the function where we find results from every page (with while loop), check in case results are available, add them in the events array (push technique) and set request params newer start value (meaning that number from where we wish to see different results on next page).

While no more results on a page (else statement) we stop a loop and return events array:

While-no-more-results-on-a-page.jpg While-no-more-results-on-a-page-2.jpg

Now we could launch a parser:

Now-we-could-launch-a-parser.jpg

Output

Output.jpg Use Google Event API from Actowiz Solutions

This section shows a comparison between a DIY solution and Actowiz solution.

The largest difference is, you don’t have to make a parser from the scratch and preserve it.

There’s also an opportunity that a request could be blocked from Google, we deal with that on backend therefore, there’s no requirement to find out how to make that yourself or find out which proxy provider or CAPTCHA to use.

Initially, we have to install google-search-results-nodejs:

Use-Google-Event-API-from-Actowiz-Solutions.jpg

Here’s the complete code example, in case, you don’t want any explanation:

Here-s-the-complete-code-example.jpg Here-s-the-complete-code-example-2.jpg

Code Explanation

Primary, we have to declare Actowiz Solutions from google-search-results-nodejs library as well as get new search example with the API key from Actowiz Solutions:

Code-Explanation.jpg

After that, we write what is needed to search ( a searchQuery constant) and essential parameters to make a request:

After-that,-we-write-what-is-needed.jpg After-that,-we-write-what-is-needed-2.jpg

After that, we wrap a search method from Actowiz Solutions library in the promise to work further with search results:

After-that,-we-wrap-a-search.jpg

And lastly, we declare a function getResult which gets data from every page as well as return it:

And-lastly,-we-declare-a-function.jpg

With this function, we find json with different results from every page (with while loop), observe if events_results are available, add them with eventsResults array (push technique) and set request new start value (meaning a number from where we wish to get results on next page).

While no more results on a page (else statement), stop a loop as well as return an eventsResults array:

With-this-function,-we-find.jpg

After that, we run a getResults function as well as print all the collected information in a console with console.dir technique that helps you to utilize an object using the required parameters to alter default output alternatives:

After-that,-we-run-a.jpg

Output

Output-2.jpg

And that’s it, the desired data is scraped!

For more information, contact Actowiz Solutions now!

You can also contact us for your mobile app scraping and web scraping services requirements.

Research And Report

View More

Scrape Zara Stores in Germany

Research report on scraping Zara store locations in Germany, detailing methods, challenges, and findings for data extraction.

Battle of the Giants: Flipkart's Big Billion Days vs. Amazon's Great Indian Festival

In this Research Report, we scrutinized the pricing dynamics and discount mechanisms of both e-commerce giants across essential product categories.