[Chore] Improve structure of GoogleSearcher

[olivierobert]

Issue

The code business logic for the scraper is contained in one method of 30 lines with many inline variables (e.g., DOM selectors for the ads, Google URL).

Expected

Use OOP best practices to structure GoogleSearcher:

• Extract parts of the business logic into small methods with a single responsibility
• Define hard-coded values into constants

[tajul-saajan]

Extract parts of the business logic into small methods with a single responsibility

inside page.evaluate() function runs in the browser context, not in the node.js context. So it does not work when I separate the calculation of different properties in private functions. is it acceptable to define functions for calculating intended variables?

[olivierobert]

That is indeed the challenge to solve 💦 The solution is to work off a string of the scraped page instead of a process. For that, the most standard solution is to 1) fetch the HTML content of the Google search page (this can be done with fetch, axios or any HTTP request library) and 2) use https://www.npmjs.com/package/cheerio to parse the HTML page for the required content.

[tajul-saajan]

At first. I considered both axios, cheerio combination and puppeteer for scraping. Eventually, decided to use puppeteer for its convenience to use as it imitates a regular browser and supports javascript dom.

[tajul-saajan]

the last commit on the #15 pr implements searcher based on axios and cheerio