Hype for next version 4.0

[trillox9]

I'm in hype for the next release, the switch from puppeteer to cheerio will bring speed and less consuption of pc resources.
Do you know when update will available?

[0000day]

Check the board: https://github.com/jef/streetmerchant/projects/4

[Aragorn450]

But why stop using a "browser" at all? At least with an actual browser that's controlled, it looks more like a regular user. Using these other things mean it's not necessarily behaving like a browser and therefore will be more likely to look like what it is; a bot scraping service.

What would you be trying to accomplish by switching at all?

[trillox9]

Browser Is heavy and useless for check only availability... and is recognized as bot anyway.
@jef i think the best solution is python with bs4 or scaper, could you look at this?

[Aragorn450]

I can pretty much guarantee that anything other than a browser will be identified as a bot even faster than Puppeteer is. Yes, it's heavy, although no heavier than Chrome itself.

And it has a lot of good APIs for finding content and interacting with it, which means you can do things like add to cart which wouldn't be possible with a simple scraper.

If you want to try a Python-based scraper, feel free to try that. But please don't ask @jef to do that for you...

[trillox9]

Jef is already directed toward that end.
Is the best way for speed and resource savings.
You can simply not update if you prefer puppeteer, 3.5.0 works well...

[jef]

I haven't gone through all the options and perhaps puppeteer will be our best bet! Optimizing that may be something I'd rather work on.

That being said, I think changing the backend for getting these links are lower priority then having a dynamic (internal) API that updates links/people can contribute to; rather than what we have currently (static models).