-
Notifications
You must be signed in to change notification settings - Fork 451
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(integrations): add support for Scrape.do (#3228)
What is Scrape.do ? [Scrape.do](https://scrape.do) is a service that allows you to access raw data before the target website understands that you are sending bot traffic, bypassing the blocking problems you experience while scraping your target website, thanks to its rotating proxy, headless browser and captcha handling mechanisms. All the requests you send are upgraded and compared to the requests of a real user. The firewalls of the target websites have a hard time distinguishing these requests, and your success rate will thus exceed 99%. With Scrape.do, you can use a datacenter, residential or mobile proxy from the region or country you want, manage the entire process, including headless browsers, according to the needs of the target website, and save you from spending your time on crawling processes. Co-authored-by: Khaliq <[email protected]> Co-authored-by: nalanj <[email protected]>
- Loading branch information
1 parent
f413ae6
commit 3bdb8a0
Showing
7 changed files
with
155 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
--- | ||
title: Scrape.do | ||
sidebarTitle: Scrape.do | ||
provider: scrapedo | ||
--- | ||
|
||
import Overview from "/snippets/overview.mdx"; | ||
import PreBuiltTooling from "/snippets/generated/scrapedo/PreBuiltTooling.mdx"; | ||
|
||
import PreBuiltUseCases from "/snippets/generated/scrapedo/PreBuiltUseCases.mdx" | ||
|
||
<Overview /> | ||
<PreBuiltTooling /> | ||
<PreBuiltUseCases /> | ||
|
||
## Access requirements | ||
| Pre-Requisites | Status | Comment| | ||
| - | - | - | | ||
| Paid dev account | ❓ | | | ||
| Paid test account | ❓ | | | ||
| Partnership | ❓ | | | ||
| App review | ❓ | | | ||
| Security audit | ❓ | | | ||
|
||
|
||
## Setup guide | ||
|
||
_No setup guide yet._ | ||
|
||
<Tip>Need help getting started? Get help in the [community](https://nango.dev/slack).</Tip> | ||
|
||
<Note>Contribute improvements to the setup guide by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note> | ||
|
||
|
||
## Useful links | ||
|
||
- [How to generate an API Token in your Scrape.do account](https://dashboard.scrape.do/signup) | ||
- [Scrape.do API docs](https://scrape.do/documentation/#quick-start) | ||
- [Scrape.do Authentication](https://scrape.do/documentation/#token) | ||
- [Frequently Asked Technical Questions](https://scrape.do/documentation/#frequently-asked-technical-questions) | ||
|
||
<Note>Contribute useful links by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note> | ||
|
||
## API gotchas | ||
|
||
_No API gotchas yet._ | ||
|
||
<Note>Contribute API gotchas by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note> | ||
|
||
## Going further | ||
|
||
<Card title="Connect to Scrape.do" icon="link" href="/integrations/all/scrapedo/connect" horizontal> | ||
Guide to connect to Scrape.do using Connect UI | ||
</Card> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
--- | ||
title: Scrape.do - How do I link my account? | ||
sidebarTitle: Scrape.do | ||
--- | ||
|
||
# Overview | ||
|
||
To authenticate with Scrape.do, you will need: | ||
|
||
1. **API Token** - A key that is used to identify your Scrape.do account. | ||
|
||
This guide will walk you through finding these Keys within Scrape.do. | ||
|
||
### Prerequisites: | ||
|
||
- You must have a registered Scrape.do account. | ||
|
||
### Instructions: | ||
|
||
#### Step 1: Finding Your API Token | ||
|
||
1. **Log in** to your Scrape.do dashboard. | ||
2. Open the **Dashboard** menu. | ||
3. You will see your API Token in the **Quick Start** field. | ||
|
||
#### Step 2: Enter API Token in the Connect UI | ||
|
||
Once you have your **API Token**: | ||
|
||
1. Open the form where you need to authenticate with Scrape.do. | ||
2. Enter your **API Token** in the designated field. | ||
3. Submit the form, and you should be successfully authenticated. | ||
|
||
You are now connected to Scrape.do. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
## Pre-built tooling | ||
<AccordionGroup> | ||
<Accordion title="✅ Authorization"> | ||
| Tools | Status | | ||
| - | - | | ||
| Pre-built authorization (API Key) | ✅ | | ||
| Pre-built authorization UI | ✅ | | ||
| Custom authorization UI | ✅ | | ||
| End-user authorization guide | ✅ | | ||
| Expired credentials detection | ✅ | | ||
</Accordion> | ||
<Accordion title="✅ Read & write data"> | ||
| Tools | Status | | ||
| - | - | | ||
| Pre-built use-cases | 🚫 (time to contribute: <48h) | | ||
| API unification | ✅ | | ||
| 2-way sync | ✅ | | ||
| Webhooks from Nango on data modifications | ✅ | | ||
| Real-time webhooks from 3rd-party API | 🚫 (time to contribute: <48h) | | ||
| Proxy requests | ✅ | | ||
</Accordion> | ||
<Accordion title="✅ Observability & data quality"> | ||
| Tools | Status | | ||
| - | - | | ||
| HTTP request logging | ✅ | | ||
| End-to-type type safety | ✅ | | ||
| Data runtime validation | ✅ | | ||
| OpenTelemetry export | ✅ | | ||
| Slack alerts on errors | ✅ | | ||
| Integration status API | ✅ | | ||
</Accordion> | ||
<Accordion title="✅ Customization"> | ||
| Tools | Status | | ||
| - | - | | ||
| Create or customize use-cases | ✅ | | ||
| Pre-configured pagination | 🚫 (time to contribute: <48h) | | ||
| Pre-configured rate-limit handling | 🚫 (time to contribute: <48h) | | ||
| Per-customer configurations | ✅ | | ||
</Accordion> | ||
</AccordionGroup> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
## Pre-built use-cases | ||
|
||
_No pre-built use cases yet (time to contribute: <48h)_ | ||
|
||
<Tip>Not seeing the use case you need? [Build your own](https://docs.nango.dev/guides/customize/setup) independently.</Tip> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.