Skip to content

Commit

Permalink
feat(integrations): add support for Scrape.do (#3228)
Browse files Browse the repository at this point in the history
What is Scrape.do ?
[Scrape.do](https://scrape.do) is a service that allows you to access
raw data before the target website understands that you are sending bot
traffic, bypassing the blocking problems you experience while scraping
your target website, thanks to its rotating proxy, headless browser and
captcha handling mechanisms. All the requests you send are upgraded and
compared to the requests of a real user. The firewalls of the target
websites have a hard time distinguishing these requests, and your
success rate will thus exceed 99%.

With Scrape.do, you can use a datacenter, residential or mobile proxy
from the region or country you want, manage the entire process,
including headless browsers, according to the needs of the target
website, and save you from spending your time on crawling processes.

Co-authored-by: Khaliq <[email protected]>
Co-authored-by: nalanj <[email protected]>
  • Loading branch information
3 people authored Jan 6, 2025
1 parent f413ae6 commit 3bdb8a0
Show file tree
Hide file tree
Showing 7 changed files with 155 additions and 0 deletions.
54 changes: 54 additions & 0 deletions docs-v2/integrations/all/scrapedo.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
title: Scrape.do
sidebarTitle: Scrape.do
provider: scrapedo
---

import Overview from "/snippets/overview.mdx";
import PreBuiltTooling from "/snippets/generated/scrapedo/PreBuiltTooling.mdx";

import PreBuiltUseCases from "/snippets/generated/scrapedo/PreBuiltUseCases.mdx"

<Overview />
<PreBuiltTooling />
<PreBuiltUseCases />

## Access requirements
| Pre-Requisites | Status | Comment|
| - | - | - |
| Paid dev account || |
| Paid test account || |
| Partnership || |
| App review || |
| Security audit || |


## Setup guide

_No setup guide yet._

<Tip>Need help getting started? Get help in the [community](https://nango.dev/slack).</Tip>

<Note>Contribute improvements to the setup guide by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note>


## Useful links

- [How to generate an API Token in your Scrape.do account](https://dashboard.scrape.do/signup)
- [Scrape.do API docs](https://scrape.do/documentation/#quick-start)
- [Scrape.do Authentication](https://scrape.do/documentation/#token)
- [Frequently Asked Technical Questions](https://scrape.do/documentation/#frequently-asked-technical-questions)

<Note>Contribute useful links by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note>

## API gotchas

_No API gotchas yet._

<Note>Contribute API gotchas by [editing this page](https://github.com/nangohq/nango/tree/master/docs-v2/integrations/all/scrapedo.mdx)</Note>

## Going further

<Card title="Connect to Scrape.do" icon="link" href="/integrations/all/scrapedo/connect" horizontal>
Guide to connect to Scrape.do using Connect UI
</Card>
34 changes: 34 additions & 0 deletions docs-v2/integrations/all/scrapedo/connect.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
title: Scrape.do - How do I link my account?
sidebarTitle: Scrape.do
---

# Overview

To authenticate with Scrape.do, you will need:

1. **API Token** - A key that is used to identify your Scrape.do account.

This guide will walk you through finding these Keys within Scrape.do.

### Prerequisites:

- You must have a registered Scrape.do account.

### Instructions:

#### Step 1: Finding Your API Token

1. **Log in** to your Scrape.do dashboard.
2. Open the **Dashboard** menu.
3. You will see your API Token in the **Quick Start** field.

#### Step 2: Enter API Token in the Connect UI

Once you have your **API Token**:

1. Open the form where you need to authenticate with Scrape.do.
2. Enter your **API Token** in the designated field.
3. Submit the form, and you should be successfully authenticated.

You are now connected to Scrape.do.
1 change: 1 addition & 0 deletions docs-v2/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,7 @@
"integrations/all/salesforce-experience-cloud",
"integrations/all/salesloft",
"integrations/all/sap-success-factors",
"integrations/all/scrapedo",
"integrations/all/segment",
"integrations/all/sendgrid",
"integrations/all/servicem8",
Expand Down
40 changes: 40 additions & 0 deletions docs-v2/snippets/generated/scrapedo/PreBuiltTooling.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
## Pre-built tooling
<AccordionGroup>
<Accordion title="✅ Authorization">
| Tools | Status |
| - | - |
| Pre-built authorization (API Key) ||
| Pre-built authorization UI ||
| Custom authorization UI ||
| End-user authorization guide ||
| Expired credentials detection ||
</Accordion>
<Accordion title="✅ Read & write data">
| Tools | Status |
| - | - |
| Pre-built use-cases | 🚫 (time to contribute: &lt;48h) |
| API unification ||
| 2-way sync ||
| Webhooks from Nango on data modifications ||
| Real-time webhooks from 3rd-party API | 🚫 (time to contribute: &lt;48h) |
| Proxy requests ||
</Accordion>
<Accordion title="✅ Observability & data quality">
| Tools | Status |
| - | - |
| HTTP request logging ||
| End-to-type type safety ||
| Data runtime validation ||
| OpenTelemetry export ||
| Slack alerts on errors ||
| Integration status API ||
</Accordion>
<Accordion title="✅ Customization">
| Tools | Status |
| - | - |
| Create or customize use-cases ||
| Pre-configured pagination | 🚫 (time to contribute: &lt;48h) |
| Pre-configured rate-limit handling | 🚫 (time to contribute: &lt;48h) |
| Per-customer configurations ||
</Accordion>
</AccordionGroup>
5 changes: 5 additions & 0 deletions docs-v2/snippets/generated/scrapedo/PreBuiltUseCases.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## Pre-built use-cases

_No pre-built use cases yet (time to contribute: &lt;48h)_

<Tip>Not seeing the use case you need? [Build your own](https://docs.nango.dev/guides/customize/setup) independently.</Tip>
20 changes: 20 additions & 0 deletions packages/shared/providers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6016,6 +6016,26 @@ sap-success-factors:
secret: true
doc_section: '#step-4-generating-your-saml-assertion'

scrapedo:
display_name: Scrape.do
categories:
- other
auth_mode: API_KEY
proxy:
base_url: https://api.scrape.do
query:
token: ${apiKey}
docs: https://docs.nango.dev/integrations/all/scrapedo
docs_connect: https://docs.nango.dev/integrations/all/scrapedo/connect
credentials:
apiKey:
type: string
title: API Token
description: The API Token for your Scrape.do account
example: 3c12d71308a346c41d10b19a2b2ac1ea5cacb53588d
pattern: '^[a-zA-Z0-9]+$'
doc_section: '#step-1-finding-your-api-key'

salesloft:
display_name: Salesloft
categories:
Expand Down
1 change: 1 addition & 0 deletions packages/webapp/public/images/template-logos/scrapedo.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 3bdb8a0

Please sign in to comment.