Crunchbase Scraper (Old)

*** This is old and needs updating ***

Goals

• To import leads from a CSV file to a database
• To turn companies into leads and mark said companies as scraped
• To push scrape results into Airtable

How to use it

1. Build a list of companies to scrape from on crunchbase.

• Head over to Crunchbase Company Search and put in your search criteria.
• Make sure to exclude the existing lists from the search.

2. Start the list builder

rake list:build

• It will ask you a bunch of questions about the search. The answers will be stored, and uploaded into Customer.IO.

• The most important question it will ask you is a list number. It will become the folder where the list is stored, and is used by the importer.

• The list builder is watching your clipboard. Select all of the companies on the results page (Cmd + A) and copy them. Keep moving forward in the list until you are done.

• Once the list has been built its time to scrape to import the list into the database.

3. Import the list

Execute the import by calling rake import:import, with a number of list being imported.

rake import:list[2]

4. Start Scraping

Execute the scraper by invoking the scraper:scrape command, with a list number.

rake scraper:scrape[2]
`