Brickset Scraper Using Scrapy

Intro

A Web Scraper that uses Python3 and Scrapy to scrape Brickset - A LEGO database.

scrapy runspider brickset-scraper > brickset-scraper.json

Example output JSON files are here.

By default, past 5 years of bricksets are scraped.

Change the START_YEAR and END_YEAR in scraper.py to scrape desired years.

Example output has been beautified with JSON Lint for readability.

Working of this web scraper depends on the source at Brickset.

Working as on 18 Aug 2018.

Foundation for this scraper can be found here.

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
example-output		example-output
README.md		README.md
scraper.py		scraper.py