A Web Scraper that uses Python3 and Scrapy to scrape Brickset - A LEGO database.
- Python 3.x
- Scrapy
scrapy runspider brickset-scraper > brickset-scraper.json
Example output JSON files are here.
By default, past 5 years of bricksets are scraped.
Change the START_YEAR and END_YEAR in scraper.py to scrape desired years.
Example output has been beautified with JSON Lint for readability.
Working of this web scraper depends on the source at Brickset.
Working as on 18 Aug 2018.
Foundation for this scraper can be found here.