Skip to content

A Python program that takes a website address and parses its data to be saved to a text file.

Notifications You must be signed in to change notification settings

jairo8925/web-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Web Scraper

A Python program that takes a website address and parses its data to be saved to a text file. Sends an HTTP request to the website and uses BeautifulSoup to parse its data. Specifically, this program scrapes articles from https://www.nature.com/nature/articles and saves each one in a separate .txt file.

To start, provide the number of pages to specify the number of pages on which the program should look for the articles. Next, provide the type of article that the program should look for (eg. News, Correspondence, Research Highlight). After the program is done, the articles will be saved in the directories Page_1 to Page_N (N corresponds to page number), depending on what page an article was found in.

My other project (Multilingual Online Translator) that uses web scraping can be found here

About

A Python program that takes a website address and parses its data to be saved to a text file.

Topics

Resources

Stars

Watchers

Forks

Languages