README.md

urlscrapper

A web scrapper to extract the URLs embedded on the Website.

Installation

go install -v github.com/thevillagehacker/urlscrapper@latest

or

git clone https://github.com/thevillagehacker/urlscrapper.git
cd urlscrapper
go build

urlscrapper binary will be created and Move the binary to the required folder and add the path to the environment variables.

Usage

go run scrapper.go -u https://example.com

or

urlscrapper -u https://example.com

Update

• Can now output the scrapped data into a file using -o flag.
• Can now print HTTP status code of the scrapped URLs using -sc flag.

Example

Print the status code data

urlscrapper -u https://example.com -sc

Print the results to the output file

urlscrapper -u https://example.com -o out.txt

Print the results with status code to the output file

urlscrapper -u https://example.com -sc -o out.txt