JobScraper is a web application that allows users to search for job listings with specific keywords. The application scrapes data from various job boards and presents it in an easy-to-read format. It consists of a Flask backend for scraping job data and a React frontend for displaying the data in a user-friendly UI.
- Scrape job listings from a job board (e.g., Remote OK).
- Search for jobs by keyword.
- Display job title, company name, and other relevant job details.
- Dynamic data fetching and display via React.
- Fully responsive design to support multiple screen sizes.
Before you begin, ensure you have met the following requirements:
- Python 3.x or higher
- Node.js (with npm or yarn)
- Flask
- React (Vite)
- Dependencies for both Flask and React (will be installed automatically)
git clone https://github.com/your-username/jobscraper.git
cd jobscraper
Navigate to the backend folder (backend
) and create a virtual environment (optional but recommended):
cd job-scraper-backend
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
Flask
: For the backend API.requests
: For making HTTP requests.BeautifulSoup
: For scraping job listings.Flask-CORS
: To handle Cross-Origin Resource Sharing (CORS).
flask run
This will start the Flask server at http://127.0.0.1:5000/
.
Navigate to the frontend folder (frontend
) and install the necessary packages:
cd job-scraper-frontend
npm install # or use yarn install
npm run dev # or use yarn dev
This will start the React app at http://localhost:5173/
.
- Flask API should be running at
http://127.0.0.1:5000/
. - React frontend should be running at
http://localhost:5173/
.
- Open the React frontend in your browser (
http://localhost:5173/
). - Enter a job keyword (e.g., "developer", "designer", etc.) into the search input.
- Press the "Fetch Jobs" button.
- View the list of job titles and company names that match your keyword.
- The data is fetched dynamically from the backend, which scrapes job listings from job boards.
JobScraper/
├── backend/
│ ├── app.py # Flask API to scrape data and handle requests
│ ├── requirements.txt # Python dependencies
│ └── ... # Other backend-related files (e.g., config, utils, etc.)
├── frontend/
│ ├── public/ # Folder for public files like index.html, images, etc.
│ │ └── index.html # Main HTML file (template for Vite)
│ ├── src/ # Folder for React source code
│ │ ├── assets/ # Folder for images, icons, or other assets
│ │ ├── JobList.jsx # Component to display the list of jobs
│ │ ├── app.jsx # Main application component
│ │ ├── App.css # CSS specific to the app component
│ │ ├── index.css # Global styles for the entire app
│ │ └── main.jsx # Entry point for the React app
│ ├── .gitignore # Git ignore file to exclude certain files/folders from version control
│ ├── package.json # Frontend dependencies and scripts
│ ├── package-lock.json # Lockfile for npm dependencies (auto-generated)
│ ├── vite.config.js # Vite configuration file
│ └── ... # Other frontend-related files
└── README.md # Project documentation and setup instructions
- The backend scrapes data from a job board (e.g., Remote OK) and returns job titles and company names in JSON format.
- The frontend provides a simple interface where users can search for jobs and view the results in a table.
- For any issues related to CORS (Cross-Origin Resource Sharing), make sure you have
Flask-CORS
configured correctly in the backend.