Inspiration
As college students looking for internship opportunities, we realized how tedious the process was for finding internships and we wanted to streamline that process.
What it does
It's a Python and BeautifulSoup program that parses data from a HTML script into a CSV file.
How we built it
A. Import the necessary packages. (link, BeautifulSoup, Request, re, writer)
B. Identify the elements that we want to add to the CSV:
- Job position
- Company name and Location
- Link to application and Job description
C. Identify the necessary HTML tags using developer tools in the browser, and iterate through all the jobs on the page using a for loop.
D. Append them to the CSV file so the user would find the data in a structured and clean way.
Challenges we overcame/What we learned
- Getting accustomed to Git and Github
- Getting used to BeautifulSoup
- Learning how to use python to manipulate CSV files
- Finding a way to scrape a web link within a weblink
- Preventing the CSV file from being overwritten whenever the program is run
What's next for our project
- Making the web script iterate through all the webpages
- Making it scrape in real-time
- Making it applicable to other websites
- Adding filters to the script to personalize user experience
Log in or sign up for Devpost to join the conversation.