A Python web scraping tool that extracts comprehensive statistics from GitHub user profiles using Selenium WebDriver.

• 📊 Basic Profile Info: Extract name and username from GitHub profiles
• 🔗 Social Links: Scrape social media links from user profiles
• 📈 Repository Analysis: Get total repositories, stars, and commits
• 💻 Language Detection: Find programming languages used across repositories
• 👥 Social Stats: Followers, following count and ratio calculation
• 🤖 Headless Browsing: Automated Chrome browser in headless mode

• Python 3.6+
• Chrome browser installed
• ChromeDriver (managed by selenium)

1. Install required packages:

   pip install selenium

2. Ensure Chrome is installed on your system

Run the script:

python github_scraper.py

Enter the GitHub username when prompted:

PASTE YOUR ACCOUNT'S URL : username

The script contains three main classes:

Handles basic profile information extraction.

Methods:

• name() - Extracts full name and username
• socials() - Gets social media links from profile
• repo() - Collects all repository URLs

Inherits from BasicInf0 and analyzes repository details.

Methods:

• no_of_stars() - Counts stars for current repository
• no_of_commits() - Extracts commit count using regex
• no_of_languages() - Identifies programming languages
• all_repo_insider() - Iterates through all repos for analysis

Handles follower/following statistics and ratios.

Methods:

• followers_and_follwing() - Extracts follower/following counts
• no_of_repos() - Displays total repository count
• follower_to_follwing_ratio() - Calculates and prints ratio

John Doe @johndoe
Twitter : https://twitter.com/johndoe
LinkedIn : https://linkedin.com/in/johndoe
{'Python', 'JavaScript', 'HTML', 'CSS', 'Java'}
Total stars : 150
Total commits : 1250
5
1.75

1. Initialize: Creates headless Chrome browser instance
2. Get URL: Prompts user for GitHub username
3. Basic Info: Scrapes name, username, and social links
4. Repository Collection: Navigates to repositories tab and collects all repo URLs
5. Repository Analysis:
   • Visits each repository individually
   • Extracts stars, commits, and languages
   • Accumulates totals across all repositories
6. Social Analysis: Calculates follower-to-following ratio
7. Display Results: Prints all collected statistics

op = webdriver.ChromeOptions()
op.add_argument('headless')
driver = webdriver.Chrome(options=op)

# Collects repository URLs for later analysis
self.to_store_repos.append(ele1.get_attribute('href'))

# Extracts numbers from commit text
numbers = re.findall(r'\d+', commits.text)
commit_list = [int(nums) for nums in numbers]

# Uses set to avoid duplicate languages
self.total_lang = set()
self.total_lang.add(a.text)

• No Error Handling: Original code lacks try-catch blocks
• Fixed Selectors: CSS selectors may break if GitHub updates their layout
• No Rate Limiting: May overwhelm GitHub's servers with rapid requests
• Input Handling: Limited validation of user input
• Resource Management: Browser instance not properly closed

1. Element Not Found: CSS selectors may fail on different profile layouts
2. Dynamic Content: Some elements may not load immediately
3. Private Repositories: Cannot access private repo data
4. Rate Limiting: GitHub may block excessive requests

• The script navigates between different GitHub pages automatically
• Uses headless Chrome to avoid opening browser windows
• Processes all repositories sequentially which may take time for users with many repos
• Accumulates statistics across all public repositories

• Only accesses publicly available GitHub data
• Respects GitHub's public profile information
• Use responsibly and respect GitHub's terms of service
• Consider using GitHub's official API for production use

• Add error handling and exception management
• Implement delays between requests
• Add input validation
• Proper browser cleanup
• Export results to files
• Progress indicators for long operations

selenium
re (built-in)

Make sure ChromeDriver is compatible with your installed Chrome version.