Desktop GUI tool for scraping product images & metadata from e-commerce websites
_{Give it a spreadsheet of URLs. Get back a beautifully formatted workbook with images, titles, and more.}

Fresh features to make scraping smoother and faster.

   +-----------------+          +-------------------+          +------------------+
   |                 |          |                   |          |                  |
   |   INPUT FILE    |  -----→ |   web_scrappie    |  -----→ |     OUTPUT       |
   |                 |          |                   |          |                  |
   |  .xlsx / .ods   |          |  Opens browser    |          |  Excel workbook  |
   |  .xls  / .pdf   |          |  Scrolls pages    |          |  CSV / JSON      |
   |                 |          |  Extracts images  |          |  Downloaded imgs |
   +-----------------+          +-------------------+          +------------------+

_{Input File (URLs by category) → Scrape (automated browser) → Output (formatted workbook + images)}

Clean dark UI -- configure settings, pick your file, hit Start

Live progress tracking with color-coded log output

I got tired of manually copying product images and titles from retail sites. Now I maintain a spreadsheet of URLs and let this handle the rest.

• Python 3.8+
• Google Chrome installed

# Clone
git clone https://github.com/GopalGB/web-scrappie.git
cd web-scrappie

# Run (dependencies install automatically on first launch)
python web_scrappie.py

Or install dependencies manually first:

pip install -r requirements.txt
python web_scrappie.py

Your spreadsheet needs at least two columns. The tool auto-detects which column has categories and which has URLs based on header names.

• If headers aren't recognized, it assumes column 1 = category, column 2 = URL
• Multiple sheets are supported -- each sheet is processed independently
• PDFs: extracts every URL found in text and annotations

All configurable from the GUI:

The scraper tries three methods in order, using the first one that returns results:

1. Preloaded State -- Many React/Next.js sites embed product data in window.__PRELOADED_STATE__ or window.__NEXT_DATA__. Pulls structured data directly. Fastest and most reliable.

2. Product Links -- Finds all <a> tags containing <img> elements. Extracts image source and alt text as the product title.

3. Standalone Images -- Falls back to grabbing every <img> with alt text, filtering out icons and tiny spacer images.

The Excel workbook includes:

• Summary sheet -- category names, item counts, generation timestamp
• One sheet per category -- columns: #, Title, Image URL, Page URL
• Embedded thumbnails (if enabled) -- scaled to fit cells without stretching
• Styling -- alternating row colors, frozen header rows, auto-filters, clickable hyperlinks

New in v1.2.0: also exports CSV and JSON files alongside the workbook.

• Getting zero results? Uncheck headless mode -- some sites aggressively block headless browsers
• SSL errors? The tool handles certificate issues automatically, useful on corporate networks
• Small images skipped -- files under 500 bytes (tracking pixels, spacers) are filtered out
• Deduplication -- results are deduplicated by image URL within each category

MIT

Made by Gopal Bagaswar