Skip to content

Anantaverma20/Conscious-Web

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

33 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Conscious Web

A Chrome extension that helps you browse with clarity: scrape and score article links, automate page actions with natural language, fact-check content (including videos), speak commands with voice, and optionally save to Gmail, Google Docs, and more.

Features

Sidebar

  • Toggle sidebar on any page with Ctrl+Shift+S (Windows) / Cmd+Shift+S (Mac)
  • Light/dark mode – follows system preference
  • Central hub for scraping, automation, fact-checking, and chat

Scraping

  • Article links – Fetches and scores links from the current page (Browserbase + optional Redis cache)
  • Caching – Redis cache for faster repeat visits

Browser Automation

  • Natural language commands – Describe what you want in plain English:
    • "Click the login button"
    • "Hide the weather card"
    • "Remove all shorts from my YouTube feed"
    • "Remove ads from this page"
  • Self-evolving agent – On script failure, the system retries with error feedback (Planner → Writer → Daytona validation)
  • Safe Browsing Mode – Guardrails block truly destructive commands (e.g. "remove everything") while allowing targeted removals (remove shorts, ads, sidebars)

News Truth (Fact-Check)

  • Verify page, screenshot, or video – Content-based routing:
    • Video URLs (YouTube, Instagram, TikTok, etc.) → Reka Vision extracts claims from audio + visuals
    • Screenshot → Bem or Gemini image extraction
    • Text/Caption → Fastino or Gemini text extraction
  • Fact Hunter – Searches trusted sources (Reuters, AP, BBC, etc.) via Tavily
  • Truth Judge – Multi-model jury (Gemini + Reka) returns verdicts: True / False / Unverified
  • Page trust score – 0–100 aggregate
  • Deep Research – Yutori Research API for in-depth claim investigation (5–10 min); poll for results

Scout

  • Watch pages – Create scouts to monitor URLs (e.g. "alert when article updated")
  • Yutori Scouting API – Periodic checks, webhook notifications
  • Email alerts – Optional Composio webhook for email when changes detected

Voice

  • Speech-to-text – Microphone button sends audio to backend
  • Transcription – MiniMax (preferred) or OpenAI Whisper
  • Safety check – Fastino classifies voice input before execution

Composio Bridge (Optional)

  • Save to external apps – When you explicitly ask:
    • Gmail – Draft or send emails
    • Google Docs – Create or append summaries
    • Google Sheets – Add rows
    • Google Calendar – Create events
    • Notion – Create pages
  • Chat follow-up – In multi-turn chat, DOM commands (remove, hide) run automation; save/email requests run Composio

Project Structure

├── backend/                 # FastAPI app (port 8001)
│   ├── agents/
│   │   ├── automation/      # Planner, writer, checker, validator (ADK)
│   │   └── truth/           # Fact Hunter, Judge (Gemini + Reka)
│   ├── routes/              # automate, scrape, voice, verify, scout
│   └── services/            # Composio, Bem, Reka Vision, Fastino, Yutori, Daytona
├── Browserbase/             # Scraper API (port 8000) – optional
├── BrowserPlugin/           # Chrome extension (manifest v3)
│   ├── icons/               # 16, 48, 128
│   ├── background.js        # Service worker (API calls, voice upload)
│   ├── content.js           # Injects sidebar
│   └── sidebar.js           # UI logic, scrape, automate, verify, voice
├── .env.example             # Copy to .env and fill in keys
├── run_backend.bat          # Start backend on 8001
└── run_scraper.bat          # Start Browserbase scraper on 8000

Prerequisites

  • Python 3.10+ and a virtual environment
  • Chrome (or Chromium-based browser)
  • API keys (see Environment)

Setup

  1. Clone and enter the repo

    git clone https://github.com/Anantaverma20/Conscious-Web.git
    cd Conscious-Web
  2. Create a virtual environment and install dependencies

    python -m venv venv
    venv\Scripts\activate          # Windows
    # source venv/bin/activate      # macOS/Linux
    pip install -r backend/requirements.txt

    For the Browserbase scraper (port 8000):
    pip install -r Browserbase/requirements.txt

  3. Environment

    • Copy .env.example to .env
    • Fill in at least:
      • BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID (scraping)
      • GOOGLE_API_KEY (Gemini, for automation)
      • TAVILY_API_KEY (News Truth fact-check; required for Verify)
      • DAYTONA_API_KEY (automation script validation)
    • Optional: MINIMAX_API_KEY or OPENAI_API_KEY (voice), REKA_API_KEY (video fact-check + jury judge), YUTORI_API_KEY (scout + deep research), etc.
  4. Start the backend

    • Backend (required): port 8001
      run_backend.bat or:
      uvicorn backend.main:app --host 127.0.0.1 --port 8001
    • Scraper API (optional): port 8000
      run_scraper.bat or:
      uvicorn Browserbase.api:app --host 127.0.0.1 --port 8000
  5. Load the extension

    • Open Chrome → chrome://extensions
    • Enable Developer mode
    • Click Load unpacked and select the BrowserPlugin folder

Usage

  • Toggle sidebar: Ctrl+Shift+S (Windows) / Cmd+Shift+S (Mac), or use the extension icon
  • Article links: Open the sidebar; it scrapes the page and shows links (backend on 8001; scraper on 8000 if used)
  • Automation: Type a command (e.g. "Remove all shorts", "Hide the sidebar") and click Run or press Enter
  • Voice: Click the microphone, speak your command, click again to stop; then Run to execute
  • Verify Truth: Click the shield icon to fact-check the current page or screenshot
  • Deep Research: After verifying, use Deep Research for a claim to run an in-depth Yutori investigation (5–10 min)
  • Scout: Create scouts from the sidebar to watch pages for changes

Environment

Variable Purpose
BROWSERBASE_API_KEY, BROWSERBASE_PROJECT_ID Scraping (Browserbase)
GOOGLE_API_KEY Automation (Gemini), News Truth Judge
TAVILY_API_KEY News Truth – Fact Hunter search
DAYTONA_API_KEY Automation – script validation
REKA_API_KEY Video fact-check (Reka Vision), Truth Judge jury
YUTORI_API_KEY Scout (page watching), Deep Research
BEM_API_KEY News Truth – claim extraction (optional; Gemini fallback)
FASTINO_API_KEY PII redaction, claim extraction, safety classification
MINIMAX_API_KEY or MINI_MAX_API_KEY Voice transcription (preferred)
OPENAI_API_KEY Voice transcription (Whisper fallback)
COMPOSIO_API_KEY, COMPOSIO_ENTITY_ID Gmail, Docs, Sheets, Notion, Calendar
REDIS_URL Optional: cache scrape results

Full list and comments are in .env.example. Do not commit .env.

API Endpoints

Endpoint Description
POST /automate Run automation pipeline (body: url, command, page_context)
POST /scrape Scrape URL (body: url, refresh_cache?)
GET /scrape Scrape URL (query: url)
POST /verify Fact-check page, screenshot, or video (body: url, content?, image_base64?)
POST /verify/deep-research Start deep research for a claim
GET /verify/deep-research/{task_id} Poll deep research result
POST /scout Create scout to watch a page
GET /scout/list List scouts
DELETE /scout/{id} Delete scout
POST /voice/transcribe Transcribe voice audio
GET /health Health check
GET /health/truth Truth config diagnostic

License

See repository for license information.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors