TruthDose

Fighting health misinformation and disinformation online.

Comment

Inspiration

Having suffered from a chronic hormonal health condition, I've always had to weed through so much information online to understand and identify the myths from actual health information which has been proven. Health misinformation is a global challenge in today's world. 800 people died during COVID-19 from drinking methanol, believing misinformation that it would cure the virus.

To address the challenge of health misinformation, we built TruthDose. When you're reading an article or health post online, you can simply highlight the text and use TruthDose to check if there's actual research backing it up, without leaving the page.

What it does

TruthDose is a Chrome extension that instantly fact-checks health claims with peer-reviewed scientific research. Here's how it works:

Two Ways to Verify:

  1. Highlight any health claim on a webpage, right-click, and select "Verify this claim"
  2. Open the extension directly and type in any health claim you want to check

What You Get:

  • Truth Score (0-100%): Overall scientific consensus on the claim

  • Top Research Papers: Ranked using our intelligent hybrid system that balances:

    • Recent papers which have a high citation
    • Highly-cited established research

  • Meta data from abstracts:

    • Study population, demographic
    • Sample size
    • Statistical significance (p-values, effect sizes)
    • Study methodology (systematic review, RCT, case study, etc.)

  • Plain-English Summaries: Summaries of complex abstracts

  • Sentiment Analysis: Does each paper SUPPORT, CONTRADICT, or remain NEUTRAL on the claim input by the user

  • Translation Support: Currently supporting Spanish

Why the demographics matter: There's a huge representation gap in medical research—not all communities and genders are represented equally. Women are often overmedicated because drug trials are done primarily on men. It's crucial to know who was studied, not just what was studied.

How we built it

Tech Stack:

  • Chrome Extension (Manifest V3) with content scripts for text highlighting
  • Chrome Built-in AI APIs :
    • *Prompt API *: Query rephrasing, metadata extraction, sentiment analysis, and final analysis generation
    • Summarizer API: Condenses research abstracts into simple, key findings
    • Translator API: Translates results into Spanish
  • PubMed API: Gold-standard medical literature with keyword matching
  • Semantic Scholar API: Cross-disciplinary papers with citation velocity tracking

Our Pipeline:

  1. Query Rephrasing: Convert health claims provided into optimized phrases for searching papers using Prompt API
  2. Hybrid Paper Search: Search PubMed + Semantic Scholar in parallel (up to 10 papers each)
  3. Paper Enrichment: Enrich PubMed papers with Semantic Scholar citation data, deduplicate by DOI
  4. Quality Ranking: Rank papers by our custom quality score:
    • 60% Influential Citations (log scale, filters self-citations)
    • 40% Recency Score (1.0 for current year, 0.0 for 20+ years old)
  5. Metadata Extraction: Use Prompt API to extract study demographics, sample size, and statistical significance from abstracts
  6. Abstract Summarization: Summarize all abstracts in parallel using Summarizer API
  7. Sentiment Analysis: Analyze each paper with Prompt API to determine if it supports/contradicts/is neutral to the claim, with confidence scoring
  8. Truth Score Calculation: Weighted formula based on paper quality scores and sentiment confidence
  9. Final Analysis: Generate comprehensive fact-check summary using Prompt API
  10. Translation (if selected by user): Translate results using Translator API (currently works only for Spanish)

Challenges we ran into

  1. Speed Optimization: On-device AI processing is powerful but slower than cloud APIs. The extension currently takes 40-60s.

  2. Query Rephrasing and non-deterministic nature of LLMs: Had to try different prompts to see which ones provided better rephrased query majority of the time.

  3. API Rate Limits: Both PubMed and Semantic Scholar have rate limits. We implemented intelligent caching and request batching to stay within limits while maintaining responsiveness. However, we still hit the semantic scholar rate limit quickly. It falls back to PubMed if that happens.

  4. Learning Chrome Extension Development: This was our first Chrome extension! Understanding Manifest V3, service workers, content scripts took time, especially when integrating with Chrome's Built-in AI APIs.

Accomplishments that we're proud of

  1. Addressing a Critical Problem: We're tackling health misinformation head-on in an era where false health claims can lead to death.

  2. Privacy-First Design: Unlike other fact-checking tools that send your queries to servers, TruthDose processes everything locally using Chrome's Built-in AI. Your sensitive health queries never leave your device - no logging , no data collection, no privacy compromise.

  3. Highlighting Representation Gaps: By extracting and displaying study demographics, we're making it transparent when medical research doesn't represent all populations. This empowers users to make more informed decisions about whether research findings apply to them.

  4. Intelligent Ranking: Our quality score balances influential citations (peer validation) with recency (breakthrough research), so users see both established evidence and cutting-edge findings.

What we learned

  1. Chrome's Built-in AI APIs are Transformative: The Prompt API with Gemini Nano enables sophisticated NLP tasks entirely on-device, opening new possibilities for privacy-sensitive applications. This changes the game for health tech, where privacy is paramount.

  2. Client-Side AI Changes UX Patterns: We can now be proactive with AI features—analyzing content as users browse without worrying about API costs, rate limits, or privacy violations. This enables new interaction patterns that weren't possible before.

  3. Optimization Matters with On-Device Models: Working with on-device models made us realize the that we need to be cautious about the amount of processing / api calls we make.

  4. Medical Research Has Serious Representation Gaps: Building this tool made us confront how underrepresented women, minorities, and non-Western populations are in clinical trials. This isn't just an abstract problem—it has real consequences for people's health decisions.

  5. Hybrid Approaches Work Best: Combining PubMed's gold-standard medical literature with Semantic Scholar's broader, more recent cross-disciplinary papers gives better coverage than either source alone.

What's next for TruthDose

Misinformation to "Missed Information":

While misinformation is a huge problem, there's also the challenge of "missed" information. Many health remedies from regional traditions, home remedies, and alternative medicine might actually work but lack formal research funding. These approaches are often based on generations of community experience.

Community Insights Feature:

Our next step is to add a crowdsourced community insights component that analyzes real experiences from social media:

  • Data Sources: Reddit health communities, YouTube health channels, TikTok wellness content, health forums
  • Smart Analysis: Use Chrome's AI APIs to process comments and identify patterns in what actually helps people
  • Sentiment & Demographics: Aggregate experiences by demographics to show "this helped 70% of women with PCOS who tried it"
  • Credibility Scoring: Weight insights by user history, community reputation, and consistency across sources

Additional Features:

  • Multimodal Input: Leverage Prompt API's image support to fact-check health infographics and charts shared on social media
  • Expanded Data Sources: Integrate FDA adverse event reports, MedlinePlus, clinical trial databases
  • Hybrid AI Option: For users who consent, offer cloud-based analysis for more comprehensive research synthesis
  • Symptom Tracker: Help users track their own health experiments and correlate with community insights
  • Personalized Health Profiles: Save anonymously which demographics users want to prioritize in research results

Our Vision: Combine the rigor of peer-reviewed research with the wisdom of lived community experiences, creating a more complete picture of what works—especially for underrepresented populations. Because everyone deserves to make informed health decisions based on both science AND real-world experience.

Built With

Share this project:

Updates