ifyShop

shop with confidence

Comment

🧐 Inspiration

We’ve all been there: scrolling through social media, you see a stunning mechanical keyboard or a perfect lamp setup. But the moment of inspiration is instantly killed by frustration. "What model is that?" "Is it actually good?" "Is this random website selling it a scam?"

In an era of drop-shipping scams, fake 5-star reviews, and "fast fashion" waste, shopping has become a minefield. We were inspired to build ifyShop not just as a search engine, but as a guardian. We wanted to use AI for Good, to protect consumers from financial scams, highlight sustainable choices through an "Eco-Score," and bring radical transparency to e-commerce. We set out to bridge the gap between "seeing something cool" and "making a confident, safe purchase."

💭 What it does

ifyShop is an intelligent visual shopping agent that turns any image into a deeply researched, safety-first buying guide. It employs a sophisticated Two-Stage Agentic Pipeline:

  1. Visual Discovery (The Eye): Users upload an image or screenshot. The agent uses Gemini 2.0 Flash Vision to detect objects and understand context (e.g., "gaming setup"), the information of which will be sent to Google Lens for specific identification of products.
  2. Market Scouting (The Explorer): Once an item is identified, the agent autonomously dispatches a "Scout" to crawl the web (via Tavily and SerpAPI) for live pricing, availability, and alternatives. It uses Snowflake Vector Search to recall previously verified high-quality products.
  3. The Scam Shield (AI For Good): Perhaps our most impactful feature; a dedicated "Skeptic" agent analyzes seller reputation and review patterns to flag potential drop-shipping scams or bot-generated praise. It also calculates an Eco-Score based on material durability and sustainability signals.
  4. Personalized Verdict: Finally, the system synthesizes all data: pricing, specs, real reviews, and user preferences into a clear "Buy," "Consider," or "Avoid" recommendation.

👷‍♂️ How we built it

We engineered a modular, agentic system orchestrated by LangGraph.

  • The Brains (Google Gemini): We leveraged Gemini 2.0 Flash for every node in our graph. Its multimodal capabilities allowed us to pass images directly into the context window for detection, while its reasoning speed was crucial for our parallel "Scout" and "Skeptic" agents.
  • ** Product Identification (Google Lens via SerpAPI) ** Once a target object is confirmed, we delegate exact product identification to Google Lens. This dramatically reduces hallucinations and ensures real-world accuracy
  • The Memory (Snowflake): We implemented Snowflake Vector Search to store embeddings of verified, high-quality products. This allows the agent to "remember" good finds and perform semantic similarity searches against a curated catalog, reducing hallucinations.
  • The Orchestrator: We used LangGraph to define a cyclic state graph: State = { Image, BoundingBox, ScoutData, SkepticVerdict }. The flow moves from Vision Node → Scout Node → Skeptic Node → Analysis Node.
  • Parallel Execution: To minimize latency, the "Scout" (Search) and "Skeptic" (Critique) nodes run in parallel using Python's ThreadPoolExecutor.
  • Frontend: Built with React and Vite, featuring an interactive "Lens-like" overlay where users can click bounding boxes drawn by the AI.

🚧 Challenges we ran into

  • Agent Challenge: Managing multiple AI agents is difficult. Initially, our "Scout" agent would get distracted by irrelevant accessories. We had to implement strict Pydantic models to force the agents to return structured JSON data, keeping them focused on the specific target.
  • Latency: Deep research takes time. Our initial prototype took 45+ seconds. We solved this by parallelizing the "Scout" and "Skeptic" nodes using Python's ThreadPoolExecutor, cutting response time down to 20 seconds.

🏆 Accomplishments that we're proud of

  1. The "Skeptic" Agent: We are incredibly proud of the node that specifically looks for fake reviews. Seeing the AI correctly identify that a 5-star product had "suspiciously repetitive phrasing" was a huge win for our "AI for Good" goal.
  2. Technically Impressive Workflow: Successfully merging "External" search results (live web) with "Internal" knowledge (Snowflake Vector Store) to give the best recommendation.
  3. Interactive Vision: Building a seamless chat-to-vision interface where you can say "What's that red thing?" and have the AI highlight the correct object in the image.

📚 What we learned

  1. Multimodality is the future of Search: Text queries are limiting. Letting the model "see" the product context (e.g., knowing a keyboard is in a cramped office vs. a pro studio) changes the recommendation entirely.
  2. Agent Specialization: Breaking the problem into small, specialized agents (one just for looking, one just for pricing, one just for critiquing) yielded far better results than one giant prompt.
  3. Trust is Key: Users don't just want a link; they want verification. Building the "Skeptic" agent taught us that safety is a feature.

⏭️ What's next for ifyShop

  1. AR Try-On: Integrating Augmented Reality to see how that detected lamp would look in your room.
  2. Browser Extension: Bringing the "Skeptic" agent directly to Amazon/Shopify pages as a sidebar assistant.
  3. Collaborative Shopping: Allowing groups to share a "Cart" where the AI acts as a mediator to find products that everyone agrees on.

Built With

Share this project:

Updates