MedMarket

Inspiration

We were discussing about the challenges and brainstorming ideas and one of our friend just came up with an idea that we did not expect at all. Betting on morality! And hell nahhhh cause it sounds SOOOOOOO wrong, but yet the idea is quite funny and well, it raised us a question: How can machine looking at ultrasound and see bacteria or they can spot out what human might miss out. Thus, MedMarket was born—a somewhat satirically dystopian yet technically serious attempt to combine high-tech medical imaging analysis with the thrill of a prediction market.

What it does

MedMarket is a real-time prediction market interface for medical imaging streams. It allows users to:

Watch Live Ultrasounds: Users view a real-time stream of medical imaging (e.g., echocardiograms) where an AI overlay identifies and tracks biological structures or anomalies.
Micro-Betting on Outcomes: Using Solana (SOL), users can place "Yes" or "No" bets on specific medical questions (e.g., "Is this dude gonna die?", representing the severity of the findings) with real-time odds.
Interact: A live chat and AI assistant provide community interaction and automated insights into the medical data being streamed.

How we built it

We architected a high-performance system to handle real-time video processing and crypto transactions:

Frontend: Built with Next.js and Tailwind CSS, featuring a responsive 4-quadrant layout (Viewer, Chat, Betting Panel, AI Assistant). We used components like LiveBroadcastViewer for the stream and SolanaDeposit for wallet integration.
Backend: A robust Go (Golang) server manages WebSocket connections for the chat and broadcast coordination. It acts as the central hub, routing video frames to the AI service and validating Solana transactions.
AI/ML Engine: A separate FastAPI service runs the SAM (Segment Anything Model). It processes video frames in real-time, performing adaptive segmentation and tracking of regions of interest (ROI) across frames to visualize bacteria or anomalies.
Blockchain: We integrated Solana for fast, low-cost betting. The frontend connects to wallets like Phantom, and the Go backend verifies on-chain transaction signatures to confirm bets.

Challenges we ran into

Real-time AI Synchronization: Running a heavy model like SAM3 frame-by-frame while maintaining a smooth video stream was difficult. We had to implement session-based state management to propagate bounding boxes between frames to avoid re-calculating everything from scratch.
Model Compatibility: We encountered bugs with torch.compile in the SAM3 implementation, which required us to "monkey-patch" the library to ensure stable inference.
Cross-Service Communication: Orchestrating the flow of data between the User (Frontend) -> Go Server -> Python AI Service and back was a complex pipeline to debug, especially ensuring low latency for live betting.

Accomplishments that we're proud of

Seamless Live Segmentation: We successfully got the AI to overlay segmentation masks on the video stream in near real-time, which looks incredibly cool and futuristic.
Crypto Integration: Building a functioning betting system where users can deposit SOL and see it verified instantly by our backend.
Unified UI: The dashboard effectively combines four distinct complex features (video, chat, betting, AI chat) into a single, cohesive, and aesthetically pleasing interface.

What we learned

latency Matters: In a betting application, even milliseconds of delay between the video feed and the betting window can be critical.
The Power of SOTA Models: Working with SAM taught us the capabilities and limitations of modern foundation models in video tracking contexts.
Blockchain Verification: We gained deep insight into how to securely verify Solana transactions on the backend to prevent fraud in a prediction market.

What's next for MedMarket

Production Database: Implementing a robust PostgreSQL database to track user balances and betting history persistently (currently simplified for the hackathon).
Advanced Betting Logic: Moving beyond simple binary bets to more complex parimutuel markets based on specific AI findings (e.g., "Will the bacteria count exceed 1000?").
Extended Medical Models: Training or fine-tuning models for specific pathologies beyond generic segmentation to provide true medical diagnostic value alongside the market.