PICO Scholar

Code available: https://github.com/datagero/pico-scholar

Inspiration

Systematic reviews in biomedicine are essential for advancing scientific knowledge but are often slow and labor-intensive. The inspiration for PICO Scholar stemmed from the need to accelerate this process while maintaining rigor. By leveraging AI-driven technologies, we aimed to create a solution that streamlines reviews, enhances accuracy, and ensures researchers work with the most up-to-date information and best AI models for their specific needs.

Recent research efforts, such as the work in "Enhancing Abstract Screening Classification in Evidence-Based Medicine: Incorporating Domain Knowledge into Pre-trained Models (2024)", have made significant progress in this area. TiDB’s scalable, efficient data handling and integration capabilities enable us to bring these research findings and transformer-adapter models into practical, real-world applications, making a meaningful impact for researchers and clinicians alike.

What it does

PICO Scholar automates the entire systematic review workflow, from planning to reporting. PICO, which stands for Population, Intervention, Comparison, Outcome, is a widely used method for evaluating research papers. Our platform integrates with databases like PubMed to ingest literature into our TiDB Datastore and uses employs advanced AI tools, such as TiDB Vector Search and LlamaIndex, for PICO extraction and metadata management. The system is designed with AI models that can be fine-tuned, and our data models are built to support feedback loops that improve these AI models over time. This is crucial for handling specialized domains and for enhancing entity extraction, embeddings, and ranking systems, ultimately reducing the time required for systematic review

How we built it

We built PICO Scholar using a combination of advanced AI tools and robust technologies. TiDB Serverless powers scalable data handling, while LlamaIndex manages document processing. We utilized parameter-efficient fine-tuning to adapt models for specialized tasks and developed multiple indexes for embeddings stored in TiDB VectorStore at different granularities. The system design includes feedback loops to continuously refine these AI models, improving accuracy and adaptability. A RAG-powered chatbot allows researchers to explore selected full papers in depth. The platform is fully containerized for easy deployment, with semantic search capabilities that evolve through user feedback, ensuring ongoing improvement.

Challenges we ran into

The main challenge we faced was serving and Dockerizing the app, as our team had limited experience with this process. However, we successfully used Gitpod to host our application through a simple docker-compose command. While our app includes RAG components and a serving endpoint, due to time constraints, these were left out from the network since they use a Streamlit port. Users are welcome to try these features out locally.

We also built pipelines to load PubMed data at scale, including both abstracts and full documents, but were only able to load a small portion of the potential due to time constraints. Additionally, integrating these AI tools with various back-end and front-end technologies and ensuring their seamless collaboration posed significant technical challenges.

Accomplishments that we're proud of

We are proud of several key accomplishments in developing PICO Scholar. Firstly, there is the successful integration of advanced AI tools into a user-friendly platform that significantly accelerates the systematic review process. Secondly, we also take pride in our platform’s scalability, which allows it to handle large datasets and diverse research needs efficiently.

What we learned

Throughout this project, we learned the importance of balancing cutting-edge technology with user-centric design. While AI can greatly enhance efficiency and accuracy, the value of human expertise and feedback cannot be overstated. We had the opportunity to gather insights from two PhD academics who expressed interest in using our tool for systematic research, reinforcing the need for user-focused development.

What's next for PICO Scholar

Looking ahead, we plan to enhance PICO Scholar by integrating more collaborative features and user management tools, enabling team-based research and further streamlining the systematic review process. We aim to integrate our tool with the best AI models available and load it with a significant amount of freely available academic literature, making it useful for a wide range of researchers.

We also intend to keep improving the sophistication of our data gathering and AI feedback loops. Our vision is to support multilingual AI fine-tuned models, helping to bridge the AI data gap and bring the tool to researchers in need. Ultimately, our goal is to make PICO Scholar the go-to tool for accelerating and enhancing systematic reviews in biomedicine and beyond.