Engel Score Predictor using NER, RAG, and LLM Models

Up close visualization of the Knowledge Graph in NEO4J
Front-end UI of the final product

Inspiration

Looking towards how neurologists classify their epilepsy patients is time-consuming and inefficient. Seeing this pain point in our healthcare system we took to automating that process by using clinical notes to generate an Engel Score, based on the MGH Epilepsy Surgery Outcome Scales criteria.

What it does

This model takes clinical notes developed in prose or email style that are recorded during a doctor-patient visit. The model then views these notes, looking at the symptoms to create an Engel score (which classifies the efficacy of the epilepsy treatment on the patient).

How we built it

We built it using the Anthropic API first to turn our preliminary data into NER, Name Entities, and Relations. This then allows us to create a knowledge graph of symptoms, Engel Scores, and reasons using the JSON file of NER values. This knowledge graph is created and visualized using the NEO4J Aura database and knowledge graph tool. We are then able to extract relationships between the entities of symptoms. We then use an RAG algorithm to query the knowledge graph with another clinic note, which will be parsed into a fine-tuned model of Gemini. Gemini will then return an Engel score based on the relationships we extracted from the synthetic data set and create an explanation of the model that developed this conclusion.

Challenges we ran into

As we were creating this project we ran into challenges with the syntax of all these various models and dependencies. Learning how to use NEO4J, Cypher, Streamlit, Gemini, and Anthropic, but also the background of RAG, LLM, and Knowledge Tree algorithms. But with research and determination, we pulled through with our final product.

Accomplishments that we're proud of

We are proud of the visualization of the knowledge graph using NEO4J and our end product, a deliverable that can be implemented in a hospital setting.

What's next for Engel Score Predictor using NER, RAG, and LLM Models

Moving forward, to expand the functionality of Engel Score Predictor, with more data we can create an application that tracks a user's Engel score over a period of time. This can provide the doctor with metrics about a certain drug or the path the person is on to recovery or success. Furthermore, to create a more polished product, utilizing a database system like Azure so we can continue to train the model, and keep track of all the patient data we collect.