Dermalab

Inspiration

dermalab was inspired by a very common problem that doctors face: disease misdiagnoses. Skin diseases are often misdiagnosed by dermatologists because conditions often contain many similarities in appearance. Doctors - or even patients at times -, unaided with advanced technology, are often less sure of skin disease diagnoses, which may result in incorrect treatment, or even failure of treatment altogether. Skin disease misdiagnosis is primarily relevant in low-income communities where there is a lack of medical doctors with the appropriate specialty and expertise.

Machine learning (ML) and Artificial Intelligence (AI) technologies can be used to detect differences more accurately than the human eye but currently aren’t widely adopted due to a lack of algorithms that are able to conduct a variety of tasks for disease diagnosis. Current algorithms are specialized to a single task in this journey and are often difficult to use, for both patients and doctors.

Therefore, we strove to create an algorithm that was able to successfully leverage these technologies while catering to the entirety of the disease diagnosis process. We wanted to enable doctors to have more confidence in their diagnoses as well as provide helpful, understandable explanations to patients during this journey. dermalab is what we've produced to solve these issues, and we hope you enjoy it!

What it does

dermalab helps fill the aforementioned gaps via real-time skin condition classification, disease severity calculation, and spread predictions using cutting-edge technology. We've consolidated all of our amazing features on a website that anybody can use!

Users first have an option of uploading pictures of the affected area to the website. Our machine learning algorithm will then classify and identify the disease or condition that is present, if any. However, we don't stop there! Users also receive a summary of their conditions - without all of the hard-to-understand medical jargon. That way, both patients and doctors are on the same page when it comes to understanding a disease diagnosis.

Our web page also gives users the choice of completing a questionnaire to return the severity level of the disease. We provide explanations for each severity level to ensure that this process is empathetic to patients in the best way possible. Our deep learning algorithm also outputs the predicted disease spread for doctors as well. This was done via further analysis through deep learning and MATLAB to understand region, rate, and direction of disease growth. We used the InceptionResNetV2 model, with lesion images and their annotated masks to train the deep learning model, to perform semantic segmentation on unseen data. Once the semantic segmentation of lesions are done, thereby performing binary masks, MATLAB could be used to perform an onion ring segmentation of the region, where dilation and erosion of mask is done and the difference is taken.

Combined, the variety of services that dermalab provides a robust interface for doctors and patients during disease diagnoses, and helps prevents misdiagnoses.

Check it out here!

How we built it

dermalab can be primarily be split into the frontend and backend. The frontend consists of the web application and the backend contains the ML/AI/deep learning/LLM models.

The backend has four objectives: A. Diagnosis: Classification of skin diseases B. Evaluation: Calculation of disease severity level C. Prognosis: Prediction of disease spread D. Explanation: Understandable relaying of information to patients

A. Diagnosis: Classification of skin diseases dermalab uses machine learning and image processing to identify the type of disease that a person is suffering from.

We trained a random forest model on a dataset consisting of multiple labeled types of skin diseases
Weights of trained model were saved and loaded with the user-provided image containing affected region.
Model returns the class the image falls under,
Success! Identification of the diseases through the image provided is outputted to the user.

B. Evaluation: Calculation of disease severity level While it is certainly helpful to identify the diseases the person might have, the severity of the diseases plays a key role in determining what measures to take next.

Users are asked to give their medical history and the severity level of their disease is given.
Random Forest model is trained with cross validation + medical history dataset of patients with Eryhemato-Squamous dataset
Results vary from 0 to 5, where 0 indicates the absence of disease and 5 indicates the greatest severity.

C. Prognosis: Prediction of disease spread dermalab also predicts disease progression.

Further analysis via deep learning and MATLAB to understand region, rate, and direction of disease growth.
InceptionResNetV2 model with lesion images and annotated masks for semantic segmentation
Onion ring segmentation of affected region performed via MATLAB
More informed doctors making better decisions for treatment plans

D. Explanation: Understandable relaying of information to patients dermalab supports powerful physician-patient interactions. When it comes to disease classification, the Metaphor API returns up-to-date information about disease, overcoming a limitation of ChatGPT. In terms of disease severity (0 to 5), Llama 2.0 and Replicate provide explanation of severity, allowing us to show that LLMs can be used for language simplification.

Other Tools!

MATLAB dermalab uses MATLAB as a major tool to analyse the spread of lesions in skin: semantic segmentation further segmented to onion ring segmentations, providing information about origination, growth rate, and direction.

Metaphor We use Metaphor's API because we believe that medical communication requires the most recent research. Patients have a right to access the most recent information available, and Metaphor's API is the perfect solution.

The frontend was connected using Flask and Github.

Challenges we ran into

One challenge we ran into was running the Metaphor API. It took a lot of troubleshooting, debugging, and asking for help to resolve the issue, but we're glad we did!

Another challenge was the lack of annotated dermatological data. Since we aren't medical professionals that are qualified to annotate available data, we had to get creative when it came to finding data that could achieve our objectives.

Accomplishments that we're proud of

We're proud, first and foremost, that we were able to bring dermalab to completion. One accomplishment was linking the frontend (web page) and the backend (multiple ML models) to create a seamless interface for the user. Another major accomplishment was troubleshooting with Metaphor's API until it was able to output the results we wanted.

What we learned

We're so grateful for everything that we learned!

Combining frontend and backend via Flask
Using Metaphor's API to return improved responses from ChatGPT
Employing machine learning, deep learning, and MATLAB for dermatology applications

What's next for dermalab

We want dermalab to be a tool for all doctors and patients. To get there, we want to refine our algorithm to be able to take into account more disease classes, so that it can classify a greater variety of diseases.