The aim of our project is to develop a system that allows individuals to obtain a preliminary analysis of skin problems through an image, serving as a pre-consultation step before seeing a medical professional. We leveraged pretrained models and utilized the DoRA methods to enhance the accuracy and reliability of the analysis, while exploring if good results can be achieved with less computational resources.

Skin and subcutaneous diseases are common health problems worldwide and are the leading causes of the global disease burden. Early detection of skin diseases sometimes is crucial and lifesaving. Using deep learning as a tool for detection can benefit us in many aspects such as quicker, cheaper, and accessible skin problem analysis and assisting dermatologists classifying skin problems.

In this project we examined the use of pretrained models – DINOV2 (ViT) and ResNet-101 (CNN). We aimed to see if we can get better results using large pretrained models with fine tuning methods (DoRA).

Figure 1: Block Diagram.

We used a Kaggle dataset that contains 27,153 skin disease labeled images with 10 different disease classes:

1. Eczema – 1,677 images
2. Melanoma – 15,750 images
3. Atopic Dermatitis – 1,250 images
4. Basal Cell Carcinoma (BCC) – 3,323 images
5. Melanocytic Nevi (NV) – 7,970 images
6. Benign Keratosis-like Lesions (BKL) – 2,624 images
7. Psoriasis pictures Lichen Planus and related diseases – 2,000 images
8. Seborrheic Keratoses and other Benign Tumors – 1,800 images
9. Tinea Ringworm Candidiasis and other Fungal Infections – 1,700 images
10. Warts Molluscum and other Viral Infections – 2,103 images

Figure 2: Classes images.

To run the code, follow these steps:

• Download the Dataset:

  • Go to the Kaggle dataset and download the dataset. You can use Kaggle's API or download it manually.

• Upload to Google Drive:

  • Upload the downloaded classes folders to your Google Drive account.

  • Place the classes folders in the directory 046211/project within your Google Drive.

  • Ensure that the path to the dataset in your code matches the Google Drive directory:

    dataset_path = '/content/drive/MyDrive/046211/project'

• Run the Code:

  • Once the classes folders is correctly placed in the specified directory and your notebook is set up to access Google Drive, you can run the code as usual.

We recommend using a GPU on Google Colab for running the code, as it is very compute-intensive.

Comparing the two pretrained models with Dora fine-tuning results:

We observed improved results with the DINOV2 model overall; however, the ResNet-101 model performs better in three specific classes. As expected, the classes with more images show better performance.

Figure 3: ResNet101 – Training and Validation loss and accuracy.

Figure 4: DINOV2 – Training and Validation loss and accuracy.

The loss and accuracy curves are improving with more epochs, showing better performance. Notably, the DINOV2 curves demonstrate greater consistency between training and validation. In contrast, the ResNet101 curves exhibit more noise and instability.

Comparing to a scientific paper:

We compared our results with a study that used our dataset with various models. Our findings suggest that training full models without pre-trained fine-tuning methods is more effective. The study reported model accuracies ranging from 91.75% to 99.59%, while our model's accuracy stands at 77.3% and 73.3%.

• Pretrained Models: We anticipated achieving better results with pretrained models through fine-tuning compared to developing a new architecture from scratch. However, our findings indicate that training full models yields superior results. Nonetheless, full model training requires substantial computational resources and time, which were constraints for our project.
• ResNet101 VS DINOV2: Our comparison revealed that DINOV2 (ViT) outperforms ResNet101 (CNN) in terms of accuracy and stability when using pretrained models with fine-tuning. This disparity is likely due to DINOV2 being trained on 142 million images compared to ResNet101’s 1.4 million, marking a significant difference in training data volume.
• Classes Accuracy: As demonstrated in the table and graph, accuracy improves with an increasing number of images. For precise diagnostic results in specific classes, acquiring more data related to those diseases is crucial.

For future improvements, we propose gathering additional data and integrating it into our models. We also plan to explore the addition of DORA layers in the middle of the models and experiment with more hyperparameter tuning using tools like Optuna.

1. Skin Diseases Image Dataset
   Kaggle. Link

2. Enhancing Skin Disease Classification and Privacy Preservation through Federated Learning-Based Deep Learning
   Raj Gaurang Tiwari, Himani Maheshwari, Vinay Gautam, Ambuj Kumar Agarwal, Naresh Kumar Trivedi. 2023 International Conference on Artificial Intelligence for Innovations in Healthcare Industries (ICAIIHI). Link

3. DINOv2: Self-Supervised Vision Transformers
   Meta. Link

4. ResNet-101 Model Reference
   Microsoft. (n.d.). ResNet-101. Link

5. DoRA: Improving on LoRA’s Parameter-Efficient Fine-Tuning
   LM Po. Medium, 2024. Link