• 

  Model layers

Inspiration

Equity and ethics in the medical field have been ever-increasing in importance, especially when it comes to educating physicians on more diverse sets of patients based on race and sex. A clear example of where work is still needed is in skin cancer, where African American and Hispanic patients are much less likely to be diagnosed than Caucasian patients. Modern AIs have attempted to use image processing to diagnose skin lesions with various cancerous or benign conditions, but they have been predominantly trained on datasets of white skin, thus transferring the bias in underdiagnosing dark-skinned patients into these models. To rectify this, we set out to create our own model trained on a more skin-tone-neutral dataset, which we achieved both by diversifying our dataset and through our image processing design.

How we built our project

To achieve a more inclusive and accurate model, we first needed image datasets of skin lesions with relatively consistent frame size relative to the lesion as well as information on whether the lesions were cancerous or not. Datasets from the International Skin Imaging Collaborative (ISIC) proved most diverse and useful, with extra metadata containing diagnoses of eight categories and general lesion location. We then merged these datasets with other smaller and more widely applied datasets such as the Human Against Machine 10,000 (HAM10000) dataset, which had more specific diagnosis categories and lesion location categories. Merging these sets also required manually combining these categories and rewriting metadata. With a complete set of over 43,000 images, we then converted each image into grayscale and masked them to remove indication of skin tone and highlight the outline and depth of the lesion, which are the general two primary modes of diagnosis. To do this, the image was first imported to the script in color. To minimize interference of hair with masking and training, an isolation algorithm was utilized to blur out the hair. Following this, the image was then converted to grayscale and then equalized using a pixel-color distribution histogram. Equalizing the images is what allows us to use our model over varying skin colors. Finally, the possibly cancerous mark was identified in the image. Contours were made using the mark, and a mask was placed around the image to block out extraneous parts of the image. The very last step was to filter out any unusable images as we did not want to train the model on images that were masked in an unidentifiable way. After filtering, our usable image count in our dataset was around 15K. The model is built using the Keras API with TensorFlow as the backend. It takes as input images of size 224x224 pixels, along with the patient's age, sex, and the location of the skin lesion. The ResNet50 architecture is used as the base model for lesion extraction. The base model is frozen so that only the fully connected layers are trained during the training process and speeding up the process. The fully connected layers include age, sex, and localization as inputs to the model, which are combined with the extracted features from the base model. The model's architecture is optimized using the Keras Tuner API, which searches for the optimal set of hyperparameters such as the number of layers, number of units per layer, learning rate, dropout rate, etc. The model is trained using binary cross-entropy loss and optimized using the Adam optimizer. The accuracy of the model is evaluated using the test dataset. Finally, the trained model is saved for later use in the final website. Finally, in order to present the information in a user friendly and accessible way, we coded a website that interacts with our trained model through an API and a GUI, which accepts the inputs needed and displays a result of the model’s diagnosis. In addition, the website outlines the background of our work and the rationale behind the project, as well as a bit of information about each of our roles in this project.

Challenges Faced

Procuring a representative and sizable dataset to train and test on was more difficult than expected. Although the HAM10000 is a large and detailed set, it is predominantly light-skinned. Merging this with other ISIC and publicly available datasets proved immensely useful in diversifying our data, and also solved the issue of having a dataset too small to effectively train. Each image dataset contained between 40-70% of images with metadata labeling the diagnosis as “Unknown, thus rendering them unusable. If we only used one dataset for our model, we would have approximately 5000 images for training, validating, and testing, which is relatively small to be effective. Image equalization and masking prior to training was necessary to remove the factor of background skintone and thus remove race bias. The biggest challenge when it came to this aspect was differing image quality within the dataset as well as the presence of dense hair in some photos. For the most part, we were able to overcome this challenge by using an isolation algorithm to blur out the hair in order to make the mark more distinct. After analyzing the dataset, it became apparent that a significant portion of the images lacked a diagnosis, rendering them unusable for training the model. Consequently, the number of images available for training was reduced from 43k to 15k. Preprocessing the images to mask and equalize them proved challenging due to the presence of numerous objects such as hair that required careful consideration. Additionally, identifying a suitable computational resource to process the reduced dataset and train the model presented a formidable challenge. Fortunately, a team member was able to secure access to a high-performance computer featuring an Quadro RTX 6000 processor for the model training.

What we learned

To plan ahead, we did not plan our resources ahead and did not have access to the better processor for a long time. The lesson learned is to write a plan for what resources we have prior to starting.

Built With

• python
• tensorflow