Table of Contents

• Background
• plant village dataset
• Files in this repository
• Prerequisites
• How to run
• Results
• References

Project goal: Detection of plant species and diseases from images. We conducted extensive research on various architectures to optimize our results. Initially, we engineered a custom CNN network, achieving high test accuracy with clean data. However, encountering challenges posed by noisy data prompted us to pivot towards pre-trained models. Leveraging transfer learning with established models like VGG, AlexNet, DenseNet, and ResNet, we applied methodologies such as fine-tuning, feature extraction, DoRA, and LoRA. Ultimately, the adoption of the vision transformers architecture emerged as the most effective solution in addressing noise sensitivity. Throughout each phase, we trained the models and assessed their performance across validation and test sets.

We utilized a dataset comprising 38 classes encompassing both diseased and healthy plants. This dataset was sourced from the following link: PlantVillage. example image:

Running Locally Press "Download ZIP" under the green button Clone or download or use git to clone the repository using the following command: git clone https://github.com/LeeBenyamin/Sherleaf-Holmes-project.git (in cmd/PowerShell in Windows or in the Terminal in Linux/Mac)

Open the folder in Jupyter Notebook (it is recommended to use Anaconda). Notes:

1. Every variable named "local*" is a local path in the machine. Please make sure to update the path so it will match to your desired path on your machine.
2. If you are planning to use (load) one of our transfer learning models, due to large size we attach a few links to download these models from our drive:
   • ViT model - https://drive.google.com/file/d/1ItpGNj26819r5OcIZlj5NmGzm8Z-OhZM/view?usp=sharing
   • VGG feature extract - https://drive.google.com/file/d/1GNAxZfA_gxzQUY6qCt4QslBzio8lZhPb/view
   • VGG DoRA - https://drive.google.com/file/d/1kXopEeiHgmqEamhHYg9bIjbJ7QT-eVKU/view
   • VGG LoRA - https://drive.google.com/file/d/1uPnSM4SYSvQkzYWadDlW6nKN_KwS231v/view
   • AlexNet feature extract - https://drive.google.com/file/d/1xy6bmQIGIcqThpWSQeQCasT57reCXfxY/view
   • AlexNet fine-tuning - https://drive.google.com/file/d/1ryd6MQXx5iDCUMK8mdFMJtpOEllZd1VP/view
   • AlexNet DoRA - https://drive.google.com/file/d/13uOFWHGgLFAbml7VBl6xevnQszf75Cot/view
   • AlexNet LoRA - https://drive.google.com/file/d/1uPnSM4SYSvQkzYWadDlW6nKN_KwS231v/view
   • DenseNet feature extract - https://drive.google.com/file/d/1yOkUElPrwqZbiG2erSsb8t9nU7Q4XiTF/view
   • DenseNet DoRA - https://drive.google.com/file/d/1-PmXx5U9PK3NZYmImjRf94UyyIRi52vX/view
   • DenseNet LoRA - https://drive.google.com/file/d/16iFAKxAuSIUUjlmalgi3AsrSusoBlqnB/view
   • ResNet feature extract - https://drive.google.com/file/d/1j2n3Qx3JF-NyiEVsMYlkhp4qWWRFgXsQ/view
   • ResNet fine-tuning - https://drive.google.com/file/d/12SfGD_TYtaBdQo_CuqJPrdUzNpQhDZyh/view
   • ResNet DoRA - https://drive.google.com/file/d/1sYKoUvD5tpkEyAXBwS2DugvlFDvHe38Q/view
   • ResNet LoRA - https://drive.google.com/file/d/1YLmFJOlRHI_FwGevCBarv0gaMvfZx5Y_/view

our cnn results:

In the following graph, you can observe the test accuracy achieved using Optuna and Random Search.

transfer learning results:

The graph illustrates the relationship between accuracy test scores and training time for each model. Notably, the fine-tuned versions of AlexNet and ResNet demonstrate the highest accuracy among all models.

vison transformer results:

tables results:

"An Image is Worth 16x16 Words" paper:https://arxiv.org/abs/2010.11929

vision transformer architecture explanation: https://www.youtube.com/watch?v=qU7wO02urYU

LoRA paper: https://arxiv.org/abs/2106.09685

DoRA paper: https://arxiv.org/abs/2402.09353

icons made by https://www.flaticon.com/