Every year, millions of apples are harvested in the Netherlands, which must be sorted before they go to consumers. This is done partly by the sorting machine and partly by hand. Two students from Fontys argue, based on their research findings, that with the help of deep learning it should be possible to determine the quality of apples and then separate the apples on the basis of these factors. Using a roller conveyor and sorting cameras, the machine ensures that the apples with the right sizes come out through the right exits. It is then human labor to check the apples for quality.
It seemed to me that the worst offenders were often car companies and their suppliers. The software development teams in these organizations seemed stuck in a kind of through-the-looking-glass world called Autosar, where common words and concepts seemed to have a different meaning. When we talked about event-driven, service-oriented systems, we often got blank stares. I could never quite grasp the nature of the problem or the communication gap. And so we never got a foothold in the automotive world.
Felix van der Heijden and Jelle Stappers, then fourth-year mechatronics students at Fontys Hogeschool Techniek & Logistiek in Venlo, were commissioned by the Stappers Baarlo fruit-growing company to investigate whether the apple quality sorting process could be automated. In doing so, they wanted to make use of deep learning technology. Deep learning is a form of artificial intelligence that allows objects to be detected automatically. This is achieved by placing a camera with a deep learning application in the sorting machine, allowing the machine itself to distinguish the good apples from the bad ones. After this, it can perform actions to sort by quality.
To detect objects, the software must first know what the object in question looks like. To do this, objects are labeled. In this case, it was apples and quality factors, whose labels are then given to the deep learning algorithm so that the system can "train. During this training, images are compared with each other to assign weights to various properties, such as color and shape. The final result is a "classes" and a "weight" file that the algorithm can use to recognize objects.
The software / deep learning application Yolo (you only look once) was chosen, because knowledge of it had already been acquired. In combination with Qt Creator (an Integrated Development Environment), it should be possible to detect the quality of apples.
"In order to recognize apples, a data set had to be created first. To do this as accurately as possible, the practical situation with camera was used."
It was decided to switch from Yolo to Tensorflow, as this system offers more support and is more widely used. Multiple neural networks are available in Tensorflow. For testing, Mobilenet was used. Detecting objects in an image takes 20 ms with this network. Two datasets were then realized: one to detect apples and one dataset to detect quality. Tensorflow proved to be better at detecting individual apples, but due to lack of data from rejected apples, it was not possible to properly detect all quality factors.
From the study, the students conclude that Tensorflow is more suitable for use in business. The system is better supported and more robust than Yolo, and it works faster and more accurately. Due to the lack of a large and evenly distributed data set, damage prediction within this study failed. The students indicate that this could be improved by creating a dataset where each class consists of the same number of labels. They stopped short of doing this themselves.
The conclusion is hopeful: based on the findings from the study, it should be possible to use deep learning to determine the quality of apples and separate apples based on these factors.
Fontys Expertise Center High Tech Systems and Materials (HTSM) connects Fontys students and researchers with high-tech companies in the Brainport area.
Together they conduct applied research and develop innovative technologies that benefit both education and business. In addition, the Expertise Center HTSM offers various forms of lifelong learning for professionals working in the high-tech sector.
Willem: "However, we know that successful AI projects require more than just technology. You only achieve the best solution together with the customer!" And that, then, is the immediate challenge for all companies: get help!
Dutch