Fruit ripeness estimation models have for decades depended on spectral index
features or colour-based features, such as mean, standard deviation, skewness,
colour moments, and/or histograms for learning traits of fruit ripeness.
Recently, few studies have explored the use of deep learning techniques to
extract features from images of fruits with visible ripeness cues. However, the
blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible
traits of ripeness when mature and therefore poses great difficulty to fruit
pickers. The mature blackberry, to the human eye, is black before, during, and
post-ripening. To address this engineering application challenge, this paper
proposes a novel multi-input convolutional neural network (CNN) ensemble
classifier for detecting subtle traits of ripeness in blackberry fruits. The
multi-input CNN was created from a pre-trained visual geometry group 16-layer
deep convolutional network (VGG16) model trained on the ImageNet dataset. The
fully connected layers were optimized for learning traits of ripeness of mature
blackberry fruits. The resulting model served as the base for building
homogeneous ensemble learners that were ensemble using the stack generalization
ensemble (SGE) framework. The input to the network is images acquired with a
stereo sensor using visible and near-infrared (VIS-NIR) spectral filters at
wavelengths of 700 nm and 770 nm. Through experiments, the proposed model
achieved 95.1% accuracy on unseen sets and 90.2% accuracy with in-field
conditions. Further experiments reveal that machine sensory is highly and
positively correlated to human sensory over blackberry fruit skin texture.
Fruit ripeness estimation is a complex task that has traditionally relied on spectral index features or color-based features. These features, such as mean, standard deviation, skewness, and color moments, provide important information about the ripeness of fruits. However, in the case of blackberries, these visible traits are not reliable indicators of ripeness. Blackberries appear black both before and after ripening, making it difficult for fruit pickers to determine their level of maturity.
To overcome this challenge, this paper proposes a novel approach using a multi-input convolutional neural network (CNN) ensemble classifier. The CNN is created by utilizing a pre-trained VGG16 model, which is a deep convolutional network trained on the ImageNet dataset. The fully connected layers of the VGG16 model are optimized specifically for learning the traits of ripeness in mature blackberry fruits.
A distinctive aspect of this approach is that it utilizes images acquired through a stereo sensor using visible and near-infrared (VIS-NIR) spectral filters. These filters operate at wavelengths of 700 nm and 770 nm, providing a unique perspective on the fruit’s ripeness.
Through experiments, the proposed model achieved impressive accuracy rates. It achieved 95.1% accuracy on unseen sets, which indicates its ability to generalize well to new data. Additionally, it achieved 90.2% accuracy under in-field conditions, demonstrating its robustness in real-world scenarios.
In further experiments, the study found a strong positive correlation between machine sensory data and human sensory evaluation based on blackberry fruit skin texture. This highlights the potential of using machine learning techniques to accurately determine the ripeness of blackberries, even when visible cues are not reliable.
It is worth noting that this study showcases the multi-disciplinary nature of the concepts involved. It combines computer vision (specifically CNNs) with sensor technology (stereo sensor and VIS-NIR spectral filters) and agricultural science (fruit ripeness evaluation). By harnessing the power of these different disciplines, researchers have developed a model that can accurately detect subtle traits of ripeness in blackberry fruits. This multi-disciplinary approach holds great promise for future advancements in fruit ripeness estimation and other related fields.