Color Distribution Analysis for Ripeness Prediction of Golden Apollo Melon

Human visual perception on color of melon fruit for ripeness judgement is a complex phenomenon that depends on many factors, making the judgement is often inaccurate and inconsistent. The objective of this study is to develop an image processing algorithm that can be used for distinguishing ripe melons from unripe ones based on their skin color. The image processing algorithm could then be used as a pre-harvest tool to facilitate farmers with enough information for making decisions about whether or not the melon is ready to harvest. Four sample groups of Golden Apollo melon were harvested at four different age, with 55 fruits in each group. Using the color distribution as results of the image analysis, the first two groups of the samples can be separated from other groups with minimal overlap, but they cannot be separated in the other two groups. The color image analysis of the melons in combination with discriminant analysis could be used to distiguish between harvesting age groups with an average accuracy of 86%.


Introduction
Indonesia has many different tropical fruits grown in various regions throughout the country. Melon is one such fruit, famous for it's sweet and pleasant taste, which is high in demand especially in dry season. As a non-climacteric fruit, melon needs to be harvested when it is already in ripe stage. As a consequence, melon farmers need to harvest daily during the season because melons do not ripen all at the same time. Such selection is an acquired skill without which the farmers would tend to receive lower prices for their melons when the collectors do the total soluble solids (TSS) spot checks to determine ripeness level.
Human visual perception of a melon's color or ripeness is a complex phenomenon that depends on many, internal and external factors. One very important internal factor is an individual farmer's visual perception which is usually varies among others. Among the external factors are the composition of the object in relation to light reflectance, illumination environment, the distance and angles of illumination, and viewing position. For many people as buyers' point of view, appearance is a primary criterion when making food purchases [1]. This phenomenon is particularly strong for fresh fruit and vegetables. Product appearance is typically evaluated by such charcateristics as size, shape, form, color, freshness and absence of visual defects [2]. Of these characteristics, color is a particularly important sensory attribute of product quality. Therefore, color is an important grading criteria for most food, especially fresh products [3]. A number of reports have used image processing to evaluate various agricultural products such as coffee bean quality [4], development of a mango grading machine [5] and building a vision system to monitor harvested paddy grain quality during head-feeding combine harvester operation [6].
Normally, consumers initially judge a food product by its color, followed by other attributes such as taste and/or aroma. That is why the color of fresh products affects consumer purchase decisions [7,8]. Thus, research to objectively assess food color is a field with a growing number of applications [9][10][11][12]. With these increasing consumer requirements for high quality products, the food industry continues to search for more effective systems to rapidly and objectively assess color and non-destructively predict the quality of fresh fruit and other products. In image, a color is usually composed of three color planes, red, green, and blue [13]. The objective of this study is to develop an image processing algorithm that can be used for distinguishing ripe melons from unripe ones based on their skin color. The image processing algorithm could then be used as a pre-harvest tool to facilitate farmers with enough information for making decisions if the melon is ready to harvest.

Research Method
Four sample groups of Golden Apollo melon were obtained from a farmer in Sragen District, Central Java Province, and categorized according to harvesting ages (46, 53, 60, and 67 Days After Planting or DAP, with 55 fruits in each group). At first, each melon was placed in an enclosure (70×50×80 cm 3 ) made from plywood and white thick clothe at one side, with four 5 W 235 lm cool daylight Philips lamps located at each of the four upper corners to illuminate inside the enclosure (Figure 1a). A black clothe was placed on the bottom, while the insides of the enclosure were covered by white papersat three sides and a white clothe at one side.
Images of the melon were captured using a DFK21BUCO3 (Imaging Source) with 2.8-12 mm CS-mount lens (Computar) color CCD camera mounted 40 cm above the floor of the enclosure, and saved in jpeg format with a 744 by 480 pixels resolution. After the images had been captured, the TSS was measured using a PR-210 type Atago refractometer ( Figure 1b Feature extraction is an important process in the classification [14,15], but we have to do some pre-processing to facilitate it. The pre-processing applied to the captured images were binarization and binary opening and closing operations to remove noises, and masking of the color image with the result of binary image with eliminated background. The binarization was performed by converting the color image to grayscale, and pixels were assigned as parts of object if their intensities were 60 or more, and the rest pixels were considered as background and they were eliminated. To remove the fruit stem, an opening operation using a structuring element of 7 by 7 pixels was conducted on the resulted binary image, followed by closing operation using the same structuring element to keep the object in original size. Then the resulting clean binary image from these operations was used to obtain a color image of the melon minus the background by masking the binary image into the original image, as shown in Figure 2. Finally, the color of the fruit in normalized RGB (red-green-blue symbolized by r, g, and b) and HSI (hue-saturation-intensity symbolized by H, S, and I) color models were extracted from the masked image. The resulting of color distribution in each image was used to identify parameters that were correlated with harvesting ages and fruit sweetness (as measured TSS of the melon flesh). The two color distributions (rgb and HSI) of the melons were determined and 1661 processed using discriminant analysis to search for color attributes that can be used to distinguish melon according to harvesting age (DAP), for real-time field applications just before harvest as a final goal of this research. Figure 2. Image processing applied to extract color of melon fruits; a) original color image of melon captured by CCD camera, b) binary image obtained by grayscale thresholding, c) binary image after opening and closing with 7 by 7 pixels structuring element, and d) result of masking for background elimination prior to color extraction

Results and Analysis
The relationship between average color values in RGB color model and harvesting age and the relationship between average color values in HSI color model are shown in Figure 3. The values of RGB components were displayed after normalization to 0 to 1 range. From the figures, it is clear that the red component of the RGB color model increased with harvesting age, though the incremental increase declined from harvesting age 60 to 67 DAP. The green component shows no clear differences between harvesting ages, while the blue component responded opposite to that of the red; decreasing with increase in harvesting age. This indicates that the color of melon skin changed from a blueish-green to a reddish-green as harvesting age increased. However, the change from 60 to 67 DAP was smaller than that for earlier harvesting ages. Similar phenomena are also shown in HSI color model, where values of hue and saturation showing relationships with harvesting ages, while intensity shows positive relationship with high overlapping. A similiar phenomenon was observed for TSS values, as shown in Figure 4. The TSS increased rapidly from 46 to 60 DAP, but flattened off from 60 to 67 DAP, indicating the melons were already entering the ripening stage by 60 DAP. This trend in TSS values is consistent with melons being a non-climacteric fruit entering the ripening stage [16]. From a harvest point of view, it is recommended that Golden Apollo melon be harvested between 60 and 67 DAP, since they are already ripe, so they still have long enough shelf life for distribution before consumption. (c) (d) Figure 5. Color distribution in RGB and HSI color models of melon at several harvesting age as results of image analysis; a) green-red distribution, b) blue-red distribution, c) saturation-hue distribution, and d) intensity-hue distribution The color distribution as results of the image analysis for the two color models are shown in Figure 5. In the RGB color model as shown in Figure 5, it can be seen that the red and blue components of the melons can be separated at 46 and 53 DAP groups from other groups with minimal overlap, but they cannot be separated from the other two groups (60 and 67 DAP). This means that the RGB color distribution cannot be used to distinguish melons at 60 and 67 DAP, because they are all grouped together. The same phenomenon was also observed in the HSI color model, where hue and saturation components could be separated at 46 and 53 DAP with minimal overlap, but not at 60 and 67 DAP. This also means that HSI color distribution cannot be used to distinguish melons at 60 and 67 DAP, because they all grouped together in one group.
To overcome the problem with direct separation using color distribution as explained above, Discriminant Analysis (DA) was applied as a new approach for ripeness separation. In applying DA, each group member was analyzed to check any non-collinearity between members. Tolerance values for the green and blue components of the RGB color model were larger than 0.1 (0.6876 for green and 0.7553 for blue), while for HSI only intensity was larger than 0.1 (0.5439). It means that green and blue of the RGB color model and intensity of the HSI color model can be further analyzed, while the rest color components from the two color models cannot. The VIF values were less than 10.0 for all three components as shown in Table 1, indicate that all the three components are suitable for DA since there is no collinearity betwen them. In general, the basic idea is to compute the theoretically expected value for each data point based on the distribution. The Fisher's box test shows that the p-value (0.0001) was less than alpha (0.05), meaning that the data for the variables are not homogenous; a requirement for data processing using by DA method. And finally, the Fisher discriminant function coefficients, as shown in Table 2, were used to calculate the probability values to demonstrate and show that the covariance matrices of green, blue, and intensity data were not homogenous. Four equations were developed using variables and coefficients determined before. Then the four equations were tested by inputting the values of G, B, and I extracted from each image, with the resulting values ranging from a 0 to 1 probability. The four equations are: (1) (3) In every case, only one function among the four functions (equations 1 to 4) gave a value of 1 or near to 1 as a result, and the other three functions gave 0 or near to zero values. The melons were grouped into the group where the function gave a value of 1 or near to 1. The results of harvesting ages classification using the four equations are shown in Table 3. From Table 3, it is clear that DA could discriminate harvesting age into four groups with an accuracy of 100%, 96%, 63% and 85% for the 46, 53, 60 and 67 DAP groups respectively. As can be seen from these results, DA could accurately discriminate for the 46 and 53 DAP groups, but not accurate for the 60 and 67 DAP groups. The reason for this is simply because no clear change in melon skin color occurred at 60 and 67 DAP, and reflected in the relatively smaller change in TSS during this period as shown in Figure 4. It suggests melons harvested at 60 DAP are already ripe and only a small increase in TSS value can be expected for later harvest.
Another way to ensure the results of the Fisher classication method is with F1, F2, and F3. To find what function will be usefull for DA, the discrimination functions were plotted as shown in Figure 6a. This shows that F1 alone is enough for discrimintaion, with a slight increase to 99.86% discrimination when F2 was included, while F3 was remain unsed. Another criterion is the bi-plot correlation as shown in Figure 6b, which shows the three variables (green, blue, and intensity) are not inline each other for the two functions (F1 and F2), indicating the variables are eligible for discrimination. Canonical discriminant function coefficients as shown in Table 4 were used to develop a Canonical discriminant function, the results of which are plotted and shown in Figure 7.   Figure 7. Plot of discrimination function to predict a group of samples All methods of data processing show that ripening at Golden Apollo melon used as samples in this study happened at 60 days after planting. Functions developed using DA can be used to discriminate that ripeness state from other two groups which are not ripen (46 and 53 DAP). Harvesting after 60 DAP might not suitable because increasing in TSS and color was not significant, while wasting time and increasing risk of any damage of melon by insect and diseases. If that the case, there is no necessity to distinguish 60 and 67 DAP since farmers are suggested to harvest the fruits when they entered the ripening stage at 60 DAP.

Conclusion
From total soluble solid content, Golden Apollo melon fruits started to ripen since 60 days after planting, with total soluble solids increasing slightly after that. Color image analysis of the melons in combination with discriminant analysis could be used to distiguish between harvesting ages of 46, 53, 60, and 67 days after planting with an average accuracy of 86%. However, the developed method could not distinguish melons harvested at 60 and 67 days after planting with a high accuracy, as both groups of melons had already entered the ripening stage.