Impact of cosine similarity function on SVM algorithm for public opinion mining about national sports week 2024 on X

Abstract

National Sports Week (Indonesian: Pekan Olahraga Nasional PON, abbreviated as PON) is a multi-sport event held every four years in Indonesia.  It has been held in Aceh and North Sumatra in 2024. There were many issues and public opinions about the event on social media X and it became a trending topic. The opinion can be feedback maintained or improved for upcoming PON. This research analyzes the sentiment of public opinion about PON on X social media using the Support Vector Machine (SVM) algorithm. Usually, SVM algorithm has good performance with Kernel function. Unfortunately, the function does not design as text similarity function. This study proposed cosine similarity to substituted Kernel function on the algorithm. The dataset obtained from X social media through web scraping techniques, labeled as positive, neutral, or negative sentiment. The dataset goes through data pre-processing stages, such as text cleaning, tokenization, and removal of irrelevant words. The analysis was completed using two scenarios: the baseline SVM algorithm and the SVM algorithm with cosine similarity function. The results showed that the model with cosine similarity function improved performance by 3.3-6.3%, with 88.73% accuracy, 88.3% precision, 89.3% recall, and 88.3% F1 score. The analysis also identified negative sentiments related to referee performance and specific sports. In contrast, positive sentiment focused on support for the contingent and appreciation for medals. This study confirms the value of sentiment analysis as an evaluation method that can provide insights for organizers of major sporting events like PON, particularly in improving dissatisfied aspects while maintaining favorable features.