PERBANDINGAN KINERJA PRE-TRAINED INDOBERT-BASE DAN INDOBERT-LITE PADA KLASIFIKASI SENTIMEN ULASAN TIKTOK TOKOPEDIA SELLER CENTER DENGAN MODEL INDOBERT

Authors

  • Wildan Amru Hidayat Universitas Muhammadiyah Malang
  • Vinna Rahmayanti Setyaning Nastiti Universitas Muhammadiyah Malang

DOI:

https://doi.org/10.30656/jsii.v11i2.9168

Abstract

Era digital telah membawa revolusi dalam dunia e-commerce dengan mengintegrasikan platform media sosial dan platform e-commerce, yang menghasilkan inovasi seperti aplikasi TikTok Tokopedia Seller Center. Aplikasi ini menggabungkan platform e-commerce dengan fitur media sosial, memungkinkan pengguna untuk mengelola penjualan sekaligus memperluas jangkauan pasar dan mempromosikan produk melalui video pendek yang interaktif pada platform media sosial TikTok. Dengan adanya inovasi fitur baru dalam aplikasi ini, penelitian ini melakukan analisis sentimen untuk memahami persepsi dan ulasan berbahasa Indonesia dari para pengguna aplikasi TikTok Tokopedia Seller Center menggunakan model deep learning IndoBERT. Data ulasan dikumpulkan menggunakan teknik scraping pada Google Play Store sebanyak 3.145 ulasan yang dilabeli secara manual menjadi 1.755 klasifikasi sentimen negatif dan 1390 klasifikasi sentimen positif. Tahapan preprocessing seperti teks cleaning, case folding, normalisasi teks, dan stopword removal dilakukan untuk memberihkan data teks sebelum digunakan untuk pelatihan model. Data yang sudah dibersihkan terbagi menjadi 64% data training sebesar 2.012 data, 16% data validation sebesar 504 data, dan 20% data testing sebesar 629 data. Dua varian pre-trained model IndoBERT, yaitu Indobert-base-p2 versi besar dan Indobert-lite-base-p2 versi lebih ringan digunakan dalam penelitian ini untuk pemrosesan bahasa alami khusus bahasa Indonesia. Hasil penelitian menunjukkan bahwa komparasi model IndoBERT dengan kedua pre-trained menunjukkan bahwa pre-trained Indobert-base-p2 mendapatkan hasil akurasi yang lebih unggul dibandingkan Indobert-lite-base-p2, dengan akurasi sebesar 97%, presisi sebesar 97%, recall sebesar 97%, dan f1-score sebesar 97%, sedangkan pre-trained Indobert-lite-base-p2 dengan akurasi sebesar 94%, presisi sebesar 94%, recall sebesar 94%, dan f1-score sebesar 94%.

References

A. Andini, D. Ramadani, F. H. Jafar, and R. E. Mayasari, “Legal Review of Tik Tok Shop Re-Operation on The Tik Tok Social Media Application,†vol. 1, no. 1, pp. 1–9, 2024.

M. Isnan, G. N. Elwirehardja, and B. Pardamean, “Sentiment Analysis for TikTok Review Using VADER Sentiment and SVM Model,†Procedia Comput. Sci., vol. 227, pp. 168–175, 2023, doi: 10.1016/j.procs.2023.10.514.

M. E. Purbaya, D. Putra Rakhmadani, M. Puspa Arum, and L. Zian Nasifah, “Comparison of Kernel Support Vector Machines in Conducting Sentiment Analysis Review of Buying Chips on the Shopee E- Marketplace in Indonesian,†in 2022 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS), IEEE, Nov. 2022, pp. 435–440. doi: 10.1109/ICIMCIS56303.2022.10017546.

Z. A. Diekson, M. R. B. Prakoso, M. S. Q. Putra, M. S. A. F. Syaputra, S. Achmad, and R. Sutoyo, “Sentiment analysis for customer review: Case study of Traveloka,†Procedia Comput. Sci., vol. 216, pp. 682–690, 2023, doi: 10.1016/j.procs.2022.12.184.

T. Willianto, Supryadi, and A. Wibowo, “Sentiment Analysis on E-commerce Product using Machine Learning and Combination of TF-IDF and Backward Elimination,†Int. J. Recent Technol. Eng., vol. 8, no. 6, pp. 2862–2867, Mar. 2020, doi: 10.35940/ijrte.F7889.038620.

M. J. Hossain, D. Das Joy, S. Das, and R. Mustafa, “Sentiment Analysis on Reviews of E-commerce Sites Using Machine Learning Algorithms,†in 2022 International Conference on Innovations in Science, Engineering and Technology (ICISET), IEEE, Feb. 2022, pp. 522–527. doi: 10.1109/ICISET54810.2022.9775846.

S. Jafar Sidiq and A. Nur Rachman, “Analysis Of Twitter User Sentiment To Tiktok Shop Using Naïve Bayes And Decision Tree Algorithms,†Int. J. Appl. Inf. Syst. Informatics, vol. 1, no. 1, Nov. 2023, doi: 10.37058/jaisi.v1i1.8990.

J. Mantik et al., “Application Of N-Gram On K-Nearest Neighbor Algorithm To Sentiment Analysis Of TikTok Shop Shopping Features,†J. Mantik, vol. 6, no. 3, pp. 2685–4236, 2022.

C. M. T. Y. M. H. W. M. P. Dhuhita, “Sentiment Analysis on TikTok Shop Reviews Using Long Short-Term Memory Method to Find Business Opportunity,†Inf. J. Ilm. Bid. Teknol. Inf. dan Komun., no. Vol. 9 No. 1 (2024), pp. 1–7, 2024, [Online]. Available: https://ejournal.unitomo.ac.id/index.php/inform/article/view/6524/3258

N. Z. Al Habesyah, R. Herteno, F. Indriani, I. Budiman, and D. Kartini, “Sentiment Analysis of TikTok Shop Closure in Indonesia on Twitter Using Supervised Machine Learning,†J. Electron. Electromed. Eng. Med. Informatics, vol. 6, no. 2, pp. 148–156, Apr. 2024, doi: 10.35882/jeeemi.v6i2.381.

M. A. Hadiwijaya, F. P. Pirdaus, D. Andrews, S. Achmad, and R. Sutoyo, “Sentiment Analysis on Tokopedia Product Reviews using Natural Language Processing,†in 2023 International Conference on Informatics, Multimedia, Cyber and Informations System (ICIMCIS), IEEE, Nov. 2023, pp. 380–386. doi: 10.1109/ICIMCIS60089.2023.10348996.

H. Jayadianti, W. Kaswidjanti, A. T. Utomo, S. Saifullah, F. A. Dwiyanto, and R. Drezewski, “Sentiment analysis of Indonesian reviews using fine-tuning IndoBERT and R-CNN,†Ilk. J. Ilm., vol. 14, no. 3, pp. 348–354, Dec. 2022, doi: 10.33096/ilkom.v14i3.1505.348-354.

W. M. Baihaqi and A. Munandar, “Sentiment Analysis of Student Comment on the College Performance Evaluation Questionnaire Using Naïve Bayes and IndoBERT,†JUITA J. Inform., vol. 11, no. 2, p. 213, Nov. 2023, doi: 10.30595/juita.v11i2.17336.

P. Kaur, “Sentiment analysis using web scraping for live news data with machine learning algorithms,†Mater. Today Proc., vol. 65, pp. 3333–3341, 2022, doi: 10.1016/j.matpr.2022.05.409.

S. G. C. G and B. S. -, “Grid Search Tuning of Hyperparameters in Random Forest Classifier for Customer Feedback Sentiment Prediction,†Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 9, 2020, doi: 10.14569/IJACSA.2020.0110920.

M. Khader, A. Awajan, and G. Al-Naymat, “The Effects of Natural Language Processing on Big Data Analysis: Sentiment Analysis Case Study,†in 2018 International Arab Conference on Information Technology (ACIT), IEEE, Nov. 2018, pp. 1–7. doi: 10.1109/ACIT.2018.8672697.

C. Slamet, A. R. Atmadja, D. S. Maylawati, R. S. Lestari, W. Darmalaksana, and M. A. Ramdhani, “Automated Text Summarization for Indonesian Article Using Vector Space Model,†IOP Conf. Ser. Mater. Sci. Eng., vol. 288, p. 012037, Jan. 2018, doi: 10.1088/1757-899X/288/1/012037.

F. Hemmatian and M. K. Sohrabi, “A survey on classification techniques for opinion mining and sentiment analysis,†Artif. Intell. Rev., vol. 52, no. 3, pp. 1495–1545, Oct. 2019, doi: 10.1007/s10462-017-9599-6.

J. Singh and P. Tripathi, “Sentiment analysis of Twitter data by making use of SVM, Random Forest and Decision Tree algorithm,†in 2021 10th IEEE International Conference on Communication Systems and Network Technologies (CSNT), IEEE, Jun. 2021, pp. 193–198. doi: 10.1109/CSNT51715.2021.9509679.

Y. Xu and R. Goodacre, “On Splitting Training and Validation Set: A Comparative Study of Cross-Validation, Bootstrap and Systematic Sampling for Estimating the Generalization Performance of Supervised Learning,†J. Anal. Test., vol. 2, no. 3, pp. 249–262, Jul. 2018, doi: 10.1007/s41664-018-0068-2.

J. H. Computer, S. M. Honova, V. P. Computer, C. A. Setiawan, I. H. Parmonangan, and Diana, “Sentiment Analysis of Skincare Product Reviews in Indonesian Language using IndoBERT and LSTM,†in 2023 IEEE 9th Information Technology International Seminar (ITIS), IEEE, Oct. 2023, pp. 1–6. doi: 10.1109/ITIS59651.2023.10420222.

J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,†NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.

A. Nayak, H. Timmapathini, K. Ponnalagu, and V. Gopalan Venkoparao, “Domain adaptation challenges of BERT in tokenization and sub-word representations of Out-of-Vocabulary words,†in Proceedings of the First Workshop on Insights from Negative Results in NLP, Stroudsburg, PA, USA: Association for Computational Linguistics, 2020, pp. 1–5. doi: 10.18653/v1/2020.insights-1.1.

H. D. Sharma and P. Goyal, “An Analysis of Sentiment: Methods, Applications, and Challenges,†in RAiSE-2023, Basel Switzerland: MDPI, Dec. 2023, p. 68. doi: 10.3390/engproc2023059068.

K. S. Nugroho, A. Y. Sukmadewa, H. W. DW, F. A. Bachtiar, and N. Yudistira, “BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews,†Jul. 2021, doi: 10.1145/3479645.3479679.

R. Qasim, W. H. Bangyal, M. A. Alqarni, and A. Ali Almazroi, “A Fine-Tuned BERT-Based Transfer Learning Approach for Text Classification,†J. Healthc. Eng., vol. 2022, pp. 1–17, Jan. 2022, doi: 10.1155/2022/3498123.

K. Bhowmick and V. Sarvaiya, “A Comparative Study Of The Different Classification Algorithms On Football Analytics,†Int. J. Adv. Res., vol. 9, no. 08, pp. 392–407, Aug. 2021, doi: 10.21474/IJAR01/13280.

M. Totox and H. F. Pardede, “Exploring the Effectiveness of Deep Learning in Analyzing Review Sentiment,†JIKO (Jurnal Inform. dan Komputer), vol. 6, no. 2, Aug. 2023, doi: 10.33387/jiko.v6i2.6372.

Downloads

Published

2024-09-15

Issue

Vol. 11 No. 2 (2024)

Section

Articles

License

Creative Commons License

  1. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

  2. Author(s)' Warranties

    The author warrants that the article is original, written by stated author(s), has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author(s).

  3. Information
    Notice about change in the copyright policy of the journal 'Jurnal Sistem Informasi (JSiI)' : "From Vol 1, onwards the copyright of the article published in the journal 'Jurnal Sistem Informasi' will be retained by the author"