Evaluasi Kinerja Pendekatan Latent Semantic Indexing dan Kombinasi Latent Semantic Indexing - K Nearest Neighbor pada Klasifikasi Dokumen Beban Kerja Dosen

Abstract

The Lecturer Workload (BKD) document is an academic administrative document categorized based on higher education's Tridharma (triple dharma) activities, making it highly valuable as a dataset for document classification research. However, the characteristics of BKD documents which feature diverse academic terms and an imbalanced class distribution pose a distinct challenge in the classification process. This study aims to evaluate and compare the performance of two classification approaches based on Latent Semantic Indexing (LSI): LSI + Cosine Similarity and LSI + K-Nearest Neighbor (KNN). As a benchmark, TF-IDF + KNN is utilized as a baseline method to analyze the impact of LSI on classification performance. The dataset consists of 352 Postgraduate BKD documents from Universitas Negeri Medan, which underwent text extraction, preprocessing (case folding, cleaning, tokenization, stopword removal, stemming, normalization, and BKD keyword enrichment), TF-IDF weighting, and dimensionality reduction using LSI. Evaluation was conducted using Stratified 3-Fold Cross Validation with Accuracy, Precision, Recall, F1-Score, ROC-AUC, and Average Precision (AP) as metrics. The results indicate that the LSI + Cosine Similarity approach delivers the best performance, achieving an Accuracy of 86.3%, F1-Weighted of 0.865, AUC-Macro of 0.965, and AP-Macro of 0.887. Meanwhile, LSI + KNN achieved 84.6% Accuracy, and the TF-IDF + KNN baseline reached 82.6% Accuracy. These findings demonstrate that semantic representation using LSI successfully enhances the quality of BKD document classification, while the Cosine Similarity-based approach proves more stable than KNN on datasets characterized by varied academic terminology and imbalanced class distributions.