Identifikasi Penggunaan Chat GPT Pada Esai TOEFL Menggunakan Metode Long Short Term Memory
DOI:
https://doi.org/10.47065/bulletincsr.v6i3.772Keywords:
Identification; Chat GPT; LSTM; Human; TOEFLAbstract
The use of Artificial Intelligent (AI) technology is increasing along with technological developments. One of the technologies that is often used is Chat GPT (Generative Pre-trained Transformers). Chat GPT is an application used for many things such as source of information, write an essay, and answer TOEFL essay questions. Because of its easiness, people will excessively use this that can cause people to lose creativity because they do not understand the material context and rely too much on the AI text result, which poses academic risks. Teachers also have difficulty to distinguish between AI and human text writing. Therefore, this research is to identify whether TOEFL essay are result of human text or GPT. This research used the Long Short Term Memory (LSTM) method to identify the use of GPT in TOEFL essay. This research also used 3 different split data configurations to find the best results. This research consists of 2 TOEFL essay datasets with the same prompt and has total of 220 data samples. The LSTM method is a modification of algorithm Recurrent Neural Network (RNN) and part of Deep Learning. The LSTM method involves memory cell controlled by three gates, such as input gate, forgot fate, output gate, and the hidden state. The gates are used to decide and control the information added, deleted, and removed from memory cell. The results of this research is a system that can help teachers detect the use of GPT in TOEFL essay. This research successfully identified the use of GPT in TOEFL essay in a 70:30 data split configuration with a loss score of 25,07%, accuracy score of 89,83%, and prediction score of 64,32%. Therefore, it is hoped that this system can help teachers identify the use of GPT and facilitate the assessment of TOEFL essay.
Downloads
References
I. Belcic and C. Stryker, “What is GPT (generative pre-trained transformer)? | IBM.” Accessed: May 31, 2025. [Online]. Available: https://www.ibm.com/think/topics/gpt
J. Fleckenstein, J. Meyer, T. Jansen, S. D. Keller, O. Köller, and J. Möller, “Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays,” Computers and Education: Artificial Intelligence, vol. 6, Jun. 2024, doi: 10.1016/j.caeai.2024.100209.
A. Fiedler and J. Döpke, “Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses,” International Review of Economics Education, vol. 49, p. 100321, Jun. 2025, doi: 10.1016/J.IREE.2025.100321.
A. Mizumoto, S. Yasuda, and Y. Tamura, “Identifying ChatGPT-generated texts in EFL students’ writing: Through comparative analysis of linguistic fingerprints,” Applied Corpus Linguistics, vol. 4, no. 3, Dec. 2024, doi: 10.1016/j.acorp.2024.100106.
Y. Luan and S. Lin, “Research on Text Classification Based on CNN and LSTM,” 2019 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), pp. 352–355, Mar. 2019, doi: 10.1109/ICAICA.2019.887345410.1109/ICAICA.2019.8873454.
T. Rahman et al., “Human vs AI: Evaluating the Effectiveness of Deep Learning in AI Text Detection,” in 2024 27th International Conference on Computer and Information Technology, ICCIT 2024 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2024, pp. 2736–2741. doi: 10.1109/ICCIT64611.2024.11021749.
R. An, Y. Yang, F. Yang, and S. Wang, “Use prompt to differentiate text generated by ChatGPT and humans,” Machine Learning with Applications, vol. 14, p. 100497, Dec. 2023, doi: 10.1016/j.mlwa.2023.100497.
GlobalExam, “TOEFL Writing - Online Training.” Accessed: Jun. 08, 2025. [Online]. Available: https://global-exam.com/blog/en/toefl-writing-online-training/
Toeflessays.Com and G. Gao, “Answers to All TOEFL Essay Questions.” Accessed: Jun. 28, 2025. [Online]. Available: https://www.goodreads.com/book/show/41726611-answers-to-all-toefl-essay-questions
M. Goodine, “Master the TOEFL Writing Section in 2025 | Test Resources.” Accessed: Jun. 08, 2025. [Online]. Available: https://www.toeflresources.com/writing-section/
S. Zhu and F. Chollet, “Understanding masking & padding | TensorFlow Core.” Accessed: Jan. 05, 2025. [Online]. Available: https://www.tensorflow.org/guide/keras/understanding_masking_and_padding#padding_sequence_data
Krithika, “Introduction to FastText Embeddings and its Implication -.” Accessed: Jan. 05, 2025. [Online]. Available: https://www.analyticsvidhya.com/blog/2023/01/introduction-to-fasttext-embeddings-and-its-implication/
J. Barnard, “What Are Word Embeddings? | IBM.” Accessed: Jan. 05, 2025. [Online]. Available: https://www.ibm.com/think/topics/word-embeddings
T. Mikolov, E. Grave, P. Bojanowski, C. Puhrsch, and A. Joulin, “Advances in Pre-Training Distributed Word Representations,” LREC 2018 - 11th International Conference on Language Resources and Evaluation, pp. 52–55, Dec. 2017, Accessed: Feb. 04, 2025. [Online]. Available: https://arxiv.org/abs/1712.09405v1
A. S. Gillis, “What is data splitting and why is it important?” Accessed: Jan. 09, 2025. [Online]. Available: https://www.techtarget.com/searchenterpriseai/definition/data-splitting
M. Bansal, A. Goyal, and A. Choudhary, “A comparative analysis of K-Nearest Neighbor, Genetic, Support Vector Machine, Decision Tree, and Long Short Term Memory algorithms in machine learning,” Decision Analytics Journal, vol. 3, p. 100071, Jun. 2022, doi: 10.1016/J.DAJOUR.2022.100071.
V. Gallan, “LSTM (Long Short Term Memory). Sebelum mempelajari tentang LSTM, kita… | by Varellino Gallan | Bina Nusantara IT Division | Medium.” Accessed: Feb. 09, 2025. [Online]. Available: https://medium.com/bina-nusantara-it-division/lstm-long-short-term-memory-d29779e2ebf8
A. Satyo Bayangkari Karno, “Prediksi Data Time Series Saham Bank BRI Dengan Mesin Belajar LSTM (Long ShortTerm Memory),” Journal of Information and Information Security (JIFORTY), vol. 1, no. 1, pp. 1–8, 2020, [Online]. Available: http://ejurnal.ubharajaya.ac.id/index.php/jiforty
M. Umer et al., “Impact of convolutional neural network and FastText embedding on text classification,” Multimed Tools Appl, vol. 82, no. 4, pp. 5569–5585, Feb. 2023, doi: 10.1007/S11042-022-13459-X/FIGURES/3.
K. Chowdhury, “10 Hyperparameters to keep an eye on for your LSTM model — and other tips | by Kuldeep Chowdhury | Geek Culture | Medium,” Geek Culture. Accessed: Feb. 10, 2025. [Online]. Available: https://medium.com/geekculture/10-hyperparameters-to-keep-an-eye-on-for-your-lstm-model-and-other-tips-f0ff5b63fcd4
M. A. H. Wadud, M. M. Kabir, M. F. Mridha, M. A. Ali, M. A. Hamid, and M. M. Monowar, “How can we manage Offensive Text in Social Media - A Text Classification Approach using LSTM-BOOST,” International Journal of Information Management Data Insights, vol. 2, no. 2, p. 100095, Nov. 2022, doi: 10.1016/J.JJIMEI.2022.100095.
N. K. Hemalatha, R. N. Brunda, G. S. Prakruthi, B. V. B. Prabhu, A. Shukla, and O. S. J. Narasipura, “Sugarcane leaf disease detection through deep learning,” Deep Learning for Sustainable Agriculture, pp. 297–323, Jan. 2022, doi: 10.1016/B978-0-323-85214-2.00003-3.
P. K. Yechuri and S. Ramadass, “Classification of Image and Text Data Using Deep Learning-Based LSTM Model,” Traitement du Signal, vol. 38, no. 6, pp. 1809–1817, Dec. 2021, doi: 10.18280/ts.380625.
englishclubmskh, “An Essay Collection For TOEFL,” 2018. Accessed: Jun. 28, 2025. [Online]. Available: https://englishclubmskh.wordpress.com/wp-content/uploads/2018/09/toefl-essays.pdf
S. K. Ahmed, “How to choose a sampling technique and determine sample size for research: A simplified guide for researchers,” Oral Oncology Reports, vol. 12, p. 100662, Dec. 2024, doi: 10.1016/J.OOR.2024.100662.
fchollet, “The Sequential model.” Accessed: Jul. 01, 2025. [Online]. Available: https://keras.io/guides/sequential_model/
“Keras Input Layer - GeeksforGeeks.” Accessed: Jul. 01, 2025. [Online]. Available: https://www.geeksforgeeks.org/deep-learning/keras-input-layer/
B. Krishnamurthy, “ReLU Activation Function Explained | Built In.” Accessed: Jul. 03, 2025. [Online]. Available: https://builtin.com/machine-learning/relu-activation-function
B. Janeczko and G. Srivastava, “The use of deep learning in image analysis for the study of oncology,” Internet of Multimedia Things (IoMT): Techniques and Applications, pp. 133–150, Jan. 2022, doi: 10.1016/B978-0-32-385845-8.00011-3.
Bila bermanfaat silahkan share artikel ini
Berikan Komentar Anda terhadap artikel Identifikasi Penggunaan Chat GPT Pada Esai TOEFL Menggunakan Metode Long Short Term Memory
ARTICLE HISTORY
How to Cite
Issue
Section
Copyright (c) 2026 Karina Natasya Darmawan, Silvester Dian Handy Permana, Ketut Bayu Yogha Bintoro
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
