Intermediate-task transfer learning for Indonesian NLP tasks
Penulis: Adrianus Saga Ekakristi, Alfan Farizki Wicaksono, Rahmad Mahendra
Informasi
JurnalNatural Language Processing Journal
PenerbitElsevier, Elsevier Ltd
Halaman100161
Tahun Publikasi2025
ISSN29497191
Jenis SumberGoogle Scholar
Sitasi
Scopus: 1
Google Scholar: 1
PubMed: 1
Abstrak
Transfer learning, a common technique in recent Natural Language Processing (NLP) research, involves pre-training a model on a large, unlabeled dataset using self-supervised methods and then fine-tuning it on a smaller, labeled dataset for a specific task. Recent studies have demonstrated that introducing an additional training step between pre-training and fine-tuning can further enhance model performance. This method is called intermediate-task transfer learning (ITTL). Although this approach can potentially improve performance in the target task, choosing an intermediate task that leads to the highest performance increase remains challenging. Furthermore, despite the extensive research on intermediate training methods in English NLP, the application of these techniques to Indonesian language processing is still relatively understudied. In this study, we apply the ITTL method to nine Indonesian NLP …
Dokumen & Tautan
