Intermediate-task transfer learning for Indonesian NLP tasks
Author: Adrianus Saga Ekakristi, Alfan Farizki Wicaksono, Rahmad Mahendra
Informations
JournalNatural Language Processing Journal
PublisherElsevier, Elsevier Ltd
Pages100161
Publication Year2025
ISSN29497191
Source TypeGoogle Scholar
Citations
Scopus: 1
Google Scholar: 1
PubMed: 1
Abstract
Transfer learning, a common technique in recent Natural Language Processing (NLP) research, involves pre-training a model on a large, unlabeled dataset using self-supervised methods and then fine-tuning it on a smaller, labeled dataset for a specific task. Recent studies have demonstrated that introducing an additional training step between pre-training and fine-tuning can further enhance model performance. This method is called intermediate-task transfer learning (ITTL). Although this approach can potentially improve performance in the target task, choosing an intermediate task that leads to the highest performance increase remains challenging. Furthermore, despite the extensive research on intermediate training methods in English NLP, the application of these techniques to Indonesian language processing is still relatively understudied. In this study, we apply the ITTL method to nine Indonesian NLP …
Documents & Links
