Indonesia is one of the world's largest palm oil producers. The area of oil palm plantations in Indonesia is increasing every year. However, large-scale land clearing for oil palm plantations is considered a cause of deforestation, negatively impacting the environment and society. Therefore, it is necessary to manage plantations sustainably, preserving forests and biodiversity. One solution to this problem is applying remote sensing technology that remotely monitors the ages and health levels of trees in oil palm plantations. In this fashion, trees detected as unproductive can be immediately replaced with new ones without having to clear new land again. There are three approaches to remote tree detection from previous studies: classical digital image processing, classical machine learning, and deep learning. Deep learning is an approach that is currently widely applied to object detection because of its accuracy and speed. This study proposes a deep-learning-based approach to detect oil palm trees by remote sensing using the RGB feature. This research consists of several stages: data collection, bounding box annotation, train/test split, model training, and evaluation. The data were acquired from an oil palm plantation in Kalimantan, Indonesia. The object-detection-based deep learning method was YOLO Version 5 (YOLO-V5). In this study, we compared all YOLO-V5 network models, namely, YOLO-V5s (small), YOLO-V5m (medium), YOLO-V5l (large), and YOLO-V5x (extra-large). The oil palm tree detection model with the highest evaluation metric used YOLO-V5s (YOLO-V5 small network) with a batch size of 32, i.e., the values of mAP50, mAP@[0.5, 0.95], and F1-Score respectively were 0.851, 0.457, and 0.785. This is preliminary research that is expected to continue to be developed so that it becomes a robust model for detecting oil palm trees in Indonesia and can be implemented in real-time on GIS software and drones. © 2023 ACM.