Intrusion Detection Systems (IDS) deployed in real-world environments must operate under severe class imbalance, evolving attack strategies, and non-stationary traffic distributions. Conventional supervised and deep learning-based IDS rely on fixed decision functions, limiting their adaptability to zero-day attacks and concept drift. This paper proposes a hybrid CNN-LSTM-DQN framework combined with a stability-aware evaluation methodology. The CNN-LSTM backbone extracts spatio-temporal representations, while a Deep Q-Network (DQN) learns adaptive detection policies using an ARMF-aware reward formulation. The framework is evaluated on eleven experimental stages (E1-11), including supervised baselines, reinforcement learning optimization, zero-day generalization (LOAO), and drift scenarios. Experimental results show that supervised models have high recall (up to 98.39 ) but generate too many alerts (ARMF up to 44,845). The reinforcement learning model with prioritized experience replay (E7) achieves a more balanced performance with a recall of 91.40 and an ARMF of 1,031. The proposed PER-based approach significantly improves detection performance while maintaining low alert rates, achieving a recall of 42.47 compared to naive reinforcement learning (E5). Further evaluation in real drifting conditions showed robust recall (89 - 92) with a tolerable number of alerts (ARMF≈ 1,189). These results indicate that adaptive policy learning enables a more effective trade-off between detection performance and operational cost, while ARMF-based evaluation provides a practical complement to accuracy metrics for real-world IDS deployment. Copyright (c) 2026 IIUM Press. This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. https://creativecommons.org/licenses/by-nc/4.0/