A preliminary study on feature-based test smell detection in Java unit tests

  • Thi Van Nguyen
Keywords: JUnit, leave-one-project-out evaluation, machine learning, software quality, static features, test smell detection

Abstract

Maintaining test suites is difficult when test code contains recurring design symptoms such as assertion-heavy tests, tests that exercise several behaviours, hidden dependencies on external resources, or duplicated test logic. This preliminary study examines whether such symptoms can be approximated using a lightweight staticanalysis and machine-learning pipeline for Java/JUnit test code. The proposed workflow parses JUnit test methods, derives rule-based candidate labels for four test smells, checks a stratified validation subset, extracts smell-oriented static features, and trains one-vs-rest classifiers based on Logistic Regression, Linear SVM, Random Forest, XGBoost, and stacking. We evaluate the workflow on three Apache Java libraries containing 9,721 JUnit test methods. The preliminary results are interpreted descriptively rather than as evidence of statistical superiority. Stacking is competitive in the within-project setting, whereas XGBoost gives the strongest observed leave-oneproject-out performance among the learned models. The findings suggest that featurebased learning is a feasible screening mechanism for JUnit test smell detection, while also showing that label construction and cross-project transfer require cautious interpretation.

Author Biography

Thi Van Nguyen

Phenikaa School of Computing, Phenikaa University, Nguyen Trac Street, Duong Noi Ward, Hanoi, Vietnam

References

[1] D. Campos, L.A. Martins, C.I.M. Bezerra, et al. (2023), “Investigating developers’ contributions to test smell survivability: A study of open-source projects”, Proceedings of The 8th Brazilian Symposium on Systematic and Automated Software Testing, pp.41-50, DOI: 10.1145/3624032.3624044.
[2] W.M. Aljedaani, A.S. Peruma, A. Aljohani, et al. (2021), “Test smell detection tools: A systematic mapping study”, Proceedings of the 25th International Conference on Evaluation and Assessment in Software Engineering, pp.170-180, DOI: 10.1145/3463274.3463335.
[3] A. Panichella, S. Panichella, G. Fraser, et al. (2022), “Test smells 20 years later: Detectability, validity, and reliability”, Empirical Software Engineering, 27, DOI: 10.1007/s10664-022-10207-5.
[4] D. Campos, L.R. Soares, I. do Carmo Machado (2021), “Developers’ perception on the severity of test smells: An empirical study”, arXiv, https://arxiv.org/abs/2107.13902, accessed 10 June 2026.
[5] M. Marabesi, A. García-Holgado, F.J. García-Peñalvo (2024), “Exploring the connection between the TDD practice and test smells - a systematic literature review”, Comput., 13(3), DOI: 10.3390/computers13030079.
[6] B. Zhang, P. Liang, X. Zhou, et al. (2024), “A comprehensive evaluation of parameter-efficient fine-tuning on code smell detection”, ACM Transactions on Software Engineering and Methodology, DOI: 10.1145/3806390.
[7] M.A. Hasan, T. Ahammed (2024), “Understanding the occurrences of test smells in open source and industrial software systems: An empirical study on Python projects”, QuASoQ2024, https://ceur-ws.org/Vol-3864/quasoq-2024-paper-03.pdf.
[8] E. Soares, M. Aranda, N.D.C. Oliveira, et al. (2023), “Manual tests do smell! Cataloging and identifying natural language test smells”, 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp.1-11, DOI: 10.48550/arXiv.2308.01386.
[9] H. Damasceno, C.I.M. Bezerra, E.F. Coutinho, et al. (2022), “Analyzing test smells refactoring from a developer’s perspective”, Proceedings of the XXI Brazilian Symposium on Software Quality, pp.1-10, DOI: 10.1145/3571473.3571487.
[10] R. Rwemalika, S. Habchi, M. Papadakis, et al. (2023), “Smells in system user interactive tests”, Empirical Software Engineering, 28(2), DOI: 10.1007/s10664-022- 10251-1.
[11] V. Pontillo, D. Amoroso d’Aragona, F. Pecorelli, et al. (2024), “Machine learningbased test smell detection”, Empirical Software Engineering, 29, DOI: 10.1007/s10664-023- 10436-2.
[12] R. Melo, P. Simoes, R. Gheyi, et al. (2025), “Agentic LMs: Hunting down test smells”, IEEE Software, 43, pp.32-40, DOI: 10.1109/MS.2025.3621356.
[13] S. Souza, A. Santana, E. Figueiredo, et al. (2026), “Beyond strict rules: Assessing the effectiveness of large language models for code smell detection”, arXiv, DOI:10.48550/arXiv.2601.09873.
[14] M. Mosquera, R. Bojorque, P. Flores (2025), “Understanding code smell detection through hyperparameter optimization and metric correlation analysis”, IEEE Access, 13, pp.217750-217768, DOI: 10.1109/ACCESS.2025.2505678.
[15] I. Nakamura, Y. Kashiwa, B. Lin, et al. (2026), “Understanding self-admitted technical debt in test code: An empirical study”, ACM Transactions on Software Engineering and Methodology, 35(9), pp.1-35, DOI: 10.1145/3786791.
[16] H.K.V. Tran, N. bin Ali, M. Unterkalmsteiner, et al. (2025), “Quality attributes of test cases and test suites - importance & challenges from practitioners’ perspectives”, Software Quality Journal, 33, DOI: 10.1007/s11219-024-09698-w.
[17] M.A. Hasan, T. Ahammed (2025), “Prioritizing test smells: An empirical evaluation of quality metrics and developer perceptions”, 2025 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp.815-820, DOI: 10.1109/ICSME64153.2025.00085.
[18] A. Bodea (2022), “Pytest-smell: A smell detection tool for Python unit tests”, Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pp.793-796, DOI: 10.1145/3533767.3543290.
Published
2026-09-25