Automated Hyperparameter Optimization Using Optuna for EfficientNet-Based Medical Image Classification
A Case Study on Acute Lymphoblastic Leukemia Detection
Abstract
Manual hyperparameter tuning remains a significant bottleneck in developing robust deep learning models for medical applications. This study presents a comprehensive analysis of Optuna's Tree-structured Parzen Estimator (TPE) for automated hyperparameter optimization of EfficientNet-B2 architecture in Acute Lymphoblastic Leukemia (ALL) cell classification. Using the C-NMC dataset comprising 10,661 training and 1,867 test images, we conducted 20 optimization trials with architecture-specific search spaces targeting learning rate (1×10⁻⁵ to 1×10⁻²), dropout rates (0.1-0.5), weight decay (1×10⁻⁶ to 1×10⁻²), and hidden layer sizes (256-1024 neurons). Results demonstrate that learning rate dominates optimization importance (55%) followed by dropout regularization (34%). The framework achieved optimal configuration with 96.86% validation accuracy, reducing manual tuning time by approximately 90% while maintaining its performance (86.72% test accuracy, 0.92 AUC-ROC). Statistical analysis across multiple runs shows consistent performance with coefficient of variation of 1.96%, validating the reliability of TPE-based optimization for medical imaging applications.
References
R. Aggarwal et al., "Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis," npj Digit. Med., vol. 4, no. 1, pp. 1-23, Mar. 2021.
X. Zhao et al., "A review of convolutional neural networks in computer vision," Artif. Intell. Rev., vol. 57, no. 4, pp. 1-45, Apr. 2024.
C. Janiesch, P. Zschech, and K. Heinrich, "Machine learning and deep learning," Electron. Mark., vol. 31, no. 3, pp. 685-695, Sep. 2021.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, "Optuna: A next-generation hyperparameter optimization framework," in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Anchorage, AK, USA, Aug. 2019, pp. 2623-2631.
S. Watanabe, "Tree-structured Parzen estimator: Understanding its algorithm components and their roles for better empirical performance," arXiv preprint arXiv:2304.11127, Apr. 2023.
J. H. C. Chang, M. M. Poppe, C. H. Hua, K. J. Marcus, and N. Esiashvili, "Acute lymphoblastic leukemia," Pediatr. Blood Cancer, vol. 68, no. S2, Mar. 2021.
M. Tan and Q. V. Le, "EfficientNet: Rethinking model scaling for convolutional neural networks," in Proc. 36th Int. Conf. Mach. Learn., Long Beach, CA, USA, Jun. 2019, pp. 6105-6114.
D. Agarwal et al., "Automated medical diagnosis of Alzheimer's disease using an EfficientNet convolutional neural network," J. Med. Syst., vol. 47, no. 1, pp. 1-12, Feb. 2023.
A. Gupta, "Leukemia classification dataset," Kaggle, 2021. [Online]. Available: https://www.kaggle.com/datasets/andrewmvd/leukemia-classification
R. Escobar Díaz Guerrero et al., "A data augmentation methodology to reduce class imbalance in histopathology images," J. Imag. Inform. Med., vol. 37, no. 4, pp. 1767-1782, Aug. 2024.
T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, "Focal loss for dense object detection," in Proc. IEEE Int. Conf. Comput. Vis., Venice, Italy, Oct. 2017, pp. 2980-2988.
C. Huang et al., "Learning deep representation for imbalanced classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Las Vegas, NV, USA, Jun. 2016, pp. 5375-5384.
A. Tafvizi, B. Avci, and M. Sundararajan, "Attributing AUC-ROC to analyze binary classifier performance," arXiv preprint arXiv:2205.11781, May 2022.
J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," J. Mach. Learn. Res., vol. 13, pp. 281-305, Feb. 2012.
M. Ragab et al., "A comprehensive systematic review of YOLO for medical object detection (2018 to 2023)," IEEE Access, vol. 12, pp. 57815-57836, Apr. 2024.
