Hyperparameter Tuning System for Deep Learning in a Fog Computing Environment

doi:10.22716/sckt.2026.14.3.009

All Issue

2026 Vol.14, Issue 3 Preview Page Next Page

Research Article

Hyperparameter Tuning System for Deep Learning in a Fog Computing Environment 포그 컴퓨팅에서 딥러닝을 위한 하이퍼파라미터 조정 시스템: Jong-Sub Lee¹ and Seok-Jae Moon²
이종섭¹, 문석재²; ¹Professor, Semyung University, College of General Education
²Professor, Kyunghee University, Humanitas college

¹세명대학교 교양대학 교수, ²경희대학교 후마니타스칼리지 교수

30 September 2026. pp. 117-127

PDF

Abstract

딥러닝 모델의 성능 향상을 위해서는 학습률, 배치 크기, 드롭아웃 등의 하이퍼파라미터를 효과적으로 최적화해야 하지만, 기존 클라우드 기반 방식은 데이터 전송에 따른 네트워크 부하, 지연 시간 및 프라이버시 문제를 가진다. 본 논문에서는 이러한 문제를 해결하기 위해 다수의 포그 노드를 분산·병렬로 활용하는 딥러닝 하이퍼파라미터 조정 시스템을 제안한다. 제안 시스템은 포그 게이트웨이와 다수의 포그 워커로 구성된 마스터–워커 구조를 기반으로 랜덤 서치와 베이지안 최적화를 단계적으로 적용하며, 노드의 자원 상태를 고려한 작업 배정, 조기 종료 및 장애 예외 처리를 지원한다. 시스템 성능을 평가하기 위해 CIFAR-10 데이터셋과 ResNet 기반 모델을 사용하고 포그 워커 수를 3개, 5개, 7개, 9개로 변화시키며 실험하였다. 실험 결과, 포그 워커가 3개에서 9개로 증가할 때 하이퍼파라미터 탐색 시간은 76.4분에서 40.7분으로 감소하여 약 46.7%의 시간 단축 효과를 보였다. 반면 중복 탐색, 결함률 및 평균 응답 시간이 증가하여 노드 수가 많아질수록 통신·동기화 비용과 네트워크 오버헤드에 따른 확장성의 한계가 나타났다. 따라서 제안 시스템은 자원 제약적인 포그 환경에서 효율적인 분산 하이퍼파라미터 탐색이 가능함을 확인하였으며, 향후 이기종 노드의 자원 상태를 고려한 비동기식 작업 분배와 통신 비용 절감 기법에 대한 연구가 필요하다.

Hyperparameter optimization plays a critical role in improving the performance of deep learning models; however, conventional cloud-based approaches can incur substantial communication overhead, latency, and privacy concerns due to centralized data transmission and processing. To address these limitations, this paper presents a distributed hyperparameter tuning system for deep learning in a fog computing environment. The proposed system employs a master–worker architecture consisting of a fog gateway and multiple fog workers, and integrates Random Search and Bayesian Optimization in a sequential tuning process. It also incorporates resource-aware task allocation, early stopping, and fault-handling mechanisms to improve computational efficiency and system robustness under resource-constrained conditions. The system was evaluated using the CIFAR-10 dataset and a ResNet-based model with 3, 5, 7, and 9 fog workers. Experimental results show that increasing the number of fog workers from 3 to 9 reduced the hyperparameter search time from 76.4 minutes to 40.7 minutes, corresponding to a 46.7% reduction. However, the increase in duplicate searches, fault rate, and average response time indicates that communication overhead and synchronization costs limit linear scalability as the number of fog nodes increases. Therefore, the proposed system demonstrates that efficient distributed hyperparameter exploration is feasible in resource-constrained fog environments, while future work should investigate asynchronous task allocation that accounts for the resource states of heterogeneous nodes and techniques for reducing communication overhead.

Keywords

Fog Computing

Deep Learning

Hyperparameter Optimization

Distributed Computing

Random Search

Bayesian Optimization

References

S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A Survey of Deep Learning Techniques for Autonomous Driving,” Journal of Field Robotics, Vol. 37, No. 3, pp. 362-386, 2020. DOI: 10.1002/rob.21918
10.1002/rob.21918
G. Litjens, T. Kooi, B. Ehteshami Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez, “A Survey on Deep Learning in Medical Image Analysis,” Medical Image Analysis, vol. 42, pp. 60-88, 2017. DOI: 10.1016/j.media.2017.07.005
10.1016/j.media.2017.07.005
J. Wang, Y. Ma, L. Zhang, R. X. Gao, and D. Wu, “Deep Learning for Smart Manufacturing: Methods and Applications,” Journal of Manufacturing Systems, vol. 48, pp. 144-156, 2018. DOI: 10.1016/j.jmsy.2018.01.003
10.1016/j.jmsy.2018.01.003
L. Yang and A. Shami, “On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice,” Neurocomputing, vol. 415, pp. 295-316, 2020. DOI: 10.1016/j.neucom.2020.07.061
10.1016/j.neucom.2020.07.061
F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog Computing and Its Role in the Internet of Things,” in Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, 2012. DOI: 10.1145/2342509.2342513
10.1145/2342509.2342513
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet of Things Journal, Vol. 3, No. 5, pp. 637-646, 2016. DOI: 10.1109/JIOT.2016.2579198
10.1109/JIOT.2016.2579198
C.-H. Hong and B. Varghese, “Resource Management in Fog/Edge Computing: A Survey on Architectures, Infrastructure, and Algorithms,” ACM Computing Surveys, Vol. 52, No. 5, Article 97, pp. 1-37, 2019. DOI: 10.1145/3326066
10.1145/3326066

L. Li, K. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar, “Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization,” Journal of Machine Learning Research, Vol. 18, No. 185, pp. 1-52, 2018.

K. Li, A. Jamieson, E. Rostamizadeh, J. Gonina, M. Ben-tzur, B. Hardt, Recht, and A. Talwalkar, “A System for Massively Parallel Hyperparameter Tuning,” Proceedings of Machine Learning and Systems (MLSys), Vol. 2, 2020.
S. Falkner, A. Klein, and F. Hutter, “BOHB: Robust and Efficient Hyperparameter Optimization at Scale,” in Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR, Vol. 80, pp. 1437-1446, 2018.
R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, and I. Stoica, “Tune: A Research Platform for Distributed Model Selection and Training,” arXiv preprint arXiv:1807.05118, 2018.
T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623-2631, 2019. DOI: 10.1145/3292500.3330701
10.1145/3292500.3330701
D. Salinas, M. Seeger, A. Klein, V. Perrone, M. Wistuba, and C. Archambeau, “Syne Tune: A Library for Large Scale Hyperparameter Tuning and Reproducible Research,” in Proceedings of the First International Conference on Automated Machine Learning, PMLR, Vol. 188, 2022.
Y. Li, Y. Shen, H. Jiang, W. Zhang, J. Li, J. Liu, C. Zhang, and B. Cui, “Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale,” Proceedings of the VLDB Endowment, Vol. 15, No. 6, pp. 1256-1265, 2022. DOI: 10.14778/3514061.3514071
10.14778/3514061.3514071
M. Khodak, R. Tu, T. Li, L. Li, M.-F. Balcan, V. Smith, and A. Talwalkar, “Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing,” Advances in Neural Information Processing Systems (NeurIPS), Vol. 34, 2021.
Z. Wang, W. Kuang, C. Zhang, B. Ding, and Y. Li, “FedHPO-Bench: A Benchmark Suite for Federated Hyperparameter Optimization,” in Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR, Vol. 202, pp. 35908-35948, 2023.
K. Kuo, P. Thaker, M. Khodak, J. Nguyen, D. Jiang, A. Talwalkar, and V. Smith, “On Noisy Evaluation in Federated Hyperparameter Tuning,” Proceedings of Machine Learning and Systems (MLSys), Vol. 5, 2023.
H. Woisetschläger, A. Erben, R. Mayer, S. Wang, and H.-A. Jacobsen, “FLEdge: Benchmarking Federated Learning Applications in Edge Computing Systems,” in Proceedings of the 25th International Middleware Conference (Middleware ’24), pp. 88-102, 2024. DOI: 10.1145/3652892.3700751
10.1145/3652892.3700751
Y. Saadati and M. H. Amini, “Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism,” arXiv preprint arXiv:2411.12244, 2024.
L. Toderean, M. Daian, T. Cioara, I. Anghel, V. Michalakopoulos, E. Sarantinopoulos, and E. Sarmas, “Heuristic Based Federated Learning with Adaptive Hyperparameter Tuning for Households Energy Prediction,” Scientific Reports, Vol. 15, Article 12564, 2025. DOI: 10.1038/s41598-025-96443-3
10.1038/s41598-025-96443-3 40221586 PMC11993608

Information

Publisher :The Society of Convergence Knowledge
Publisher(Ko) :융복합지식학회
Journal Title :The Society of Convergence Knowledge Transactions
Journal Title(Ko) :융복합지식학회논문지
Volume : 14
No :3
Pages :117-127
DOI :https://doi.org/10.22716/sckt.2026.14.3.009

[1] S. Grigorescu, B. Trasnea, T. Cocias, and G. Macesanu, “A Survey of Deep Learning Techniques for Autonomous Driving,” Journal of Field Robotics, Vol. 37, No. 3, pp. 362-386, 2020. DOI: 10.1002/rob.21918
10.1002/rob.21918

[2] G. Litjens, T. Kooi, B. Ehteshami Bejnordi, A. A. A. Setio, F. Ciompi, M. Ghafoorian, J. A. W. M. van der Laak, B. van Ginneken, and C. I. Sánchez, “A Survey on Deep Learning in Medical Image Analysis,” Medical Image Analysis, vol. 42, pp. 60-88, 2017. DOI: 10.1016/j.media.2017.07.005
10.1016/j.media.2017.07.005

[3] J. Wang, Y. Ma, L. Zhang, R. X. Gao, and D. Wu, “Deep Learning for Smart Manufacturing: Methods and Applications,” Journal of Manufacturing Systems, vol. 48, pp. 144-156, 2018. DOI: 10.1016/j.jmsy.2018.01.003
10.1016/j.jmsy.2018.01.003

[4] L. Yang and A. Shami, “On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice,” Neurocomputing, vol. 415, pp. 295-316, 2020. DOI: 10.1016/j.neucom.2020.07.061
10.1016/j.neucom.2020.07.061

[5] F. Bonomi, R. Milito, J. Zhu, and S. Addepalli, “Fog Computing and Its Role in the Internet of Things,” in Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing, 2012. DOI: 10.1145/2342509.2342513
10.1145/2342509.2342513

[6] W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet of Things Journal, Vol. 3, No. 5, pp. 637-646, 2016. DOI: 10.1109/JIOT.2016.2579198
10.1109/JIOT.2016.2579198

[7] C.-H. Hong and B. Varghese, “Resource Management in Fog/Edge Computing: A Survey on Architectures, Infrastructure, and Algorithms,” ACM Computing Surveys, Vol. 52, No. 5, Article 97, pp. 1-37, 2019. DOI: 10.1145/3326066
10.1145/3326066

[8] K. Li, A. Jamieson, E. Rostamizadeh, J. Gonina, M. Ben-tzur, B. Hardt, Recht, and A. Talwalkar, “A System for Massively Parallel Hyperparameter Tuning,” Proceedings of Machine Learning and Systems (MLSys), Vol. 2, 2020.

[9] S. Falkner, A. Klein, and F. Hutter, “BOHB: Robust and Efficient Hyperparameter Optimization at Scale,” in Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR, Vol. 80, pp. 1437-1446, 2018.

[10] R. Liaw, E. Liang, R. Nishihara, P. Moritz, J. E. Gonzalez, and I. Stoica, “Tune: A Research Platform for Distributed Model Selection and Training,” arXiv preprint arXiv:1807.05118, 2018.

[11] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623-2631, 2019. DOI: 10.1145/3292500.3330701
10.1145/3292500.3330701

[12] D. Salinas, M. Seeger, A. Klein, V. Perrone, M. Wistuba, and C. Archambeau, “Syne Tune: A Library for Large Scale Hyperparameter Tuning and Reproducible Research,” in Proceedings of the First International Conference on Automated Machine Learning, PMLR, Vol. 188, 2022.

[13] Y. Li, Y. Shen, H. Jiang, W. Zhang, J. Li, J. Liu, C. Zhang, and B. Cui, “Hyper-Tune: Towards Efficient Hyper-parameter Tuning at Scale,” Proceedings of the VLDB Endowment, Vol. 15, No. 6, pp. 1256-1265, 2022. DOI: 10.14778/3514061.3514071
10.14778/3514061.3514071

[14] M. Khodak, R. Tu, T. Li, L. Li, M.-F. Balcan, V. Smith, and A. Talwalkar, “Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing,” Advances in Neural Information Processing Systems (NeurIPS), Vol. 34, 2021.

[15] Z. Wang, W. Kuang, C. Zhang, B. Ding, and Y. Li, “FedHPO-Bench: A Benchmark Suite for Federated Hyperparameter Optimization,” in Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR, Vol. 202, pp. 35908-35948, 2023.

[16] K. Kuo, P. Thaker, M. Khodak, J. Nguyen, D. Jiang, A. Talwalkar, and V. Smith, “On Noisy Evaluation in Federated Hyperparameter Tuning,” Proceedings of Machine Learning and Systems (MLSys), Vol. 5, 2023.

[17] H. Woisetschläger, A. Erben, R. Mayer, S. Wang, and H.-A. Jacobsen, “FLEdge: Benchmarking Federated Learning Applications in Edge Computing Systems,” in Proceedings of the 25th International Middleware Conference (Middleware ’24), pp. 88-102, 2024. DOI: 10.1145/3652892.3700751
10.1145/3652892.3700751

[18] Y. Saadati and M. H. Amini, “Hyper-parameter Optimization for Federated Learning with Step-wise Adaptive Mechanism,” arXiv preprint arXiv:2411.12244, 2024.

[19] L. Toderean, M. Daian, T. Cioara, I. Anghel, V. Michalakopoulos, E. Sarantinopoulos, and E. Sarmas, “Heuristic Based Federated Learning with Adaptive Hyperparameter Tuning for Households Energy Prediction,” Scientific Reports, Vol. 15, Article 12564, 2025. DOI: 10.1038/s41598-025-96443-3
10.1038/s41598-025-96443-3 40221586 PMC11993608

The Society of Convergence Knowledge Transactions ISSN:2287-8920(Print) 융복합지식학회논문지

All Issue