Issue

Vol. 11, No. 3, August 2026 (Article in Progress)

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Federated Learning and Deep Reinforcement Learning Synergy: Opportunities for Multi-Cloud Serverless Deployment

https://doi.org/10.22219/kinetik.v11i3.2694
I Gusti Ngurah Wikranta Arsa Arsa
Institut Teknologi dan Bisnis STIKOM Bali
Arief Setyanto
Universitas Amikom Yogyakarta
Andi Sunyoto
Universitas Amikom Yogyakarta
Alva Hendi Muhammad
Universitas Amikom Yogyakarta

Corresponding Author(s) : I Gusti Ngurah Wikranta Arsa Arsa

arsa@stikom-bali.ac.id

Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, Vol. 11, No. 3, August 2026 (Article in Progress)

Abstract

The Development of distributed computing has enabled the use of multi-cloud and serverless computing, which are beneficial due to their flexibility, scalability, and cost efficiency. There are, of course, pertinent challenges associated with these computing paradigms, such as resource heterogeneity, cold-start latency, vendor lock-in, and privacy. Recent trends in Federated Learning (FL) and Deep Reinforcement Learning (DRL) hold promise in solving these issues. FL systems enable decentralised, privacy-preserving model training across heterogeneous systems, while DRL systems enable adaptive models for real-time decision-making to optimise system resources and improve performance. This Systematic Literature Review (SLR) covers the years 2020 to early 2026 and examines the intersection of FL and DRL in multi cloud serverless computing, following the PRISMA methodology. A primary analysis of 50 quality studies was undertaken to answer four privacy-related resource management questions. The results showed FL improves privacy and scalability using decentralised training. Consolidating the Federated DRL and Multi-Agent stacks enhances the system by achieving a better trade-off and optimization among latency, energy, and operational efficiency. However, a few gaps still exist, such as the absence of a more holistic framework, elusiveness in cross-system integration and collaboration, and a lack of concrete real-world applications. More work is needed to build a cohesive Federated Learning framework to improve sustainability and security in the multi-cloud, serverless systems of the future. This examination provides a solid foundation for the Development of innovative, privacy-preserving, and dynamic resource management in future cloud computing environments.

Keywords

multi-cloud serverless federated learning deep reinforcement learning PRISMA optimization
Arsa, I. G. N. W. A., Arief Setyanto, Andi Sunyoto, & Alva Hendi Muhammad. (2026). Federated Learning and Deep Reinforcement Learning Synergy: Opportunities for Multi-Cloud Serverless Deployment. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 11(3). https://doi.org/10.22219/kinetik.v11i3.2694
Download Citation
Endnote/Zotero/Mendeley (RIS)
BibTeX
References
  1. R. Vayyala, “Serverless Data Management Architectures for Multi Cloud Environments,” Proc. 7th Int. Conf. Intell. Sustain. Syst. ICISS 2025, pp. 593–598, 2025, doi: 10.1109/ICISS63372.2025.11076490.
  2. Y. Chen, B. Liu, W. Lin, Y. Guo, and Z. Peng, “CASR: Optimizing cold start and resources utilization in serverless computing,” Futur. Gener. Comput. Syst., vol. 170, Sep. 2025, doi: 10.1016/j.future.2025.107851.
  3. T. Singh, V. Shreshth, A. Raj, S. Swain, A. Bandyopadhyay, and N. Sharma, “Incentive Mechanisms for Federated Learning in Multi-Cloud Environments,” Proc. - 2025 7th Int. Conf. Comput. Intell. Commun. Technol. CCICT 2025, pp. 302–307, 2025, doi: 10.1109/CCICT65753.2025.00054.
  4. P. Tam, R. Corrado, C. Eang, and S. Kim, “Applicability of Deep Reinforcement Learning for Efficient Federated Learning in Massive IoT Communications,” Appl. Sci., vol. 13, no. 5, Mar. 2023, doi: 10.3390/app13053083.
  5. W. Khalaifat, W. Elmedany, and H. Alryalat, “Privacy and security of federated learning in resource-constrained Internet of Things environment: Systematic literature review,” Internet Things (The Netherlands), vol. 33, Sep. 2025, doi: 10.1016/j.iot.2025.101679.
  6. H. Qiu et al., “SIMPPO: A Scalable and Incremental Online Learning Framework for Serverless Resource Management,” in SoCC 2022 - Proceedings of the 13th Symposium on Cloud Computing, Nov. 2022, pp. 306–322, doi: 10.1145/3542929.3563475.
  7. Y. Liu, F. Li, and C. Kong, “A generic framework for minimizing cold start times in serverless applications via resource serialization,” J. Supercomput., vol. 81, no. 12, Aug. 2025, doi: 10.1007/s11227-025-07710-z.
  8. Z. Shojaee Rad and M. Ghobaei-Arani, “Federated serverless cloud approaches: A comprehensive review,” Comput. Electr. Eng., vol. 124, May 2025, doi: 10.1016/j.compeleceng.2025.110372.
  9. K. A. Ali, O. A. Fadare, and F. Al-Turjman, “Dynamic Resource Allocation (DRA) in Cloud Computing,” in Sustainable Civil Infrastructures, vol. Part F4042, Springer Science and Business Media B.V., 2025, pp. 1033–1049.
  10. L. Albshaier, S. Almarri, and A. Albuali, “Federated Learning for Cloud and Edge Security: A Systematic Review of Challenges and AI Opportunities,” Electron., vol. 14, no. 5, Mar. 2025, doi: 10.3390/electronics14051019.
  11. C. Prigent, A. Costan, G. Antoniu, and L. Cudennec, “Enabling federated learning across the computing continuum: Systems, challenges and future directions,” Futur. Gener. Comput. Syst., vol. 160, pp. 767–783, Nov. 2024, doi: 10.1016/j.future.2024.06.043.
  12. D. Ritter, “Cost-aware process modeling in multiclouds,” Inf. Syst., vol. 108, p. 101969, 2022, doi: 10.1016/j.is.2021.101969.
  13. N. Singh and M. Adhikari, “SelfFed : Self-adaptive Federated Learning with Non-IID data on Heterogeneous Edge Devices for Bias Mitigation and Enhance Training,” Inf. Fusion, vol. 118, no. October 2024, p. 102932, 2025, doi: 10.1016/j.inffus.2025.102932.
  14. S. Demil and M. R. Abdmeziem, “Internet of Things Incentivizing task offloading in IoT : A distributed auctions-based DRL approach,” Internet of Things, vol. 30, no. December 2024, p. 101493, 2025, doi: 10.1016/j.iot.2025.101493.
  15. O. Bushehrian and A. Moazeni, “Deep reinforcement learning ‑ based optimal deployment of IoT machine learning jobs in fog computing architecture,” Computing, vol. 107, no. 1, pp. 1–25, 2025, doi: 10.1007/s00607-024-01353-3.
  16. J. Yu, R. Zhou, B. Li, L. Wu, and S. Member, “Intelligent Frameworks for Minimizing Job Completion Time in Clustered Federated Learning,” pp. 1–14, 2025, doi: 10.1109/TON.2025.3600674.
  17. D. Ayepah-mensah, G. Sun, G. O. Boateng, S. Anokye, and G. Liu, “Blockchain-Enabled Federated Learning-Based Resource Allocation and Trading for Network Slicing in 5G,” IEEE/ACM Trans. Netw., vol. 32, no. 1, pp. 654–669, 2024, doi: 10.1109/TNET.2023.3297390.
  18. C. Wang, T. Yao, T. Fan, S. Peng, C. Xu, and S. Yu, “Modeling on Resource Allocation for Age-Sensitive Mobile-Edge Computing Using Federated Multiagent Reinforcement Learning,” IEEE Internet Things J., vol. 11, no. 2, pp. 3121–3131, 2024, doi: 10.1109/JIOT.2023.3294535.
  19. S. M. Rajagopal, M. Supriya, and R. Buyya, “FedSDM: Federated learning based smart decision making module for ECG data in IoT integrated Edge–Fog–Cloud computing environments,” Internet of Things (Netherlands), vol. 22, no. April, p. 100784, 2023, doi: 10.1016/j.iot.2023.100784.
  20. N. Yellas, B. Addis, S. Boumerdassi, R. Riggio, and S. Secci, “Function Placement for In-network Federated Learning,” Comput. Networks, vol. 256, no. February 2024, p. 110900, 2025, doi: 10.1016/j.comnet.2024.110900.
  21. N. Hudson, H. Khamfroush, M. Baughman, D. E. Lucani, K. Chard, and I. Foster, “QoS-aware edge AI placement and scheduling with multiple implementations in FaaS-based edge computing,” vol. 157, no. September 2023, pp. 250–263, 2024.
  22. W. Feng, X. Zuo, R. Zhang, Y. Zhu, and C. Wang, “Federated Deep Reinforcement Learning for Multimodal Content Caching in Edge-Cloud Networks,” IEEE Trans. Netw. Sci. Eng., vol. 12, no. 3, pp. 2188–2201, 2025, doi: 10.1109/TNSE.2025.3545924.
  23. A. A. Okine, N. Adam, F. Naeem, and G. Kaddoum, “FedRoute : A Multi-Server Federated Meta-DRL Routing Scheme for Tactical Air-Ground WSNs,” IEEE Open J. Commun. Soc., vol. 6, no. April, pp. 4176–4193, 2025, doi: 10.1109/OJCOMS.2025.3567024.
  24. A. S. M. S. Sagar, A. Haider, and H. S. Kim, “A hierarchical adaptive federated reinforcement learning for efficient resource allocation and task scheduling in hierarchical IoT network,” Comput. Commun., vol. 229, no. July 2024, p. 107969, 2025, doi: 10.1016/j.comcom.2024.107969.
  25. W. Hou, H. Wen, S. Member, N. Zhang, and S. Member, “Adaptive Training and Aggregation for Federated Learning in Multi-Tier Computing Networks,” IEEE Trans. Mob. Comput., vol. 23, no. 5, pp. 4376–4388, 2024, doi: 10.1109/TMC.2023.3289940.
  26. C. Sun, S. Member, X. Li, and J. Wen, “Federated Deep Reinforcement Learning for Recommendation-Enabled Edge Caching in Mobile Edge-Cloud Computing Networks,” IEEE J. Sel. Areas Commun., vol. 41, no. 3, pp. 690–705, 2023, doi: 10.1109/JSAC.2023.3235443.
  27. W. Fan, P. Chen, X. Chun, and Y. Liu, “MADRL-Based Model Partitioning, Aggregation Control, and Resource Allocation for Cloud-Edge-Device Collaborative Split Federated Learning,” IEEE Trans. Mob. Comput., vol. 24, no. 6, pp. 5324–5341, 2025, doi: 10.1109/TMC.2025.3530482.
  28. A. R. Malipatil, M. E. Paramasivam, D. Gulyamova, and A. Saravanan, “Energy-Efficient Cloud Computing Through Reinforcement Learning-Based Workload Scheduling,” vol. 16, no. 4, pp. 645–656, 2025.
  29. L. Xiao, H. Shan, J. Zhu, R. Mao, and S. Pan, “FD3QN : A Federated Deep Reinforcement Learning Approach for Cross-Domain Resource Cooperative Scheduling in Hybrid Cloud Architecture,” vol. 49, pp. 127–146, 2025.
  30. M. C. Sekhar, P. Kumaraswamy, N. Yamsani, G. B. K, and R. Kotoju, “FedTaskRL : A Reinforcement Learning-Based Framework for Efficient Task Scheduling in Federated Cloud Environments,” vol. 12, no. 7, pp. 74–89, 2025.
  31. N. Ma, A. Tang, Z. Xiong, and F. Jiang, “A deep multi-agent reinforcement learning approach for the micro-service migration problem with affinity in the cloud,” Expert Syst. Appl., vol. 273, no. February, p. 126856, 2025, doi: 10.1016/j.eswa.2025.126856.
  32. S. B. Tadele, W. Yahya, B. Kar, Y. D. Lin, Y. C. Lai, and F. G. Wakgra, “Optimizing the Ratio-Based Offloading in Federated Cloud-Edge Systems: A MADRL Approach,” IEEE Trans. Netw. Sci. Eng., vol. 12, no. 1, pp. 463–475, 2025, doi: 10.1109/TNSE.2024.3501398.
  33. S. Najafli, A. Toroghi, and H. Babak, “A novel reinforcement learning ‑ based hybrid intrusion detection system on fog ‑ to ‑ cloud computing,” J. Supercomput., vol. 80, no. 18, pp. 26088–26110, 2024, doi: 10.1007/s11227-024-06417-x.
  34. B. Brik, S. Member, and M. Esseghir, “On Adjusting Data Throughput in IoT Networks : Game Approach,” IEEE Internet Things J., vol. 11, no. 7, pp. 11368–11380, 2024, doi: 10.1109/JIOT.2023.3330408.
  35. S. Cho, “DRL-Enabled Hierarchical Federated Learning Optimization for Data Heterogeneity Management in Multi-Access Edge Computing,” IEEE Access, vol. 12, no. September, pp. 147209–147219, 2024, doi: 10.1109/ACCESS.2024.3473008.
  36. H. Zhou, H. Wang, Z. Yu, G. Bin, M. Xiao, and J. Wu, “Federated Distributed Deep Reinforcement Learning for Recommendation-Enabled Edge Caching,” IEEE Trans. Serv. Comput., vol. 17, no. 6, pp. 3640–3656, 2024, doi: 10.1109/TSC.2024.3433579.
  37. S. Vadigi, K. Sethi, D. Mohanty, and S. Prasad, “Journal of Information Security and Applications Federated reinforcement learning based intrusion detection system using dynamic attention mechanism,” vol. 78, no. September, 2023.
  38. J. Shi, C. Li, Y. Guan, P. Cong, and J. Li, “Multi-UAV-assisted computation offloading in DT-based networks : A distributed deep reinforcement learning approach,” Comput. Commun., vol. 210, no. April, pp. 217–228, 2023, doi: 10.1016/j.comcom.2023.07.041.
  39. P. Zhang, N. Chen, G. S. Member, and S. Li, “Multi-Domain Virtual Network Embedding Algorithm Based on Horizontal Federated Learning,” vol. 18, pp. 3363–3375, 2023.
  40. D. Qiao, S. Guo, S. Member, D. Liu, and S. Long, “Adaptive Federated Deep Reinforcement Learning for Proactive Content Caching in Edge Computing,” IEEE Trans. Parallel Distrib. Syst., vol. 33, no. 12, pp. 4767–4782, 2022, doi: 10.1109/TPDS.2022.3201983.
  41. M. Bazargani, S. Tarkesh, E. Behnam, and H. Reza, “AFedSLL-LDL : a framework based-on federated self-supervised learning and lightweight deep learning for attack detection in serverless edge computing,” 2025.
  42. G. Nagabushnam and K. Hoon, Faddeer : a deep multi-agent reinforcement learning-based scheduling algorithm for aperiodic tasks in heterogeneous fog computing networks, vol. 28, no. 6. Springer US, 2025.
  43. M. Abdullah, R. Munir, W. Iqbal, and S. Ahmad, “Cost and performance-effective dynamic VM type selection in auto-scaling using reinforcement learning,” 2025.
  44. P. Zhang, E. Wang, M. Guizani, K. Liu, J. Wang, and L. Tan, “Privacy-Preserving Task Offloading in Vehicular Edge Computing,” IEEE Trans. Veh. Technol., vol. PP, no. Xx, pp. 1–13, 2025, doi: 10.1109/TVT.2025.3588204.
  45. T. Zeng, X. Zhang, J. Duan, C. Yu, C. Wu, and X. Chen, “An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed,” IEEE Trans. Parallel Distrib. Syst., vol. 35, no. 5, pp. 720–731, 2024, doi: 10.1109/TPDS.2024.3360438.
  46. A. H. D. Mendes, M. J. F. Rosa, M. A. Marotta, A. Araujo, A. C. M. A. Melo, and C. G. Ralha, “MAS-Cloud+: A novel multi-agent architecture with reasoning models for resource management in multiple providers,” Futur. Gener. Comput. Syst., vol. 154, no. February 2023, pp. 16–34, 2024, doi: 10.1016/j.future.2023.12.022.
  47. Z. Zhang, K. Ning, and G. Wu, “Enhancing multi-cloud service deployment with SkyCap: A loss-aware coordinator in sky computing,” Ad Hoc Networks, vol. 157, no. January, p. 103460, 2024, doi: 10.1016/j.adhoc.2024.103460.
  48. C. Jiang and L. Su, “Federated learning with three-way decisions for privacy-preserving multicloud resource scheduling,” Appl. Soft Comput., vol. 183, no. October 2024, p. 113634, 2025, doi: 10.1016/j.asoc.2025.113634.
  49. S. Salar, S. Mobina, R. Craciunescu, S. Maiduc, M. Bayram, and B. Arasteh, “Adaptive Resource Scheduling in Multi-Cloud Computing Using Recurrent Neural Forecasting and Memory-Based Metaheuristic Optimization,” 2025.
  50. A. John, J. Kawash, and R. Alhajj, Predictive container orchestration in the cloud using artificial intelligence techniques, vol. 107, no. 7. Springer Vienna, 2025.
  51. B. Kumar, A. Verma, and P. Verma, A multivariate transformer ‑ based monitor ‑ analyze ‑ plan ‑ execute ( MAPE ) autoscaling framework for dynamic resource allocation in cloud environment, vol. 107, no. 3. Springer Vienna, 2025.
  52. D. Dong, “Agent-based cloud simulation model for resource management,” J. Cloud Comput., vol. 12, no. 1, 2023, doi: 10.1186/s13677-023-00540-5.
  53. H. Zhang, J. Wang, H. Zhang, and C. Bu, “Security computing resource allocation based on deep reinforcement learning in serverless multi-cloud edge computing,” Futur. Gener. Comput. Syst., vol. 151, no. September 2023, pp. 152–161, 2024, doi: 10.1016/j.future.2023.09.016.
  54. M. Emami Khansari and S. Sharifian, “A deep reinforcement learning approach towards distributed Function as a Service (FaaS) based edge application orchestration in cloud-edge continuum,” J. Netw. Comput. Appl., vol. 233, no. August 2024, p. 104042, 2025, doi: 10.1016/j.jnca.2024.104042.
  55. G. R. Russo, V. Cardellini, and F. Lo Presti, “A framework for offloading and migration of serverless functions in the Edge – Cloud Continuum,” vol. 100, no. March, 2024.
  56. M. Zhou, B. Zheng, and L. Pan, “Balancing function performance and cluster load in serverless computing: A reinforcement learning solution,” J. Netw. Comput. Appl., vol. 243, no. March, 2025.
  57. A. Mampage, S. Karunasekera, and R. Buyya, “Deep reinforcement learning for application scheduling in resource-constrained , multi-tenant serverless computing environments,” Futur. Gener. Comput. Syst., vol. 143, pp. 277–292, 2023, doi: 10.1016/j.future.2023.02.006.
  58. G. R. Russo, D. Ferrarelli, D. Pasquali, V. Cardellini, and F. Lo Presti, “QoS-aware offloading policies for serverless functions in the Cloud-to-Edge continuum,” Futur. Gener. Comput. Syst., vol. 156, no. July 2023, pp. 1–15, 2024, doi: 10.1016/j.future.2024.02.019.
  59. J. Kaur, I. Chana, and A. Bala, “MADQL : Multi-Agent Deep Q-Learning for Optimized Job Scheduling in Serverless Computing,” Arab. J. Sci. Eng., 2025, doi: 10.1007/s13369-025-10461-x.
  60. A. Z. M. Ghobaei-arani and L. Esmaeili, “An efficient function placement approach in serverless edge computing,” Computing, vol. 107, no. 3, pp. 1–58, 2025, doi: 10.1007/s00607-025-01438-7.
  61. S. Nastic, “Self ‑ Provisioning Infrastructures for the Next Generation Serverless Computing,” SN Comput. Sci., 2024, doi: 10.1007/s42979-024-03022-w.
Read More
Author biographies is not available.
Download this PDF file
Statistic
Read Counter : 0

Downloads

Download data is not yet available.