Skip to main content
Springer Nature Link
Log in

Netherite: efficient execution of serverless workflows

  • Special Issue Paper
  • Published:
View saved research
The VLDB Journal Aims and scope Submit manuscript

Abstract

Serverless applications are popular because they can provide scalability and load-based billing with minimal developer effort. Beyond simple stateless functions (FaaS), modern serverless programming models such as Durable Functions (DF) now provide stateful abstractions, such as workflows, tasks and actors, whose state and progress is implicitly and continuously persisted. This automatic resilience greatly simplifies development, but also creates performance challenges, such as a large number of IOps. To address these challenges, we introduce Netherite, a novel architecture for executing serverless workflows on an elastic cluster. Netherite groups the numerous application objects into a smaller number of partitions, and pipelines the state persistence of each partition. This improves latency and throughput, as it enables workflow steps to group commit, even if causally dependent. Moreover, Netherite leverages FASTER’s hybrid log approach to support larger-than-memory application state, and to enable efficient partition movement between compute hosts. Our evaluation shows that (a) Netherite achieves lower latency and higher throughput than the original DF engine, by more than an order of magnitude in some cases, and (b) that Netherite has lower latency than some commonly used alternatives, like AWS Step Functions or cloud storage triggers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from €37.37 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price includes VAT (Netherlands)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

Notes

  1. We define causality of two transitions A, B as follows: B is causally dependent on A if B consumes a message produced by A, if B reads an instance state written by A, or if there is a transitive chain of such dependencies.

  2. Our.NET interface also offers an interface-and-proxy-based syntax for calling entities that provides type checking for operations and arguments. It takes a bit more space so we chose the untyped syntax for this small demonstration example.

  3. Source at: https://github.com/aws-samples/lambda-refarch-imagerecognition

  4. Source at: https://github.com/FINRAOS/maskopy

  5. Blob in Azure and S3 in AWS.

  6. Durable Objects closed beta was announced on September 28th, 2020.

  7. Akka Serverless was formerly known as Lightbend CloudState.

References

  1. Apache: Apache Airflow. https://airflow.apache.org/ (2022)

  2. Apache: Apache Kafka. http://kafka.apache.org (2022)

  3. Apache: Apache OpenWhisk. https://openwhisk.apache.org/ (2022)

  4. Apers, P.M.: Data allocation in distributed database systems. ACM Trans. Database Syst. 13(3), 263–304 (1988)

    Article  Google Scholar 

  5. Armstrong, J.: The development of Erlang. ICFP 97, 196–203 (1997)

    Article  Google Scholar 

  6. AWS: AWS Lambda—Serverless Compute—Amazon Web Services. https://aws.amazon.com/lambda/ (2022a)

  7. AWS: AWS Step Functions. https://aws.amazon.com/step-functions/ (2022b)

  8. Azure: Azure Functions. https://azure.microsoft.com/en-us/services/functions/ (2022a)

  9. Azure: Azure Functions Host repository. https://github.com/Azure/azure-functions-host (2022b)

  10. Azure: Azure Functions PowerShell repository. https://github.com/Azure/azure-functions-powershell-worker (2022c)

  11. Azure: Azure Logic Apps Service. https://azure.microsoft.com/en-us/services/logic-apps/ (2022d)

  12. Azure: Durable Functions Extension (C-sharp) repository. https://github.com/Azure/azure-functions-durable-extension (2022e)

  13. Azure: Durable Functions JavaScript repository. https://github.com/Azure/azure-functions-durable-js (2022f)

  14. Azure: Durable Functions Python repository. https://github.com/Azure/azure-functions-durable-python (2022g)

  15. Azure: What are Durable Functions? Microsoft Azure. https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview (2022h)

  16. Azure: What is Azure Event Hubs? Microsoft Azure. https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-about (2022i)

  17. Azure : Generally available: New storage backend for Durable Functions—Microsoft Netherite & MSSQL. https://aka.ms/DFNetheriteMssqlGA (2023)

  18. Bernstein, P.: Actor-oriented database systems. In: Proceedings of the 2018 IEEE 34th International Conference on Data Engineering, IEEE Computer Society, pp. 13–14, https://www.microsoft.com/en-us/research/publication/actor-oriented-database-systems/ (2018)

  19. Burckhardt, S., Gillum, C., Justo, D., Kallas, K., McMahon, C., Meiklejohn, C.S.: Durable functions: Semantics for stateful serverless. Proc. ACM Program. Lang. 5, 133 (2021)

    Article  Google Scholar 

  20. Bykov, S., Geller, A., Kliot, G., Larus, J.R., Pandya, R., Thelin, J.: Orleans: Cloud Computing for Everyone. In: Proceedings of the 2nd ACM Symposium on Cloud Computing, ACM, p. 16 (2011)

  21. Ceri, S., Negri, M., Pelagatti, G.: Horizontal data partitioning in database design. In: Proceedings of the 1982 ACM SIGMOD international conference on Management of data, pp. 128–136 (1982)

  22. Ceri, S., Navathe, S., Wiederhold, G.: Distribution design of logical database schemas. IEEE Trans. Softw. Eng. 4, 487–504 (1983)

    Article  Google Scholar 

  23. Chandramouli, B., Prasaad, G., Kossmann, D., Levandoski, J., Hunter, J., Barnett, M.: FASTER: A concurrent key-value store with in-place updates. In: Proceedings of the 2018 International Conference on Management of Data, pp. 275–290 (2018)

  24. Cloudflare: Using Durable Objects, Cloudflare Docs. https://developers.cloudflare.com/workers/learning/using-durable-objects (2022a)

  25. Cloudflare: Workers Durable Objects Beta: A New Approach to Stateful Serverless. https://blog.cloudflare.com/introducing-workers-durable-objects/ (2022b)

  26. CloudState: CloudState-Towards Stateful Serverless by Jonas Bonér. https://www.youtube.com/watch?v=DVTf5WQlgB8 (2022)

  27. DeCandia, G., Hastorun, D., Jampani, M., Kakulapati, G., Lakshman, A., Pilchin, A., Sivasubramanian, S., Vosshall, P., Vogels, W.: Dynamo: Amazon’s highly available key-value store. ACM SIGOPS Oper. Syst. Rev. 41(6), 205–220 (2007)

    Article  Google Scholar 

  28. Escriva, R., Wong, B., Sirer, E.G.: Hyperdex: A distributed, searchable key-value store. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 25–36 (2012)

  29. Fission: Fission: Open source, Kubernetes-native Serverless Framework. https://fission.io/ (2022)

  30. Fn: Fn Flow. https://github.com/fnproject/flow/ (2022)

  31. Fouladi, S., Wahby, R.S., Shacklett, B., Balasubramaniam, K.V., Zeng, W., Bhalerao, R., Sivaraman, A., Porter, G., Winstein, K.: Encoding, fast and slow: Low-latency video processing using thousands of tiny threads. In: 14th \(\{USENIX\}\) Symposium on Networked Systems Design and Implementation (\(\{NSDI\}\) 17), pp. 363–376 (2017)

  32. Fouladi, S., Romero, F., Iter, D., Li, Q., Chatterjee, S., Kozyrakis, C., Zaharia, M., Winstein, K.: From laptop to lambda: Outsourcing everyday jobs to thousands of transient functional containers. In: 2019 \(\{USENIX\}\) Annual Technical Conference (\(\{USENIX\} \{ATC\}\) 19), pp. 475–488 (2019)

  33. Ghemawat, S., Gobioff, H., Leung, S.T.: The google file system. In: Proceedings of the Nineteenth ACM Symposium on Operating Systems Principles, pp. 29–43 (2003)

  34. Goldstein, J., Abdelhamid, A.S., Barnett, M., Burckhardt, S., Chandramouli, B., Gehring, D., Lebeck, N., Meiklejohn, C., Minhas, U.F., Newton, R., Peshawaria, R., Zaccai, T., Zhang, I.: AMBROSIA: providing performant virtual resiliency for distributed applications. Proc. VLDB Endow. 13(5), 588–601 (2020)

    Article  Google Scholar 

  35. Google: Google Cloud Composers. https://cloud.google.com/composer/ (2022)

  36. Google: Google Cloud Functions. https://cloud.google.com/functions/docs/ (2022a)

  37. Google: Google Workflows. https://cloud.google.com/workflows (2022b)

  38. Haller, P.: On the integration of the actor model in mainstream technologies: the Scala perspective. In: Proceedings of the 2nd Edition on Programming Systems, Languages and Applications Based on Actors, Agents, and Decentralized Control Abstractions, ACM, pp. 1– 6 (2012)

  39. Hellerstein, J.M., Faleiro, J.M., Gonzalez, J., Schleier-Smith, J., Sreekanti, V., Tumanov, A., Wu, C.: Serverless computing: One step forward, two steps back. In: CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13-16, 2019, http://cidrdb.org/cidr2019/papers/p119-hellerstein-cidr19.pdf (2019)

  40. Jangda, A., Pinckney, D., Brun, Y., Guha, A.: Formal foundations of serverless computing. Proceedings of the ACM on Programming Languages (PACMPL) 3(OOPSLA) (2019)

  41. Jia, Z., Witchel, E.: Boki: Stateful serverless computing with shared logs. In: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, Association for Computing Machinery, New York, NY, SOSP ’21, pp. 691–707. https://doi.org/10.1145/3477132.3483541 (2021a)

  42. Jia, Z., Witchel, E.: Nightcore: efficient and scalable serverless computing for latency-sensitive, interactive microservices. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Association for Computing Machinery, New York, NY, ASPLOS ’21, pp. 152–166, https://doi.org/10.1145/3445814.3446701, (2021b)

  43. Jonas, E., Pu, Q., Venkataraman, S., Stoica, I., Recht, B.: Occupy the cloud: Distributed computing for the 99%. In: Proceedings of the 2017 Symposium on Cloud Computing, pp. 445–451 (2017)

  44. Jonas, E., Schleier-Smith, J., Sreekanti, V., Tsai, C., Khandelwal, A., Pu, Q., Shankar, V., Carreira, J., Krauth, K., Yadwadkar, N.J., Gonzalez, J.E., Popa, R.A., Stoica, I., Patterson, D.A.: Cloud Programming Simplified: A Berkeley View on Serverless Computing. arXiv:1902.03383 (2019)

  45. Kallas, K., Zhang, H., Alur, R., Angel, S., Liu, V.: Executing microservice applications on serverless, correctly. Proc. ACM Program. Languages 7, 367–395 (2023)

    Article  Google Scholar 

  46. Kuchler, T., Giardino, M., Roscoe, T., Klimovic, A.: Function as a function. In: Proceedings of the 2023 ACM Symposium on Cloud Computing, pp. 81–92 (2023)

  47. Lahiri, S.: Complexity of Word Collocation Networks: A Preliminary Structural Analysis. In: Proceedings of the Student Research Workshop at the 14th Conference of the European Chapter of the Association for Computational Linguistics, Association for Computational Linguistics, Gothenburg, Sweden, pp. 96–105, http://www.aclweb.org/anthology/E14-3011 (2014)

  48. Leitner, P., Wittern, E., Spillner, J., Hummer, W.: A mixed-method empirical study of function-as-a-service software development in industrial practice. J. Syst. Softw. 149, 340–359 (2019). https://doi.org/10.1016/j.jss.2018.12.013. (https://www.sciencedirect.com/science/article/pii/S0164121218302735)

    Article  Google Scholar 

  49. Li, H., Ghodsi, A., Zaharia, M., Shenker, S., Stoica, I.: Tachyon: Reliable, memory speed storage for cluster computing frameworks. In: Proceedings of the ACM Symposium on Cloud Computing, pp. 1–15 (2014)

  50. Li, T., Chandramouli, B., Burckhardt, S., Madden, S.: Darq matter binds everything: Performant and composable cloud programming via resilient steps. Proc. ACM Manag. Data https://doi.org/10.1145/3589262 (2023)

  51. Liu, D.H., Levy, A., Noghabi, S., Burckhardt, S.: Doing more with less: Orchestrating serverless applications without an orchestrator. In: 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pp. 1505–1519 (2023)

  52. Microsoft Research: FASTER. https://github.com/microsoft/FASTER (2022)

  53. Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M.I., et al Ray: A distributed framework for emerging AI applications. In: 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pp. 561–577 (2018)

  54. Navathe, S., Ceri, S., Wiederhold, G., Dou, J.: Vertical partitioning algorithms for database design. ACM Trans. Database Syst. 9(4), 680–710 (1984)

    Article  Google Scholar 

  55. Netflix: Netflix Conductor. https://netflix.github.io/conductor/ (2022)

  56. Perron, M., Castro Fernandez, R., DeWitt, D., Madden, S.: Starling: A scalable query engine on cloud functions. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 131–141 (2020)

  57. Qi, S., Liu, X., Jin, X.: Halfmoon: Log-optimal fault-tolerant stateful serverless computing. In: Proceedings of the 29th Symposium on Operating Systems Principles, pp. 314–330 (2023)

  58. Rajan RAP (2018) Serverless architecture - a revolution in cloud computing. In: 2018 Tenth International Conference on Advanced Computing (ICoAC), pp. 88–93

  59. Sacca, D., Wiederhold, G.: Database partitioning in a cluster of processors. ACM Trans. Database Syst. 10(1), 29–56 (1985)

    Article  Google Scholar 

  60. Schleier-Smith, J.: Serverless foundations for elastic database systems. In: CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13–16, http://cidrdb.org/cidr2019/gongshow/abstracts/cidr2019_140.pdf (2019)

  61. Shahrad, M., Fonseca, R., Goiri, I., Chaudhry, G.I., Batum, P., Cooke, J., Laureano, E., Tresness, C., Russinovich, M., Bianchini, R.: Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In: Proceedings of the USENIX Annual Technical Conference (ATC), USENIX (2020)

  62. Shilkov, M.: Temporal: Open Source Workflows as Code. https://mikhail.io/2020/10/temporal-open-source-workflows-as-code/ (2020)

  63. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), IEEE, pp. 1–10 (2010)

  64. Sreekanti, V., Wu, C., Chhatrapati, S., Gonzalez, J.E., Hellerstein, J.M., Faleiro, J.M.: A fault-tolerance shim for serverless computing. In: Proceedings of the Fifteenth European Conference on Computer Systems, Association for Computing Machinery, New York, NY, EuroSys ’20, https://doi.org/10.1145/3342195.3387535 (2020a)

  65. Sreekanti, V., Wu, C., Lin, X.C., Schleier-Smith, J., Gonzalez, J.E., Hellerstein, J.M., Tumanov, A.: Cloudburst: Stateful functions-as-a-service. Proceedings of the VLDB Endowment pp. 2438–2452 (2020b)

  66. Stojkovic, J., Xu, T., Franke, H., Torrellas, J.: Specfaas: Accelerating serverless applications with speculative function execution. In: 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA), IEEE, pp. 814–827 (2023)

  67. Temporal: Temporal. https://temporal.io/ (2022)

  68. van Eyk, E., Toader, L., Talluri, S., Versluis, L., Uţă, A., Iosup, A.: Serverless is more: From PaaS to present cloud computing. IEEE Internet Comput. 22(5), 8–17 (2018). https://doi.org/10.1109/MIC.2018.053681358

    Article  Google Scholar 

  69. Wu, C., Sreekanti, V., Hellerstein, J.M.: Transactional causal consistency for serverless computing. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, New York, NY, SIGMOD ’20, pp. 83–97, https://doi.org/10.1145/3318464.3389710 (2020)

  70. Zeebe: Zeebe: A Workflow Engine for Microservices Orchestration. https://zeebe.io/ (2022)

  71. Zhang, H., Cardoza, A., Chen, P.B., Angel, S., Liu, V.: Fault-tolerant and transactional stateful serverless workflows. In: 14th \(\{USENIX\}\) Symposium on Operating Systems Design and Implementation (\(\{OSDI\}\) 20) (2020a)

  72. Zhang, W., Fang, V., Panda, A., Shenker, S.: Kappa: A programming framework for serverless computing. In: Proceedings of the 11th ACM Symposium on Cl oud Computing, ACM, p. 16 (2020b)

Download references

Author information

Authors and Affiliations

  1. Microsoft Research, Redmond, USA

    Sebastian Burckhardt & Badrish Chandramouli

  2. Microsoft Azure, Redmond, USA

    Chris Gillum, David Justo & Connor McMahon

  3. University of California, Los Angeles, Los Angeles, USA

    Konstantinos Kallas

  4. Carnegie Mellon University, Pittsburgh, USA

    Christopher S. Meiklejohn

  5. University of Washington, Seattle, USA

    Xiangfeng Zhu

Authors
  1. Sebastian Burckhardt
    View author publications

    Search author on:PubMed Google Scholar

  2. Badrish Chandramouli
    View author publications

    Search author on:PubMed Google Scholar

  3. Chris Gillum
    View author publications

    Search author on:PubMed Google Scholar

  4. David Justo
    View author publications

    Search author on:PubMed Google Scholar

  5. Konstantinos Kallas
    View author publications

    Search author on:PubMed Google Scholar

  6. Connor McMahon
    View author publications

    Search author on:PubMed Google Scholar

  7. Christopher S. Meiklejohn
    View author publications

    Search author on:PubMed Google Scholar

  8. Xiangfeng Zhu
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Burckhardt.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Burckhardt, S., Chandramouli, B., Gillum, C. et al. Netherite: efficient execution of serverless workflows. The VLDB Journal 34, 25 (2025). https://doi.org/10.1007/s00778-024-00898-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1007/s00778-024-00898-1

Keywords