Problem Description
VDMS supports an ad-hoc method using which users of VDMS can serve user-defined functions (UDF) locally or on a remote server. Setting up the UDF environment in the manner above suffers from the following problems
- The user will have to manually set up the local/remote UDF environment.
- User has to specify the URL to the remote server explicitly in the query itself
- There is no notion of automatically tackling points of failure in the system
- Resource management (required vs available remote servers) by VDMS is missing
Solution
- Support for an orchestration framework (like Kubernetes) in VDMS should be included, which automatically spins up the required number of remote servers.
- The addition of a scheduler inside VDMS could help make decisions about where a particular query could be served.
- Kubernetes gives some off-the-shelf solutions to deploy applications in such a way that points of failure are minimized.
- A controller inside VDMS could help in resource management in order to create/delete remote servers and services to support remote operations.
- An easy setup script and config file are required to spin up the cluster with the least effort from the VDMS user/administrator.