The way to do this, I think:
- Add fields to the initiator JSON payload containing stuff like
same.yaml, steps, requirements.txt, etc. So basically include everything.
- Change the executor to accept context blobs which contain serialised global namespaces (like we do for kubeflow/aml)
- Change orchestrator to fan-out each step and thread the contexts through.