Skip to content

We should support a "meta-stage" set-up where you can run multiple transformations in a single stage at once, without caching the files in between. #56

@mmcdermott

Description

@mmcdermott

This would make things likely a fair bit faster and less disk expensive, especially when compiling multiple small transformations one after the other.

This can also tie into #55 in that both present fundamental improvements to the pipeline runnability and API.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Computational PerformanceIssues relating to efficient computational performance of MEDS_transforms pipelinesPipeline Configuration and Stage ManagementIssues relating to proper definition and usability of different stages in a pipelineRunnerFor things about the multi-stage, single-script RunnerUsability / Interfacepriority:highA high priority issue.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions