PROTOTYPE: Magic files to access search results/modified files#357
PROTOTYPE: Magic files to access search results/modified files#357mrnugget wants to merge 1 commit into
Conversation
eseliger
left a comment
There was a problem hiding this comment.
Code-wise this looks good.
I also think there are valid use-cases for such a pass-in to the container, for example the case where argument lists would become too long for bash to process.
However, I think it is heavier to use and (at least from my perspective) a little harder to understand. Probably mostly because I personally never saw such a pattern before, at least not such an implicit pattern.
I was reminded of kubernetes secrets/configmaps when thinking about this and I thought: Why not make a hybrid of this file-based stuff and of the variables syntax, while making the use of files a little more explicit as well.
Example:
steps:
- image: golang:1.14
command: go fmt ${{ search_results }} # I assume there are only a few, so I pass it in directly, for the sake of simplicity.
- image: node:14
command: </modified-files xargs npm run prettier -c /prettier/conf.js
mount:
# Allows for arbitrary files that are needed.
/prettier.conf.js: |
useCurly: true
other super important config value: oh yeah
# As well as files that contain the content of a yaml variable
/modified-files: ${{ join modified_files "\n" }} # hell that's a lot of files, need to put them in a fileThat way
- it is explicit where files are mounted that hold the variables content
- is similar to how kubernetes would do it
- Also covers the case where the user needs to pass-in a config file or something
- Allows to use both variables AND files
I haven't really thought this through, but I could imagine it is similarly easy to implement and would take the best of the two prototypes.
|
Cool idea! Like Erik, I wonder if this is just a bit less user-friendly than templates. Here's a (possibly terrible) idea... what if this: |
That can be done using |
Yeah, I thought about something similar, but what I ended up thinking about was more along the lines of But I think we should provide more primitives, because those will allow user to do that and more. I'll try to see if I can implement Erik's idea today. |
|
Closing in favor of #361. |
NOTE FOR REVIEWERS: This is a prototype and should be treated as such, which means: let's review idea/design/feasibility instead of concrete implementation details (but if you do have concrete feedback on the implementation, feel free to leave it as such).
This is another prototype, next to #334, to solve the same thing: allowing
step.runcommands to access specific files.What this prototype does is to create magic files inside the container. The files contain lists of files, one filename per line. By using standard shell scripting, the
runcommands can then access these files.Example campaign spec:
Screenshot
Details
This branch here builds on top of #334. But instead of using templating, it simply makes the data available in temp files that are then mounted into the container.
Drawbacks