one thing to note is that when stopping the docker container with CTRL+C, it seems to finish the current sleep, and won't actually terminate for another 60 seconds. This doesn't seem ideal. I suppose we could pass in a sleep time, so that we can set it to 1 for development purposes. I'd love to hear other thoughts about this.

I mean, this is so lightweight anyway that checking every 10-15 seconds would be fine too, and eliminate that problem.

The long term solution to this is IMO to figure out how to call vDB with a set of commands and do it's own concurrency with goroutines, since the current system is bad practice anyway.. :)

Agree that this is no problem and potentially cutting down the interval could make sense.

I think enabling vdb to be invoked with a set of commands is a cool idea but I'd also be in favor of isolating each command in its own container (and breaking up execute so that we can have separate containers extracting logs vs extracting diffs vs transforming logs vs transforming diffs). Thinking that should make it easier to restart an isolated process that fails without needing to restart everything (rather than the current approach which means we have to restart header sync due to an unrelated fatal error in the event transformer execution)

cool, thanks for the feedback guys! i'll drop the checking interval to 10 seconds.

and yep, i definitely agree that we need to rework how we're kicking off vdb. I was also thinking along the lines of breaking things up into different containers. but i like the idea of breaking our current commands down even further as well. 👍

I mean, this is so lightweight anyway that checking every 10-15 seconds would be fine too, and eliminate that problem.

The long term solution to this is IMO to figure out how to call vDB with a set of commands and do it's own concurrency with goroutines, since the current system is bad practice anyway.. :)