The current log-ingestor doesn't handle killed compression jobs. When that case happens, the log-ingestor will poll for results forever.
As confirmed by @junhaoliao, we will redesign the compression side flow so that "killed" is not a recoverable state. To address this issue, we only need to check whether the state is killed, and quit with error messages printed in the log.
As discussed in #1978, a killed compression job should also trigger the ingested files' state associated with the compression job to be "failed".
The current log-ingestor doesn't handle killed compression jobs. When that case happens, the log-ingestor will poll for results forever.
As confirmed by @junhaoliao, we will redesign the compression side flow so that "killed" is not a recoverable state. To address this issue, we only need to check whether the state is killed, and quit with error messages printed in the log.
As discussed in #1978, a killed compression job should also trigger the ingested files' state associated with the compression job to be "failed".