This is currently used for 4676 ingest and is only supported for map reduce ingest currently. We should additionally support this use case for local file ingest.