Update:

when we call softUpdateDocument for a segment that already has some deleted doc, it will iterate all the deleted doc use ReadersAndUpdates#MergedDocValues#onDiskDocValues, but he has to iterate the array twice, the first time is
Lucene90DocValuesConsumer#writeValues will compute gcd, min, max. the second time is IndexedDISI#writeBitSet, this creates some waste, we can remove the first iterate for soft delete, this can speed up about 53% for updates.

Benchmark code:

  public static void main(final String[] args) throws Exception {
    long min = Long.MAX_VALUE;
    for (int i = 0; i < 5; i++) {
      min = Math.min(doWrite(), min);
    }
    System.out.println("BEST:" + min);
  }

static long doWrite() throws IOException {
    Random rand = new Random(5);
    Directory dir = new ByteBuffersDirectory();
    IndexWriter writer =
        new IndexWriter(
            dir,
            new IndexWriterConfig(null)
                .setSoftDeletesField("_soft_deletes")
                .setMaxBufferedDocs(IndexWriterConfig.DISABLE_AUTO_FLUSH));
    int maxDoc = 4096 * 100;

    for (int i = 0; i < maxDoc; i++) {
      Document doc = new Document();
      doc.add(new StringField("id", String.valueOf(i), Field.Store.NO));

      writer.addDocument(doc);
      if (i > 0 && i % 5000 == 0) {
        writer.commit();
      }
    }

    System.out.println("start update");
    long t0 = System.currentTimeMillis();

    for (int i = 0; i < maxDoc; i += 2) {
      Document doc = new Document();
      writer.softUpdateDocument(
          new Term("id", String.valueOf(i)), doc, new NumericDocValuesField("_soft_deletes", 1));
      if (i > 0 && i % 100 == 0) {
        writer.commit();
      }
    }
    long tookMs = System.currentTimeMillis() - t0;
    System.out.println("update took:" + (System.currentTimeMillis() - t0));

    IOUtils.close(writer, dir);
    return tookMs;
  }

@jpountz Would you please take a look when you get a chance?

Update: revert changes about IndexedDISI#advance to keep things as simple as possible.

I understand the idea, I'm a bit less happy about special-casing the soft deletes field in our doc values file format. I don't have a better suggestion though...

Thanks @jpountz , it isn't perfect, there seems to be no better way to avoid probing data and know early on that all values are the same, I am just trying to improve, but i haven't thought of a better way yet

I get some great suggestion from @gf2121 , we can get if the doc values has single value from ReadersAndUpdates#onDiskDocValues and ReadersAndUpdates#updatesToApply this can be avoid use special-casing for soft deletes field in doc values, is it looks better? @jpountz

Thanks @easyice !

As we are exposing API like softUpdateDocument(Term term, Iterable<? extends IndexableField> doc, Field... softDeletes). We cannot guarantee that all soft delete fields have the same value 1 in Lucene. This patch looks more reasonable as we remove this assertion. But I am not sure if it is the right answer to touch the NumericDocValues interface for this scene, Let's see what other people think :)

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!

This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution!