Skip to content

Remove IncrementalIndex template modifier#11160

Merged
jihoonson merged 2 commits intoapache:masterfrom
liran-funaro:pr-ii-remove-template
Oct 27, 2021
Merged

Remove IncrementalIndex template modifier#11160
jihoonson merged 2 commits intoapache:masterfrom
liran-funaro:pr-ii-remove-template

Conversation

@liran-funaro
Copy link
Contributor

@liran-funaro liran-funaro commented Apr 26, 2021

This patch removes the template modifier from IncrementalIndex<AggregatorType> . It is no longer required:

  • private final AggregatorType[] aggs moved to OffheapIncrementalIndex (it is only used there).
  • iterableWithPostAggregations() defined as abstract and implemented in OnheapIncrementalIndex/OffheapIncrementalIndex.
  • initAggs() returns void.

Rationale

It is not required. Especially since OffheapIncrementalIndex will be removed by #11124.

}

@Override
public Iterable<Row> iterableWithPostAggregations(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the reason for moving this method from IncrementalInde to here and OffheapIncrementalIndex?

Copy link
Contributor Author

@liran-funaro liran-funaro Oct 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When this method was in the generic IncrementalIndex, it has to use the generic AggregatorType by calling the following abstract generic functions:

AggregatorType[] getAggsForRow(int rowOffset);
Object getAggVal(AggregatorType agg, int rowOffset, int aggPosition);

These functions were forced and didn't really fit each of the implementations.
getAggsForRow() didn't fit OffheapIncrementalIndex since it only have one aggregator instance (not one for each row).
getAggVal() didn't fit OnheapIncrementalIndex since rowOffset and aggPosition is only relevant to BufferAggregator.

That said, other than that, the implementation is identical in both incremental-index implementations.
It might be possible to find a more generic implementation. But since we are removing OffheapIncrementalIndex, I don't see the point of making this effort here.
Once this PR and PR #11124 will be merged, I will try to generify this function for both OnheapIncrementalIndex and OakIncrementalIndex (#10001).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thank you for the explanation.

# Conflicts:
#	processing/src/main/java/org/apache/druid/segment/incremental/IncrementalIndex.java
#	processing/src/main/java/org/apache/druid/segment/incremental/OffheapIncrementalIndex.java
@liran-funaro
Copy link
Contributor Author

@jihoonson I updated this PR after the merger of #11124.

Copy link
Contributor

@jihoonson jihoonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks @liran-funaro.

}

@Override
public Iterable<Row> iterableWithPostAggregations(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thank you for the explanation.

@jihoonson jihoonson merged commit 9ca8f1e into apache:master Oct 27, 2021
@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants