-
Notifications
You must be signed in to change notification settings - Fork 190
[BUG] SQL Join timeout isn't applied when using PIT #3820
Description
What is the bug?
When using a JOIN timeout hint with a PIT-enabled cluster, the PIT will get deleted before the timeout expires.
Caused by: java.lang.IllegalArgumentException: Point In Time id: [y_T7MwDnSG-rblpw9Xa6mQ] doesn't exist
This is due to a 1-minute keepalive
sql/opensearch/src/main/java/org/opensearch/sql/opensearch/setting/OpenSearchSettings.java
Line 61 in 35518dc
| timeValueMinutes(1), |
Used by PIT at
sql/legacy/src/main/java/org/opensearch/sql/legacy/pit/PointInTimeHandlerImpl.java
Line 59 in 35518dc
| LocalClusterState.state().getSettingValue(SQL_CURSOR_KEEP_ALIVE), false, indices); |
How can one reproduce the bug?
Steps to reproduce the behavior:
- Create indices that require more than 60s to join on (>5 mil records each)
- Configure the cluster to use PIT joins per
sql/docs/dev/opensearch-pagination.md
Line 497 in 35518dc
"plugins.sql.pagination.api" : "true" - Run a join query with a longer join timeout hint
/*! JOIN_TIME_OUT(90) */ - See in the logs that the join failed because the PIT no longer exists (deleted by the 1m keepalive)
What is the expected behavior?
The JOIN_TIME_OUT should also update the keepalive
What is your host/environment?
- OS: Latest
Do you have any screenshots?
N/A
Do you have any additional context?
Add any other context about the problem.