-
Notifications
You must be signed in to change notification settings - Fork 190
[BUG] Test failure with q41.ppl when match exists #4845
Copy link
Copy link
Closed
Labels
PPLPiped processing languagePiped processing languagebugSomething isn't workingSomething isn't workingclickbenchpushdownpushdown related issuespushdown related issuestestingRelated to improving software testingRelated to improving software testing
Description
Query Information
PPL Command/Query:
source=hits
| where CounterID = 62 and EventDate >= '2013-07-01 00:00:00' and EventDate <= '2013-07-31 00:00:00' and IsRefresh = 0 and TraficSourceID in (-1, 6) and RefererHash = 3594120000172545465
| stats bucket_nullable=false count() as PageViews by URLHash, EventDate
| sort - PageViews
| head 10 from 100
# Remember to sanitize any sensitive fields or values
Expected Result:
{
"schema": [
{
"name": "PageViews",
"type": "bigint"
},
{
"name": "URLHash",
"type": "bigint"
},
{
"name": "EventDate",
"type": "timestamp"
}
],
"datarows": [
... expected a hit here
],
"total": 0,
"size": 0
}
Actual Result:
Failure in logs:
[2025-11-21T15:02:32,298][ERROR][o.o.s.p.r.RestPPLQueryAction] [runTask-0] Error happened during query handling
java.lang.RuntimeException: java.sql.SQLException: exception while executing query: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:217) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at java.base/java.security.AccessController.doPrivileged(AccessController.java:74) ~[?:?]
at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$1(OpenSearchExecutionEngine.java:209) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.client.OpenSearchNodeClient.schedule(OpenSearchNodeClient.java:222) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.execute(OpenSearchExecutionEngine.java:207) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.executor.QueryService.lambda$executeWithCalcite$1(QueryService.java:105) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at java.base/java.security.AccessController.doPrivileged(AccessController.java:74) ~[?:?]
at org.opensearch.sql.executor.QueryService.lambda$executeWithCalcite$0(QueryService.java:96) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.calcite.CalcitePlanContext.run(CalcitePlanContext.java:121) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.executor.QueryService.executeWithCalcite(QueryService.java:93) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.executor.QueryService.execute(QueryService.java:70) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.executor.execution.QueryPlan.execute(QueryPlan.java:66) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$submit$0(OpenSearchQueryManager.java:28) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$withCurrentContext$0(OpenSearchQueryManager.java:42) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:916) [opensearch-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619) [?:?]
at java.base/java.lang.Thread.run(Thread.java:1447) [?:?]
Caused by: java.sql.SQLException: exception while executing query: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
at org.apache.calcite.avatica.Helper.createException(Helper.java:56) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.Helper.createException(Helper.java:41) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:579) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
... 17 more
Caused by: java.lang.IllegalArgumentException: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.parseDateTimeString(OpenSearchExprValueFactory.java:308) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.createOpenSearchDateType(OpenSearchExprValueFactory.java:340) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.parse(OpenSearchExprValueFactory.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.construct(OpenSearchExprValueFactory.java:182) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.response.OpenSearchResponse.lambda$handleAggregationResponse$0(OpenSearchResponse.java:232) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215) ~[?:?]
at java.base/java.util.AbstractList$RandomAccessSpliterator.tryAdvance(AbstractList.java:708) ~[?:?]
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:297) ~[?:?]
at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206) ~[?:?]
at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161) ~[?:?]
at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:303) ~[?:?]
at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:669) ~[?:?]
at org.opensearch.sql.opensearch.storage.scan.OpenSearchIndexEnumerator.moveNext(OpenSearchIndexEnumerator.java:127) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.EnumerableDefaults$SkipWhileEnumerator.moveNext(EnumerableDefaults.java:4141) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.EnumerableDefaults$TakeWhileEnumerator.moveNext(EnumerableDefaults.java:4055) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.EnumerableDefaults$TakeWhileEnumerator.moveNext(EnumerableDefaults.java:4055) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<init>(Linq4j.java:666) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:99) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:91) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:575) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
... 17 more
Dataset Information
Dataset/Schema Type
Test data currently in repo, updated values so that filters have a hit -
{"index":{}}
{"WatchID":"9110818468285196899","JavaEnable":0,"Title":"","GoodEvent":1,"EventTime":"2013-07-14 20:38:47","EventDate":"2013-07-15","CounterID":62,"ClientIP":-1216690514,"RegionID":839,"UserID":"-2461439046089301801","CounterClass":0,"OS":0,"UserAgent":0,"URL":"","Referer":"https://example.org/about","IsRefresh":0,"RefererCategoryID":0,"RefererRegionID":0,"URLCategoryID":0,"URLRegionID":0,"ResolutionWidth":0,"ResolutionHeight":0,"ResolutionDepth":0,"FlashMajor":0,"FlashMinor":0,"FlashMinor2":"","NetMajor":0,"NetMinor":0,"UserAgentMajor":0,"UserAgentMinor":"�O","CookieEnable":0,"JavascriptEnable":0,"IsMobile":0,"MobilePhone":0,"MobilePhoneModel":"","Params":"","IPNetworkID":3793327,"TraficSourceID":6,"SearchEngineID":0,"SearchPhrase":"","AdvEngineID":0,"IsArtifical":0,"WindowClientWidth":0,"WindowClientHeight":0,"ClientTimeZone":-1,"ClientEventTime":"1971-01-01 14:16:06","SilverlightVersion1":0,"SilverlightVersion2":0,"SilverlightVersion3":0,"SilverlightVersion4":0,"PageCharset":"","CodeVersion":0,"IsLink":0,"IsDownload":0,"IsNotBounce":0,"FUniqID":"0","OriginalURL":"","HID":0,"IsOldCounter":0,"IsEvent":0,"IsParameter":0,"DontCountHits":0,"WithHash":0,"HitColor":"5","LocalEventTime":"2013-07-15 10:47:34","Age":0,"Sex":0,"Income":0,"Interests":0,"Robotness":0,"RemoteIP":-1001831330,"WindowName":-1,"OpenerName":-1,"HistoryLength":-1,"BrowserLanguage":"�","BrowserCountry":"�\f","SocialNetwork":"","SocialAction":"","HTTPError":0,"SendTiming":0,"DNSTiming":0,"ConnectTiming":0,"ResponseStartTiming":0,"ResponseEndTiming":0,"FetchTiming":0,"SocialSourceNetworkID":0,"SocialSourcePage":"","ParamPrice":"0","ParamOrderID":"","ParamCurrency":"NH\u001C","ParamCurrencyID":0,"OpenstatServiceName":"","OpenstatCampaignID":"","OpenstatAdID":"","OpenstatSourceID":"","UTMSource":"","UTMMedium":"","UTMCampaign":"","UTMContent":"","UTMTerm":"","FromTag":"","HasGCLID":0,"RefererHash":"3594120000172545465","URLHash":"-8417682003818480435","CLID":0}
``
Index Mapping
same as clickbench_index_mapping.jsonSample Data
see above
## Bug Description
**Issue Summary:**
I'm running CalcitePPLClickBenchIT, query 41 fails when i update the test data in `clickbench.json` to match filters - included below.
**Steps to Reproduce:**
1. update test date to include a hit
2. Run test with q41 only
3.
**Impact:**
test failure
## Environment Information
**OpenSearch Version:**
main
**Additional Details:**
<!-- Any other relevant environment information -->
<!-- Exclude sensitive infrastructure details -->
## Screenshots
<!-- If applicable, add screenshots to help explain your problem -->
<!-- ⚠️ IMPORTANT: Ensure screenshots don't contain sensitive information -->
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
PPLPiped processing languagePiped processing languagebugSomething isn't workingSomething isn't workingclickbenchpushdownpushdown related issuespushdown related issuestestingRelated to improving software testingRelated to improving software testing
Type
Projects
Status
Done