Skip to content

[BUG] Test failure with q41.ppl when match exists #4845

@mch2

Description

@mch2

Query Information

PPL Command/Query:

source=hits
| where CounterID = 62 and EventDate >= '2013-07-01 00:00:00' and EventDate <= '2013-07-31 00:00:00' and IsRefresh = 0 and TraficSourceID in (-1, 6) and RefererHash = 3594120000172545465
| stats bucket_nullable=false count() as PageViews by URLHash, EventDate
| sort - PageViews
| head 10 from 100
# Remember to sanitize any sensitive fields or values

Expected Result:

{
  "schema": [
    {
      "name": "PageViews",
      "type": "bigint"
    },
    {
      "name": "URLHash",
      "type": "bigint"
    },
    {
      "name": "EventDate",
      "type": "timestamp"
    }
  ],
  "datarows": [
  ... expected a hit here
],
  "total": 0,
  "size": 0
}

Actual Result:
Failure in logs:

[2025-11-21T15:02:32,298][ERROR][o.o.s.p.r.RestPPLQueryAction] [runTask-0] Error happened during query handling
java.lang.RuntimeException: java.sql.SQLException: exception while executing query: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
        at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:217) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:74) ~[?:?]
        at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$1(OpenSearchExecutionEngine.java:209) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.client.OpenSearchNodeClient.schedule(OpenSearchNodeClient.java:222) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.execute(OpenSearchExecutionEngine.java:207) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.executor.QueryService.lambda$executeWithCalcite$1(QueryService.java:105) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at java.base/java.security.AccessController.doPrivileged(AccessController.java:74) ~[?:?]
        at org.opensearch.sql.executor.QueryService.lambda$executeWithCalcite$0(QueryService.java:96) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.calcite.CalcitePlanContext.run(CalcitePlanContext.java:121) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.executor.QueryService.executeWithCalcite(QueryService.java:93) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.executor.QueryService.execute(QueryService.java:70) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.executor.execution.QueryPlan.execute(QueryPlan.java:66) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$submit$0(OpenSearchQueryManager.java:28) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$withCurrentContext$0(OpenSearchQueryManager.java:42) [opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:916) [opensearch-3.4.0-SNAPSHOT.jar:3.4.0-SNAPSHOT]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:1447) [?:?]
Caused by: java.sql.SQLException: exception while executing query: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
        at org.apache.calcite.avatica.Helper.createException(Helper.java:56) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.Helper.createException(Helper.java:41) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:579) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        ... 17 more
Caused by: java.lang.IllegalArgumentException: Construct TIMESTAMP from "2013-07-15 00:00:00" failed, unsupported format.
        at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.parseDateTimeString(OpenSearchExprValueFactory.java:308) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.createOpenSearchDateType(OpenSearchExprValueFactory.java:340) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.parse(OpenSearchExprValueFactory.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.data.value.OpenSearchExprValueFactory.construct(OpenSearchExprValueFactory.java:182) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.response.OpenSearchResponse.lambda$handleAggregationResponse$0(OpenSearchResponse.java:232) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215) ~[?:?]
        at java.base/java.util.AbstractList$RandomAccessSpliterator.tryAdvance(AbstractList.java:708) ~[?:?]
        at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.lambda$initPartialTraversalState$0(StreamSpliterators.java:297) ~[?:?]
        at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.fillBuffer(StreamSpliterators.java:206) ~[?:?]
        at java.base/java.util.stream.StreamSpliterators$AbstractWrappingSpliterator.doAdvance(StreamSpliterators.java:161) ~[?:?]
        at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:303) ~[?:?]
        at java.base/java.util.Spliterators$1Adapter.hasNext(Spliterators.java:669) ~[?:?]
        at org.opensearch.sql.opensearch.storage.scan.OpenSearchIndexEnumerator.moveNext(OpenSearchIndexEnumerator.java:127) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.EnumerableDefaults$SkipWhileEnumerator.moveNext(EnumerableDefaults.java:4141) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.EnumerableDefaults$TakeWhileEnumerator.moveNext(EnumerableDefaults.java:4055) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.EnumerableDefaults$TakeWhileEnumerator.moveNext(EnumerableDefaults.java:4055) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.<init>(Linq4j.java:666) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.Linq4j.enumeratorIterator(Linq4j.java:99) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.linq4j.AbstractEnumerable.iterator(AbstractEnumerable.java:33) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.MetaImpl.createCursor(MetaImpl.java:91) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:186) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:64) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:43) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:575) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$2(OpenSearchExecutionEngine.java:213) ~[opensearch-sql-3.4.0.0-SNAPSHOT.jar:3.4.0.0-SNAPSHOT]
        ... 17 more

Dataset Information

Dataset/Schema Type

Test data currently in repo, updated values so that filters have a hit -

{"index":{}}
{"WatchID":"9110818468285196899","JavaEnable":0,"Title":"","GoodEvent":1,"EventTime":"2013-07-14 20:38:47","EventDate":"2013-07-15","CounterID":62,"ClientIP":-1216690514,"RegionID":839,"UserID":"-2461439046089301801","CounterClass":0,"OS":0,"UserAgent":0,"URL":"","Referer":"https://example.org/about","IsRefresh":0,"RefererCategoryID":0,"RefererRegionID":0,"URLCategoryID":0,"URLRegionID":0,"ResolutionWidth":0,"ResolutionHeight":0,"ResolutionDepth":0,"FlashMajor":0,"FlashMinor":0,"FlashMinor2":"","NetMajor":0,"NetMinor":0,"UserAgentMajor":0,"UserAgentMinor":"�O","CookieEnable":0,"JavascriptEnable":0,"IsMobile":0,"MobilePhone":0,"MobilePhoneModel":"","Params":"","IPNetworkID":3793327,"TraficSourceID":6,"SearchEngineID":0,"SearchPhrase":"","AdvEngineID":0,"IsArtifical":0,"WindowClientWidth":0,"WindowClientHeight":0,"ClientTimeZone":-1,"ClientEventTime":"1971-01-01 14:16:06","SilverlightVersion1":0,"SilverlightVersion2":0,"SilverlightVersion3":0,"SilverlightVersion4":0,"PageCharset":"","CodeVersion":0,"IsLink":0,"IsDownload":0,"IsNotBounce":0,"FUniqID":"0","OriginalURL":"","HID":0,"IsOldCounter":0,"IsEvent":0,"IsParameter":0,"DontCountHits":0,"WithHash":0,"HitColor":"5","LocalEventTime":"2013-07-15 10:47:34","Age":0,"Sex":0,"Income":0,"Interests":0,"Robotness":0,"RemoteIP":-1001831330,"WindowName":-1,"OpenerName":-1,"HistoryLength":-1,"BrowserLanguage":"�","BrowserCountry":"�\f","SocialNetwork":"","SocialAction":"","HTTPError":0,"SendTiming":0,"DNSTiming":0,"ConnectTiming":0,"ResponseStartTiming":0,"ResponseEndTiming":0,"FetchTiming":0,"SocialSourceNetworkID":0,"SocialSourcePage":"","ParamPrice":"0","ParamOrderID":"","ParamCurrency":"NH\u001C","ParamCurrencyID":0,"OpenstatServiceName":"","OpenstatCampaignID":"","OpenstatAdID":"","OpenstatSourceID":"","UTMSource":"","UTMMedium":"","UTMCampaign":"","UTMContent":"","UTMTerm":"","FromTag":"","HasGCLID":0,"RefererHash":"3594120000172545465","URLHash":"-8417682003818480435","CLID":0}
``

Index Mapping

same as clickbench_index_mapping.json

Sample Data
see above


## Bug Description
**Issue Summary:**
I'm running CalcitePPLClickBenchIT, query 41 fails when i update the test data in `clickbench.json` to match filters - included below.

**Steps to Reproduce:**
1. update test date to include a hit
2. Run test with q41 only
3.

**Impact:**
test failure

## Environment Information
**OpenSearch Version:**
main

**Additional Details:**
<!-- Any other relevant environment information -->
<!-- Exclude sensitive infrastructure details -->

## Screenshots
<!-- If applicable, add screenshots to help explain your problem -->
<!-- ⚠️ IMPORTANT: Ensure screenshots don't contain sensitive information -->

Metadata

Metadata

Assignees

Labels

PPLPiped processing languagebugSomething isn't workingclickbenchpushdownpushdown related issuestestingRelated to improving software testing

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions