-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted #25983
Copy link
Copy link
Closed
Description
Steps to reproduce:
List the minimal actions needed to reproduce the behaviour.
- Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted.
- To be honest, I'm not sure whether this issue should be placed on https://github.com/influxdata/influxdb or https://github.com/apache/arrow-java. But considering that no one has ever written unit tests for InfluxDB 3 Core on https://github.com/apache/arrow-java, I think I should confirm it on the InfluxDB side first.
- I created a minimal unit test at https://github.com/linghengqian/influxdb-3-core-jdbc-test/blob/master/src/test/java/io/github/linghengqian/TimeDifferenceTest.java. To execute it, just install
SDKMAN!andDocker CEin advance, then,
sdk install java 21.0.6-ms
git clone git@github.com:linghengqian/influxdb-3-core-jdbc-test.git
cd ./influxdb-3-core-jdbc-test/
sdk use java 21.0.6-ms
./mvnw -T 1C -Dtest=TimeDifferenceTest clean testClick me to view the core logic of the unit test🥯🥨🍟🧂🥖🥚🍔🦪🍜🍘
@Testcontainers
public class TimeDifferenceTest {
private final Instant magicTime = Instant.now().minusSeconds(10);
@Container
private final GenericContainer<?> container = new GenericContainer<>("quay.io/influxdb/influxdb3-core:911ba92ab4133e75fe2a420e16ed9cb4cf32196f")
.withCommand("serve --node-id local01 --object-store file --data-dir /home/influxdb3/.influxdb3")
.withExposedPorts(8181);
@Test
void test() throws Exception {
try (InfluxDBClient client = InfluxDBClient.getInstance(
"http://" + container.getHost() + ":" + container.getMappedPort(8181),
null,
"mydb")) {
writeData(client);
queryDataByInfluxDbClient(client);
queryDataByJdbcDriver();
}
}
private void writeData(InfluxDBClient client) {
Point point = Point.measurement("home")
.setTag("location", "London")
.setField("value", 30.01)
.setTimestamp(magicTime);
client.writePoint(point);
}
private void queryDataByInfluxDbClient(InfluxDBClient client) {
try (Stream<PointValues> stream = client.queryPoints("select time,location,value from home order by time desc limit 10",
QueryOptions.DEFAULTS)) {
List<PointValues> list = stream.toList();
assertThat(list.size(), is(1));
PointValues p = list.getFirst();
assertThat(p.getField("value", Double.class), is(30.01));
assertThat(p.getTag("location"), is("London"));
assertThat(p.getTimestamp(), is(NanosecondConverter.convert(magicTime, WritePrecision.NS)));
}
}
private void queryDataByJdbcDriver() throws SQLException {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setJdbcUrl("jdbc:arrow-flight-sql://" + container.getHost() + ":" + container.getMappedPort(8181) + "/?useEncryption=0&database=mydb");
try (HikariDataSource hikariDataSource = new HikariDataSource(hikariConfig);
Connection connection = hikariDataSource.getConnection()) {
ResultSet resultSet = connection.createStatement().executeQuery("select time,location,value from home order by time desc limit 10");
assertThat(resultSet.next(), is(true));
assertThat(resultSet.getString("location"), is("London"));
assertThat(resultSet.getString("value"), is("30.01"));
assertThat(Timestamp.from(magicTime).getTime(), is(magicTime.toEpochMilli()));
assertThat(resultSet.getString("time"), notNullValue());
assertThat(resultSet.getTimestamp("time").getTime(), is(magicTime.toEpochMilli()));
}
}
}- Obviously, there is no concept of ZoneId or Offset for
java.time.Instant, which is a fixed point on the continuous timeline. The timestamp of the fixed time is inserted through the InfluxDB Client, and then the same timestamp can be obtained by executing SQL to the InfluxDB Client. However, executing the same SQL through the Flight JDBC Driver will obtain a timestamp that is inconsistent with the original one. That is, this assertion will fail.
assertThat(resultSet.getTimestamp("time").getTime(), is(magicTime.toEpochMilli()));- Some early investigation came from The documentation or JavaDoc of
InfluxDBClientshould document the existence ofNanosecondConverterInfluxCommunity/influxdb3-java#222 .
Expected behaviour:
Describe what you expected to happen.
- Timestamps queried from InfluxDB 3 Core via the JDBC driver are consistent with the inserted timestamps.
Actual behaviour:
Describe What actually happened.
- Timestamps queried from InfluxDB 3 Core via JDBC Driver are inconsistent with those inserted.
Environment info:
- Please provide the command you used to build the project, including any
RUSTFLAGS.- It seems that this is not needed, I use the Docker Image
quay.io/influxdb/influxdb3-core:911ba92ab4133e75fe2a420e16ed9cb4cf32196fdirectly in my unit test.
- It seems that this is not needed, I use the Docker Image
- System info: Run
uname -srmor similar and copy the output here (we want to know your OS, architecture etc).Linux 5.15.167.4-microsoft-standard-WSL2 x86_64
- If you're running IOx in a containerised environment then details about that would be helpful.
$ docker --version
Docker version 27.5.1, build 9f9e405- Other relevant environment details: disk info, hardware setup etc.
- Considering that I execute unit tests under WSL, it’s hard to say there’s anything special about this. However, my WSL time zone environment is set to
Asia/Shanghai.
- Considering that I execute unit tests under WSL, it’s hard to say there’s anything special about this. However, my WSL time zone environment is set to
Config:
Copy any non-default config values here or attach the full config as a gist or file.
- It doesn't seem necessary.
Logs:
Include snippet of errors in logs or stack traces here.
Sometimes you can get useful information by running the program with the RUST_BACKTRACE=full environment variable.
Finally, the IOx server has a -vv for verbose logging.
- The unit test will dynamically create a Docker container, so I think I only need to provide the log for the unit test.
Click me to view the complete Log🥯🥨🍟🧂🥖🥚🍔🦪🍜🍘
$ ./mvnw -T 1C -Dtest=TimeDifferenceTest clean test
[INFO] Scanning for projects...
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 16
[INFO]
[INFO] ----------< io.github.linghengqian:influxdb-3-core-jdbc-test >----------
[INFO] Building influxdb-3-core-jdbc-test 1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ influxdb-3-core-jdbc-test ---
[INFO] Deleting /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/target
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ influxdb-3-core-jdbc-test ---
[INFO] skip non existing resourceDirectory /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ influxdb-3-core-jdbc-test ---
[INFO] No sources to compile
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ influxdb-3-core-jdbc-test ---
[INFO] skip non existing resourceDirectory /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/src/test/resources
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ influxdb-3-core-jdbc-test ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 7 source files with javac [debug target 21] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.2:test (default-test) @ influxdb-3-core-jdbc-test ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running io.github.linghengqian.TimeDifferenceTest
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.BaseAllocator <clinit>
信息: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true.
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.DefaultAllocationManagerOption getDefaultAllocationManagerFactory
信息: allocation manager type not specified, using netty as the default type
2月 08, 2025 11:51:56 上午 org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.CheckAllocator reportResult
信息: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 3.760 s <<< FAILURE! -- in io.github.linghengqian.TimeDifferenceTest
[ERROR] io.github.linghengqian.TimeDifferenceTest.test -- Time elapsed: 3.696 s <<< FAILURE!
java.lang.AssertionError:
Expected: is <1738986704151L>
but: was <1738929104151L>
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)
at io.github.linghengqian.TimeDifferenceTest.queryDataByJdbcDriver(TimeDifferenceTest.java:83)
at io.github.linghengqian.TimeDifferenceTest.test(TimeDifferenceTest.java:47)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TimeDifferenceTest.test:47->queryDataByJdbcDriver:83
Expected: is <1738986704151L>
but: was <1738929104151L>
[INFO]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.060 s (Wall Clock)
[INFO] Finished at: 2025-02-08T11:51:58+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-test) on project influxdb-3-core-jdbc-test: There are test failures.
[ERROR]
[ERROR] See /home/linghengqian/TwinklingLiftWorks/git/public/influxdb-3-core-jdbc-test/target/surefire-reports for the individual test results.
[ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExceptionReactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels