test(load): time series load tests for single node and 3-node cluster#4450
Conversation
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 40 |
🟢 Coverage ∅ diff coverage · -7.01% coverage variation
Metric Results Coverage variation ✅ -7.01% coverage variation Diff coverage ✅ ∅ diff coverage Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (7cae7b7) 128345 95039 74.05% Head commit (e3dc3d0) 156028 (+27683) 104605 (+9566) 67.04% (-7.01%) Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#4450) 0 0 ∅ (not applicable) Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Code Review
This pull request introduces integration load tests for ArcadeDB's time series functionality, including single-server and three-node Raft cluster replication tests, along with a TimeSeriesDatabaseWrapper helper class to drive the workload. The code review highlights several critical resource leak issues where connections, servers, database result sets, and input streams are not properly closed. It is highly recommended to address these by wrapping the affected resources in try-finally or try-with-resources blocks to ensure proper cleanup and prevent connection exhaustion during load testing.
| executor.submit(() -> { | ||
| final TimeSeriesDatabaseWrapper w = new TimeSeriesDatabaseWrapper(server, protocol); | ||
| final long base = 1_000_000_000L + threadIndex * 100_000_000L; | ||
| w.ingestSeries("sensor-" + threadIndex, "region-" + (threadIndex % 3), base, POINTS_PER_THREAD); | ||
| w.close(); | ||
| }); |
There was a problem hiding this comment.
If ingestSeries throws an exception, the TimeSeriesDatabaseWrapper instance w is never closed, leaking the RemoteDatabase connection and its associated HTTP/gRPC client resources. Wrap the execution in a try-finally block to guarantee resource cleanup.
executor.submit(() -> {
final TimeSeriesDatabaseWrapper w = new TimeSeriesDatabaseWrapper(server, protocol);
try {
final long base = 1_000_000_000L + threadIndex * 100_000_000L;
w.ingestSeries("sensor-" + threadIndex, "region-" + (threadIndex % 3), base, POINTS_PER_THREAD);
} finally {
w.close();
}
});There was a problem hiding this comment.
Done in 08e86f5 - the writer is wrapped in try-finally so w.close() (which closes the RemoteDatabase and its HttpClient/gRPC channel) runs even if ingestion throws.
| executor.submit(() -> { | ||
| final TimeSeriesDatabaseWrapper w = new TimeSeriesDatabaseWrapper(servers.getFirst(), protocol); | ||
| final long base = 1_000_000_000L + threadIndex * 100_000_000L; | ||
| w.ingestSeries("sensor-" + threadIndex, "region-" + (threadIndex % 3), base, POINTS_PER_THREAD); | ||
| w.close(); | ||
| }); |
There was a problem hiding this comment.
If ingestSeries throws an exception, the TimeSeriesDatabaseWrapper instance w is never closed, leaking the RemoteDatabase connection and its associated HTTP/gRPC client resources. Wrap the execution in a try-finally block to guarantee resource cleanup.
executor.submit(() -> {
final TimeSeriesDatabaseWrapper w = new TimeSeriesDatabaseWrapper(servers.getFirst(), protocol);
try {
final long base = 1_000_000_000L + threadIndex * 100_000_000L;
w.ingestSeries("sensor-" + threadIndex, "region-" + (threadIndex % 3), base, POINTS_PER_THREAD);
} finally {
w.close();
}
});There was a problem hiding this comment.
Done in 08e86f5 - same try-finally around the writer wrapper here.
| public void createDatabase() { | ||
| final RemoteServer httpServer = new RemoteServer(server.host(), server.httpPort(), "root", PASSWORD); | ||
| httpServer.setConnectionStrategy(RemoteHttpComponent.CONNECTION_STRATEGY.FIXED); | ||
|
|
||
| if (httpServer.exists(DATABASE)) { | ||
| logger.info("Dropping existing database {}", DATABASE); | ||
| httpServer.drop(DATABASE); | ||
| } | ||
| logger.info("Creating database {}", DATABASE); | ||
|
|
||
| final int maxAttempts = 30; | ||
| for (int attempt = 1; attempt <= maxAttempts; attempt++) { | ||
| try { | ||
| httpServer.create(DATABASE); | ||
| return; | ||
| } catch (final ServerIsNotTheLeaderException e) { | ||
| if (e.getLeaderAddress() != null || attempt == maxAttempts) | ||
| throw e; | ||
| logger.info("Leader address not yet known (attempt {}/{}), retrying in 2s...", attempt, maxAttempts); | ||
| } catch (final Exception e) { | ||
| if (attempt == maxAttempts) | ||
| throw e; | ||
| logger.info("Database creation attempt {}/{} failed ({}), retrying in 2s...", attempt, maxAttempts, e.getMessage()); | ||
| } | ||
| sleep(2_000); | ||
| } | ||
| } |
There was a problem hiding this comment.
The RemoteServer instance httpServer is created locally but never closed. Since RemoteServer extends RemoteHttpComponent which manages an internal HttpClient and thread pool, failing to close it leaks these resources. Wrap the logic in a try-finally block to ensure httpServer.close() is called.
public void createDatabase() {
final RemoteServer httpServer = new RemoteServer(server.host(), server.httpPort(), "root", PASSWORD);
try {
httpServer.setConnectionStrategy(RemoteHttpComponent.CONNECTION_STRATEGY.FIXED);
if (httpServer.exists(DATABASE)) {
logger.info("Dropping existing database {}", DATABASE);
httpServer.drop(DATABASE);
}
logger.info("Creating database {}", DATABASE);
final int maxAttempts = 30;
for (int attempt = 1; attempt <= maxAttempts; attempt++) {
try {
httpServer.create(DATABASE);
return;
} catch (final ServerIsNotTheLeaderException e) {
if (e.getLeaderAddress() != null || attempt == maxAttempts)
throw e;
logger.info("Leader address not yet known (attempt {}/{}), retrying in 2s...", attempt, maxAttempts);
} catch (final Exception e) {
if (attempt == maxAttempts)
throw e;
logger.info("Database creation attempt {}/{} failed ({}), retrying in 2s...", attempt, maxAttempts, e.getMessage());
}
sleep(2_000);
}
} finally {
httpServer.close();
}
}There was a problem hiding this comment.
Done in 08e86f5 - createDatabase now closes the RemoteServer in a finally block. Confirmed RemoteServer extends RemoteHttpComponent, which owns its own HttpClient and whose close() calls shutdownNow()+close(), so this prevents leaking that client and its NIO thread (a connection distinct from the long-lived query db).
| try { | ||
| db.command("sql", | ||
| "CREATE TIMESERIES TYPE " + TYPE_NAME | ||
| + " TIMESTAMP ts TAGS (sensor_id STRING, region STRING) FIELDS (temperature DOUBLE, humidity DOUBLE)"); | ||
| return; | ||
| } catch (final Exception e) { |
There was a problem hiding this comment.
The ResultSet returned by db.command is never closed, which can leak database cursors and connection resources. Use try-with-resources to ensure it is closed automatically.
| try { | |
| db.command("sql", | |
| "CREATE TIMESERIES TYPE " + TYPE_NAME | |
| + " TIMESTAMP ts TAGS (sensor_id STRING, region STRING) FIELDS (temperature DOUBLE, humidity DOUBLE)"); | |
| return; | |
| } catch (final Exception e) { | |
| try (final ResultSet rs = db.command("sql", | |
| "CREATE TIMESERIES TYPE " + TYPE_NAME | |
| + " TIMESTAMP ts TAGS (sensor_id STRING, region STRING) FIELDS (temperature DOUBLE, humidity DOUBLE)")) { | |
| return; | |
| } catch (final Exception e) { |
There was a problem hiding this comment.
Not applied. RemoteDatabase.query/command return an InternalResultSet that eagerly materializes the entire HTTP/gRPC response into an in-memory list; its close() only does content.clear() (see InternalResultSet.java). No server-side cursor or network connection is held by the result set, so there is nothing to leak here and try-with-resources would change only when a small in-memory list becomes garbage. This is also consistent with the sibling DatabaseWrapper, which does not close these result sets.
| try { | ||
| db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME).next(); | ||
| return; | ||
| } catch (final Exception e) { |
There was a problem hiding this comment.
The ResultSet returned by db.query is never closed, which can leak database cursors and connection resources. Use try-with-resources to ensure it is closed automatically.
| try { | |
| db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME).next(); | |
| return; | |
| } catch (final Exception e) { | |
| try (final ResultSet rs = db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME)) { | |
| rs.next(); | |
| return; | |
| } catch (final Exception e) { |
There was a problem hiding this comment.
Not applied. RemoteDatabase.query/command return an InternalResultSet that eagerly materializes the entire HTTP/gRPC response into an in-memory list; its close() only does content.clear() (see InternalResultSet.java). No server-side cursor or network connection is held by the result set, so there is nothing to leak here and try-with-resources would change only when a small in-memory list becomes garbage. This is also consistent with the sibling DatabaseWrapper, which does not close these result sets.
| try { | ||
| db.command("sql", | ||
| "INSERT INTO " + TYPE_NAME + " SET ts = ?, sensor_id = ?, region = ?, temperature = ?, humidity = ?", | ||
| ts, sensorId, region, temperature, humidity); | ||
| } catch (final Exception e) { |
There was a problem hiding this comment.
The ResultSet returned by db.command is never closed, which can leak database cursors and connection resources. Since this is called for every SQL insert, it will leak thousands of cursors during load tests. Use try-with-resources to ensure it is closed automatically.
| try { | |
| db.command("sql", | |
| "INSERT INTO " + TYPE_NAME + " SET ts = ?, sensor_id = ?, region = ?, temperature = ?, humidity = ?", | |
| ts, sensorId, region, temperature, humidity); | |
| } catch (final Exception e) { | |
| try (final ResultSet rs = db.command("sql", | |
| "INSERT INTO " + TYPE_NAME + " SET ts = ?, sensor_id = ?, region = ?, temperature = ?, humidity = ", | |
| ts, sensorId, region, temperature, humidity)) { | |
| // Execute command and auto-close ResultSet | |
| } catch (final Exception e) { |
There was a problem hiding this comment.
Not applied. RemoteDatabase.query/command return an InternalResultSet that eagerly materializes the entire HTTP/gRPC response into an in-memory list; its close() only does content.clear() (see InternalResultSet.java). No server-side cursor or network connection is held by the result set, so there is nothing to leak here and try-with-resources would change only when a small in-memory list becomes garbage. This is also consistent with the sibling DatabaseWrapper, which does not close these result sets. Separately, the suggested snippet is incorrect: it drops a bind parameter - temperature = ?, humidity = with only four arguments - which would break the INSERT.
| public long countPoints() { | ||
| final ResultSet rs = db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME); | ||
| return ((Number) rs.next().getProperty("cnt")).longValue(); | ||
| } |
There was a problem hiding this comment.
The ResultSet returned by db.query is never closed, which can leak database cursors and connection resources. Use try-with-resources to ensure it is closed automatically.
| public long countPoints() { | |
| final ResultSet rs = db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME); | |
| return ((Number) rs.next().getProperty("cnt")).longValue(); | |
| } | |
| public long countPoints() { | |
| try (final ResultSet rs = db.query("sql", "SELECT count(*) AS cnt FROM " + TYPE_NAME)) { | |
| return ((Number) rs.next().getProperty("cnt")).longValue(); | |
| } | |
| } |
There was a problem hiding this comment.
Not applied. RemoteDatabase.query/command return an InternalResultSet that eagerly materializes the entire HTTP/gRPC response into an in-memory list; its close() only does content.clear() (see InternalResultSet.java). No server-side cursor or network connection is held by the result set, so there is nothing to leak here and try-with-resources would change only when a small in-memory list becomes garbage. This is also consistent with the sibling DatabaseWrapper, which does not close these result sets.
| while (true) { | ||
| final ResultSet rs = db.query("sql", sql, sensorId); | ||
| if (rs.hasNext()) { | ||
| final Result r = rs.next(); | ||
| return new AggResult(toDouble(r.getProperty("avgT")), toDouble(r.getProperty("minT")), toDouble(r.getProperty("maxT")), | ||
| ((Number) r.getProperty("cnt")).longValue()); | ||
| } | ||
| if (System.currentTimeMillis() >= deadline) |
There was a problem hiding this comment.
The ResultSet returned by db.query is never closed, which can leak database cursors and connection resources. Since this is called in a polling loop, it will leak a ResultSet on every iteration until the aggregation succeeds. Use try-with-resources to ensure it is closed automatically.
| while (true) { | |
| final ResultSet rs = db.query("sql", sql, sensorId); | |
| if (rs.hasNext()) { | |
| final Result r = rs.next(); | |
| return new AggResult(toDouble(r.getProperty("avgT")), toDouble(r.getProperty("minT")), toDouble(r.getProperty("maxT")), | |
| ((Number) r.getProperty("cnt")).longValue()); | |
| } | |
| if (System.currentTimeMillis() >= deadline) | |
| while (true) { | |
| try (final ResultSet rs = db.query("sql", sql, sensorId)) { | |
| if (rs.hasNext()) { | |
| final Result r = rs.next(); | |
| return new AggResult(toDouble(r.getProperty("avgT")), toDouble(r.getProperty("minT")), toDouble(r.getProperty("maxT")), | |
| ((Number) r.getProperty("cnt")).longValue()); | |
| } | |
| } |
There was a problem hiding this comment.
Not applied. RemoteDatabase.query/command return an InternalResultSet that eagerly materializes the entire HTTP/gRPC response into an in-memory list; its close() only does content.clear() (see InternalResultSet.java). No server-side cursor or network connection is held by the result set, so there is nothing to leak here and try-with-resources would change only when a small in-memory list becomes garbage. This is also consistent with the sibling DatabaseWrapper, which does not close these result sets. Each iteration's InternalResultSet is eligible for GC immediately; close() remains a no-op clear().
| try { | ||
| if (conn.getResponseCode() != 200) | ||
| throw new IOException("Latest query failed: HTTP " + conn.getResponseCode()); | ||
| final JSONObject json = new JSONObject(new String(conn.getInputStream().readAllBytes(), StandardCharsets.UTF_8)); | ||
| final JSONArray latest = json.getJSONArray("latest"); | ||
| return latest.getLong(0); |
There was a problem hiding this comment.
The InputStream returned by conn.getInputStream() is not closed, which can leak connection resources or prevent connection reuse. Use try-with-resources to ensure it is closed automatically.
try {
if (conn.getResponseCode() != 200)
throw new IOException("Latest query failed: HTTP " + conn.getResponseCode());
final JSONObject json;
try (java.io.InputStream is = conn.getInputStream()) {
json = new JSONObject(new String(is.readAllBytes(), StandardCharsets.UTF_8));
}
final JSONArray latest = json.getJSONArray("latest");
return latest.getLong(0);There was a problem hiding this comment.
Done in 08e86f5 - the input stream is now read inside try-with-resources.
| final int status = conn.getResponseCode(); | ||
| if (status != 204) { | ||
| String err = ""; | ||
| if (conn.getErrorStream() != null) | ||
| err = new String(conn.getErrorStream().readAllBytes(), StandardCharsets.UTF_8); | ||
| throw new IOException("Line protocol write failed: HTTP " + status + " " + err); | ||
| } |
There was a problem hiding this comment.
The InputStream returned by conn.getErrorStream() is not closed, which can leak connection resources or prevent connection reuse. Use try-with-resources to ensure it is closed automatically.
| final int status = conn.getResponseCode(); | |
| if (status != 204) { | |
| String err = ""; | |
| if (conn.getErrorStream() != null) | |
| err = new String(conn.getErrorStream().readAllBytes(), StandardCharsets.UTF_8); | |
| throw new IOException("Line protocol write failed: HTTP " + status + " " + err); | |
| } | |
| final int status = conn.getResponseCode(); | |
| if (status != 204) { | |
| String err = ""; | |
| try (java.io.InputStream es = conn.getErrorStream()) { | |
| if (es != null) { | |
| err = new String(es.readAllBytes(), StandardCharsets.UTF_8); | |
| } | |
| } | |
| throw new IOException("Line protocol write failed: HTTP " + status + " " + err); | |
| } |
There was a problem hiding this comment.
Done in 08e86f5 - the error stream is now read inside try-with-resources.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4450 +/- ##
============================================
+ Coverage 64.88% 65.62% +0.73%
============================================
Files 1649 1612 -37
Lines 128345 124322 -4023
Branches 27525 26910 -615
============================================
- Hits 83282 81583 -1699
+ Misses 33335 31228 -2107
+ Partials 11728 11511 -217 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
…ster Add Testcontainers-based load tests dedicated to the time series engine, covering a single standalone server and a 3-node Raft HA cluster. Ingestion is driven over three protocols (InfluxDB line protocol HTTP endpoint, and SQL INSERT over both HTTP and gRPC RemoteDatabase) via a new TimeSeriesDatabaseWrapper that reuses the existing ContainersTestTemplate infrastructure. The single-node test asserts the exact total point count, time-bucket aggregation extremes, and the tag-filtered latest value. The 3-node test verifies replication consistency by converging the unfiltered count across all nodes, then asserts per-node aggregation extremes and latest value. Verification accounts for time series behaviors observed against the server: tag-filtered queries (count and timeBucket aggregation) reflect only the asynchronously-sealed subset of points, so exact totals are asserted via the unfiltered count while aggregation is asserted on stable min/max extremes over a sealed bulk sensor with polling; schema presence is polled to absorb replication lag; and the RID-less SQL INSERT response is tolerated.
Address code review resource-cleanup findings: - Wrap the per-thread writer wrapper usage in try-finally so the RemoteDatabase (and its HttpClient / gRPC channel) is closed even if ingestion throws. - Close the RemoteServer in createDatabase via try-finally; RemoteServer owns its own HttpClient (RemoteHttpComponent) and would otherwise leak the client and its NIO thread, separate from the long-lived query connection. - Close the HttpURLConnection input and error streams via try-with-resources in latestTimestamp and postLineProtocol. ResultSet try-with-resources was intentionally not added: RemoteDatabase materializes responses eagerly into an in-memory InternalResultSet whose close() only clears the list, so no server cursors or connections are held.
3c3b061 to
08e86f5
Compare
|
Code Review Good addition overall - the parameterized approach across three protocols, the well-documented async assertion strategy, and the reuse of ContainersTestTemplate infrastructure are all solid. A few issues to address before merging. BUGS / CORRECTNESS Resource leaks when assertions fail In SingleServerTimeSeriesLoadTestIT, admin.close() is called at the end of the test body but not in a finally block. If any assertion fails (e.g. assertAggregateExtremes), the remote connection is leaked. Same problem in ThreeNodesTimeSeriesLoadTestIT for ingest, r0, r1, r2. Implement AutoCloseable on TimeSeriesDatabaseWrapper and use try-with-resources, or wrap in a try/finally. MISSING @tag("slow") ANNOTATIONS From CLAUDE.md: "Use @tag("slow") for functional regression tests that take noticeably long (large batches, multi-second elapsed time, big payloads)." Both test classes ingest 50,000+ points, poll for minutes, and the 3-node test requires ~10 GB of Docker resources. They need @tag("slow") plus import org.junit.jupiter.api.Tag on both test classes. SILENT EXCEPTION SWALLOWING IN assertThatPointCountIs The existing DatabaseWrapper.assertThatUserCountIs tracks lastException and propagates it with an AssertionError if the deadline expires without a successful count. The new implementation discards all exceptions silently. If the connection is broken throughout the 30s window, the test will report "-1 != expectedTotal" with no indication of why queries kept failing. Mirror the existing pattern. STYLE / CONVENTION ISSUES Inconsistent API style in ThreeNodesTimeSeriesLoadTestIT The lambda body uses servers.getFirst() (Java 21 SequencedCollection API), but directly above in the same method, the outer code uses servers.get(0). The existing SingleServerLoadTestIT uses getFirst(), so standardise on that. Float equality comparison in assertAggregateExtremes The check URL parameter not encoded in latestTimestamp The tag value (e.g. "sensor_id:verify") contains a colon and is appended raw to the URL query string. Use URLEncoder.encode(tag, StandardCharsets.UTF_8) to be safe against future tag values that contain characters requiring percent-encoding. SQL protocol batching asymmetry Line protocol batches 500 points per HTTP call; SQL protocol sends one INSERT per round-trip (10,000 per thread). The three protocol cases therefore measure fundamentally different throughput characteristics and the SQL cases will be considerably slower. Consider batching SQL inserts via sqlscript, or add a Javadoc note so readers know the timings are not comparable across protocols. MINOR
SUMMARY
|
…-insert sample loss Documents a reproducible data-loss: under many concurrent single-row INSERTs into a time series type (each its own transaction, as over the wire), serialized per-shard appends can read a stale per-page sample count and write the same row slot, so one sample is silently overwritten (count comes up short, no error). Reproduces reliably at high concurrency with a single shard; the batched line-protocol path is unaffected. Kept @disabled because the fix is a delicate change to the time series append path (making the slot read-modify-write durable under the per-shard append lock without breaking HA WAL shipping). Enable once that is fixed.
Code ReviewGood addition - the time series load tests fill a real gap and the three-protocol parameterisation ( Required fixes1. Missing
// SingleServerTimeSeriesLoadTestIT.java
@Tag("slow")
class SingleServerTimeSeriesLoadTestIT extends ContainersTestTemplate { …
// ThreeNodesTimeSeriesLoadTestIT.java
@Tag("slow")
class ThreeNodesTimeSeriesLoadTestIT extends ContainersTestTemplate { …2. URL encoding missing in
url.append("&tag=").append(tag);The values passed from the tests include a colon ( 3. Interrupt handling in the monitoring loop does not break Both } catch (final InterruptedException e) {
Thread.currentThread().interrupt();
// no break!
}Re-setting the interrupt flag without breaking means the loop re-enters, calls 4. Interrupt handling in
Minor / suggestions5.
6. The comment says "Skip compareAllDatabases()" but 7. final long deadline = System.currentTimeMillis() + 30_000;
while (true) {
final ResultSet rs = db.query("sql", sql, sensorId);
if (rs.hasNext()) { … return; }
if (System.currentTimeMillis() >= deadline)
throw new IllegalStateException(…);
sleep(1_000); // no second deadline check before sleeping
}If the deadline expires between the 8. Insertion failures in the SQL path are only logged and counted; they don't propagate. This means the final 9. SQL path does not batch
Nit
|
… encoding - TimeSeriesDatabaseWrapper implements AutoCloseable; both ITs use try-with-resources for the top-level wrappers so connections close even when an assertion fails. - assertThatPointCountIs now tracks and surfaces the last exception (mirrors DatabaseWrapper.assertThatUserCountIs) instead of silently reporting -1. - assertAggregateExtremes uses isCloseTo(within(1e-9)) and a single polling level; aggregate() is now a single query (returns null if no rows) so the retry no longer compounds. - latestTimestamp URL-encodes the tag query parameter. - Standardised servers.getFirst() over get(0) in the 3-node test. - Documented that the protocols are not throughput-comparable (line protocol batches 500/call; SQL sends one INSERT per round-trip, kept to exercise the single-row path).
|
Addressed in e3dc3d0. Per item:
Note: the SQL-protocol cases currently fail at |
Code ReviewThis PR adds solid Testcontainers-based load tests for the time series engine. The structure is clean and follows the existing Correctness / Bugs1. Interrupt handling in retry loops (
// createDatabase() retry sleep
try {
Thread.sleep(2_000);
} catch (final InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}Same fix applies to 2. Silent error swallowing in Both methods catch any 3. final JSONArray latest = json.getJSONArray("latest");
return latest.getLong(0);If the server returns Design / Maintainability4. The override body is just 5. Executor wait loop has no upper-bound timeout Both load tests use 6. The method implements its own poll loop with Awaitility.await()
.atMost(60, TimeUnit.SECONDS)
.pollInterval(2, TimeUnit.SECONDS)
.untilAsserted(() -> {
final AggResult agg = aggregate(sensorId);
assertThat(agg).isNotNull();
assertThat(agg.min()).isCloseTo(expectedMin, within(tolerance));
assertThat(agg.max()).isCloseTo(expectedMax, within(tolerance));
});would unify the polling style and provide better failure messages on timeout. 7. Magic number
private static final int VERIFICATION_POINTS = 4;
...
final long expectedTotal = bulkPoints + VERIFICATION_POINTS;or sharing the constant from The
|
Summary
Adds Testcontainers-based load tests dedicated to the time series engine, covering a single standalone server and a 3-node Raft HA cluster. Reuses the existing
ContainersTestTemplateinfrastructure unchanged.Ingestion is exercised over three protocols via a new
TimeSeriesDatabaseWrapper:POST /api/v1/ts/{db}/write)INSERTover HTTPRemoteDatabaseINSERTover gRPCRemoteDatabaseEach test is
@ParameterizedTestover those three protocols.What is verified
SingleServerTimeSeriesLoadTestIT): concurrent bulk ingestion, exact total point count, time-bucket aggregation extremes over a sealed bulk sensor, and the tag-filtered latest value.ThreeNodesTimeSeriesLoadTestIT): replication consistency by converging the unfiltered point count across all nodes (Awaitility), then per-node aggregation extremes and latest value.compareAllDatabases()is skipped (non-persistent containers), matchingThreeNodesLoadTestIT.Notes on time series query semantics (informed the assertions)
Verified against the server image:
WHERE sensor_id = ?, both plaincount(*)andts.timeBucket(...) GROUP BY) reflect only the asynchronously sealed subset of points; the unfilteredcount(*)and the/latestendpoint see all data. So exact totals are asserted via the unfiltered count, while aggregation is asserted on stablemin/maxextremes over a sealed bulk sensor (with polling to absorb seal/replication lag).bucketis a reserved SQL word, so the time-bucket alias istbucket.INSERTinto a time series type returns a record without an@rid, which the remote client can reject after the server commits; this is tolerated in the ingest path (no data loss).Test plan
Run against
arcadedata/arcadedb:latest:All 6 parameterized cases (3 single-node + 3 cluster) pass locally. The 3-node test needs ~10 GB free for Docker.