Conversation
Still remote database missing and more checks on calling lock after a page was loaded/modified in a tx
LockManager requester now is configurable, so in case of remote call, it's the session id instead of the Thread object. This allow to unlock resources from a different thread.
There was a problem hiding this comment.
Summary of Changes
Hello @lvca, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly enhances transaction management by introducing explicit locking capabilities for database buckets and types. This new mechanism allows transactions to acquire pessimistic locks, thereby avoiding the default retry behavior of implicit locking. Additionally, a new LOCK SQL statement has been added to provide a direct way for users to manage these explicit locks. The changes also include foundational updates to the transaction manager to support more flexible locking requesters and general code refactoring for improved maintainability.
Highlights
- Explicit Transaction Locking: Introduced
TransactionExplicitLockinterface andLocalTransactionExplicitLockimplementation to allow explicit, pessimistic locking of database buckets and types within transactions. This aims to prevent transaction retries caused by implicit locking. - New SQL
LOCKStatement: Added a newLOCKSQL command to the grammar, enabling users to explicitly lock types and buckets directly via SQL. - Transaction Manager Enhancements: Modified the
TransactionManagerto support generic requester objects (instead of justThreadinstances) for file locking, improving flexibility for explicit locking scenarios (e.g., HTTP sessions). - Transaction Context Updates: Enhanced
TransactionContextto manage explicit locks, ensuring proper acquisition, release, and cleanup during transaction lifecycle events (commit, rollback, kill). - Remote Database Support: Extended remote database operations to support explicit locking, allowing remote clients to utilize the new locking mechanisms.
- Code Refactoring and Cleanup: Performed various code cleanups, including import consolidation, minor formatting adjustments, and removal of unused code, improving overall code maintainability.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request significantly enhances transaction management by introducing explicit locking mechanisms, allowing users to acquire pessimistic locks on database buckets and types. The new LOCK SQL statement provides a clear interface for this feature. The changes also include necessary refactoring in the TransactionManager to support generic requesters for locks, which is a robust improvement for multi-threaded environments. Comprehensive test cases have been added to validate the new functionality and its error handling. Overall, the changes improve the robustness and control over transactions.
| protected void explicitLock(final Set<Integer> filesToLock) { | ||
| if (explicitLockedFiles != null) | ||
| throw new TransactionException("Explicit lock already acquired"); | ||
|
|
||
| if (!indexChanges.isEmpty() || !immutablePages.isEmpty() || // | ||
| (modifiedPages != null && !modifiedPages.isEmpty()) || // | ||
| (newPages != null && !newPages.isEmpty()) // | ||
| ) | ||
| throw new TransactionException("Explicit lock must be acquired before any modification"); | ||
|
|
||
| explicitLockedFiles = lockFilesInOrder(filesToLock); | ||
| } |
There was a problem hiding this comment.
The explicitLock method includes crucial checks to ensure the integrity of the explicit locking mechanism. Specifically, it prevents acquiring an explicit lock if one is already active or if any modifications (index changes, modified pages, new pages) have already occurred within the transaction. This strict enforcement ensures that pessimistic locks are established on a clean state, preventing potential race conditions or inconsistent states. This is a strong and correct design choice for pessimistic locking.
| if (explicitLockedFiles != null) { | ||
| // CHECK THE LOCKED FILES ARE ALL LOCKED ALREADY | ||
| if (!explicitLockedFiles.containsAll(modifiedFiles)) { | ||
| final HashSet<Integer> left = new HashSet<>(modifiedFiles); | ||
| left.removeAll(explicitLockedFiles); | ||
| throw new TransactionException( | ||
| "Cannot commit transaction because not all the modified resources were locked: " + left); | ||
| } |
There was a problem hiding this comment.
The check if (!explicitLockedFiles.containsAll(modifiedFiles)) within commit1stPhase is vital for correctness when explicit locks are in use. It ensures that all files that were actually modified during the transaction were also explicitly locked. If a transaction attempts to modify a resource that was not part of its explicit lock set, it correctly throws a TransactionException. This prevents accidental modifications to unlocked resources, which is a core tenet of pessimistic locking.
| case "name": | ||
| final String newTypeName = identifierValue.getStringValue(); | ||
| final int bucketSize = type.getInvolvedBuckets().size(); | ||
| final int bucketSize = type.getBuckets(false).size(); |
There was a problem hiding this comment.
The change from type.getInvolvedBuckets().size() to type.getBuckets(false).size() for calculating bucketSize in the ALTER TYPE NAME statement has a functional implication. getInvolvedBuckets() would include buckets from polymorphic subtypes, while getBuckets(false) only includes buckets directly associated with the type itself. If the subsequent logic for renaming a type relies on the total number of buckets within the type's hierarchy (including inherited ones), this change could lead to incorrect behavior or allow operations that should be restricted. For example, a type might have no direct buckets but its subtypes do, and this change would make bucketSize zero, potentially bypassing checks. Please ensure this change aligns with the intended behavior for ALTER TYPE NAME across the entire type hierarchy.
| private final AtomicLong transactionIds = new AtomicLong(); | ||
| private final AtomicLong logFileCounter = new AtomicLong(); | ||
| private final LockManager<Integer, Thread> fileIdsLockManager = new LockManager<>(); | ||
| private final LockManager<Integer, Object> fileIdsLockManager = new LockManager<>(); |
There was a problem hiding this comment.
Changing the generic type of fileIdsLockManager from Thread to Object is a necessary and correct adjustment. This allows the LockManager to handle a more generalized requester object, which is crucial for supporting explicit locking in scenarios where the lock initiator might not always be the current thread (e.g., an HTTP session ID). This enhances the flexibility and correctness of the locking mechanism.
| * the new acquired locks are released. | ||
| */ | ||
| public List<Integer> tryLockFiles(final Collection<Integer> fileIds, final long timeout) { | ||
| public List<Integer> tryLockFiles(final Collection<Integer> fileIds, final long timeout, final Object requester) { |
There was a problem hiding this comment.
The addition of the requester parameter to tryLockFiles and its subsequent propagation to tryLockFile and unlockFilesInOrder is a significant improvement. This ensures that locks are correctly associated with the specific entity (e.g., a user session) that initiated the transaction, rather than just the executing thread. This is vital for accurate lock management in concurrent and distributed environments, preventing issues like incorrect unlocking or deadlocks when multiple sessions might share threads.
| if (requester == null) | ||
| throw new IllegalArgumentException("Requester is null"); |
There was a problem hiding this comment.
Adding a null check for the requester parameter in the tryLock method is a good defensive programming practice. Since the requester type has been generalized to Object, it's important to explicitly validate that it's not null before proceeding, preventing potential NullPointerExceptions in downstream operations that rely on the requester object.
There was a problem hiding this comment.
Pull Request Overview
Introduces explicit transaction locks, extends SQL grammar for LOCK statements, and updates related handlers and tests.
- Adds
TransactionExplicitLockinterface and implementations for explicit pessimistic locking - Updates SQL parser and grammar to recognize and execute
LOCKstatements - Enhances server and network layers to propagate a generic requester for file locks
- Adds integration and unit tests for explicit lock behavior
Reviewed Changes
Copilot reviewed 24 out of 26 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| server/src/test/java/com/arcadedb/remote/RemoteDatabaseJavaApiIT.java | Adds an end-to-end test for remote explicit locking |
| engine/src/test/java/com/arcadedb/query/sql/method/collection/SQLMethodTransformTest.java | Stubs out acquireLock in a test harness (returns null) |
| engine/src/main/java/com/arcadedb/database/TransactionContext.java | Implements new explicit lock state, requester tracking, and cleanup |
| engine/src/main/java/com/arcadedb/database/LocalTransactionExplicitLock.java | Implements bucket/type-to-file mapping and delegation to TransactionContext |
| engine/src/main/java/com/arcadedb/query/sql/parser/LockStatement.java | Adds AST node and execution logic for the LOCK SQL statement |
Comments suppressed due to low confidence (2)
engine/src/main/java/com/arcadedb/database/TransactionContext.java:138
- [nitpick] This public
lock()method lacks Javadoc. Consider adding a comment explaining its role in initiating an explicit lock within the transaction lifecycle.
public LocalTransactionExplicitLock lock() {
engine/src/main/java/com/arcadedb/database/LocalTransactionExplicitLock.java:19
- The generated-file header references
OAlterClassStatement.java, which is unrelated to this class. Please update or remove the incorrect header to prevent confusion.
/* Generated By:JJTree: Do not edit this line. OAlterClassStatement.java Version 4.3 */
|
|
||
| @Override | ||
| public LocalTransactionExplicitLock acquireLock() { | ||
| return null; |
There was a problem hiding this comment.
Returning null from acquireLock will lead to NullPointerExceptions when the test invokes locking logic. Consider returning a valid LocalTransactionExplicitLock or a mock implementation to avoid NPEs and ensure the test runs as intended.
| return null; | |
| return new LocalTransactionExplicitLock() { | |
| @Override | |
| public void lock() { | |
| // Mock lock behavior | |
| } | |
| @Override | |
| public void unlock() { | |
| // Mock unlock behavior | |
| } | |
| }; |
|
|
||
| } catch (Exception e) { | ||
| caughtExceptions.incrementAndGet(); | ||
| e.printStackTrace(); |
There was a problem hiding this comment.
[nitpick] Avoid using e.printStackTrace() in tests. Instead, let the exception propagate or use a logger/assertion to capture failures so the test framework reports errors cleanly.
| assertThat(rid[0].asVertex().getInteger("id")).isEqualTo(CONCURRENT_THREADS * TOT); | ||
| assertThat(committed.get()).isEqualTo(CONCURRENT_THREADS * TOT); | ||
| assertThat(caughtExceptions.get()).isEqualTo(0); | ||
| assertThat(committed.get() + caughtExceptions.get()).isEqualTo(TOT * CONCURRENT_THREADS); |
There was a problem hiding this comment.
[nitpick] The test opens a RemoteDatabase instance at the start but never closes it. Consider adding database.close() at the end of the test to properly clean up resources.
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferences |
* First version of explicit tx lock. Still remote database missing and more checks on calling lock after a page was loaded/modified in a tx * test: fixed test case * test: refactoring test case * test: added more test cases to implicit tx locking * Extended SQL BEGIN adding LOCK TYPE|BUCKET * Supported SQL `lock` command to be used also by RemoteDatabase API LockManager requester now is configurable, so in case of remote call, it's the session id instead of the Thread object. This allow to unlock resources from a different thread. * test: added missing db.close() (cherry picked from commit 3e03494)
This pull request introduces explicit locking mechanisms for transactions, adds a new SQL grammar feature for
LOCKstatements, and includes minor refactoring and cleanup across the codebase. The changes enhance transaction management by allowing explicit locks on database buckets and types, while also improving code maintainability and readability.Transaction Management Enhancements:
TransactionExplicitLockinterface and its implementationLocalTransactionExplicitLockto allow explicit locking of buckets and types in transactions, avoiding the retry mechanism of implicit locking. (engine/src/main/java/com/arcadedb/database/TransactionExplicitLock.java- [1]engine/src/main/java/com/arcadedb/database/LocalTransactionExplicitLock.java- [2]TransactionContextto support explicit locking, including methods for acquiring and managing these locks, and ensuring proper cleanup during transaction reset or kill operations. (engine/src/main/java/com/arcadedb/database/TransactionContext.java- [1] [2] [3] [4] [5] [6]TransactionManagerto support locking files with a generic requester object instead of just threads, enabling compatibility with explicit locking. (engine/src/main/java/com/arcadedb/engine/TransactionManager.java- [1] [2]SQL Grammar Updates:
LOCKSQL statement in the grammar definition, including syntax for locking types and buckets. (engine/src/main/grammar/SQLGrammar.jjt- [1] [2] [3]Code Refactoring and Cleanup:
RemoteDatabaseJavaApiTestto use wildcard imports for cleaner code. (e2e/src/test/java/com/arcadedb/e2e/RemoteDatabaseJavaApiTest.java- e2e/src/test/java/com/arcadedb/e2e/RemoteDatabaseJavaApiTest.javaL32-R35)BasicDatabase. (engine/src/main/java/com/arcadedb/database/BasicDatabase.java- engine/src/main/java/com/arcadedb/database/BasicDatabase.javaL22)LocalDatabaseandTransactionContextfor better readability. (engine/src/main/java/com/arcadedb/database/LocalDatabase.java- [1] [2];engine/src/main/java/com/arcadedb/database/TransactionContext.java- [3]These changes collectively improve the robustness of transaction handling, extend the SQL feature set, and enhance the maintainability of the codebase.