Conversation
- Updated all instances of direct model queries (e.g., Label, Source, Bulletin, Actor, Incident, Location) to utilize db.session.query for improved session management.
This commit introduces a new file `pagination.py` that implements a flexible pagination system for SQLAlchemy queries. It supports both cursor-based and offset-based pagination, allowing for efficient data retrieval. The `PaginationResult` class is defined to encapsulate the results, including items, total count, next cursor, and items per page. The `paginate_query` function handles the logic for pagination based on the provided parameters.
This update simplifies the pagination implementation by removing unnecessary code related to cursor-based pagination and consolidating the logic for both cursor and offset pagination.
…nation prototype): - Simplified pagination in the `api_bulletins` function by removing the page parameter and utilizing cursor-based pagination exclusively. - Enhanced the response structure to directly return pagination results using the new `to_dict` method from `PaginationResult`. - Updated the frontend to handle pagination more effectively, including a new computed property for item length and improved loading behavior. - Improved error handling and loading state management during data fetching. This update streamlines the API and enhances the user experience with more efficient data retrieval.
Unified query conditions into a single list for clarity. Streamlined parameter handling for consistent filters. Simplified return structure, ensuring compatibility. Enhances search functionality and supports future updates.
…y handling - Enhanced the `api_bulletins` function to streamline query execution and pagination logic, utilizing cursor-based pagination exclusively. - Updated the response structure to include total count, pagination details, and items in a more efficient format. - Refactored the `SearchUtils` class to simplify nested query handling, allowing for clearer and more maintainable query construction. - Improved overall performance and clarity in the bulletin search functionality, setting the stage for future enhancements.
…rameters and improved clarity in pagination metadata handling.
…earch - Enhanced nested queries handling: combine conditions from multiple queries, allowing for clearer and more maintainable query construction. - Streamlined the handling of logical operations (`and`/`or`) when processing nested queries.
…alization, jsonify
This reverts commit 48b400f.
…rformance (Backward ID index scan)
…model for search-critical foreign keys (assigned_to_id, first_peer_reviewer_id, second_peer_reviewer_id) - Add indexes to Event model for search-critical foreign keys (location_id, eventtype_id) - Create migration script with production-safe CONCURRENTLY indexes - Include junction table indexes for many-to-many search relationships - Based on BulletinSearchBox.js search pattern analysis
…or model for search-critical foreign keys (assigned_to_id, first_peer_reviewer_id, second_peer_reviewer_id, origin_place_id) - Create migration script with production-safe CONCURRENTLY indexes - Include junction table indexes for actor many-to-many search relationships - Based on ActorSearchBox.js search pattern analysis
…Incident model for search-critical foreign keys (assigned_to_id, first_peer_reviewer_id, second_peer_reviewer_id) - Create migration script with production-safe CONCURRENTLY indexes - Include junction table indexes for incident many-to-many search relationships - Based on IncidentSearchBox.js search pattern analysis
## Jira Issue 1. [BYNT-1381](https://syriajustice.atlassian.net/browse/BYNT-1381) ## Description Implements comprehensive migration for Actors and Incidents, covering SearchUtils refactoring from (queries, ops) tuple to unified query objects, cursor-based pagination with dual query strategy, complete database index audit (all 50 indexes introduced in branch `search-v4` migrations are also defined on ORM-level), and minimal serialization patterns ## Checklist - [x] Tests added/updated - [x] Documentation updated (if needed) - [ ] New strings prepared for translations ## API Changes (if applicable) - [ ] Permissions checked - [ ] Endpoint tests added ## Additional Notes [Any other relevant information] [BYNT-1381]: https://syriajustice.atlassian.net/browse/BYNT-1381?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: tarekio <7659394+tarekio@users.noreply.github.com>
…t to valid config value
Add permission checks to search endpoints before returning item data. Users without access now get restricted_json() instead of full details. Fixes issue where users could see titles, assigned users, and status of restricted bulletins, actors, and incidents in search results.
Replace NOT ILIKE ALL() with individual indexed searches to resolve 21-42 second timeouts. Uses subquery approach enabling GIN trigram index usage for better performance.
Use direct NOT ILIKE conditions instead of NOT IN subquery to avoid correlated subquery performance issues. Generates WHERE NOT (search ILIKE '%term1%') AND NOT (search ILIKE '%term2%') instead of WHERE id NOT IN (SELECT id FROM ... OR ...).
…cessary direct NOT conditions and optimized the exclude logic for better performance.
Replace slow NOT IN subquery pattern with direct NOT ILIKE ALL array syntax to leverage GIN trigram index efficiently. Performance improvement: 34 seconds -> ~400ms for complex exclusions
Use fast array containment (@>) for exact tag matches (0.06ms vs 1.8s) while preserving ILIKE wildcards for partial searches when needed. Performance improvement: 30,000x faster for exact tag searches
- Exact Match checked: Use fast array containment (@>) for 9533x performance improvement - Exact Match unchecked: Use ILIKE for partial/wildcard matching behavior - Preserve Any checkbox functionality for OR vs AND logic Performance: 572ms -> 0.06ms for exact tag searches
tarekio
approved these changes
Aug 3, 2025
5 tasks
tarekio
pushed a commit
that referenced
this pull request
Aug 25, 2025
Due to recent changes in #97 an update was made to the latest vuetify version that included [some changes](https://vuetifyjs.com/en/getting-started/release-notes/?version=v3.7.4), that caused the dialog to ignore custom sizes when the prop fullscreen was used. This update changes the width through the content styles
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement Cursor-Based Pagination for Bulletins with Performance Enhancements
Overview
Replaces traditional offset-based pagination with cursor-based infinite scroll for bulletins, significantly improving performance for large datasets.
Key Changes
id < cursorfor efficient pagination, eliminating expensive COUNT queriesTechnical Improvements
Impact
Ready for review and testing with production-scale bulletin datasets.