Improve test infrastructure and security updates#162
Merged
Conversation
- Add modern pyproject.toml with PEP 621 project metadata - Migrate dependencies from requirements/*.txt to pyproject.toml - Add optional dependency groups: dev (testing/tools) and ai (ML/OCR) - Preserve existing Black formatter configuration - Generate uv.lock for reproducible builds - Require Python 3.12+ for full compatibility including AI features
Dependencies now managed in pyproject.toml with UV
Add environment-based configuration selection and dedicated test config to fix test failures. - Add config.test.json with test-optimized settings - Enable environment variable config selection via BAYANAT_CONFIG_FILE - Fix test client redirect behavior for proper HTTP status code testing - Resolve email domain validation in test factories
Replace exact version pins (==) with compatible ranges (>=) to enable automatic security updates. Updated all dependencies to use minimum version requirements while allowing patch and minor version updates. Key improvements: - Replace pypdf==5.5.0 with pypdf>=6.0.0 (fixes GHSA-7hfw-26vp-jp8m security vulnerability) - Enable automatic security patches for cryptography, certifi, and other critical packages - Use Flask>=3.1.0,<4.0 for major version compatibility - Allow UV to manage exact versions via lock file for reproducible builds Security updates applied: - pypdf: 5.5.0 → 6.0.0 (security fix) - cryptography: 45.0.3 → 45.0.6 - certifi: 2025.4.26 → 2025.8.3
Replace pip-based dependency management with UV in production and test containers. - Install UV in base images via official installer - Use 'uv sync --frozen' for exact dependency resolution - Copy UV-built virtual environments to final containers - Maintain security updates from UV lock file (pypdf 6.0.0, etc.) Benefits: - Consistent dependency management between local and Docker - Faster container builds (UV vs pip) - Automatic security patch integration
Add AI extras to test Docker builds and implement pytest.importorskip for graceful handling of missing optional dependencies. - Install AI extras in test containers (--extra ai) for comprehensive testing - Add pytest.importorskip to whisper tests for graceful local development - Tests now properly skip when optional dependencies unavailable - Docker tests validate full functionality when dependencies present Follows testing best practices for optional dependencies.
- Replace pip-audit GitHub action with native uv workflow - Add pip-audit as dev dependency in pyproject.toml - Audit pyproject.toml directly instead of exported requirements - Update trigger paths to monitor pyproject.toml and uv.lock - Simplify workflow by removing requirements.txt export step
- Remove AI dependencies to avoid llvmlite Python 3.12 compatibility issues - Security audit only needs core and dev dependencies, not AI extras - Prevents build failures in CI while maintaining security scanning
- Install libimage-exiftool-perl package required by pyexifinfo - Fixes build failure during dependency installation in CI - pyexifinfo setup.py checks for exiftool during build time
- Exclude AI extras to avoid llvmlite Python 3.12 compatibility issues - Keep essential system dependencies (exiftool, ffmpeg, postgis) for core functionality - Tests don't require AI features like Whisper/OCR
…in permissions Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
- remove conflicting nginx image specification - use custom nginx build with prod target - add missing nginx logs tmpfs mount for bitnami compatibility
- Fix config leakage in user access control by using Flask runtime config - Replace direct Config import with current_app.config + has_app_context() - Update tests to mock Flask config instead of imported config objects - Fix flaky rate limiting test with limiter.reset() for proper isolation - Organize imports: move Flask imports to top of files for clean structure Resolves test failures when production config differs from test config. Tests now properly isolated from production configuration changes.
Add numba>=0.61.0 to resolve Python version compatibility issue. openai-whisper was pinned to numba==0.53.1 which only supports Python 3.6-3.9, but numba 0.61.0+ added Python 3.12/3.13 support. This enables uv sync --extra ai to work correctly.
Replace direct Config class import with proper Flask app context handling to ensure test configurations are properly isolated from main config.json. Changes: - Use has_app_context() to detect Flask context availability - When app context exists: use current_app.config values - When no context (CLI): fallback to ConfigManager defaults - Import Flask utilities at module top level - Use correct ConfigManager default values (min_length=10, min_score=3) This fixes test failures when config.json password settings are modified, ensuring tests use their TestConfig values instead of production config.
Automatically uses current_app.config in Flask context (tests/requests) or falls back to Config class attributes (CLI/standalone). Fixes test isolation issues where config.json changes affected tests. Updated password validation to use new method.
Update config usage throughout codebase to use the smart Config.get() method that automatically detects Flask app context for proper test isolation. Changes: - Replace Config as cfg imports with Config.get() calls - Fix module-level config initialization (OAuth client, signer) - Ensure tests use TestConfig while production uses Config - Maintain backwards compatibility with existing functionality Files updated: - Admin models (Activity, Notification) - Validation models and views - Data import utilities - Export models - User models and views - Backup and database utilities This resolves test isolation issues where config.json changes would cause test failures.
- Add Config.get() method that routes to current_app.config during tests - Fix ACTIVITIES_LIST naming for proper Flask config attribute copying - Update Activity model to use Config.get() pattern consistently - Fix user tests to patch app.config instead of Config class attributes - Remove unnecessary production config patching from test setup - Achieve complete test/production configuration isolation
- Replace patch.object(cfg, ACCESS_CONTROL_RESTRICTIVE, False) with patch.dict(current_app.config, {ACCESS_CONTROL_RESTRICTIVE: False})
- Ensures consistent runtime configuration patching across all tests
- Maintains proper test configuration isolation
- Fixed in actors, bulletins, and incidents test modules
- Remove inheritance and add all required config keys with test values - Fix validation compatibility: SECURITY_PASSWORD_LENGTH_MIN=8, SECURITY_ZXCVBN_MINIMUM_SCORE=3 - Add GEO_MAP_DEFAULT_CENTER dict format for validation model compatibility - Use fakeredis and in-memory Celery to avoid external dependencies
- Add uv.lock for reproducible dependency resolution - Update pyproject.toml with modern Python project configuration - Remove legacy requirements/ directory (dev.txt, main.txt, opt.txt) - Update GitHub Actions pip-audit workflow to use UV commands - Update Docker configurations to use UV instead of pip - Update both production and test Dockerfiles for UV compatibility This completes the migration from pip/requirements.txt to UV package management for both direct Python workflows and Docker-based CI/CD.
- Update SQLALCHEMY_DATABASE_URI to use environment variables for Docker compatibility - Adjust Redis configuration to consistently use fakeredis for all test scenarios - Ensure session management utilizes fakeredis to avoid external dependencies
gen-env.sh hardcodes POSTGRES_DB=bayanat but docker-compose-test.yml creates bayanat_test database, causing 'database does not exist' error.
Empty config.test.json was missing required validation fields.
Hardcode POSTGRES_DB to "bayanat_test" in TestConfig to ensure consistent test database naming across all test environments (local and Docker).
tarekio
approved these changes
Aug 25, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.