| 2026-01-07 09:49:50 by Thomas Klausner | Files touched by this commit (2525) |
Log message: *: recursive bump for icu 78.1 |
| 2025-12-12 02:14:20 by Takahiro Kambe | Files touched by this commit (3) | |
Log message: mail/rspamd: update to 3.14.2 3.14.2 (2025-12-10) * [Feature] DMARC: Add --recheck-rua option to dmarc_report for RUA filtering at send time * [Feature] Metadata exporter: Add multipart and msgpack formatters * [Feature] Milter headers: Add remove_ar_from option for selective Authentication-Results header removal * [Fix] Security: Backport security fixes from libucl 0.9.3 * [Fix] HTTP: Handle early server responses during request write * [Fix] HTTP: Only apply early response handling for HTTP clients * [Fix] MIME: Prevent splitting UTF-8 sequences in header encoding * [Fix] URL: Normalize URLs with multiple slashes between host and path * [Fix] URL: Use locale-independent patterns in URL encoding * [Fix] Milter headers: Allow default_headers_order to be configured * [Fix] Milter headers: Fix hierarchical domain matching in remove_ar_from map lookups * [Fix] Aliases: Enable plugin by default to restore plus-addressing * [Fix] DCC: Handle nil opts in plugin initialization * [Fix] Neural: Fix Lua 5.4 compatibility issues * [Fix] Reputation: Fix whitelist schema and selector-aware checking * [Fix] Logging: Fix off-by-one when at the maximum tag length * [Fix] Logging: Null-terminate the log tag when copying the header * [Fix] Prometheus: Use double type for rspamd_scan_time_average metric * [Fix] Build: FreeBSD 15 inotify build compatibility * [Fix] Build: Remove deprecated OpenSSL engine.h include * [Fix] Lua: Use 0 instead of LUA_OK for lua_pcall result comparison (Lua 5.1 compat) * [Fix] Lua: Handle Lua 5.4 require returning two values * [Fix] Lua: Use math.floor for Lua 5.4 integer division compatibility * [Fix] Lua: Improve loadstring error handling for Lua 5.4 compatibility * [Fix] Lua: Use userdata __gc for UCL objects in all Lua versions * [Fix] Lua: Use ipairs for ordered iteration in header checks * [Fix] Lua: Fix other places of Lua 5.4 rounding issues * [Fix] Performance: Avoid repeated simdutf implementation detection on each call * [Minor] Update public suffix list * [Minor] Debian: Allow skipping ASAN build * [Test] Add Lua 5.4 and cffi-lua support for testing * [Test] Fix Python 3.12+ compatibility in test scripts * [Test] Fix TCP tests for HTTP/1.1 compliance |
| 2025-12-09 15:07:29 by Takahiro Kambe | Files touched by this commit (3) | |
Log message: mail/rspamd: update to 3.14.1 3.14.1 (2025-12-01) * [Feature] Composites: Add inverted index for fast composite rule evaluation * [Feature] Composites: Add bloom filter for fast negative symbol lookups * [Feature] Composites: Add statistics tracking and control protocol command * [Feature] Composites: Precompute atom types at config time * [Feature] Multimap: Add combinator option for selector rules * [Feature] SPF: Add rspamadm spf_flatten tool with macro preservation * [Feature] URL: Add deep processing architecture with C-to-Lua filter consultation * [Feature] URL: Add obfuscated URL detection to url_suspect plugin * [Feature] URL: Add rspamd_util.decode_html_entities for HTML entity decoding * [Feature] lua_shape: Add new validation library as tableshape replacement * [Feature] lua_shape: Add T.callable() type for function validation * [Feature] lua_shape: Add callable defaults support * [Feature] Whitelist: Auto-mark symbols with SYMBOL_TYPE_FINE flag * [Fix] url_suspect: Fix plugin causing massive false positives * [Fix] url_suspect: Optimize for high URL volume messages * [Fix] Network: Prevent infinite loop in split_networks_into_chunks() * [Fix] Memory: Fix leak in custom tokenizer result handling * [Fix] Composites: Fix group matchers handling in inverted index * [Fix] Composites: Improve atom polarity detection in inverted index * [Fix] Composites: Copy expression string to memory pool for Lua composites * [Fix] lua_shape: Fix transform logic and tableshape compatibility * [Fix] lua_shape: Fix registry to recursively resolve nested schemas * [Fix] lua_shape: Improve error safety * [Fix] Settings: Keep groups_*/symbols_* fields for runtime processing * [Fix] URL: Encode redirect URLs to handle unencoded spaces and special characters * [Fix] external_relay: Fix mixins and confighelp * [Fix] RBL: Fix plugin transform schemas * [WebUI] Update D3 libs with bug fixes and validation * [WebUI] Restore hover colors for symbols * [WebUI] Fix hover behavior outside status tables * [Minor] Whitelist: Use contemporary API for maps * [Minor] Migrate all plugins and libraries from tableshape to lua_shape * [Minor] Simplify configuration by removing use_*_map flags * [Minor] Add plugins registry with reworked mixins * [Test] Add comprehensive tests for URL deep processing * [Test] Isolate url_suspect tests with symbols_enabled |
| 2025-11-16 22:39:50 by Thomas Klausner | Files touched by this commit (6) | |
Log message: rspamd: update to 3.14.0. 3.14.0: 10 Nov 2025 * [Feature] Fuzzy check: Add HTML fuzzy hashing for structural similarity matching * [Feature] Fuzzy check: Add per-rule text_hashes toggle for HTML-only fuzzy rules * [Feature] Fuzzy check: Add structured checks configuration with backward \ compatibility * [Feature] Fuzzy storage: Implement full TCP protocol support with auto-switch * [Feature] Fuzzy check: Add TCP connection management and error handling * [Feature] URL: Add task:get_cta_urls() API for proper CTA domain extraction * [Feature] URL: Move CTA processing into dedicated module * [Feature] URL: Add url:get_hash() method for efficient deduplication without \ string conversion * [Feature] GPT: Add web search context support with Redis caching * [Feature] HTML: Add infrastructure for async URL rewriting with Lua bindings * [Feature] HTML: Add task:rewrite_html_urls() and task:get_html_urls() Lua API * [Feature] WebUI: Implement dark mode with theme toggle and auto detection * [Feature] Aliases: Add advanced resolution with loop detection for \ converging paths * [Feature] Milter: Add ESMTP argument parsing with Lua API access * [Feature] Milter: Add per-recipient ESMTP args parsing and metadata access * [Feature] Milter: Support array of positions for remove_headers operations * [Feature] Proxy: Add client IP preservation in message headers through chain * [Feature] Rspamc: Add milter.add_headers object format support to --mime * [Feature] Configwizard: Add Postfix integration wizard using postconf utility * [Feature] Build: Add comprehensive BSD workflows (FreeBSD, NetBSD, OpenBSD) \ with Lua version selection * [Feature] Build: Add automated code review GitHub Actions workflow with \ Rspamd-specific guidelines * [Feature] Build: Add Docker-based integration test suite with ASAN and real \ corpus * [Feature] Build: Add automatic public suffix list synchronization * [Feature] Multimap: Add support for symbols with leading numerals * [Feature] DMARC: Add Auto-Reply-To and Precedence headers to prevent \ out-of-office replies * [Feature] Platform: Add NetBSD memory usage tracking support * [Feature] Utilities: Add fuzzy Redis migration utility * [Feature] Bayes: Allow skipping local/authenticated mail in autolearn condition * [Feature] ARC: Add DKIM signing key API for flexible ARC signing * [Feature] Logger: Add type specifiers support for better formatting * [Feature] Heap: Add rspamd_heap_push_slot to eliminate double allocation * [Fix] DNS: Preserve req->pos during reply validation to prevent packet \ truncation on UDP-to-TCP retransmits * [Fix] DNS: Regenerate transaction ID before copying to TCP buffer to avoid \ collisions * [Fix] DNS: Fix nameserver round-robin when using /etc/resolv.conf * [Fix] DNS: Fix TCP uninitialized memory leak * [Fix] DMARC: Add batching and forced GC for Redis connections to prevent \ pool exhaustion * [Fix] DMARC: Validate and normalize batch_size to prevent fractional \ indexing and loop errors * [Fix] DMARC: Refactor reporting to use helper functions and async maps * [Fix] Allocator: Fix jemalloc/system malloc mixing in getline() to prevent \ crashes * [Fix] Allocator: Fix allocator mismatches in hiredis * [Fix] Allocator: Fix allocator mismatches in libucl * [Fix] Hyperscan: Use runtime version instead of compile-time for database \ validation * [Fix] Hyperscan: Auto-recreate invalid unserialized cache files on version \ mismatch * [Fix] Memory: Fix leaks in fuzzy storage khash tables * [Fix] Memory: Fix leaks in upstream address parsing * [Fix] Memory: Fix leaks in *-any address parsing * [Fix] Memory: Fix OpenSSL providers cleanup * [Fix] Memory: Fix UCL object memory leak in Lua integration * [Fix] Memory: Fix stat metadata tokenization leak * [Fix] Fuzzy TCP: Fix double-release in fuzzy_tcp_session * [Fix] Fuzzy TCP: Fix refcount leak in destructor * [Fix] Fuzzy TCP: Fix timeout handling and buffer overflow * [Fix] Fuzzy TCP: Fix endianness mismatch in framing protocol * [Fix] Fuzzy TCP: Fix race conditions and fd reuse bugs * [Fix] Fuzzy TCP: Use pure ev_timer for session timeouts * [Fix] Fuzzy TCP: Fix server replies and client event handling * [Fix] Shutdown: Keep srv events active during shutdown to track auxiliary \ processes * [Fix] ARC: Restore strict header ordering to comply with RFC 8617 * [Fix] ARC: Add ed25519 key support * [Fix] Composites: Implement two-phase evaluation for postfilter dependencies * [Fix] Composites: Use null-terminated string for symbol lookup * [Fix] URL: Refactor extraction to prevent DoS with hash-based deduplication * [Fix] URL: Add 50k URL limit with warning for DoS protection * [Fix] URL: Skip HTML_DISPLAYED URLs in CTA detection * [Fix] URL: Fix CTA priority preservation in extract_specific_urls * [Fix] Bayes: Improve Redis server discovery * [Fix] Bayes: Only bypass learn when header value matches * [Fix] ESMTP: Robust per-recipient parsing in milter with safe cursor advance * [Fix] ESMTP: Refcount ESMTP args in proxy_session_refresh to avoid use-after-free * [Fix] ESMTP: Correct Lua stack cleanup in lua_task_get_rcpt_esmtp_args * [Fix] HTML: Correct attribute value offset calculation for URL rewriting * [Fix] HTML: Add HTML entity encoding for URL rewriting * [Fix] HTML: Fix segfault due to incorrect HTML features access * [Fix] HTML: Fix frequency-based ordering in domain hashing * [Fix] HTML: Fix shingles hash generation bugs * [Fix] HTML: Fix memory leaks in shingles generation * [Fix] HTML: Fix memory management in html_cta.process_html_links * [Fix] HTML: Fix CSS class normalization in fuzzy tokens * [Fix] HTML: Fix cache key collision between text and HTML fuzzy hashes * [Fix] OpenBSD: Fix kinfo_proc structure member names * [Fix] OpenBSD: Disable Hyperscan (not available) * [Fix] FreeBSD: Fix zstd package name * [Fix] FreeBSD: Add IGNORE_OSVERSION for package version mismatches * [Fix] NetBSD: Setup pkgin and PKG_PATH before installing packages * [Fix] NetBSD: Fix missing dependencies and package names * [Fix] BSD: Remove -j flag from ninja in all BSD workflows * [Fix] Multimap: Handle symbols with leading numerals * [Fix] Aliases: Prevent creation of malformed email addresses * [Fix] Aliases: Fix alias loop detection for converging paths * [Fix] Aliases: Fix is_local_domain to support backend objects * [Fix] Aliases: Correct to_local when no recipients present * [Fix] Aliases: Fix set_addr validation to prevent malformed addresses * [Fix] MIME: Remove Authentication-Results and anonymize envelope-from in \ Received headers * [Fix] Mempool: Prevent double-free in destructor cleanup * [Fix] Rspamadm: Unbreak dnstool command * [Fix] Integration tests: Fix ASAN configuration and startup diagnostics * [Minor] Replace GHashTable with khash in fuzzy_check.c and \ lua_textpart_get_cta_urls * [Minor] Update cache key prefix to match module name in llm_search_context * [Minor] Fix llm_search_context to follow Rspamd idioms * [Minor] Refactor llm_search_context to use lua_cache module * [Minor] Address review comments in various modules * [Minor] Fix droid usage * [Minor] Use GPT-5 Codex for code reviews * [Minor] Update libucl with automatic stack management * [Rework] Prioritize CTA URLs in redirector and Lua helpers * [Rework] RBL configuration: Add new from selectors, content_urls checks, and \ lower_utf8 for hashed domains * [Rework] Make Bayes learn guards configurable * [Rework] Refactor element visibility control to use Bootstrap classes * [Rework] Use postconf utility for Postfix configuration in configwizard * [Rework] Remove Lua-level HTTP header parsing in ESMTP args getters * [Rework] Add CFG_REF_* macros with debug logging for config refcounting * [Rework] Move OpenSSL providers from global to libs_ctx * [Rework] Convert heap to fully intrusive kvec-based implementation * [Rework] Add specialized pool types for long-lived and short-lived allocations * [Rework] Improve memory pool destructors with smart preallocation based on \ pool type * [Project] Restrict code review workflow to authorized maintainers * [Project] Add Claude Code and Cursor AI assistant configuration * [WebUI] Replace Glyphicons with FontAwesome SVG icons * [WebUI] Update CodeJar to version 4.3.0 * [WebUI] Update Node.js and ESLint * [WebUI] Update D3-based visualization libs * [WebUI] Replace deprecated alert-error class with alert-danger * [WebUI] Add search syntax hint to history table filter input * [WebUI] Fix theme toggle default to auto * [WebUI] Keep classifiers list when request is skipped * [WebUI] Repopulate classifier dropdown * [WebUI] Add comment for removeEventListener * [WebUI] Fix icon rendering race condition in tab initialization * [Test] Add comprehensive Lua unit tests for HTML URL rewriting * [Test] Add unit tests for HTML URL rewriting patch engine * [Test] Add functional tests for HTML fuzzy hashing * [Test] Add ARC chain verification tests with multiple signatures * [Test] Add e2e for classifier dropdown population * [Test] Multimap symbol with leading numerals * [Test] Sync public suffix list automatically * [Test] Update JS linters * [Test] Fix integration test environment variable passing * [Test] Add detailed error output for integration test failures |
| 2025-11-04 12:42:45 by Thomas Klausner | Files touched by this commit (3) |
Log message: rspamd: improve NetBSD support, from upstream Bump PKGREVISION. |
| 2025-11-02 22:17:43 by Thomas Klausner | Files touched by this commit (6) | |
Log message:
rspamd: update to 3.13.2.
3.13.2
What's Changed
[Feature] Add user/domain context support for LLM-based classification by \
@vstakhov in #5647
Implement Metadefender hash lookup module by @vstakhov in #5656
Add vault kv version 2 support by @vstakhov in #5654
Fix duplicate symbol in once_received plugin by @vstakhov in #5658
[Fix] Enforce server-controlled HTTP map refresh intervals by @vstakhov in #5660
[Fix] Propagate unused Redis Sentinel options by @fatalbanana in #5597
Fix rspamd dkim key loading for ed25519 by @vstakhov in #5664
Fix dkim relaxed bodyhash calculation for spaces by @vstakhov in #5662
Feat: Added rua address exclusion in dmarc.lua by @croessner in #5653
[Feature] Add separate encryption keys for read/write operations in \
fuzzy_check by @vstakhov in #5665
3.13.1
Added
Archive module: Full support for encrypted ZIP archives, including both \
ZipCrypto and AES encryption; both reading and writing of AES-encrypted ZIP \
archives is supported with updated Lua bindings using libarchive for flexible \
compatibility with all standard ZIP encryption schemes
Encrypted maps: Support for encrypted maps to enable new map distribution \
scenarios
Redis TLS: Configurable TLS connections in Redis backend for improved \
compatibility in secure environments
Improved
MIME encoding refactoring: Major overhauls and multiple fixes for MIME \
encoding logic, including improved handling and decoding of UTF-8 in MIME \
headers, resulting in more robust email processing and better compatibility
Learning system: Numerous fixes to learn checks and autolearn flag handling, \
prevention of duplicate message learning, and extended multiclass learning test \
coverage
Map helpers alignment: Map helpers now enforced to be aligned to 64 bytes to \
prevent unaligned memory access errors on certain platforms
CLI enhancements: Enhanced secretbox CLI and additional security test coverage
Platform compatibility: Improved compatibility with Lua versions above 5.1 \
and better support for 32-bit platforms
Fixed
Critical fixes: Fixed bug when converting zero-length strings to numbers
XML parsing: Fixed XML prolog detection in lua_magic module
Build issues: Fixed build issues on 32-bit platforms
Empty input handling: Addressed issues with empty input handling in lua_magic
Test stability: Improved stability of automated testing with multiple \
miscellaneous test fixes
Compatibility: Minor compatibility improvements and bugfixes (buffer \
allocation, missing cmath include, etc.)
This release introduces archive module extensibility with full encrypted archive \
support including AES, new map distribution capabilities, secure integration \
options with Redis TLS, robust email and message processing improvements, and \
bugfixes for broader platform compatibility. This is recommended as a major \
stability and feature update.
3.13.0
Highlights & Major Features Since 3.12.1
1. Multiclass Bayes Classification ([#5547](#5547))
Bayesian classifiers now support multi-class differentiation—labels like \
spam, ham, transactional, newsletters, phishing, and more (2-20 classes \
supported).
Efficient: all classes for a message handled in a single Redis call.
Backward compatible with old config (is_spam); new config enables named \
classes and labels.
Autolearn and Lua API support multiclass workflows.
Fully class-aware Redis caching.
Examples:
rspamc learn_class:transactional receipt.eml or rspamc \
learn_class:newsletter newsletter.eml
Lua API: task:get_multiclass_result() gives class probabilities and confidence.
2. Neural Module Overhaul ([#5579](#5579))
Complete rework into a provider-based architecture: combines symbols, LLM \
embeddings (OpenAI, etc.), and planned providers (Bayes/FastText, in the \
future).
Pluggable fusion—multiple feature types can be combined for richer, more \
accurate classification.
Trained normalization (unit/zscore/none), used consistently at \
training/inference.
Redis-backed caching for LLM embeddings to control cost/latency.
Configurable via providers, versioned for safe upgrades, fully backwards \
compatible.
3. Multimap Selectors & Regex Enhancements ([#5615](#5615))
Powerful, SA-style “selector” rules in multimap module for regex \
filtering on message fields.
Dedicated selector field, integrated with Hyperscan and regex cache.
Example: selector FROM_CORP from:domain =~ /corp\.example$/i
4. MIME & HTML Feature Extraction ([#5619](#5619), [#5608](#5608))
MIME parser detects part types automatically.
HTML parser project extracts more features for downstream modules.
5. HTTP, DNS, Upstream Improvements ([#5614](#5614), [#5603](#5603), [#5601](#5601))
Flexible HTTP timeout config and handling.
Upstream reliability: probe mode, less need for forced revive.
DNS nameserver resolution moved to getaddrinfo.
6. Modernization & Maintenance ([#5592](#5592), [#5598](#5598), \
[#5580](#5580), others)
Standardized on C++20; builds, test, and CI improvements (ARM support, \
modern fallback maps).
Regular code cleaning, bugfixes, and RPM tweaks.
7. WebUI & UX ([#5606](#5606), [#5607](#5607))
E2E scan test flows in WebUI.
Bootstrap upgrade, Bayes class management from the web interface.
8. GPT & LLM Integrations ([#5612](#5612), [#5572](#5572))
Improved handling of GPT model parameters and prompts.
Initial support for OpenAI GPT-5 and other models.
Notable Bugfixes & Maintenance
DCC plugin rewritten ([#5602](#5602)), optimized.
DKIM relaxed body canonicalization ([#5593](#5593)), multimap, WebUI and \
configuration reliability increased.
Numerous minor bugfixes, build and CI improvements.
Projects & Modules Affected
Core: Multiclass Bayes, Neural/LLM fusion
Filtering: Multimap selectors, regex & Hyperscan
Protocols: HTTP, DNS
Web: WebUI, Bootstrap
Plugins: DCC, DKIM, GPT, Neural
|
| 2025-10-23 22:40:24 by Thomas Klausner | Files touched by this commit (2999) |
Log message: *: recursive bump for pcre2 Running an old binary against the new pcre doesn't work: /usr/pkg/lib/libpcre2-8.so.0: version PCRE2_10.47 required by \ /usr/pkg/lib/libglib-2.0.so.0 not defined |
| 2025-07-01 16:54:25 by Amitai Schleier | Files touched by this commit (3) | |
Log message: rspamd: update to 3.12.1. Changes: * [Feature] Add /bayes/classifiers HTTP endpoint * [Feature] Further improvements in scheduling next checks * [Fix] Another fix for maps concurrent load * [Fix] Do not add log tag header in milter logic * [Fix] Do not explicitly add Connection header if it's there * [Fix] Fix proxy headers duplication * [Fix] Fix several issues with the lua_logger * [Fix] Make logger more graceful when dealing with format arguments * [Fix] Try to avoid incomplete writes * [Rework] Eliminate maps locking |