fix: show General forum topic messages (coalesce NULL reply_to_top_id)#122
Conversation
The viewer's topic list already displays General with its true message count via ``coalesce(reply_to_top_id, 1)`` in ``get_forum_topics()``, but clicking the General tab rendered an empty pane even when the sidebar said "N messages". Cause: ``get_messages_paginated()`` filters with strict ``Message.reply_to_top_id == topic_id``, so ``?topic_id=1`` never matches messages whose ``reply_to_top_id`` is NULL. Pre-v6.2.0 messages and pre-forum messages all have NULL — Telegram's own client groups them under General. Fix: apply the same ``coalesce(reply_to_top_id, 1) == topic_id`` filter in the messages query so General sees its NULL-top rows while non-General topics still filter strictly. Regression tests in ``tests/test_general_topic_messages.py`` spin up an in-memory SQLite DB via the ORM metadata and cover: - ``topic_id=1`` returns NULL-top messages - non-General topic IDs still filter strictly - no ``topic_id`` returns all messages regardless of ``reply_to_top_id``
📝 WalkthroughWalkthroughUpdated get_messages_paginated() to treat NULL Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (3)
tests/test_general_topic_messages.py (2)
29-61: Shared fixture + type hint would trim duplication.All three tests repeat the
adapter, engine, chat_id = await _make_adapter_with_messages()/try/finally engine.dispose()dance. Apytest_asyncio.fixtureyielding the tuple and disposing on teardown would drop ~15 lines and remove the chance of forgettingdispose()in a future test. Also, the helper is missing a return type annotation (per repo guideline: type hints for function signatures).As per coding guidelines: "Use type hints for function signatures".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_general_topic_messages.py` around lines 29 - 61, Add a pytest_asyncio.fixture that yields the (adapter, engine, chat_id) tuple created by the existing helper and disposes the engine in the fixture teardown; update tests to use that fixture instead of calling _make_adapter_with_messages() and manual try/finally engine.dispose(). Also add a return type annotation to the helper _make_adapter_with_messages (e.g., -> Tuple[DatabaseAdapter, AsyncEngine, int]) to satisfy the repo typing guideline and keep the helper for reuse; ensure the fixture imports pytest_asyncio and references DatabaseAdapter and engine.dispose() for cleanup.
53-54: Nit: rename the placeholder text."sexy topic msg" / "another sexy msg" reads oddly in a regression test. Something neutral like
"topic-47 msg#1"keeps future readers focused on the assertion, not the wording.Proposed tweak
- (3, datetime(2024, 3, 1, 10), "sexy topic msg", 47), - (4, datetime(2024, 3, 2, 10), "another sexy msg", 47), + (3, datetime(2024, 3, 1, 10), "topic-47 msg `#1`", 47), + (4, datetime(2024, 3, 2, 10), "topic-47 msg `#2`", 47),🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/test_general_topic_messages.py` around lines 53 - 54, Replace the odd placeholder strings in the test data tuples (currently "sexy topic msg" and "another sexy msg") with neutral, descriptive placeholders so the test reads clearly; update the two tuples (the ones with message text and topic id 47) to use something like "topic-47 msg `#1`" and "topic-47 msg `#2`" to keep focus on the assertion rather than the wording.src/db/adapter.py (1)
1101-1104: LGTM — coalesce matchesget_forum_topicscounting logic.Filter now treats
NULL reply_to_top_idas General (id=1), so sidebar counts and message pane agree. Non-General topics remain strict sincecoalesce(NULL, 1) == 47is false.One heads-up: if an index exists on
reply_to_top_id, wrapping it incoalesce(...)will prevent its use on most engines. Likely negligible given thechat_idfilter already narrows the scan, but worth keeping in mind if message volumes grow.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/db/adapter.py` around lines 1101 - 1104, The current use of func.coalesce(Message.reply_to_top_id, 1) in stmt.where disables index use on reply_to_top_id; replace the coalesce predicate with an explicit conditional that preserves indexability: filter by Message.reply_to_top_id == topic_id for non-General topics, and for topic_id == 1 add an additional OR branch that matches Message.reply_to_top_id IS NULL (i.e., treat NULL as General) so the query uses a direct column comparison on Message.reply_to_top_id instead of wrapping it in coalesce; update the stmt.where call accordingly (referencing Message.reply_to_top_id, stmt.where, and the current topic_id handling).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@src/db/adapter.py`:
- Around line 1101-1104: The current use of
func.coalesce(Message.reply_to_top_id, 1) in stmt.where disables index use on
reply_to_top_id; replace the coalesce predicate with an explicit conditional
that preserves indexability: filter by Message.reply_to_top_id == topic_id for
non-General topics, and for topic_id == 1 add an additional OR branch that
matches Message.reply_to_top_id IS NULL (i.e., treat NULL as General) so the
query uses a direct column comparison on Message.reply_to_top_id instead of
wrapping it in coalesce; update the stmt.where call accordingly (referencing
Message.reply_to_top_id, stmt.where, and the current topic_id handling).
In `@tests/test_general_topic_messages.py`:
- Around line 29-61: Add a pytest_asyncio.fixture that yields the (adapter,
engine, chat_id) tuple created by the existing helper and disposes the engine in
the fixture teardown; update tests to use that fixture instead of calling
_make_adapter_with_messages() and manual try/finally engine.dispose(). Also add
a return type annotation to the helper _make_adapter_with_messages (e.g., ->
Tuple[DatabaseAdapter, AsyncEngine, int]) to satisfy the repo typing guideline
and keep the helper for reuse; ensure the fixture imports pytest_asyncio and
references DatabaseAdapter and engine.dispose() for cleanup.
- Around line 53-54: Replace the odd placeholder strings in the test data tuples
(currently "sexy topic msg" and "another sexy msg") with neutral, descriptive
placeholders so the test reads clearly; update the two tuples (the ones with
message text and topic id 47) to use something like "topic-47 msg `#1`" and
"topic-47 msg `#2`" to keep focus on the assertion rather than the wording.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: ffdd2f96-3f14-492c-b1a7-22625e4b735d
📒 Files selected for processing (2)
src/db/adapter.pytests/test_general_topic_messages.py
|
Hey @tondeaf, thanks for this excellent bug fix! The root cause analysis in the PR description is really well done — identifying the mismatch between I ran a thorough review (code quality, security, test adequacy) and the fix looks great overall. A few items to address before merge: Code Quality (1 low-severity note)LOW — if topic_id == 1:
stmt = stmt.where(or_(Message.reply_to_top_id == 1, Message.reply_to_top_id.is_(None)))
else:
stmt = stmt.where(Message.reply_to_top_id == topic_id)That said, the current approach is consistent with Test Adequacy (1 missing edge case)Please add a test for explicit (6, datetime(2024, 5, 1, 10), "explicit general msg", 1),Then update CodeRabbitCodeRabbit flagged that the PR description doesn't follow the repo's PR template (Type of Change checkboxes, Database Changes section, etc.). Would be nice to restructure the description to match the template format. SecurityNo concerns — the change is a parameterized SQLAlchemy expression, Thanks again for the contribution! 🙏 |
Address PR GeiserX#122 review: the existing seed data only had NULL reply_to_top_id rows for the General topic, but post-forum-enable Telegram sets reply_to_top_id=1 explicitly on new General messages. Add a row with reply_to_top_id=1 and assert it appears alongside the NULL rows under topic_id=1, proving the coalesce predicate matches both representations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/test_general_topic_messages.py`:
- Around line 29-64: Replace the fragile DatabaseManager.__new__ pattern in
_make_adapter_with_messages with a normal constructor call using
DatabaseManager(database_url="sqlite+aiosqlite://") so __init__ runs and
attributes are set correctly, and add an explicit return type hint on
_make_adapter_with_messages: async def _make_adapter_with_messages() ->
tuple[DatabaseAdapter, AsyncEngine, int]: to reflect the returned
(DatabaseAdapter, engine, chat_id) tuple; keep the rest of the seeding logic
unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 93c58d48-8952-40db-bfc3-4ea4c2fd0bda
📒 Files selected for processing (1)
tests/test_general_topic_messages.py
| async def _make_adapter_with_messages(): | ||
| """Spin up an in-memory SQLite DB seeded with a forum chat + mixed-topic messages.""" | ||
| engine = create_async_engine( | ||
| "sqlite+aiosqlite://", | ||
| poolclass=StaticPool, | ||
| connect_args={"check_same_thread": False}, | ||
| ) | ||
|
|
||
| async with engine.begin() as conn: | ||
| await conn.run_sync(Base.metadata.create_all) | ||
|
|
||
| db_manager = DatabaseManager.__new__(DatabaseManager) | ||
| db_manager.engine = engine | ||
| db_manager.database_url = "sqlite+aiosqlite://" | ||
| db_manager._is_sqlite = True | ||
| db_manager.async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) | ||
|
|
||
| chat_id = -1002010842190 | ||
| async with db_manager.async_session_factory() as session: | ||
| session.add(Chat(id=chat_id, type="group", title="Forum Chat", is_forum=1)) | ||
| rows = [ | ||
| # (id, date, text, reply_to_top_id) | ||
| (1, datetime(2024, 1, 1, 10), "pre-forum general #1", None), | ||
| (2, datetime(2024, 1, 2, 10), "pre-forum general #2", None), | ||
| (3, datetime(2024, 3, 1, 10), "sexy topic msg", 47), | ||
| (4, datetime(2024, 3, 2, 10), "another sexy msg", 47), | ||
| (5, datetime(2024, 4, 1, 10), "photos topic", 144), | ||
| # Post-forum-enable: Telegram sets reply_to_top_id=1 explicitly on | ||
| # new General messages. Must appear alongside NULL rows under topic_id=1. | ||
| (6, datetime(2024, 5, 1, 10), "post-forum general (explicit 1)", 1), | ||
| ] | ||
| for mid, dt, body, top in rows: | ||
| session.add(Message(id=mid, chat_id=chat_id, date=dt, text=body, reply_to_top_id=top)) | ||
| await session.commit() | ||
|
|
||
| return DatabaseAdapter(db_manager), engine, chat_id |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
ast-grep --pattern $'class DatabaseManager:
$$$
'
fd -e py . tests | xargs rg -nP -C2 'DatabaseManager\.__new__|DatabaseManager\('Repository: GeiserX/Telegram-Archive
Length of output: 27480
Replace DatabaseManager.__new__ with normal instantiation and add return type hint.
The DatabaseManager.__new__(DatabaseManager) plus manual attribute injection is unnecessary and fragile. DatabaseManager.__init__ accepts a database_url parameter, so just use:
db_manager = DatabaseManager(database_url="sqlite+aiosqlite://")This avoids circumventing initialization and is clearer.
Also add the return type hint to the helper:
async def _make_adapter_with_messages() -> tuple[DatabaseAdapter, AsyncEngine, int]:(Optional: The three tests rebuild engine + schema + seed each time. A pytest_asyncio fixture would reduce boilerplate and consolidate cleanup, but the current approach works.)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/test_general_topic_messages.py` around lines 29 - 64, Replace the
fragile DatabaseManager.__new__ pattern in _make_adapter_with_messages with a
normal constructor call using
DatabaseManager(database_url="sqlite+aiosqlite://") so __init__ runs and
attributes are set correctly, and add an explicit return type hint on
_make_adapter_with_messages: async def _make_adapter_with_messages() ->
tuple[DatabaseAdapter, AsyncEngine, int]: to reflect the returned
(DatabaseAdapter, engine, chat_id) tuple; keep the rest of the seeding logic
unchanged.
|
Addressed your feedback in the latest commit. Summary of what changed and why: Test row for explicit
Index-usage alternative (LOW)
PR description template
Ready for another look. |
|
Thanks @tondeaf! This fix is now live in v7.6.4. 🎉 The coalesce alignment between I also shipped a follow-up in #132 that built on your test suite: renamed the test data to neutral names, converted to a pytest fixture, removed redundant decorators, and added edge case tests for nonexistent topics, |
Summary
coalesce(reply_to_top_id, 1) == topic_idpredicate inget_messages_paginated()thatget_forum_topics()already uses for counting, so the General topic pane stops returning empty when its messages have NULLreply_to_top_id(pre-v6.2.0 / pre-forum) or explicitreply_to_top_id=1(post-forum-enable).Type of Change
Database Changes
scripts/Data Consistency Checklist
chat_idvalues use marked format (via_get_marked_id()) — N/A, no chat_id construction_strip_tz()before DB operations — N/A, no datetime handlingTesting
python -m pytest tests/ -v)ruff check .)ruff format --check .)Security Checklist
topic_idflows through SQLAlchemy bound parametersDeployment Notes
The bug
In the web viewer, a forum chat's topic list correctly shows the General topic with its true message count (e.g. "67 messages"), but clicking the General tab renders an empty message pane.
Reproduce:
Sidebar shows the correct count; main pane is empty.
Root cause
get_forum_topics()insrc/db/adapter.pyalready handles this correctly when counting General messages:…with a comment that explains: "Messages with
reply_to_top_id=NULLare treated as General topic (id=1), since pre-v6.2.0 messages and pre-forum messages lack topic assignment and Telegram's client displays them under General."But
get_messages_paginated()uses a strict equality filter:So
?topic_id=1never matches the NULL-top rows, and the General pane is empty despite the sidebar claim.The fix
Apply the same
coalesce(reply_to_top_id, 1) == topic_idfilter inget_messages_paginated(). General sees its NULL-top messages; non-General topics still filter strictly.One-line change + three regression tests (
tests/test_general_topic_messages.py) covering:topic_id=1returns NULL-top and explicitreply_to_top_id=1messages (post-review fix-up)topic_idreturns everything regardless ofreply_to_top_idTests use
Base.metadata.create_allon an in-memory SQLite engine so column drift can't silently break them.Verification against a real DB
On the affected forum chat (pre-forum + forum messages mixed):
GET /api/chats/<id>/messages?topic_id=1— before:[]; after: returns the General messages.GET /api/chats/<id>/messages?topic_id=<other>— still returns only rows matching that exact topic.Test plan
pytest tests/test_general_topic_messages.py— 3/3 pass