Skip to content

fix: Pipeline stability and logic audit#114

Merged
Kavirubc merged 8 commits intomainfrom
fix/pipeline-stability-audit
May 8, 2026
Merged

fix: Pipeline stability and logic audit#114
Kavirubc merged 8 commits intomainfrom
fix/pipeline-stability-audit

Conversation

@Kavirubc
Copy link
Copy Markdown
Contributor

@Kavirubc Kavirubc commented May 8, 2026

Summary

This PR resolves 12 functional and logic bugs across the pipeline's stage ordering, data flow, transfer routing, and gatekeeper. All tests and builds pass.

Fixes

Data Integrity

  • L5: Unify ID strategy (align pipeline indexer UUID seed with bulk indexer) to prevent duplicate Qdrant points.
  • L6: Add field to pipeline indexer payload so the duplicate detector receives actual body text instead of empty strings.

Pipeline Flow

  • L3: Ensure the indexer always runs as a post-pipeline step even if the main pipeline skips gracefully, keeping VDB state synced.
  • L2: Prevent transfer_check from overwriting existing transfer decisions made by llm_router.
  • L4: Skip the triage pipeline for unknown slash commands (e.g. /status).

Duplicate Detection & Actions

  • L8: Skip duplicate detection and quality checks on pr_comment events.
  • L9: When applying labels for duplicates, filter out triage labels so only potential-duplicate is applied.
  • L10: Set original_repo metadata during rule-based transfers so responses correctly show 'Transferred from'.

Gatekeeper & Infrastructure

  • L12: Restrict sender.loginCommentAuthor mapping to primary-actor events to stop the gatekeeper from skipping human-opened but bot-labeled issues.
  • L1: Cache embeddings in context to avoid redundant API calls between similarity search and LLM routing.
  • L7: Deduplicate VDB router counts by issue number to avoid multi-chunk bias from bulk-indexed issues.
  • L11: Add documentation warning about ephemeral local filesystem usage in pending_action_scheduler.

Verification

  • go build ./... and go vet ./... are clean.
  • go test ./... passes (including fixes to the VDB router deduplication test).

Summary by CodeRabbit

  • New Features

    • Embeddings are now cached during pipeline execution to reduce redundant API calls.
    • Indexing is now processed as a post-pipeline step, allowing it to run even when the main pipeline is gracefully skipped.
  • Bug Fixes

    • Improved handling of pull request comments in quality assessment and duplicate detection.
    • Fixed deduplication of repository suggestions when indexing creates multiple entries per issue.
    • Refined comment author attribution logic for specific issue actions.
  • Documentation

    • Added documentation clarifying pending action scheduler persistence behavior and storage requirements.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 8, 2026

Warning

Rate limit exceeded

@Kavirubc has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 46 minutes and 48 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d37d8e31-dc29-48e4-a5b5-464c1812d57c

📥 Commits

Reviewing files that changed from the base of the PR and between 2e27033 and da79e45.

📒 Files selected for processing (18)
  • cmd/simili/commands/batch.go
  • cmd/simili/commands/learn.go
  • cmd/simili/commands/pipeline_runner.go
  • cmd/simili/commands/process.go
  • internal/core/config/config.go
  • internal/core/pipeline/pipeline.go
  • internal/steps/action_executor.go
  • internal/steps/auto_closer.go
  • internal/steps/command_handler.go
  • internal/steps/duplicate_detector.go
  • internal/steps/indexer.go
  • internal/steps/llm_router.go
  • internal/steps/pending_action_scheduler.go
  • internal/steps/quality_checker.go
  • internal/steps/similarity.go
  • internal/steps/transfer_check.go
  • internal/transfer/vdb_router.go
  • internal/transfer/vdb_router_test.go
📝 Walkthrough

Walkthrough

The PR refactors configuration handling to always apply defaults via a new ApplyDefaults() method, restructures pipeline execution to treat indexer as a post-step conditional on ErrSkipPipeline, standardizes Qdrant point IDs using MD5-based UUIDs, introduces embedding caching across steps, refines event filtering to skip PR comments, and deduplicates VDB router results by issue identity.

Changes

Config and Pipeline API Contracts

Layer / File(s) Summary
Config and Pipeline API
internal/core/config/config.go, internal/core/pipeline/pipeline.go
New exported ApplyDefaults() method wraps internal defaulting logic. Pipeline.Run now returns ErrSkipPipeline on graceful exit instead of nil, allowing callers to conditionally run post-pipeline steps.

Config Validation and Application

Layer / File(s) Summary
Config Validation Requirements
internal/core/config/config.go
Validate() unconditionally requires core Qdrant fields (qdrant.url, qdrant.api_key, qdrant.collection) and embedding.api_key, removing prior conditional VDB-only logic.
Defaults Application in All Paths
cmd/simili/commands/batch.go, cmd/simili/commands/process.go, internal/core/config/config.go
ApplyDefaults() is called unconditionally after config load success, fallback, or when no config file exists.

Pipeline Structure and Sequencing

Layer / File(s) Summary
Step Separation and Post-Pipeline Execution
cmd/simili/commands/pipeline_runner.go
ExecutePipeline splits steps into main pipeline (all except indexer) and post-pipeline (indexer). Main pipeline runs first; post steps run conditionally even if main pipeline gracefully skips via ErrSkipPipeline. Post-step errors are recorded non-fatally into Result.Errors.

Point ID Generation

Layer / File(s) Summary
Deterministic UUID Generation
cmd/simili/commands/learn.go, internal/steps/indexer.go
Learn command generates uuid.NewMD5(org/repo/file); indexer generates uuid.NewMD5(org/repo#number-chunk-0). Both use github.com/google/uuid; crypto/sha256 removed from learn imports. Indexer also stores issue_number and embedding text in payload.

Embedding Optimization

Layer / File(s) Summary
Embedding Caching and Reuse
internal/steps/llm_router.go, internal/steps/similarity.go
LLMRouter and SimilaritySearch check ctx.Metadata["issue_embedding"] for cached embeddings; when absent, they generate and store embeddings for downstream reuse, eliminating redundant API calls.

Event Filtering and Control Flow

Layer / File(s) Summary
Command Handler and Event Skipping
internal/steps/command_handler.go, internal/steps/duplicate_detector.go, internal/steps/quality_checker.go
Command handler returns ErrSkipPipeline after processing slash commands. DuplicateDetector and QualityChecker skip both issue_comment and pr_comment events.
Transfer and Author Refinements
internal/steps/transfer_check.go, cmd/simili/commands/process.go
TransferCheck skips evaluation when ctx.TransferTarget already set. Process refines CommentAuthor population to specific issue actions (opened, edited, closed, reopened).

Label Application and Transfer Routing

Layer / File(s) Summary
Duplicate Label Filtering
internal/steps/action_executor.go
When IsDuplicate is true, SuggestedLabels are filtered to only potential-duplicate; otherwise all suggested labels are applied.
VDB Result Deduplication
internal/transfer/vdb_router.go, internal/transfer/vdb_router_test.go
SuggestTransfer deduplicates by issue identity using seenIssues map keyed by org/repo#issue_number (from payload issue_number or number field). Avoids overcounting when bulk-indexing creates multiple points per issue. Test helper derives issue_number from result id.

Implementation Details

Layer / File(s) Summary
Batch Qdrant Initialization
cmd/simili/commands/batch.go
VectorStore is conditionally initialized only when qURL is set and not localhost:6334; otherwise skips initialization and logs info in verbose mode.
Auto-closer Formatting and Documentation
internal/steps/auto_closer.go, internal/steps/pending_action_scheduler.go
Auto-closer undergoes whitespace and formatting normalization with no functional logic changes. Pending scheduler documents that pending actions are stored on ephemeral filesystem in GitHub Actions and references TODO for persistent storage integration.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~65 minutes

Possibly related PRs

  • similigh/simili-bot#48: Modifies indexer point ID generation and payload shape with deterministic MD5 IDs and renamed/added payload fields like issue_number and embedding text.
  • similigh/simili-bot#90: Introduces VDB-based routing that this PR's deduplication logic in vdb_router.go directly refines and extends.
  • similigh/simili-bot#86: Introduces AutoCloser step and potential-duplicate labeling workflow that this PR refines with filtered label application for duplicates.

Suggested labels

fix, reliability, maintenance, bug

Suggested reviewers

  • Sachindu-Nethmin

Poem

🐰 A rabbit hops through IDs so clean,
MD5 uuids now convene,
Embeddings cached along the way,
Pipelines structured day by day,
Duplicates deduplicated right—no more a mess! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: Pipeline stability and logic audit' is directly related to the main changes—the PR fixes 12 functional/logic issues across pipeline ordering, data flow, transfer routing, and duplicate detection to improve stability and correctness.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/pipeline-stability-audit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🧹 Nitpick comments (1)
cmd/simili/commands/pipeline_runner.go (1)

35-41: ⚡ Quick win

Prefer errors.Is over == for the ErrSkipPipeline sentinel.

All three comparisons rely on ErrSkipPipeline flowing through unwrapped, which is true today but will break the moment any step does fmt.Errorf("…: %w", pipeline.ErrSkipPipeline) — a graceful-skip would then be reported as pipeline failed: and the post-pipeline indexer for that path would also stop short. errors.Is is the idiomatic, wrap-safe form.

♻️ Proposed fix
-import (
+import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"os"
 	"time"
@@
-	if err != nil {
-		if err == pipeline.ErrSkipPipeline {
+	if err != nil {
+		if errors.Is(err, pipeline.ErrSkipPipeline) {
 			fmt.Printf("⏭️ [%s] Skipped: %s\n", s.Name(), ctx.Result.SkipReason)
 			return err
 		}
@@
-	pipelineErr := mainPipeline.Run(pCtx)
-	if pipelineErr != nil && pipelineErr != pipeline.ErrSkipPipeline {
+	pipelineErr := mainPipeline.Run(pCtx)
+	if pipelineErr != nil && !errors.Is(pipelineErr, pipeline.ErrSkipPipeline) {
 		return nil, fmt.Errorf("pipeline failed: %w", pipelineErr)
 	}
@@
-			if err := s.Run(pCtx); err != nil && err != pipeline.ErrSkipPipeline {
+			if err := s.Run(pCtx); err != nil && !errors.Is(err, pipeline.ErrSkipPipeline) {
 				// Log but don't fail the overall result for post-pipeline steps
 				pCtx.Result.Errors = append(pCtx.Result.Errors, fmt.Sprintf("post-pipeline step '%s': %v", step.Name(), err))
 			}

Also applies to: 86-89, 105-108

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/simili/commands/pipeline_runner.go` around lines 35 - 41, Replace direct
equality checks against the sentinel pipeline.ErrSkipPipeline with wrap-safe
errors.Is calls: where the code currently does if err ==
pipeline.ErrSkipPipeline (and similar checks around the
s.Name()/ctx.Result.SkipReason handling and the other two occurrences), change
them to if errors.Is(err, pipeline.ErrSkipPipeline) and import the standard
"errors" package; keep the existing logging (fmt.Printf lines that use s.Name(),
ctx.Result.SkipReason, and err.Error()) intact so graceful skips still produce
the same messages while safely handling wrapped errors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/simili/commands/batch.go`:
- Around line 148-155: The assignment line "cfg = &config.Config{}" in the error
branch after calling config.LoadWithInheritance(cfgPath, fetcher) is
mis-indented and should be moved into the if-block body to match the surrounding
statements; update the indentation so that "cfg = &config.Config{}" and the
following "cfg.ApplyDefaults()" are at the same indentation level inside the if
err != nil { ... } block (the code paths around cfg, config.LoadWithInheritance
and cfg.ApplyDefaults should align with the else branch and be
gofmt-consistent).

In `@internal/steps/auto_closer.go`:
- Around line 386-417: The comment's displayed grace period (gracePeriodDisplay)
is always rendered as hours but Run may use GracePeriodMinutesOverride, so
update the logic around gracePeriodDisplay and the comment formatting to reflect
the actual runtime override: read ac.cfg.AutoClose.GracePeriodHours and
ac.cfg.AutoClose.GracePeriodMinutesOverride (or the runtime value passed to
Run), compute a single effective duration value and choose the unit (hours vs
minutes) when building the comment string (reference the variables
gracePeriodDisplay, ac.cfg.AutoClose.GracePeriodHours,
GracePeriodMinutesOverride and the comment var) so the posted message shows the
correct number and unit matching what was used to decide the close.
- Around line 253-284: The triage-comment scan currently only skips comments
older than triageWindow but fails to exclude comments newer than since, allowing
post-label bot comments to short-circuit the search; update the timestamp check
around comment.CreatedAt in the auto-closer loop so it only considers comments
whose CreatedAt is within the window [since, triageWindow] (i.e., skip if
CreatedAt is nil or CreatedAt.Time.Before(since) or
CreatedAt.Time.After(triageWindow)), leaving the isBotUser/isBotComment checks
and subsequent reaction processing (ListIssueCommentReactions) intact so we
still detect the true triage comment correctly.

In `@internal/transfer/vdb_router_test.go`:
- Around line 49-53: The derivation of issueNum from id currently assumes every
rune in id is a digit and does rune arithmetic directly; change the loop that
computes issueNum to skip any non-digit characters (e.g., check
unicode.IsDigit(c) or '0'<=c && c<='9') so only numeric runes contribute to
issueNum, leaving issueNum unchanged for skipped characters; update the loop
that references id and the variable issueNum to perform this guard so UUID-like
or alphanumeric IDs don't produce incorrect numbers.

In `@internal/transfer/vdb_router.go`:
- Around line 93-104: The dedupe logic only handles float64 so issue numbers
deserialized as int64 become 0 and all issues collapse; update the extraction
around res.Payload["issue_number"] / res.Payload["number"] (the block that sets
issueNum and builds dedupeKey/seenIssues) to use a type-switch like in
similarity.go that handles int64, int, float64 (and json.Number/string if
present), converting the found numeric value into a numeric representation used
for the key (e.g., convert ints to float64 or format the integer) before
creating dedupeKey and marking seenIssues[dedupeKey]=true.

---

Nitpick comments:
In `@cmd/simili/commands/pipeline_runner.go`:
- Around line 35-41: Replace direct equality checks against the sentinel
pipeline.ErrSkipPipeline with wrap-safe errors.Is calls: where the code
currently does if err == pipeline.ErrSkipPipeline (and similar checks around the
s.Name()/ctx.Result.SkipReason handling and the other two occurrences), change
them to if errors.Is(err, pipeline.ErrSkipPipeline) and import the standard
"errors" package; keep the existing logging (fmt.Printf lines that use s.Name(),
ctx.Result.SkipReason, and err.Error()) intact so graceful skips still produce
the same messages while safely handling wrapped errors.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 1e5cb2f0-d5ba-4943-a225-81a7a609e359

📥 Commits

Reviewing files that changed from the base of the PR and between c26f4ce and 2e27033.

📒 Files selected for processing (18)
  • cmd/simili/commands/batch.go
  • cmd/simili/commands/learn.go
  • cmd/simili/commands/pipeline_runner.go
  • cmd/simili/commands/process.go
  • internal/core/config/config.go
  • internal/core/pipeline/pipeline.go
  • internal/steps/action_executor.go
  • internal/steps/auto_closer.go
  • internal/steps/command_handler.go
  • internal/steps/duplicate_detector.go
  • internal/steps/indexer.go
  • internal/steps/llm_router.go
  • internal/steps/pending_action_scheduler.go
  • internal/steps/quality_checker.go
  • internal/steps/similarity.go
  • internal/steps/transfer_check.go
  • internal/transfer/vdb_router.go
  • internal/transfer/vdb_router_test.go

Comment thread cmd/simili/commands/batch.go
Comment thread internal/steps/auto_closer.go Outdated
Comment thread internal/steps/auto_closer.go
Comment thread internal/transfer/vdb_router_test.go
Comment thread internal/transfer/vdb_router.go
Kavirubc added 8 commits May 8, 2026 13:46
Validate() was gated behind a needsVDB flag that made it a no-op for
most configurations, causing 7 test failures where empty required
fields (qdrant.url, qdrant.api_key, qdrant.collection, embedding.api_key)
passed validation silently.

- Remove the conditional needsVDB gate from Validate()
- Always check the 4 required fields unconditionally
- Export ApplyDefaults() so callers constructing Config{} directly
  (e.g. fallback when no config file exists) can apply sane defaults

Fixes 7 failing tests in internal/core/config.

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
…RL logic

- process.go: call ApplyDefaults() on both fallback Config{} paths to
  prevent zero-value thresholds (0.0 similarity, 0 max results)
- batch.go: same ApplyDefaults() fix, plus unify Qdrant URL fallback
  to match process.go (skip VectorStore init when no real URL is
  configured instead of defaulting to localhost:6334)
- process.go: restrict sender.login → CommentAuthor mapping to
  primary-actor events only (opened/edited/closed/reopened) to prevent
  bot-labeled issues from being incorrectly skipped by the gatekeeper

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- pipeline.go: Return ErrSkipPipeline to the caller instead of swallowing it,
  so the runner knows the pipeline was skipped gracefully.
- pipeline_runner.go: Execute the 'indexer' as a post-pipeline step that always
  runs, even if the main pipeline skips (e.g. transferred issues). This ensures
  Qdrant state stays in sync with skipped/transferred issues.

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- indexer.go: Use org/repo#number-chunk-0 format for UUID generation
  to match the bulk indexer, preventing duplicate Qdrant points.
- indexer.go: Add issue body to the 'text' payload field so similarity
  search results can provide context to the LLM duplicate detector.
- learn.go: Generate RFC 4122 UUIDs via uuid.NewMD5 instead of raw SHA256 hex.

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- transfer_check.go: Exit early if ctx.TransferTarget is already set (L2).
- transfer_check.go: Set original_repo metadata in setTransferTarget so
  the response builder correctly shows 'Transferred from' (L10).
- vdb_router.go: Deduplicate VDB results by issue number before
  calculating per-repo hit counts to prevent chunk-count bias (L7).

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- duplicate_detector.go, quality_checker.go: Skip pr_comment events
  so duplicate detection and quality checks don't run on PR reviews (L8).
- action_executor.go: When an issue is a duplicate, only apply the
  'potential-duplicate' label and filter out triage labels (L9).

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- similarity.go, llm_router.go: Cache issue embedding in context metadata
  to avoid redundant duplicate API calls (L1).
- command_handler.go: Return ErrSkipPipeline for unknown slash commands
  so they don't trigger the full triage pipeline (L4).
- pending_action_scheduler.go: Add documentation warning about ephemeral
  storage in GitHub Actions and a TODO for GitHubStateManager (L11).
- auto_closer.go: Normalize CRLF to LF.

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
- auto_closer.go: Fix triage window filtering logic
- vdb_router_test.go: Update issue number parsing loop to guard against non-digits
- vdb_router.go: Update issue number extraction to handle different numeric types
- pipeline_runner.go: Use errors.Is for ErrSkipPipeline

Signed-off-by: Kavirubc <hapuarachchikaviru@gmail.com>
@Kavirubc Kavirubc force-pushed the fix/pipeline-stability-audit branch from 2e27033 to da79e45 Compare May 8, 2026 08:16
@Kavirubc Kavirubc merged commit 3e7e690 into main May 8, 2026
6 checks passed
@Kavirubc Kavirubc deleted the fix/pipeline-stability-audit branch May 8, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants