Skip to content

[BUG] CancelTask does not stop running task execution #112

@edenreich

Description

@edenreich

Bug: CancelTask Does Not Stop Running Task Execution

Summary

The CancelTask API endpoint only updates the task state in the database but does not actually stop the running task execution. When a client calls tasks/cancel, the task continues to run in the background until completion, even though the task state shows "cancelled".

Severity

High - This breaks the fundamental expectation of task cancellation and can lead to:

  • Wasted compute resources on long-running tasks
  • Inability to stop runaway or incorrect tasks
  • Misleading task states (shows "cancelled" but still executing)

Steps to Reproduce

  1. Start an ADK agent server with a long-running task (e.g., browser screenshot, file processing)
  2. Submit a task via tasks/send that will take >10 seconds
  3. While the task is running (state: "working"), call tasks/cancel with the task ID
  4. Observe the agent logs

Expected Behavior:

  • Task execution stops immediately
  • Task state changes to "cancelled"
  • No further processing occurs

Actual Behavior:

  • Task state changes to "cancelled" in database
  • Task execution continues running in the background
  • Task eventually completes and state changes to "completed"
  • The cancel request appears to succeed but has no effect

Evidence

Timeline from Real Test Case

17:48:31.833 - Client sends CancelTask request
17:48:31.833 - ADK responds with success (state changed to "cancelled")
17:48:42.739 - Task continues processing (11 seconds later)
17:48:42.739 - Task completes and state changes to "completed"

Logs Showing the Issue

Client successfully cancels:

INFO Successfully sent cancel request to agent task_id=28711ed5-8433-471b-8d9a-92de8f259d7f

But agent continues processing:

DEBUG server/task_handler.go:162 background handler received event
      {"task_id": "28711ed5-8433-471b-8d9a-92de8f259d7f", "event_type": "adk.agent.delta"}
DEBUG server/agent_streamable.go:299 streaming completed - no tool calls executed
INFO  server/task_handler.go:175 background task status changed
      {"task_id": "28711ed5-8433-471b-8d9a-92de8f259d7f", "state": "completed"}

Root Cause Analysis

Current Implementation

In task_manager.go:446-473 (CancelTask):

func (tm *DefaultTaskManager) CancelTask(taskID string) error {
    task, exists := tm.GetTask(taskID)
    if !exists {
        return NewTaskNotFoundError(taskID)
    }

    if !tm.isTaskCancelable(task.Status.State) {
        return NewTaskNotCancelableError(taskID, task.Status.State)
    }

    // Only updates the database state - does NOT stop execution
    timestamp := time.Now().UTC().Format(time.RFC3339Nano)
    task.Status.State = types.TaskStateCanceled
    task.Status.Timestamp = &timestamp

    err := tm.storage.StoreDeadLetterTask(task)
    // ...
    return nil
}

Problem: The method only updates the task state in storage. There is no mechanism to signal the running goroutine to stop.

In task_handler.go:599 (HandleMessageStream):

eventsChan, err := streamingHandler.HandleStreamingTask(ctx, task, message)

The task is started with a context, but when CancelTask() is called, there's no way to cancel that context.

Proposed Solution

Option 1: Store Cancel Functions (Recommended)

Modify TaskManager:

type DefaultTaskManager struct {
    // ... existing fields
    runningTasks map[string]context.CancelFunc // Map taskID -> cancelFunc
    tasksMutex   sync.RWMutex
}

// When starting a task
func (tm *DefaultTaskManager) StartTask(taskID string, cancel context.CancelFunc) {
    tm.tasksMutex.Lock()
    defer tm.tasksMutex.Unlock()
    tm.runningTasks[taskID] = cancel
}

// When cancelling a task
func (tm *DefaultTaskManager) CancelTask(taskID string) error {
    // ... existing validation

    // Cancel the running context
    tm.tasksMutex.Lock()
    if cancel, exists := tm.runningTasks[taskID]; exists {
        cancel()
        delete(tm.runningTasks, taskID)
    }
    tm.tasksMutex.Unlock()

    // Update state
    task.Status.State = types.TaskStateCanceled
    // ... rest of implementation
}

// Clean up on completion
func (tm *DefaultTaskManager) CompleteTask(taskID string) {
    tm.tasksMutex.Lock()
    delete(tm.runningTasks, taskID)
    tm.tasksMutex.Unlock()
}

Modify Task Handler:

func (h *DefaultA2AProtocolHandler) HandleMessageStream(...) {
    // Create cancellable context
    ctx, cancel := context.WithCancel(ctx)
    defer cancel()

    // Register cancel function with task manager
    h.taskManager.StartTask(task.ID, cancel)

    // Start processing
    eventsChan, err := streamingHandler.HandleStreamingTask(ctx, task, message)
    // ...
}

Option 2: Periodic State Checking

Have the task handler periodically check if the task state changed to "cancelled":

func (h *Handler) HandleStreamingTask(ctx context.Context, task *Task, message *Message) {
    ticker := time.NewTicker(1 * time.Second)
    defer ticker.Stop()

    for {
        select {
        case <-ticker.C:
            currentTask, _ := h.taskManager.GetTask(task.ID)
            if currentTask.Status.State == TaskStateCanceled {
                return nil, fmt.Errorf("task was cancelled")
            }
        // ... process events
        }
    }
}

Note: Option 2 is less efficient and has lag time, but doesn't require as much refactoring.

Impact

This affects:

  • All streaming tasks (tasks/send with streaming)
  • All background tasks (HandleTask in background mode)
  • Any long-running agent operations

Additional Notes

  • The isTaskCancelable() function correctly identifies which states can be cancelled
  • The API responds successfully, giving the false impression that cancellation worked
  • This is a significant gap between the API contract and actual behavior

Recommended Priority

P1 - High Priority

This is a core functionality issue that breaks a fundamental operation. Users expect cancelled tasks to actually stop executing, especially for:

  • Expensive compute operations
  • Long-running agent tasks
  • Error recovery scenarios

Related Files

  • server/task_manager.go - Task state management
  • server/task_handler.go - Task execution
  • server/agent_streamable.go - Streaming task processing
  • types/generated_types.go - TaskState definitions

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingreleased

    Type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions