Skip to content

[BUG] _default route no longer seems to exist in 2.11.0 #5763

@kfr2

Description

@kfr2

Describe the bug
I updated a data prepper installation running 2.10.3 to 2.11.0 and the new instance fails to start after displaying an error message like:

2025-06-05T17:15:44,937 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataPrepper' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.11.0.jar!/org/opensearch/dataprepper/core/DataPrepper.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.core.DataPrepper]: Constructor threw exception; nested exception is org.opensearch.dataprepper.pipeline.parser.InvalidPipelineConfigurationException: The following routes do not exist in pipeline "entry-pipeline-us-west-2": [_default]. Configured routes include [usw2dev-test-dp-ingestion-us-west-2_route-us-west-2]

It seems like this feature addition mentioned in the 2.11.0 release notes could be related. #5106

To Reproduce

This data-prepper-config.yaml file works under 2.10.3 but not under 2.11.0:

entry-pipeline-us-west-2:
  source:
    s3:
      acknowledgments: true
      notification_type: \"sqs\"
      compression: gzip
      codec:
        newline:
      sqs:
        queue_url: \"https://sqs.us-west-2.amazonaws.com/1234567890/data-prepper-ingress-us-west-2-test\"
        maximum_messages: 10
        visibility_timeout: \"60s\"
      bucket_owners:
        usw2dev-test-dp-ingestion-us-west-2: 1234567890
      aws:
        region: \"us-west-2\"
  processor:
  route:
    - usw2dev-test-dp-ingestion-us-west-2_route-us-west-2: '/s3/bucket == \"usw2dev-test-dp-ingestion-us-west-2\"'
  sink:
    - pipeline:
        name: usw2dev-test-dp-ingestion-us-west-2-pipeline-us-west-2
        routes:
          - usw2dev-test-dp-ingestion-us-west-2_route-us-west-2
    - pipeline:
        name: default-pipeline-us-west-2
        routes:
          - _default

default-pipeline-us-west-2:
  source:
    pipeline:
      name: entry-pipeline-us-west-2
  sink:
    - opensearch:
        hosts: [ \"https://redacted.us-west-2.es.amazonaws.com\" ]
        aws:
          region: \"us-west-2\"
        index: \"data_prepper_unsorted\"

usw2dev-test-dp-ingestion-us-west-2-pipeline-us-west-2:
  source:
    pipeline:
      name: entry-pipeline-us-west-2
  processor:
    - date:
        from_time_received: true
        destination: \"@dp_timestamp\"
    - parse_json:
    - date:
        match:
          - key: date
            patterns: [\"yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'\"]
        destination: \"@timestamp\"
        source_timezone: \"Etc/UTC\"
        destination_timezone: \"Etc/UTC\"
    - delete_entries:
        with_keys: [\"date\", \"s3\"]
  sink:
    - opensearch:
        hosts: [ \"https://redacted.us-west-2.es.amazonaws.com\" ]
        aws:
          region: \"us-west-2\"
        index: \"usw2dev\"
        action: create
        max_retries: 10
        dlq:
          s3:
            bucket: \"data-prepper-ingress-us-west-2-test-dlq\"
            key_path_prefix: \"/%{yyyy}/%{MM}/%{dd}\"
            region: \"us-west-2\"

(Note that several facets like AWS account IDs have been changed.)

The _default route is described in https://docs.opensearch.org/blog/Data-Prepper-2.9.0-is-ready-for-download/

Environment (please complete the following information):

  • OS: docker image opensearchproject/data-prepper:2.11.0
  • Version 2.11.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    Done

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions