-
Notifications
You must be signed in to change notification settings - Fork 313
[BUG] _default route no longer seems to exist in 2.11.0 #5763
Copy link
Copy link
Closed
Labels
bugSomething isn't workingSomething isn't working
Milestone
Description
Describe the bug
I updated a data prepper installation running 2.10.3 to 2.11.0 and the new instance fails to start after displaying an error message like:
2025-06-05T17:15:44,937 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataPrepper' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.11.0.jar!/org/opensearch/dataprepper/core/DataPrepper.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.core.DataPrepper]: Constructor threw exception; nested exception is org.opensearch.dataprepper.pipeline.parser.InvalidPipelineConfigurationException: The following routes do not exist in pipeline "entry-pipeline-us-west-2": [_default]. Configured routes include [usw2dev-test-dp-ingestion-us-west-2_route-us-west-2]
It seems like this feature addition mentioned in the 2.11.0 release notes could be related. #5106
To Reproduce
This data-prepper-config.yaml file works under 2.10.3 but not under 2.11.0:
entry-pipeline-us-west-2:
source:
s3:
acknowledgments: true
notification_type: \"sqs\"
compression: gzip
codec:
newline:
sqs:
queue_url: \"https://sqs.us-west-2.amazonaws.com/1234567890/data-prepper-ingress-us-west-2-test\"
maximum_messages: 10
visibility_timeout: \"60s\"
bucket_owners:
usw2dev-test-dp-ingestion-us-west-2: 1234567890
aws:
region: \"us-west-2\"
processor:
route:
- usw2dev-test-dp-ingestion-us-west-2_route-us-west-2: '/s3/bucket == \"usw2dev-test-dp-ingestion-us-west-2\"'
sink:
- pipeline:
name: usw2dev-test-dp-ingestion-us-west-2-pipeline-us-west-2
routes:
- usw2dev-test-dp-ingestion-us-west-2_route-us-west-2
- pipeline:
name: default-pipeline-us-west-2
routes:
- _default
default-pipeline-us-west-2:
source:
pipeline:
name: entry-pipeline-us-west-2
sink:
- opensearch:
hosts: [ \"https://redacted.us-west-2.es.amazonaws.com\" ]
aws:
region: \"us-west-2\"
index: \"data_prepper_unsorted\"
usw2dev-test-dp-ingestion-us-west-2-pipeline-us-west-2:
source:
pipeline:
name: entry-pipeline-us-west-2
processor:
- date:
from_time_received: true
destination: \"@dp_timestamp\"
- parse_json:
- date:
match:
- key: date
patterns: [\"yyyy-MM-dd'T'HH:mm:ss.SSSSSS'Z'\"]
destination: \"@timestamp\"
source_timezone: \"Etc/UTC\"
destination_timezone: \"Etc/UTC\"
- delete_entries:
with_keys: [\"date\", \"s3\"]
sink:
- opensearch:
hosts: [ \"https://redacted.us-west-2.es.amazonaws.com\" ]
aws:
region: \"us-west-2\"
index: \"usw2dev\"
action: create
max_retries: 10
dlq:
s3:
bucket: \"data-prepper-ingress-us-west-2-test-dlq\"
key_path_prefix: \"/%{yyyy}/%{MM}/%{dd}\"
region: \"us-west-2\"
(Note that several facets like AWS account IDs have been changed.)
The _default route is described in https://docs.opensearch.org/blog/Data-Prepper-2.9.0-is-ready-for-download/
Environment (please complete the following information):
- OS: docker image opensearchproject/data-prepper:2.11.0
- Version 2.11.0
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working
Type
Projects
Status
Done