Skip to content

[APM] e2e ingest data is failing randomly #62979

@v1v

Description

@v1v

Kibana version:
master branch

commit d78711b3e8d2d072043f1ed250c540426c72f460 (HEAD -> master, upstream/master, origin/master, origin/HEAD)
Author: Walter Rafelsberger <walter@elastic.co>
Date:   Tue Mar 31 10:03:04 2020 +0200

Elasticsearch version:
master branch

{
name: "00a78790aa23",
cluster_name: "docker-cluster",
cluster_uuid: "bIAKGGRTTciu8gd5y2B20g",
version: {
number: "8.0.0-SNAPSHOT",
build_flavor: "default",
build_type: "docker",
build_hash: "b7af8524228891c305b8868ff7f748ec10cf724c",
build_date: "2020-03-30T09:57:48.811880Z",
build_snapshot: true,
lucene_version: "8.5.0",
minimum_wire_compatibility_version: "7.8.0",
minimum_index_compatibility_version: "7.0.0"
},
tagline: "You Know, for Search"
}

Server OS version:
macosx 10.14.3 (18D42)

Browser version:
N/A

Browser OS version:
N/A

Original install method (e.g. download page, yum, from source, etc.):
From source
node --version - v10.19.0
nvm --version - 0.34.0

Description of the problem including expected versus actual behavior:

e2e tests are failing with

Steps to reproduce:

## Prepare the node version
nvm use $(cat .node-version)

## Prepare the required dependencies
cd x-pack/legacy/plugins/apm/e2e
yarn install

## Clean and bootstrap the kibana dependencies
cd -
yarn kbn clean && yarn kbn bootstrap

## Run kibana
node ./scripts/kibana --no-base-path --dev --no-dev-config --config x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

Static mock data (logs: ./tmp/ingest-data.log)
✔ Successfully ingested 96 of 96 events

cat ./x-pack/legacy/plugins/apm/e2e/tmp/ingest-data.log
{"accepted":60,"errors":[{"message":"queue is full"}]}
{"accepted":0,"errors":[{"message":"queue is full"}]}

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

APM-server logs

{"level":"error","timestamp":"2020-04-08T16:43:44.523Z","logger":"request","caller":"middleware/log_middleware.go:95","message":"queue is full","request_id":"1fbfe5e3-d7fd-41bb-82a1-a51b77937d47","method":"POST","URL":"/intake/v2/events","content_length":-1,"remote_address":"pipe","user-agent":"elasticapm-go/1.7.0 go/go1.13.9","response_code":503,"error":"queue is full"}
{"level":"error","timestamp":"2020-04-08T16:43:44.523Z","logger":"request","caller":"middleware/log_middleware.go:95","message":"queue is full","request_id":"73841efbc908e5c8","method":"POST","URL":"/intake/v2/events","content_length":144047,"remote_address":"172.21.0.1","user-agent":"axios/0.19.2","trace.id":"73841efbc908e5c85dfc76c05da2c3b8","transaction.id":"73841efbc908e5c8","response_code":503,"error":"queue is full"}

Expected behavior

x-pack/legacy/plugins/apm/e2e/ingest-data/replay.js should trap the 503 httpcode errors

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions