Skip to content

[metricbeat] migrate mongodb to reporterv2 with error handling#11653

Merged
fearful-symmetry merged 2 commits intoelastic:masterfrom
fearful-symmetry:migration/mb/reporterv2error/mongo
Apr 8, 2019
Merged

[metricbeat] migrate mongodb to reporterv2 with error handling#11653
fearful-symmetry merged 2 commits intoelastic:masterfrom
fearful-symmetry:migration/mb/reporterv2error/mongo

Conversation

@fearful-symmetry
Copy link
Copy Markdown
Contributor

@fearful-symmetry fearful-symmetry commented Apr 4, 2019

See #11374

Not much to add to this one.

@fearful-symmetry fearful-symmetry added Metricbeat Metricbeat Team:Integrations Label for the Integrations team labels Apr 4, 2019
@fearful-symmetry fearful-symmetry self-assigned this Apr 4, 2019
@fearful-symmetry fearful-symmetry requested a review from a team April 4, 2019 18:15
err = errors.New("Unexpected data returned by mongodb")
logger.Error(err)
reporter.Error(err)
m.Logger().Error("Unexpected data returned by mongodb")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a change in behaviour. Previously we also sent an error event now we only log it. I think we should still send the error event.

err = errors.Wrap(err, "Mapping of the event data filed")
logger.Error(err)
reporter.Error(err)
m.Logger().Error(errors.Wrap(err, "Mapping of the event data filed"))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

err = errors.Wrapf(err, "Failed to retrieve stats for db %s", dbName)
logger.Error(err)
continue
m.Logger().Error(errors.Wrapf(err, "Failed to retrieve stats for db %s", dbName))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We lost the continue here and are not reporting the error event anymore.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

reported := reporter.Event(mb.Event{MetricSetFields: data})
if !reported {
//this means the metricset has closed, no point in trying to log anything further
return errors.New("metricset has stopped")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here we should just return nil and not report an error document. Stopping a metricset is not an error.

mongoSession, err := mongodb.NewDirectSession(m.DialInfo)
if err != nil {
logger.Error(err)
reporter.Error(err)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still missing here 😄 same in line 76.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, missed one!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, do we need to add anything there? My understanding is that we only need to add the error reporting back in where we're not actually returning, i.e, inside a loop

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is logger.Error(err) is to log the error and reporter.Error(err) is to actually report an event with error msg. So if we only have a return errors.Wrap(...) here, then we are missing the report an event part.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we return an error, it's not only logged but also reported: https://github.com/elastic/beats/blob/master/metricbeat/mb/module/wrapper.go#L243

Copy link
Copy Markdown
Contributor

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go from my perspective.

logger.Debug("error reporting event")
return
reported := reporter.Event(mb.Event{MetricSetFields: data})
if !reported {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could log on debug level that the metricset was stopped during reporting, but I think that can already be seen in other places, so +1 on this change.

Nit: I preferred the previous 1 line statement ;-)

@fearful-symmetry fearful-symmetry merged commit 3ff87df into elastic:master Apr 8, 2019
@fearful-symmetry fearful-symmetry deleted the migration/mb/reporterv2error/mongo branch April 8, 2019 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Metricbeat Metricbeat Team:Integrations Label for the Integrations team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants