Contributes to #6748

There are a few activities where lightgbm (the Python package) needs to inspect the output of one or more evaluation metrics on one or more datasets.

For example:

• early stopping
• printing evaluation results
• recording evaluation results in memory (e.g. in a dictionary) for use after training

For train() and other APIs that end up using it, it tracks those using a list of tuples like this (pseudocode):

[
  ({dataset_name}, {metric_name}, {is_higher_better}, {metric_value}).
  ...
]

cv() does something similar. However, its "metric value" is actually a mean of such values taken over all cross-validation folds. Because multiple values are being aggregated, it appends a 5th item with the standard deviation.

[
  ({dataset_name}, {metric_name}, {is_higher_better}, mean({metric_value}), stddev({metric_value}),
  ...
]

Some code in callbacks.py needs to know, given a list of such tuples, whether they were produced by cross-validation or regular train().

To facilitate that while still somewhat preserving the schema for the tuples, the cv() code:

• concatenates the first and second elements into 1
• appends the string literal "cv_agg" to the beginning of the tuple

So e.g. ("valid1", "auc", ...) becomes ("cv_agg", "valid1 auc", ...). That happens here:

https://github.com/microsoft/LightGBM/blob/480600b3afaf2a0a6f32cf417edf9567f625b2c3/python-package/lightgbm/engine.py#L580-L592

Every place dealing with such tuples then needs to deal with that, including splitting and re-combining that second element. Like this:

https://github.com/microsoft/LightGBM/blob/480600b3afaf2a0a6f32cf417edf9567f625b2c3/python-package/lightgbm/callback.py#L416-L418

This proposes changes to remove that, so that the cv() and train() tuples follow a similar schema and all the complexity of splitting and re-combining names can be removed.

It also standardizes on the names from #6749 (comment)

Notes for Reviewers

This change should be completely backwards-compatible, including with user-provided custom metric function. The code paths here are well-covered by tests (as I found out from many failed tests while developing this 😅 ).