As discussed in elastic/integrations#2076 (comment), ECS fields could be dimensions too.

One option could be to add the dimension flag directly in ECS, and import it from there. But I think that the same ECS field may be a dimension in some integrations, but not in others. As mentioned in #215, the set of dimensions should be minimal for a given data stream, so an integration including many ECS dimension fields may actually need only a subset of them to identify individual time series. For example host.id can be needed to identify time series in the system integration, but not to identify time series of pods in the kubernetes integration.

So we would need to support something like this:

- external: ecs
  name: container.id
  dimension: true

Current implementation doesn't work because it checks the type and it is not checking now external fields. It fails with this error:

Error: checking package failed: linting package failed: found 2 validation errors:
   1. file "/home/jaime/gocode/src/github.com/elastic/integrations/packages/kubernetes/data_stream/container/fields/ecs.yml" is invalid: field "container.id" of type  can't be a dimension, allowed types for dimensions: constant_keyword, keyword, long, integer, short, byte, double, float, half_float, scaled_float, unsigned_long, ip

Changes required:

• Semantic validator should accept the previous syntax. Ideally checking that the external field is of one of the allowed types. Type checking is not done by now to avoid needing to resolve the external field on validation time.
  • Allow defining an external field as dimension #248
• Elastic package should keep the dimension setting when injecting external fields. It will allow to override settings of external fields, except for some of them that will be protected.
  • Allow to override attributes from external fields elastic-package#575

@mtojek do you see problems in this plan? ^