Pinging @elastic/secops

Auditbeat's auditd module already defines fields like user.sgid. It would be better to reuse the existing fields this way users can query and get all data that has the given user ID value. You'll need move fields from auditd up to the field.common.yml.

@andrewkroh You mean extend the top-level user field, right? I don't know about that - to me it feels that euid/suid/egid/sgid are more properties of the process entity, not the user (technically, they're also different users/accounts).

I took a look at the auditd module's sample data - it does not look ECS compliant to me? Fields like user.name_map and user.uid (instead of user.id) are not in ECS, and are not correctly documented in the fields.yml either (they're documented under the auditd.user namespace, but are actually in the top-level user object). That should not be I think? In Metricbeat and the Auditbeat system module, the system tests call Libbeat's assert_fields_are_documented(evt) to make sure that does not happen. Maybe this is something we should do for auditd as well?

I don't know about that - to me it feels that euid/suid/egid/sgid are more properties of the process entity, not the user (technically, they're also different users/accounts).

I could see how they would be part of the user object if that one is treated more like "any user information" rather than "the user entity". I wonder what it should look like then? The challenge is that at the moment ECS assumes the user fields to represent a single user, with one id, one name, one email, etc. And now we have multiple values.

If we follow Auditd, it would probably be:

"user": {
  "id": "1000",
  "name": "elastic",
  "euid": "1000",
  "suid": "1000",
  "name_map": {
    "euid": "elastic",
    "suid": "elastic"
  }

But I wonder, what if we ever wanted add more information than the name? Would we need more maps?

Maybe it could allow nesting, e.g.:

"user": {
  "id": "1000",
  "name": "elastic",
  "email": "elastic@localhost",    // example additional field
  "created": "2019-01-01 19:37:56" // example additional field
  "effective": { // Nested user object
    "id": "1000",
    "name": "elastic",
    "email": "elastic@localhost",    // example additional field
    "created": "2019-01-01 19:37:56" // example additional field
  },
  "saved": { // Nested user object
    "id": "1000",
    "name": "elastic",
    "email": "elastic@localhost",    // example additional field
    "created": "2019-01-01 19:37:56" // example additional field
  }
}

The group object is also affected, but would probably just follow whatever we do for user.

Anyway, if this is a change to the ECS user object, maybe this discussion has to happen in the ECS repo?

Yeah I addressed these fields under user in my review of Auditbeat/auditd here: #10111.

Waiting on input from Nic on it before bugging you guys. But my conclusion over in that issue is that despite not being in ECS (yet?), I don't think it's a problem. Read up over there for my thinking on this.

@webmat I see that in #10111 we have the group IDs under user as well: user.gid, user.egid, etc.

Whereas here we concluded to have them under group.

I'm confused now - I think we should have one or the other, and ECS should specify which.

Personally, I like the idea of having all user information under user and not have a top-level group object at all.

To follow up on the discussion we've had elsewhere, I would also recommend populating group.id with user.egid. This will help correlation with other places that are using group.id this way.

I agree that ECS will need to:

• define how to represent user privilege escalation
• clarify or review whether a user's group ID, when seen on an event belongs at group.id or nested under the user. This work will be related to the privilege escalation work. Just like with user IDs, multiple group IDs may have to be considered before determining the effective group ID

@cwurm I also think you mentioned that the sets of IDs represented here for privilege escalation is different than what has been happening in the new system module. I agree we would benefit from harmonizing this. However since this isn't in ECS yet and we can't tackle this before FF, I would say it's a decision that is up to you and Andrew on whether to move the fields around on one of the modules to align with the other.

Although you made the point that these details may make more sense under process. than under user., which also makes sense. Perhaps we touch neither of them for now?

Oops that first paragraph was meant as a recommendation for #10111. You're already doing this here :-) I have too many PRs to review LOL

Actually this whole comment was written for #10111 :-)

I've updated to the nested user format proposed in #10111 (comment) and rebased on master.

It would be nice to have resolved user and group names for effective and saved UIDs and GIDs (user.effective.name, user.effective.group.name, user.saved.name, user.saved.group.name), but I didn't want to do it here without a cache in place for those lookups. Something for a follow-up though.

Agree with @andrewkroh.

However this PR depends on #10275

jenkins, test this