Issue

User is getting the caller IP address of type keyword for both the audit and the sign-in logs.

Summary

• As a first step I tried to understand what happens in the ingest pipeline. If the log source has IP address from this fields then it is assumed to be the same IP address and stored in the source.ip field for sign-in logs.
• Listing the IP fields of sign-in logs
  • signinlogs.properties.ipaddress
  • signinlogs.properties.ip_address
  • signinlogs.caller_ip_address
• Listing the IP fields of audit logs
  • azure.auditlogs.callerIpAddress

Observations

• The caller_ip_address in the sign-in logs should be replaced with the signinlogs.callerIpAddress or add one more field to process the callerIpAddress field.
• The log source received from the user has different IP for callerIpAddress and signinlogs.properties.ipAddress. We need to store both the values in different fields.
• While investigating further into the issue found few IP address fields defined in the fields.yml with the type keyword for audit logs. The data type for this fields has to be updated.
  • initiated_by.user.ipAddress
  • target_resources.*.ip_address
• By understanding the source log from the user we have differences in the fields that we process right now.

Audit Log

• The audit log stream has been build by using the logs initiated_by the app source. The provided log is initiated_by user source
• We need to validate the source and include the fields needed for the audit log initiated_by user.
• Adding reference to the log source initiated_by user.

Solution:
-> Issue in elastic/beats: #38834
-> PR in elastic/beats: #38835

Sign-in log

• Similarly we have couple of fields which is available in the provided source that is not defined in our datastream.
• If azure allows to add custom fields to the logs then we need to add support to the standard fields and update the documentation.

Solution:
-> Issue in elastic/integrations: elastic/integrations#8829
-> PR in elastic/integrations: elastic/integrations#8813