Skip to content

[azure_functions] Parse stringified 'properties' field in Azure Functions app logs#15595

Merged
devamanv merged 9 commits intoelastic:mainfrom
devamanv:azure-functions-invalid-json-fix
Oct 8, 2025
Merged

[azure_functions] Parse stringified 'properties' field in Azure Functions app logs#15595
devamanv merged 9 commits intoelastic:mainfrom
devamanv:azure-functions-invalid-json-fix

Conversation

@devamanv
Copy link
Copy Markdown
Contributor

@devamanv devamanv commented Oct 7, 2025

Proposed commit message

This change updates the Azure Functions ingest pipeline to correctly parse the properties field, which is often received as a stringified JSON. Previously, nested fields inside properties were not parsed and accessible in Elasticsearch.

The pipeline now:

  • Detects if the properties field is a string.
  • Converts single quotes to double quotes to form valid JSON.
  • Parses the string into a proper JSON object.
  • Makes nested fields (e.g., level, appName, message) available for mapping and querying.

This ensures that all fields inside properties are properly indexed and usable in dashboards, alerts, and queries.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • Ensure that changes are tested against the function app that generates invalid JSON
  • Ensure the fields inside the invalid JSON are correctly parsed
  • Ensure error-free ingestion of function app logs in Elasticsearch

How to test this PR locally

  • Spin up the Elastic stack and point your elastic-package to the running Kibana instance
  • Build the package locally and install the package using elastic-package install --zip azure_functions-0.11.0.zip
  • Make sure you're using a hosting plan that generates invalid JSON e.g. Function Premium was observed to exhibit such behavior
  • Navigate to the Discover section, and in the logs-* data view, add the filter data_stream.dataset: azure_functions.functionapplogs. The documents should have the fields under function field, and no pipeline errors.

Related issues

Screenshots

image

@devamanv devamanv self-assigned this Oct 7, 2025
@devamanv devamanv added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] bugfix Pull request that fixes a bug issue Integration:azure_functions Azure Functions labels Oct 7, 2025
@devamanv devamanv marked this pull request as ready for review October 7, 2025 20:22
@devamanv devamanv requested a review from a team as a code owner October 7, 2025 20:22
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@devamanv devamanv requested a review from muthu-mps October 8, 2025 08:46
{
"azure": {
"category": "FunctionAppLogs",
"event_primary_stamp_name": "waws-prod-blu-125",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the event_stamp fields be under azure.function.* or azure.* If these are generic fields then we can have at top level.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These fields are event-specific, and not to a function execution per se. That's why they have been kept at the top level.

@devamanv devamanv requested a review from muthu-mps October 8, 2025 10:40
@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

cc @devamanv

@andrewkroh andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Oct 8, 2025
@devamanv devamanv merged commit b124909 into elastic:main Oct 8, 2025
7 checks passed
@devamanv devamanv deleted the azure-functions-invalid-json-fix branch October 8, 2025 12:11
@elastic-vault-github-plugin-prod
Copy link
Copy Markdown

Package azure_functions - 0.11.0 containing this change is available at https://epr.elastic.co/package/azure_functions/0.11.0/

agithomas pushed a commit to agithomas/integrations that referenced this pull request Oct 30, 2025
tehbooom pushed a commit to tehbooom/integrations that referenced this pull request Nov 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:azure_functions Azure Functions Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants