[9.1](backport #47956) [azure-eventhub] Support for AMQP-over-WebSocket transport in the processor v2 by mergify[bot] · Pull Request #48120 · elastic/beats

mergify · 2025-12-17T00:05:35Z

Proposed commit message

Support for AMQP-over-WebSocket transport in the azure-eventhub processor v2.

Enterprise users often need to comply with network restrictions, which means using AMQP may not be an option.

In addition to AMQP-over-WebSocket support, this change allows users to run the azure-eventhub input behind an HTTPS proxy.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

No disruptive user impact. The input still defaults to AMQP transport, and unless users switch to WS transport, the behavior is unchanged.

Author's Checklist

Test with an actual proxy
Update the documentation for AMQP-over-WebSocket
Update the documentation HTTPS proxy

How to test this PR locally

I used the following Filebeat config:

# x-pack/filebeat/filebeat.yml
filebeat.inputs:
- type: azure-eventhub
  enabled: true

  # Event Hub configuration
  eventhub: "${EVENTHUB_NAME}"
  connection_string: "${EVENTHUB_CONNECTION_STRING}"
  consumer_group: "${EVENTHUB_CONSUMER_GROUP}"

  # Azure Storage configuration for checkpointing
  storage_account: "${STORAGE_ACCOUNT}"
  storage_account_connection_string: "${STORAGE_ACCOUNT_CONNECTION_STRING}"

  # Optional settings
  processor_version: ${PROCESSOR_VERSION}
  migrate_checkpoint: ${MIGRATE_CHECKPOINT}
  transport: "${TRANSPORT}"

Here are a couple of vscode configuration to test the WebSocket-only and WebSocket with HTTPS proxy scenarios (make sure to replace the placeholders YOUR * GOES HERE with actual values).

WebSocket only

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch Filebeat",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${workspaceFolder}/x-pack/filebeat/main.go",
            "args": [
                "-e",
                "-v",
                "-d",
                "*",
                "--strict.perms=false",
                "--path.home",
                "${workspaceFolder}/x-pack/filebeat",
                "-E",
                "cloud.id=<YOUR CLOUD ID GOES HERE>",
                "-E",
                "cloud.auth=<YOUR USERNAME GOES HERE>:<YOUR PASSWORD GOES HERE>",
                "-E",
                "gc_percent=100",
                "-E",
                "setup.ilm.enabled=false",
                "-E",
                "setup.template.enabled=false",
                "-E",
                "output.elasticsearch.allow_older_versions=true"
            ],
            "env": {
                "EVENTHUB_NAME": "logs",
                "EVENTHUB_CONNECTION_STRING": "<YOUR EVENT HUB CONNECTION STRING GOES HERE>",
                "EVENTHUB_CONSUMER_GROUP": "$Default",
                "STORAGE_ACCOUNT": "<YOUR STORAGE ACCOUNT NAME GOES HERE>",
                "STORAGE_ACCOUNT_KEY": "<YOUR STORAGE ACCOUNT KEY GOES HERE>",
                "STORAGE_ACCOUNT_CONNECTION_STRING": "<YOUR STORAGE ACCOUNT CONNECTION STRING GOES HERE>",
                "PROCESSOR_VERSION": "v2",
                "MIGRATE_CHECKPOINT": "true",
                "TRANSPORT": "websocket"
            }
        }
    ]
}

WebSocket with HTTPS proxy

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch Filebeat",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "${workspaceFolder}/x-pack/filebeat/main.go",
            "args": [
                "-e",
                "-v",
                "-d",
                "*",
                "--strict.perms=false",
                "--path.home",
                "${workspaceFolder}/x-pack/filebeat",
                "-E",
                "cloud.id=<YOUR CLOUD ID GOES HERE>",
                "-E",
                "cloud.auth=<YOUR USERNAME GOES HERE>:<YOUR PASSWORD GOES HERE>",
                "-E",
                "gc_percent=100",
                "-E",
                "setup.ilm.enabled=false",
                "-E",
                "setup.template.enabled=false",
                "-E",
                "output.elasticsearch.allow_older_versions=true"
            ],
            "env": {
                "EVENTHUB_NAME": "logs",
                "EVENTHUB_CONNECTION_STRING": "<YOUR EVENT HUB CONNECTION STRING GOES HERE>",
                "EVENTHUB_CONSUMER_GROUP": "$Default",
                "STORAGE_ACCOUNT": "<YOUR STORAGE ACCOUNT NAME GOES HERE>",
                "STORAGE_ACCOUNT_KEY": "<YOUR STORAGE ACCOUNT KEY GOES HERE>",
                "STORAGE_ACCOUNT_CONNECTION_STRING": "<YOUR STORAGE ACCOUNT CONNECTION STRING GOES HERE>",
                "PROCESSOR_VERSION": "v2",
                "MIGRATE_CHECKPOINT": "true",
                "TRANSPORT": "websocket",
                "HTTP_PROXY": "http://127.0.0.1:9090",
                "HTTPS_PROXY": "http://127.0.0.1:9090"
            }
        }
    ]
}

To test the proxy support, I used mitmproxy, an interactive HTTPS proxy with request inspection:

# Install
brew install mitmproxy  # macOS
# or
pip install mitmproxy

# Run the proxy
mitmproxy -p 9090

Since the Azure services require using an HTTPS proxy, you also need to set up a certificate for mitmproxy. See https://docs.mitmproxy.org/stable/concepts/certificates/ fore more details.

The mitmproxy main view:

Upgrade to AMQP-over-WebSocket:

WebSocket details:

Event hub messages received using AMQP-over-websocket through the proxy:

Related issues

Use cases

This is an automatic backport of pull request #47956 done by [Mergify](https://mergify.com).

mergify · 2025-12-17T00:05:38Z

Cherry-pick of af5d23e has failed:

On branch mergify/bp/9.1/pr-47956
Your branch is up to date with 'origin/9.1'.

You are currently cherry-picking commit af5d23e75.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   NOTICE.txt
	new file:   changelog/fragments/1765280088-azure-eventhub-v2-add-support-for-amqp-over-websocket-and-https-proxy.yaml
	modified:   go.mod
	modified:   go.sum

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   x-pack/filebeat/input/azureeventhub/auth.go
	both modified:   x-pack/filebeat/input/azureeventhub/config.go
	both modified:   x-pack/filebeat/input/azureeventhub/v2_input.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

github-actions · 2025-12-17T00:05:50Z

🤖 GitHub comments

Just comment with:

run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2025-12-17T00:05:54Z

Pinging @elastic/obs-ds-hosted-services (Team:obs-ds-hosted-services)

mergify · 2025-12-19T11:18:39Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b mergify/bp/9.1/pr-47956 upstream/mergify/bp/9.1/pr-47956
git merge upstream/9.1
git push upstream mergify/bp/9.1/pr-47956

…cessor v2 (#47956) Support for AMQP-over-WebSocket transport in the azure-eventhub processor v2. Enterprise users often need to comply with network restrictions, which means using AMQP may not be an option. In addition to AMQP-over-WebSocket support, this change allows users to run the azure-eventhub input behind an HTTPS proxy. (cherry picked from commit af5d23e) # Conflicts: # x-pack/filebeat/input/azureeventhub/auth.go # x-pack/filebeat/input/azureeventhub/config.go # x-pack/filebeat/input/azureeventhub/v2_input.go # Conflicts: # x-pack/filebeat/input/azureeventhub/auth.go # x-pack/filebeat/input/azureeventhub/config.go # x-pack/filebeat/input/azureeventhub/v2_input.go

mergify · 2025-12-22T05:03:12Z

This pull request has not been merged yet. Could you please review and merge it @zmoog? 🙏

pierrehilbert · 2025-12-22T07:57:41Z

@zmoog Don't forget this PR please.

zmoog · 2025-12-22T08:03:15Z

@zmoog Don't forget this PR please.

Thanks for the heads up, but I'm not going to forget these PRs: I promised this feature to some users.

mergify bot added backport conflicts There is a conflict in the backported pull request labels Dec 17, 2025

mergify bot requested review from a team as code owners December 17, 2025 00:05

mergify bot assigned zmoog Dec 17, 2025

mergify bot requested review from mauri870 and removed request for a team December 17, 2025 00:05

mergify bot requested a review from faec December 17, 2025 00:05

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 17, 2025

mergify bot mentioned this pull request Dec 17, 2025

[azure-eventhub] Support for AMQP-over-WebSocket transport in the processor v2 #47956

Merged

9 tasks

github-actions bot added Team:obs-ds-hosted-services Label for the Observability Hosted Services team input:azure-eventhub labels Dec 17, 2025

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Dec 17, 2025

zmoog force-pushed the mergify/bp/9.1/pr-47956 branch from 13fa47c to ceff8e8 Compare December 19, 2025 16:35

zmoog approved these changes Dec 22, 2025

View reviewed changes

zmoog merged commit 2953688 into 9.1 Dec 22, 2025
210 of 211 checks passed

zmoog deleted the mergify/bp/9.1/pr-47956 branch December 22, 2025 09:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[9.1](backport #47956) [azure-eventhub] Support for AMQP-over-WebSocket transport in the processor v2#48120

[9.1](backport #47956) [azure-eventhub] Support for AMQP-over-WebSocket transport in the processor v2#48120
zmoog merged 1 commit into9.1from
mergify/bp/9.1/pr-47956

mergify bot commented Dec 17, 2025

Uh oh!

mergify bot commented Dec 17, 2025

Uh oh!

github-actions bot commented Dec 17, 2025

Uh oh!

elasticmachine commented Dec 17, 2025

Uh oh!

mergify bot commented Dec 19, 2025

Uh oh!

mergify bot commented Dec 22, 2025

Uh oh!

pierrehilbert commented Dec 22, 2025

Uh oh!

zmoog commented Dec 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

mergify bot commented Dec 17, 2025

Proposed commit message

Checklist

Disruptive User Impact

Author's Checklist

How to test this PR locally

Related issues

Use cases

Uh oh!

mergify bot commented Dec 17, 2025

Uh oh!

github-actions bot commented Dec 17, 2025

🤖 GitHub comments

Uh oh!

elasticmachine commented Dec 17, 2025

Uh oh!

mergify bot commented Dec 19, 2025

Uh oh!

mergify bot commented Dec 22, 2025

Uh oh!

pierrehilbert commented Dec 22, 2025

Uh oh!

zmoog commented Dec 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants