Skip to content

ARP protection#1319

Merged
brianmcgillion merged 1 commit intotiiuae:mainfrom
mbssrc:arp-protect
Sep 2, 2025
Merged

ARP protection#1319
brianmcgillion merged 1 commit intotiiuae:mainfrom
mbssrc:arp-protect

Conversation

@mbssrc
Copy link
Copy Markdown
Collaborator

@mbssrc mbssrc commented Aug 14, 2025

Description of Changes

Enable static ARP in the internal network, to prevent ARP spoofing and
subsequent MITM attacks.

  • additional firewall (ebtables) rules on internal bridge, filtering out
    arp traffic; and required kernel modules
  • sysctl settings to configure arp behaviour on bridge and vm internal
    interfaces
  • splitting host networking configuration, allowing to later remove host
    networking without removing the internal bridge
  • adding static entries for all known hosts
  • adding flag for external host networking, allowing to not activate
    interfaces on the host by default and disable internal host nat

Type of Change

  • New Feature
  • Bug Fix
  • Improvement / Refactor

Related Issues / Tickets

Checklist

  • Clear summary in PR description
  • Detailed and meaningful commit message(s)
  • Commits are logically organized and squashed if appropriate
  • Contribution guidelines followed
  • Ghaf documentation updated with the commit - https://tiiuae.github.io/ghaf/
  • Author has run make-checks and it passes
  • All automatic GitHub Action checks pass - see actions
  • Author has added reviewers and removed PR draft status

Testing Instructions

Applicable Targets

  • Orin AGX aarch64
  • Orin NX aarch64
  • Lenovo X1 x86_64
  • Dell Latitude x86_64

Installation Method

  • Requires full re-installation
  • Can be updated with nixos-rebuild ... switch
  • Other:

Test Steps To Verify:

  1. Confirm ARP spoofing works on current main
  2. (optional) run sudo arp net-vm in chrome-vm before and during spoofing to see MAC address change
  3. Apply patch and confirm ARP spoofing does not work (broadcast traffic will still be visible)
  4. (optional) run sudo arp net-vm in chrome-vm before and during spoofing to see no MAC address change
  5. Check for any network performance impact

Instructions:

ssh ghaf@zathura-vm
sudo su
nix-shell -p bettercap
arp -s 192.168.100.1 02:ad:00:00:00:01
iptables -P FORWARD ACCEPT

bettercap
> net.probe on
> set arp.spoof.targets 192.168.100.102
> arp.spoof on
> net.sniff on

Then open chrome browser on desktop, go to a website and observe traffic in zathura-vm.

@enesoztrk
Copy link
Copy Markdown
Contributor

enesoztrk commented Aug 14, 2025

I was considering enabling libvirt's built-in filtering chains, which would prevent spoofing activities between VMs, if integration with Nix is possible

What do you think about it?

@mbssrc
Copy link
Copy Markdown
Collaborator Author

mbssrc commented Aug 18, 2025

I was considering enabling libvirt's built-in filtering chains, which would prevent spoofing activities between VMs, if integration with Nix is possible

* no-arp-spoofing

* no-ip-spoofing

* no-ipv6-spoofing

* clean-traffic
  Reference: https://libvirt.org/formatnwfilter.html#filtering-chains

What do you think about it?

Would be great to use a generic mechanism, but that would imply running libvirtd?

@enesoztrk
Copy link
Copy Markdown
Contributor

I haven't done thorough research yet. However, it may require libvrtd to be running on the host. We could merge this PR and investigate libvirtd later.

@mbssrc mbssrc marked this pull request as draft August 19, 2025 10:42
@mbssrc mbssrc marked this pull request as ready for review August 24, 2025 19:00
@brianmcgillion brianmcgillion added the Needs Testing CI Team to pre-verify label Aug 24, 2025
@mbssrc mbssrc requested a review from brianmcgillion August 25, 2025 08:24
@leivos-unikie
Copy link
Copy Markdown
Contributor

leivos-unikie commented Aug 25, 2025

Tested on Lenovo-X1

  • instructed arp spoofing test:
    succeeds with ghaf mainline / fails with arp-protect branch

  • ci-test-automation ipspoofing test:
    succeeds with ghaf mainline / fails with arp-protect branch
    With this PR image connection to stealer vm (chrome-vm) is completely lost during the ipspoofing test. Had to modify the test to save the outputs to /home/appuser/Unsafe share/stolen.txt to be able to check them manually after the test.

  • Checked that all vms have internet connection

  • Run ci-test-automation network test (iperf) few times, didn't notice clear change compared to mainline

  • Desktop and apps seem normal

@leivos-unikie leivos-unikie added the Tested on Lenovo X1 Carbon This PR has been tested on Lenovo X1 Carbon label Aug 25, 2025
@leivos-unikie
Copy link
Copy Markdown
Contributor

Tested arp spoofing on Orin AGX as below

Set up WiFi connection on net-vm
sudo nmcli dev wifi con "SSID" --ask

ssh ghaf@ghaf-host
sudo su
nix-shell -p bettercap
arp -s 192.168.100.1 02:ad:00:00:00:01
iptables -P FORWARD ACCEPT
bettercap
> net.probe on
> set arp.spoof.targets 192.168.100.103
> arp.spoof on
> net.sniff on

Then connected to admin-vm via ssh and sent some http requests, for example:
curl -X GET http:/www.google.com

--> bettercap reported:

HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
Accept-Ranges: none
Date: Mon, 25 Aug 2025 10:48:18 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Server: gws
Set-Cookie: AEC=AVh_V2iJ_tSinWfcMV40v-RXxneOg3PW8hJ8P0G-4yKodwbEzUqkOd6XIA; expires=Sat, 21-Feb-2026 10:48:18 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax
Vary: Accept-Encoding
Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-KW2FkHwRQ4paTH9ajPIagg' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
X-Xss-Protection: 0

arp spoofing succeeded both with ghaf mainline image and with image built from this PR

@leivos-unikie leivos-unikie added the bug on Orin AGX Cross Issues found on NVIDIA Jetson AGX Orin cross-compiled while checking this PR label Aug 25, 2025
@mbssrc
Copy link
Copy Markdown
Collaborator Author

mbssrc commented Aug 25, 2025

Tested arp spoofing on Orin AGX as below
...

The host network runs on the bridge, so it can by default see all the traffic - no need for arp spoofing. If you run net.probe on and net.sniff on you will see the traffic between the VMs (like your curl) already without spoofing.

What you could do to test the setup on orin:

  1. Watch the arp entries in admin-vm:
watch -n 1 arp
  1. Spoof from ghaf-host:
(sudo su)
nix-shell -p bettercap
arp -s 192.168.100.1 02:ad:00:00:00:01
iptables -P FORWARD ACCEPT
bettercap
# set the target as the admin-vm, and start spoofing
> set arp.spoof.targets 192.168.100.3 
> arp.spoof on

In main, you should see in the admin-vm arp table that the net-vm MAC changes to the ghaf-host MAC, with the patch nothing should happen. You can also toggle arp.spoof on and arp.spoof off on the fly to see the difference.

If you want to dig more into it, you can include the tcpdump package in the build (on-the-fly install will probably not work in admin-vm) and monitor arp traffic on ethint0 (target VM) and virbr0 (host bridge). Also, you could try to spoof the host from admin-vm, but again need to include the bettercap package in the build.

@leivos-unikie
Copy link
Copy Markdown
Contributor

Tested arp spoofing on Orin AGX as below
...

The host network runs on the bridge, so it can by default see all the traffic - no need for arp spoofing. If you run net.probe on and net.sniff on you will see the traffic between the VMs (like your curl) already without spoofing.

What you could do to test the setup on orin:

1. Watch the arp entries in admin-vm:
watch -n 1 arp
2. Spoof from ghaf-host:
(sudo su)
nix-shell -p bettercap
arp -s 192.168.100.1 02:ad:00:00:00:01
iptables -P FORWARD ACCEPT
bettercap
# set the target as the admin-vm, and start spoofing
> set arp.spoof.targets 192.168.100.3 
> arp.spoof on

In main, you should see in the admin-vm arp table that the net-vm MAC changes to the ghaf-host MAC, with the patch nothing should happen. You can also toggle arp.spoof on and arp.spoof off on the fly to see the difference.

If you want to dig more into it, you can include the tcpdump package in the build (on-the-fly install will probably not work in admin-vm) and monitor arp traffic on ethint0 (target VM) and virbr0 (host bridge). Also, you could try to spoof the host from admin-vm, but again need to include the bettercap package in the build.

I tested again on Orin AGX.

Toggling net-vm MAC address shown on ghaf-host works in main but nothing happens with your branch as you explained.

For some reason I was not able to see any http request traffic to ghaf-host with (modified) main image when running bettercap on admin-vm. Maybe if I had set up another vm for running http request it would have worked.

Anyways, the arp protection works as you described both on x86 and ARM device.

@leivos-unikie leivos-unikie added Tested on Orin AGX Cross This PR has been tested on NVIDIA Jetson AGX Orin cross-compiled and removed bug on Orin AGX Cross Issues found on NVIDIA Jetson AGX Orin cross-compiled while checking this PR Needs Testing CI Team to pre-verify labels Aug 26, 2025
Enable static ARP in the internal network, to prevent ARP spoofing and
subsequent MITM attacks.

Changes:
- additional firewall (ebtables) rules on tap interfaces, filtering out
  arp traffic
- sysctl settings to configure arp behaviour on bridge and vm internal
  interfaces
- splitting host networking configuration, allowing to later remove host
  networking without removing the internal bridge
- adding static entries for all known hosts
- adding flag for external host networking, allowing to not activate
  interfaces on the host by default and disable internal host nat

Signed-off-by: Manuel Bluhm <manuel@ssrc.tii.ae>
@mbssrc
Copy link
Copy Markdown
Collaborator Author

mbssrc commented Aug 28, 2025

Changes:

  • rebased with latest firewall changes
  • removed additional kernel modules; changed the logic to filter arp traffic on tap-* interfaces. This works on all targets without additional changes.

useful command on the host to verify the effectiveness: ebtables -L --Lc showing the package counter for the rules, can be triggered via arping or actual spoofing. tcpdump shows traffic only on originating interface, both internal bridge and other VM interfaces do not show traffic (tcpdump -i (ethint0/virbr0/tap-<vm>) -n 'arp')

Copy link
Copy Markdown
Contributor

@enesoztrk enesoztrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good. It would be good to test network services after PR's test steps.

@mbssrc mbssrc added the Needs Testing CI Team to pre-verify label Aug 28, 2025
@leivos-unikie
Copy link
Copy Markdown
Contributor

Tested on Lenovo-X1 again after the latest changes

  • arp spoofing does not succeed (ebtables -L --Lc shows the effect of spoofing at firewall)
  • Updating system time form internet works in ghaf-host and in every VM (NTP service)
  • systemctl status does not reveal any services failing in VMs

@leivos-unikie
Copy link
Copy Markdown
Contributor

Test again also on Orin AGX after the latest changes

  • arp spoofing does not succeed (ran bettercap in admin-vm)
  • Running ebtables -L --Lc in ghaf-host shows the effect of spoofing at firewall
  • Updating system time form internet works in ghaf-host and in VMs (NTP service)
  • systemctl status does not reveal any services failing in VMs

@leivos-unikie leivos-unikie removed the Needs Testing CI Team to pre-verify label Sep 2, 2025
@brianmcgillion brianmcgillion merged commit 2a48452 into tiiuae:main Sep 2, 2025
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tested on Lenovo X1 Carbon This PR has been tested on Lenovo X1 Carbon Tested on Orin AGX Cross This PR has been tested on NVIDIA Jetson AGX Orin cross-compiled

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants