Skip to content

[BUG] IP Mapping doesn't work #649

@Hazarth

Description

@Hazarth

Have you read through available documentation, open Github issues and Github Q&A Discussions?

Yes

System information

Your moosefs version and its origin (moosefs.com, packaged by distro, built from source, ...).

All were installed using the official instructions. Key rings added and installed via apt
Master: 4.57.5
Chunkserver: 4.57.5
Client: 4.57.5

Operating system (distribution) and kernel version.

Master: Ubuntu 24.04.2 LTS (6.8.0-52-generic)
Chunkserver: 22.04.5 LTS (5.15.0-134-generic)
Client: 24.04.2 LTS (6.8.0-55-generic)

Hardware / network configuration, and underlying filesystems on master, chunkservers, and clients.

Master:

  • ext4, msfmaster only

Chunkserver:

  • chunk drive is XFS, full block device is used
  • a client is also mounted here on an ext4
  • only one chunkserver for now, currently just building and testing architecture with moosefs

Client:

  • personal PC, ext4

Only HOSTS file used and duplicated on all machines:

10.5.0.1 mfsmaster
10.5.0.2 chunkserver01

How much data is tracked by moosefs master (order of magnitude)?

  • All fs objects: 27 files, 14 folders
  • Total space: 466GiB
  • Free space: 455GiB
  • RAM used: 284MiB
  • last metadata save duration: never

Describe the problem you observed.

I'm working with 1 cloud VPS and 2 home networks which I have all configured to use Wireguard as a VPN. The master is located at 10.5.0.1, the chunk server on 10.5.0.2 and my personal PC on 10.5.0.100. Mounting using sudo mfsmount /mnt/mfs -H 10.5.0.1 and I also have 10.5.0.1 mfsmaster added to my hosts file, so sudo mfsmount /mnt/mfs also works, but I'm trying to be explicit to simplify things.

This works just fine and when writing or reading all the communication is correctly routed through the VPN. However I'm also located in the same LAN that my chunkserver actually is, and I can reach it with the IP 192.168.1.59 from my local network. I want the master server to map the IP from VPN to my Local network when talking to the client and it seems the mfsipmap.cfg should be the perfect solution to this.

I configured my mfsipmap.cfg like this:

10.5.0.100
10.5.0.2 : 192.168.1.59

My understanding after reading the manpages is that this should mean that for the client with IP 10.5.0.100, the ip 192.168.1.59 should be the target if the chunkserver accessed is 10.5.0.2.

I switched that the file is loaded from /etc/mfs/mfsipmap.cfg (if I rename it or move it, mfsmaster will log that it's unable to find the file, but no such log if the file is present, so I assume it's loading it)

I also increased logging by setting SYSLOG_MIN_LEVEL to "DEBUG"

I restarted mfsmaster using sudo mfsmaster restart

I tail'd the syslogs and can confirm that my client is connecting with the IP "10.5.0.100" correctly.

I then re-mounted (and even restarted) MFS on my client, and monitored the network communication using "iftop" on both my VPN interface and my network interface, however upon accessing the MFS it always seems to be using the VPN IP at 10.5.0.2 and never even attempts connection to the local 192.168.1.59 IP.

As it is now, it's still usable for me, it's just not the most efficient routing path and the writes are kinda slow and laggy due to this (however reads are always sufficient).

No other topology, remap or multilan configuration is used, only a plain clean install of mfsmaster and the mfsipmap.cfg file with the lines as described above

Can you reproduce it? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

I assume any basic 3 machine (master/chunkserver/client) setup with IP mapping will do. Currently I think that IP mapping is broken completely.

  • I tried restarting everything
  • I tried enabling other MULTILAN_* features like BITS and CLASSES to see if they are required to load IPMAP, no success
  • I tried avoiding the VPN completely and using public IPs, no success, the FS mounted correctly via the masters public access (confirmed by watching the logs the client IP matches the expected public IP), but upon reading a media file it ended up streaming over VPN anyway
  • I tried routing on local machine via hosts file by changing the "chunkserver01" hostname to local LAN IP "192.168.1.59", no success (I figured that master is sending target IPs instead of target hostnames at this point)
  • I checked the release notes and manpages to make sure IP MAP is not Pro version only

Include any warning/errors/backtraces from the system logs.

No warnings and useful logs are generated, as technically everything works fine, just routing is not happening as expected

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions