Skip to content

Enable DNS Lookups for CIFS Volumes#39250

Merged
thaJeztah merged 1 commit intomoby:masterfrom
shuchow:706-cifs-lookup
Sep 3, 2019
Merged

Enable DNS Lookups for CIFS Volumes#39250
thaJeztah merged 1 commit intomoby:masterfrom
shuchow:706-cifs-lookup

Conversation

@shuchow
Copy link

@shuchow shuchow commented May 21, 2019

fixes docker/cli#706

This comes from an old suggestion (docker/cli#706 (comment)) on an issue we were having and has since popped up again. For NFS volumes, Docker will do an IP lookup on the volume name. This is not done for CIFS volumes, which forces you to add the volume via IP address instead. This change will enable the IP lookup also for CIFS volumes.

Signed-off-by: Shu-Wai Chow shu-wai.chow@seattlechildrens.org

- What I did

Add an IP address lookup for CIFS network volumes.

- How I did it

Change the special case triggered by an if statement to a switch cause triggered by both nfs and cifs volumes. This should enable future cases if necessary.

- How to verify it

Add a CIFS volume using a name instead of an IP Address.

- Description for the changelog

Enable DNS Lookups for CIFS Volumes

- A picture of a cute animal (not mandatory but encouraged)

This comes from an old suggestion (docker/cli#706 (comment)) on an issue we were having and has since popped up again.  For NFS volumes, Docker will do an IP lookup on the volume name.  This is not done for CIFS volumes, which forces you to add the volume via IP address instead.  This change will enable the IP lookup also for CIFS volumes.

Signed-off-by: Shu-Wai Chow <shu-wai.chow@seattlechildrens.org>
@thaJeztah thaJeztah added area/volumes Volumes kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. status/2-code-review and removed status/0-triage labels May 22, 2019
@thaJeztah
Copy link
Member

for reference; original implementation for nfs; #27329 (and related issue: #26639)

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

ping @cpuguy83

@codecov
Copy link

codecov bot commented May 22, 2019

Codecov Report

❗ No coverage uploaded for pull request base (master@12b837e). Click here to learn what that means.
The diff coverage is 33.33%.

@@            Coverage Diff            @@
##             master   #39250   +/-   ##
=========================================
  Coverage          ?   37.06%           
=========================================
  Files             ?      612           
  Lines             ?    45488           
  Branches          ?        0           
=========================================
  Hits              ?    16861           
  Misses            ?    26340           
  Partials          ?     2287

@cpuguy83
Copy link
Member

Did this get tested? Does the cifs module really use the same weird addr=:<addr> syntax?

@shuchow
Copy link
Author

shuchow commented May 22, 2019

Re: For Testing:

The Docker documentation will have to be updated since there is no current section on CIFS. We're able to create and mount CIFS volumes using the same syntax as NFS.

docker volume create
--driver local
--opt type=cifs
--opt addr=//cifs_volume_name/path
--opt o=username=USERNAMEpassword=PASSWORD
volume_name

From the Docker documentation on NFS (https://docs.docker.com/engine/reference/commandline/volume_create/):

docker volume create --driver local
--opt type=nfs
--opt o=addr=192.168.1.1,rw
--opt device=:/path/to/dir
foo

The "addr:" syntax is part of compose.

@thaJeztah
Copy link
Member

Hm, not sure that would work with the current code though; if addr=//cifs_volume_name/path, the current code will try to resolve //cifs_volume_name/path, which is not a valid hostname, so some pre-parsing will have to be done

ipAddr, err := net.ResolveIPAddr("ip", addrValue)

@shuchow
Copy link
Author

shuchow commented May 22, 2019

Sorry, copied/pasted an older command (which also works). The same exact syntax as nfs also works for cifs with the path broken out:

docker volume create
--driver local
--opt type=cifs
--opt addr=volume_name
--opt device=:/dir/another_dir
--opt o=username=USERNAME,password=PASSWORD
volume_name

The getAddress function in local.go is just looking for "addr=" for the volume_name, so that should be fine. I'm trying to dive into how CIFS is parsing out the device option for the path and resolving the ":", but it's definitely working on my end.

@Drewster727
Copy link

Hi guys-- What's the status on this? I'm running into this problem and hoping these changes get merged in.

@shuchow
Copy link
Author

shuchow commented Jun 7, 2019

Hi guys-- What's the status on this? I'm running into this problem and hoping these changes get merged in.

Hey Drew. I was going to ask the main contributors what more I can do for testing this. I don't have access to another CIFS environment to verify that the same mounting syntax for NFS works, nor have I had the chance to deep dive into the volume creation to see how it handles CIFS. I'd be happy to add more to the PR if necessary.

@thaJeztah
Copy link
Member

ping @cpuguy83 ptal

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

it would be nice to update the docs to show the newly available syntax.

Copy link
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still LGTM

let's merge 👍 thanks for being so patient!!

@thaJeztah thaJeztah merged commit a114a2c into moby:master Sep 3, 2019
@thaJeztah
Copy link
Member

@shuchow for the documentation; perhaps a block similar to the NFS example on this page; https://docs.docker.com/storage/volumes/#create-a-service-which-creates-an-nfs-volume

(source file for that is in this repository: https://github.com/docker/docker.github.io/blob/master/storage/volumes.md)

And the reference section for volume create; https://docs.docker.com/engine/reference/commandline/volume_create/#driver-specific-options

(source file for that is in the docker/cli repository: https://github.com/docker/cli/blob/master/docs/reference/commandline/volume_create.md#driver-specific-options)

@shuchow
Copy link
Author

shuchow commented Sep 3, 2019

@thaJeztah Great, thanks for the links. I'll send a PR for the documentation in a bit.

@cpuguy83
Copy link
Member

cpuguy83 commented Sep 3, 2019

Note that these options are passed directly to the kernel module for this fs type.
addr is used by nfs, I was just checking to make sure that cifs does the same thing.

Are we sure this actually works?
Just testing against mount.cifs I see a mount system call like so:

mount("\\\\1.2.3.4\\foo", ".", "cifs", 0, "ip=1.2.3.4,unc=\\\\1.2.3.4\\foo,use"...)

for this command:

mount.cifs -o user=foo,password=bar \\\\1.2.3.4\\foo /tmp

So, based purely on what I'm seeing in the ubuntu mount.cifs we need to be looking at a totally different option for cifs (as compared to nfs).

@thaJeztah thaJeztah added this to the 20.03.0 milestone Apr 2, 2020
@na4ma4
Copy link

na4ma4 commented Sep 22, 2020

There's also a prefixpath added if the UNC goes beyond the share name.

eg. //fileserver/backup/daily turns into unc=\\fileserver\backup,prefixpath=daily (and ip=, etc.)

$ sudo ./mount.cifs //fileserver/backup/rolling /backup -o 'user=foo,pass=bar'
mount(//fileserver/backup/rolling, ".", cifs, (null), ip=192.168.86.50,unc=\\fileserver\backup,user=foo,prefixpath=rolling,pass=bar)

@spinosae
Copy link

when I tried this on Docker 20.10.2, for this syntax

docker volume create
--driver local
--opt type=cifs
--opt addr=volume_name
--opt device=:/dir/another_dir
--opt o=username=USERNAME,password=PASSWORD
volume_name

I get an error during creation of volume: invalid option: "addr"

For the other syntax:

docker volume create --driver local
--opt type=cifs
--opt o=addr=hostname
--opt device=:/path/to/dir
foo

Creation was okay but when I actually mount it I got a different error: invalid argument

From the code changes I can see it is simply replacing addr=hostname to addr=ip, however the errors seemed to suggest addr was never a valid argument to mount.cifs. Could you clarify @shuchow?

@na4ma4
Copy link

na4ma4 commented Jan 18, 2021

@spinosae This still works for me in docker 20.10.2:

docker volume create 
--driver local
--opt type=cifs
--opt device=//core/backup
--opt "o=uid=1000,gid=1000,ip=172.16.20.110,unc=\\core\backup,file_mode=0664,dir_mode=0775,iocharset=utf8,cache=none,mapchars,mfsymlinks,user=bkupusr,pass=${CIFS_PASSWD}"
sync-backup

Most of the extra options can be dropped, but thats a copy of a live volume create from a daily script.

@na4ma4
Copy link

na4ma4 commented Jan 18, 2021

@spinosae I misread your comment, all this PR does is rewrite the addr= from a hostname to an IP.

so this will work:

docker volume create \
--driver local \
--opt type=cifs \
--opt device=//myserver/backup \
--opt "o=addr=myserver.localdomain.com,user=bkupusr,pass=${CIFS_PASSWD}" \
sync-backup

it will find the addr=myserver.localdomain.com look up what the IP address (say 127.0.0.1) is and replace it with that (so addr=127.0.0.1) because the kernel driver won't accept a hostname there, but requires the parameter.

So your second example should have worked if the device was a valid path to a CIFS mount eg. //server/share (but not a subdirectory like //server/share/dir, that needs the prefixpath option I've mentioned previously).

@spinosae
Copy link

@spinosae I misread your comment, all this PR does is rewrite the addr= from a hostname to an IP.

so this will work:

docker volume create \
--driver local \
--opt type=cifs \
--opt device=//myserver/backup \
--opt "o=addr=myserver.localdomain.com,user=bkupusr,pass=${CIFS_PASSWD}" \
sync-backup

it will find the addr=myserver.localdomain.com look up what the IP address (say 127.0.0.1) is and replace it with that (so addr=127.0.0.1) because the kernel driver won't accept a hostname there, but requires the parameter.

So your second example should have worked if the device was a valid path to a CIFS mount eg. //server/share (but not a subdirectory like //server/share/dir, that needs the prefixpath option I've mentioned previously).

wow, just like that, MAGIC!

so the difference is device option, instead of :/path/to/dir/ it should have been //myserver/share. What finally worked for me is:

docker volume create \
--driver local \
--opt type=cifs \
--opt device=//host.docker.internal/share \
--opt o=addr=host.docker.internal,nounix,noserverino \
source

@arobld00
Copy link

@spinosae Did you finally manage to connect via domain name?

@jefft4
Copy link

jefft4 commented Aug 9, 2023

Hi,

Was this resolved? I'm hitting the CIFS + DNS problem with a couple of containers. I tried adding the 'addr=' option as above, but I get 'invalid argument' when the container tries to start.

root@dock:/home/netadmin# docker volume create --driver local --opt type=cifs \
--opt "o=addr=dougal.mydomain.net" --opt device=//dougal/TransferTemp ttt
root@dock:/home/netadmin# docker volume inspect ttt
[
    {
        "CreatedAt": "2023-08-09T14:13:17Z",
        "Driver": "local",
        "Labels": null,
        "Mountpoint": "/var/lib/docker/volumes/ttt/_data",
        "Name": "ttt",
        "Options": {
            "device": "//dougal/TransferTemp",
            "o": "addr=dougal.mydomain.net",
            "type": "cifs"
        },
        "Scope": "local"
    }
]
root@dock:/home/netadmin# docker run -it --rm -v ttt:/test busybox sh
docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/ttt/_data': failed to mount local volume: mount //dougal/TransferTemp:/var/lib/docker/volumes/ttt/_data, data: **addr=192.168.11.22: invalid argument**.
root@dock:/home/netadmin# nslookup dougal.mydomain.net
Server:         192.168.11.1
Address:        192.168.11.1#53

Non-authoritative answer:
Name:   dougal.mydomain.net
Address: 192.168.11.22

root@dock:/home/netadmin# smbclient -L dougal.mydomain.net
Password for [WORKGROUP\root]:
Anonymous login successful

        Sharename       Type      Comment
        ---------       ----      -------
        TransferTemp    Disk
        RemoteBackup    Disk

Have the options changed, or does a FQDN for a CIFS share not work?

@thaJeztah
Copy link
Member

@jefft4 could you open a ticket with details? Note that these options are not handled by the docker daemon itself, but passed as options when doing a mount syscall, so options (including the addr= option) depend on support on the daemon's host.

@jefft4
Copy link

jefft4 commented Aug 9, 2023

@jefft4 could you open a ticket with details? Note that these options are not handled by the docker daemon itself, but passed as options when doing a mount syscall, so options (including the addr= option) depend on support on the daemon's host.

Sure, will do.

@michaelkebe
Copy link
Contributor

Does not fix the problem: #46180 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/volumes Volumes docs/revisit impact/changelog impact/documentation kind/enhancement Enhancements are not bugs or new features but can improve usability or performance. status/2-code-review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unable to create a docker volume with local CIFS driver