Hello everyone,


I've been trying to build the master branch on Ubuntu 16.04 and it currently fails 2 test, I was wondering if this was normal or if I'm missing dependencies, I also tried apt-get build-dev python3.6 and python3.7 to no avail, the build requirements install worked for python3.5 but I suspect 3.7 has different dependencies but I can't find where the documentation for the requirements are. 
2 tests failed:
    test_dtrace test_subprocess


running test_dtrace as verbose gave https://pastebin.com/ZGzzxwjk
	
[Bash] FAILED (errors=4) test test_dtrace failed 1 test failed: test_dtrace R - Pastebin.com
pastebin.com


and running test_subprocess gives


https://pastebin.com/DNjPzpgp

I discussed with Jay Yin on IRC and we understood the issue on his setup: the last entry of his PATH environment variable is a path to an existing *file*, not a directory.

In this case, subprocess.Popen() fails with ENOTDIR if the program cannot be found in any other directory of the PATH.

Copy of _posixmodule.c:
---
    /* This loop matches the Lib/os.py _execvpe()'s PATH search when */
    /* given the executable_list generated by Lib/subprocess.py.     */
    saved_errno = 0;
    for (i = 0; exec_array[i] != NULL; ++i) {
        const char *executable = exec_array[i];
        if (envp) {
            execve(executable, argv, envp);
        } else {
            execv(executable, argv);
        }
        if (errno != ENOENT && errno != ENOTDIR && saved_errno == 0) {
            saved_errno = errno;
        }
    }
    /* Report the first exec error, not the last. */
    if (saved_errno)
        errno = saved_errno;
---

If the first execv() calls with ENOENT and the last one fails with ENOTDIR, the function fails with ENOTDIR.

Attached PR 5322 fixes the issue.

Example to reproduce the bug:

$ touch file
$ PATH=$PATH:$PWD/file ./python -m test test_subprocess -m test_invalid_args -v
(...)
======================================================================
ERROR: test_invalid_args (test.test_subprocess.ContextManagerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/prog/python/master/Lib/test/test_subprocess.py", line 3050, in test_invalid_args
    stderr=subprocess.PIPE) as proc:
  File "/home/vstinner/prog/python/master/Lib/subprocess.py", line 743, in __init__
    restore_signals, start_new_session)
  File "/home/vstinner/prog/python/master/Lib/subprocess.py", line 1431, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
NotADirectoryError: [Errno 20] Not a directory: 'nonexisting_i_hope'
(...)


With PR 5322 applied, the whole Python test suite pass with PATH=$PATH:$PWD/file.

New changeset b31206a223955d614d7769f95fb979d60f77bf87 by Victor Stinner in branch 'master':
bpo-32667: Fix tests when $PATH contains a file (#5322)
https://github.com/python/cpython/commit/b31206a223955d614d7769f95fb979d60f77bf87

New changeset 255dbd2102d5dec5ffbd0b94084377e98c3b56c4 by Victor Stinner (Miss Islington (bot)) in branch '3.6':
bpo-32667: Fix tests when $PATH contains a file (GH-5322) (#5323)
https://github.com/python/cpython/commit/255dbd2102d5dec5ffbd0b94084377e98c3b56c4

New changeset 6996f284d4d90aa05c46d9fe6f38d1030454b224 by Victor Stinner in branch '2.7':
bpo-32667: Fix tests when $PATH contains a file (#5324)
https://github.com/python/cpython/commit/6996f284d4d90aa05c46d9fe6f38d1030454b224

Thank you Jay Yin for your bug report. I fixed the bug in Python 2.7, 3.6 and master.

no problem, thanks for helping and fixing the issue, I can now help contribute to python =D