Skip to content

Fix Argument inputs using shlex#72

Merged
Vizonex merged 7 commits intomainfrom
fix-processes
Oct 1, 2025
Merged

Fix Argument inputs using shlex#72
Vizonex merged 7 commits intomainfrom
fix-processes

Conversation

@Vizonex
Copy link
Owner

@Vizonex Vizonex commented Aug 25, 2025

What do these changes do?

There was a recent bug discovered in relation to agronholm/anyio#960 which will add winloop support for anyio so that users who are os-locked to windows due to numbers of things examples could be government (meaning laws that prohibit linux) or job/school/collage related. All the bugs seem to be Process related. The main Culprit of the unchecked bugs are currently unknown.

if something on my list is not checked off it means that I am hunting for the bug. There's 2 of them so far.

  • bad shell parser (Remedied via porting shlex to Cython which fixes the first bug)
  • stdin not being read at all and is instead skipping out immediately. (Cause of the bug is still unknown) uvloop doesn't have this problem strangely enough it seems my library is the only one with the bug.

EDIT: (10/1/2025) the real reason for the stdin bug was that running something like python "etc..." causes it to be read as a string and therefore all quotes during the transformation of argument ends need to be stripped out.

I have temporarily disabled GIL Releasing in response until I can perform a further diagnosis of the bug and where it happens and has been found.

This PR will be on and off depending on my work schedule and I will need reviewers depending on how bad things get, hence why this is a draft and not a full-blown pull request yet.

Are there changes in behavior for the user?

These are interior changes not exterior and have to do with problems that have never been brought to my attention before.

Is it a substantial burden for the maintainers to support this?

The shlex parser is a bit of my only concern as it means one more module for me and whoever is contributing to worry about but beyond that I have no complaints with it.

Related issue number

Bugs were first discovered in agronholm/anyio#960 when providing support with anyio for reasons that I've explained above.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you should involve AnyIO in this project, given that it's an upstream component for AnyIO. These test should be perfectly runnable with your existing test suite.

@Vizonex Vizonex marked this pull request as ready for review September 23, 2025 23:03
@Vizonex
Copy link
Owner Author

Vizonex commented Sep 23, 2025

@agronholm I figured out what might be the problem , stdint doesn't pick up as it should when working with shell commands so my question is if it's a probelm with window's cmd command. since we use cmd /c /s (For string escaping | but also via shlex) and then the rest of the command. If I can't figure this out I am planning to throw an issue on libuv about when stdin refuses to pick up when cmd is in use...

import winloop
import asyncio

async def test():
    loop = asyncio.get_event_loop()

    process = await asyncio.create_subprocess_shell(
        f'python -c "import sys; print(sys.stdin.read()[::-1])"',
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        stderr=asyncio.subprocess.PIPE
    )
    print(process.returncode is None)
    print("=== PROCESS LAUNCHED ===")

    # loop.print_debug_info()
    process.stdin.write(b'abc')
    await process.stdin.drain()

    print(process.returncode is None)

    # loop.print_debug_info()
    print("waiting...")

    process.stdin.close()
    print(await process.stdout.read())
    print(process.returncode is None)
    await process.wait()



if __name__ == "__main__":
    winloop.run(test())

@agronholm
Copy link

Best of luck 🙏

@Vizonex Vizonex changed the title Fix processes Fix Argument inputs using shlex Oct 1, 2025
@Vizonex Vizonex requested a review from agronholm October 1, 2025 17:26
@agronholm
Copy link

I don't think I'm qualified to review this PR. Have you tested it against your AnyIO PR yet?

@Vizonex
Copy link
Owner Author

Vizonex commented Oct 1, 2025

Passed in my book. Since these other tests have on/off hiccups I'll probably try clearing those in 0.2.4 if were lucky but for now I declare this a well deserved victory.

@agronholm
Copy link

I gave it a cursory look and nothing special jumped at me, so I give it a weak 👍.

@Vizonex
Copy link
Owner Author

Vizonex commented Oct 1, 2025

I don't think I'm qualified to review this PR. Have you tested it against your AnyIO PR yet?

I already have and I have some debug code you can run to test it yourself until I release 0.2.3

import winloop
import asyncio
import os
import sys


async def test():
    process = await asyncio.create_subprocess_exec(
        sys.executable, "-c", "import sys; print(sys.stdin.read()[::-1])",
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        # env=dict(os.environ.items())
        # stderr=asyncio.subprocess.PIPE
    )
    print(process.returncode is None)
    print("=== PROCESS LAUNCHED ===")

    # loop.print_debug_info()
    process.stdin.write(b'abc')
    await process.stdin.drain()

    print(process.returncode is None)

    # loop.print_debug_info()
    print("waiting...")

    process.stdin.close()
    print(await process.stdout.read())
    print(process.returncode is None)
    await process.wait()


    process = await asyncio.create_subprocess_shell(
         f'{sys.executable} -c "import sys; print(sys.stdin.read()[::-1])"',
        stdin=asyncio.subprocess.PIPE,
        stdout=asyncio.subprocess.PIPE,
        # env=dict(os.environ.items())
        # stderr=asyncio.subprocess.PIPE
    )
    print(process.returncode is None)
    print("=== PROCESS LAUNCHED ===")

    # loop.print_debug_info()
    process.stdin.write(b'abc')
    await process.stdin.drain()

    print(process.returncode is None)

    # loop.print_debug_info()
    print("waiting...")

    process.stdin.close()
    print(await process.stdout.read() == b"cba\r\n")
    print(process.returncode is None)
    await process.wait()

if __name__ == "__main__":
    winloop.run(test())

@Vizonex Vizonex merged commit e47bcda into main Oct 1, 2025
16 of 20 checks passed
@Vizonex Vizonex deleted the fix-processes branch October 1, 2025 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants