Skip to content

Add to_agen wrapper and aiofiles.os.walk.#193

Open
stankudrow wants to merge 1 commit intoTinche:mainfrom
stankudrow:extend-path-tools
Open

Add to_agen wrapper and aiofiles.os.walk.#193
stankudrow wants to merge 1 commit intoTinche:mainfrom
stankudrow:extend-path-tools

Conversation

@stankudrow
Copy link
Contributor

@stankudrow stankudrow commented Nov 11, 2024

Closes #160 .
Also may close #167 .
Potentially closes #207 .

Features:

  • add the to_agen decorator responsible for wrapping sync iterators and turning them into asynchronous ones - analogously to the to_coro decorator that makes a coroutine from a given function
  • add an async version of the os.walk iterator with tests
  • a couple of extra unrelated minor changes

@stankudrow stankudrow marked this pull request as draft November 12, 2024 08:28
@stankudrow stankudrow marked this pull request as ready for review November 12, 2024 09:14
@stankudrow stankudrow changed the title Add os.listdir and os.walk wrappers Add os.walk wrapper Nov 12, 2024
@stankudrow
Copy link
Contributor Author

#160 (comment) - the os.listdir already wrapped, the os.walk added.

@stankudrow
Copy link
Contributor Author

stankudrow commented Nov 12, 2024

@Tinche , hello. Rebased and ready for review, all checks have been passed.

Update: I found the issue #167 on an "invented-here" solution, what are your suggestions?

@Tinche
Copy link
Owner

Tinche commented Dec 4, 2024

Apologies. I have a small baby nowadays.

So the original sync os.walk is a generator, right? This has some very useful properties which we should keep. Can we make the async version an async generator? Each step can do a step of the sync one.

@stankudrow
Copy link
Contributor Author

Apologies. I have a small baby nowadays.

So the original sync os.walk is a generator, right? This has some very useful properties which we should keep. Can we make the async version an async generator? Each step can do a step of the sync one.

Hello and no need for apologies, au contraire, congratulations.

@stankudrow stankudrow requested a review from Tinche December 28, 2024 19:15
@stankudrow stankudrow marked this pull request as draft February 1, 2025 17:22
@stankudrow
Copy link
Contributor Author

Waits for the #199 to be merged.

@stankudrow stankudrow marked this pull request as ready for review February 8, 2025 15:55
@stankudrow stankudrow marked this pull request as draft February 8, 2025 16:27
@stankudrow stankudrow marked this pull request as ready for review February 8, 2025 17:44
@Martin005
Copy link

Waits for the #199 to be merged.

That PR has been merged now, can we please merge this one as well? 🙂

@stankudrow stankudrow marked this pull request as draft April 14, 2025 07:41
@stankudrow stankudrow marked this pull request as ready for review April 25, 2025 06:03
@stankudrow stankudrow marked this pull request as draft April 30, 2025 02:36
stankudrow pushed a commit to stankudrow/aiofiles that referenced this pull request Jun 20, 2025
@stankudrow stankudrow changed the title Add os.walk wrapper Add to_a(sync)gen wrapper able to turn a sync generator function into its async variant and introduce an async version of os.walk. Aug 12, 2025
@stankudrow stankudrow changed the title Add to_a(sync)gen wrapper able to turn a sync generator function into its async variant and introduce an async version of os.walk. Add to_agen wrapper and aiofiles.os.walk. Mar 6, 2026
@stankudrow stankudrow marked this pull request as ready for review March 6, 2026 22:17
@stankudrow
Copy link
Contributor Author

@Martin005 , @rrrix, @Tinche , hello.

I need your review, WIP is highlighted, but a milestone is reached, so I need to understand what is good and what can be improved, especially on the implementation of to_agen decorator. First I wanted to have the wrap written like as follows:

def wrap(cb: Callable) -> Callable:
    if inspect.isgeneratorfunction(cb):
        return _to_agen(cb)
    return _to_coro(cb)  # the former routine-only `wrap` decorator

Prior to Python 3.12 this fails because inspect.isgeneratorfunction(os.walk) is False and since Py3.12 it is True. Any ideas how to have goodies from both realms -> having one wrap which can handle routines and generator functions (or functions that return an iterator object) and are universal enough to work for Py3.9+ versions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Async os.walk attempted solution add "aiofiles.path.walk" or "aiofiles.path.listdir" Python 3.9: to_thread or not to_thread?

3 participants