Skip to content

Memory leak creating istr in multidict 6.3.0,6.3.1,6.3.2 (fixed in 6.4.0) #1131

@bdraco

Description

@bdraco

multidict==6.2.0

❯ python test_leak.py
Initial memory usage: 39.44 MB
Memory usage: 39.44 MB
Memory usage: 40.47 MB
Memory usage: 40.61 MB
Memory usage: 40.61 MB
Memory usage: 40.61 MB
Memory usage: 40.61 MB
Memory usage: 40.70 MB

multidict==6.3.2

❯ python .tmp/test_leak.py
Initial memory usage: 39.00 MB
Memory usage: 39.00 MB
Memory usage: 41.25 MB
Memory usage: 42.12 MB
Memory usage: 42.95 MB
Memory usage: 44.38 MB
Memory usage: 44.77 MB
Memory usage: 46.06 MB
Memory usage: 47.27 MB
Memory usage: 47.84 MB
Memory usage: 49.34 MB

Reproducer

import asyncio
import os

import psutil
from aiohttp import ClientSession, web


def get_memory_usage():
    process = psutil.Process(os.getpid())
    memory_info = process.memory_info()
    return memory_info.rss / (1024 * 1024)


async def handle_ping(request):
    return web.Response(text='pong')


async def run_server():
    app = web.Application()
    app.router.add_get('/ping', handle_ping)

    runner = web.AppRunner(app)
    await runner.setup()
    site = web.TCPSite(runner, 'localhost', 8080)
    await site.start()


async def run_client():
    total_requests = 10_000
    async with ClientSession() as session:
        for i in range(total_requests):
            await make_request(session, i)


async def make_request(session, request_id):
    headers = {
        'Accept': 'text/plain',
    }

    async with session.get('http://localhost:8080/ping', headers=headers) as response:
        await response.text()


async def memory_monitor():
    while True:
        print(f'Memory usage: {get_memory_usage():.2f} MB')
        await asyncio.sleep(1)


async def main():
    asyncio.create_task(memory_monitor())
    await run_server()
    await asyncio.sleep(1)  # wait server
    for _ in range(10_000):
        await run_client()


if __name__ == '__main__':
    asyncio.run(main())

Originally posted by @ermakov-oleg in #1117

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions