Skip to content

gRPC Python Server Library incompatibility with future.ProcessPoolExecutor #14436

@decebal

Description

@decebal

What version of gRPC and what language are you using?

grpc 1.9.1 , python 3.6.0

What operating system (Linux, Windows, …) and version?

under docker with phusion/baseimage , also tried on debian jessie and Kali Linux

What runtime / compiler are you using (e.g. python version or version of gcc)

python 3.6.0

What did you do?

The issue is really similar to : #13873 also maybe with #12455 . This as well: #13536

What did you expect to see?

normal server response

What did you see instead?

Got the error: E0215 12:45:01.090952877 35 tcp_server_posiix.cc:228] Failed accept4: Invalid argument in the logs while the server hangs and is made unavailable for future requests.

How to reproduce

easiest way of reproducing, which might help the other 2 issues referenced above as well, is to change this line:
https://github.com/grpc/grpc/blob/master/examples/python/route_guide/route_guide_server.py#L115
to server = grpc.server(futures.ProcessPoolExecutor(max_workers=10))

In order to get the logs I am attached an interceptor using opencensus, like this :

from opencensus.trace.samplers import always_on
from opencensus.trace.ext.grpc import server_interceptor
from opencensus.trace.exporters import print_exporter
from opencensus.trace.exporters.transports import background_thread

        sampler = always_on.AlwaysOnSampler()
        exporter = print_exporter.PrintExporter(transport=background_thread.BackgroundThreadTransport)
        tracer_interceptor = server_interceptor.OpenCensusServerInterceptor(sampler, exporter)
        server = grpc.server(
            futures.ProcessPoolExecutor(max_workers=self.max_workers),
            interceptors=(tracer_interceptor,)
        )

Anything else we should know about your project / environment?

At the moment I am considering dabeaz/curio or an asyncio alternative as temporary alternatives.
Also read the answers here https://stackoverflow.com/questions/38387443/how-to-implement-a-async-grpc-python-server , which unfortunately do not help a lot in moving the server from being thread based to a process based execution.

The need of using processes instead of threads in my architecture comes from looking to put better practices at work as well as I was looking that this piece of work can replace Gearman for me which was running in different processes. I agree there are alternative solutions, but solving this would be so much cleaner.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions