Skip to content

grpc_load_file() is flaky on MacOS when two threads attempt to read the same file #23503

@jtattermusch

Description

@jtattermusch

If two threads invoke grpc_load_file for the same filename concurrently, loading of the file might fail with this error:

E0603 07:33:03.955011000 123145516060672 verify_peer_options.cc:57]    load_file: {"created":"@1591169583.954981000","description":"Failed to load file","file":"src/core/lib/iomgr/load_file.cc","file_line":72,"filename":"src/core/tsi/test_creds/ca.pem","referenced_errors":[{"created":"@1591169583.954964000","description":"Is a directory","errno":21,"file":"src/core/lib/iomgr/load_file.cc","file_line":57,"os_error":"Is a directory","syscall":"fread"}]}
E0603 07:33:03.955061000 123145516060672 verify_peer_options.cc:57]    assertion failed: GRPC_LOG_IF_ERROR("load_file", grpc_load_file(SSL_CA_PATH, 1, &ca_slice))

Oddly, it reports "errno 21: Is a directory" but the real cause seems to be that even though each thread has called their own fopen (for reading) and they both have a different FILE* object, their invocations of fread and fseek still interfere.
(and it makes one of the reads fail occasionally).

grpc_error* grpc_load_file(const char* filename, int add_null_terminator,

Even though this conditions is rare, the grpc_load_file is a function that's used in the production code (not just in tests) and therefore should be fixed. It is mostly used for reading the files with certificates.

This is also the rootcause for verify_peer_options test being flaky (See b/160697968 for reference)

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions