storage(s3): ditch aws-sdk in favour of minio-cpp #879
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The origin of this PR was a strange
munmap_chunk(): invalid_pointerthat started appearing in our GHA CI.This error only appeared during code generation, and after narrowing it down a bit further, it seemed to happen during destruction of the S3 API.
Having AWS C++ SDK as a dependency just to interact with out MinIO S3 server always felt like an overkill. In addition, the SDK's init/destruction lifecycles have always been sketchy and error-prone. We were not using the SDK according to its guided usage neither. All in all, it felt reasonable to ditch such a heavyweight dependency and, instead, use MinIO's native C++ client.
In the process of doing so, a rather gnarly bug appeared in the interplay between
libcurl(as used by MinIO's CPP client) andcpprestsdk. Turns out this is a known issue withcpprestsdk.Alas, the project is EOL-ed, so I also ditch the dependency and make our HTTP servers inherit from faabric's native HTTP handler base class.
Unfortunately there is a cyclic dependency between both contributions that prevent me from breaking them into two PRs:
cpprestsdkto get rid of AWS SDK.cpprestsdk.