Skip to content

Fix to_boto3 method#619

Merged
mpenkov merged 4 commits intodevelopfrom
toboto3
May 25, 2021
Merged

Fix to_boto3 method#619
mpenkov merged 4 commits intodevelopfrom
toboto3

Conversation

@mpenkov
Copy link
Copy Markdown
Collaborator

@mpenkov mpenkov commented May 7, 2021

The method had a bug where it was always creating its own boto3
resource. The intention was that it re-used the resource held by the
class containing the method.

However, we've since moved away from the resource API, because it is not
multiprocessing-friendly. So, the contained class no longer holds a
boto3 resource. We now require to_boto3 to receive the resource as a
parameter. This achieves a trade-off between convenience and thread
safety:

  • The user can use the more convenient Resource API
  • The smart_open library does not have to create unsafe objects by
    itself (the user does that)

Fix #615

The method had a bug where it was always creating its own boto3
resource. The intention was that it re-used the resource held by the
class containing the method.

However, we've since moved away from the resource API, because it is not
multiprocessing-friendly. So, the contained class no longer holds a
boto3 resource. We now require to_boto3 to receive the resource as a
parameter. This achieves a trade-off between convenience and thread
safety:

- The user can use the more convenient Resource API
- The smart_open library does not have to create unsafe objects by
  itself (the user does that)

Fix #615
@mpenkov mpenkov requested a review from piskvorky May 7, 2021 04:55
@mpenkov mpenkov merged commit 56cee00 into develop May 25, 2021
@mpenkov mpenkov deleted the toboto3 branch May 25, 2021 00:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can't get properties of s3 object

1 participant