-
Notifications
You must be signed in to change notification settings - Fork 40
Constructing message from dictionary fails in protobuf 5.28.0 when message has a nested duration #518
Description
As of protobuf 5.28.0, there is a bug in proto-plus where constructing a message from a dictionary fails when the message definition has a field of type google.protobuf.duration_pb2.Duration. Note this problem only exists when duration is in a nested message. See below:
class Foo(proto.Message):
another_field: duration_pb2.Duration = proto.Field(
proto.MESSAGE,
number=1,
message=duration_pb2.Duration,
)
# this works
Foo(another_field="300s")
class Bar(proto.Message):
some_field = proto.Field(proto.MESSAGE, number=1, message=Foo)
# this raises `AttributeError: Fail to convert to Duration. Expected a timedelta like object got str: 'str' object has no attribute 'seconds'`
bar = Bar({"some_field":{"another_field":"300s"}})
There is a change in the code path of message construction in proto-plus when using protobuf 5.28.0 and newer
In 5.27.5, we have
MessageMeta.__new__ → marshal.to_proto() → BaseMarshal.get_rule() → MessageRule.to_proto() → TypeError
where the exception is TypeError: Message must be initialized with a dict: _default_package.Foo raised at the location below`
In 5.28.0, we have
MessageMeta.__new__ → marshal.to_proto() → BaseMarshal.get_rule() → MessageRule.to_proto() → AttributeError
where the exception is AttributeError: Fail to convert to Duration. Expected a timedelta like object got str: 'str' object has no attribute 'seconds'
Technically, this is not a breaking change in protobuf as they simply raised a different error. Proto-plus expected TypeError but AttributeError was raised instead.
See the re-production below:
In protobuf 5.27.5, run the code below
def test_duration_write_string_nested():
class Foo(proto.Message):
another_field: duration_pb2.Duration = proto.Field(
proto.MESSAGE,
number=1,
message=duration_pb2.Duration,
)
class Bar(proto.Message):
some_field = proto.Field(proto.MESSAGE, number=1, message=Foo)
bar = Bar({"some_field":{"another_field":"300s"}})
assert isinstance(bar.some_field.another_field, timedelta)
assert isinstance(Bar.pb(bar).some_field.another_field, duration_pb2.Duration)
assert bar.some_field.another_field.seconds == 300
assert Bar.pb(bar).some_field.another_field.seconds== 300
You'll see TypeError: Message must be initialized with a dict: _default_package.Foo raised at the location below`
proto-plus-python/proto/marshal/rules/message.py
Lines 34 to 37 in c54b31d
| try: | |
| # Try the fast path first. | |
| return self._descriptor(**value) | |
| except (TypeError, ValueError) as ex: |
In protobuf 5.27.5, the TypeError is caught, and then we successfully run the following code:
proto-plus-python/proto/marshal/rules/message.py
Lines 38 to 45 in c54b31d
| # If we have a TypeError or Valueerror, | |
| # try the slow path in case the error | |
| # was: | |
| # - an int64/string issue. | |
| # - a missing key issue in case a key only exists with a `_` suffix. | |
| # See related issue: https://github.com/googleapis/python-api-core/issues/227. | |
| # - a missing key issue due to nested struct. See: b/321905145. | |
| return self._wrapper(value)._pb |
In protobuf 5.28.0 and newer, you'll see AttributeError instead which is not caught in the code below and instead AttributeError is returned to the user
proto-plus-python/proto/marshal/rules/message.py
Lines 34 to 45 in c54b31d
| try: | |
| # Try the fast path first. | |
| return self._descriptor(**value) | |
| except (TypeError, ValueError) as ex: | |
| # If we have a TypeError or Valueerror, | |
| # try the slow path in case the error | |
| # was: | |
| # - an int64/string issue. | |
| # - a missing key issue in case a key only exists with a `_` suffix. | |
| # See related issue: https://github.com/googleapis/python-api-core/issues/227. | |
| # - a missing key issue due to nested struct. See: b/321905145. | |
| return self._wrapper(value)._pb |