Skip to content

fix int_validator not catching overflows#3112

Merged
samuelcolvin merged 2 commits intopydantic:masterfrom
ojii:int-overflow
Aug 5, 2022
Merged

fix int_validator not catching overflows#3112
samuelcolvin merged 2 commits intopydantic:masterfrom
ojii:int-overflow

Conversation

@ojii
Copy link
Copy Markdown
Contributor

@ojii ojii commented Aug 18, 2021

Change Summary

int(value) can raise an OverflowError, for example when called with very large floats.

Related issue number

#2345

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI and coverage remains at 100%
  • Documentation reflects the changes where applicable
  • changes/<pull request or issue id>-<github username>.md file added describing change
    (see changes/README.md for details)
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

@ojii
Copy link
Copy Markdown
Contributor Author

ojii commented Aug 18, 2021

please review

@PrettyWood
Copy link
Copy Markdown
Contributor

I reckon it would be better to catch the OverflowError in a separate except clause with a more explicit error. We could probably do the exact same for float

@ojii
Copy link
Copy Markdown
Contributor Author

ojii commented Sep 4, 2021

I reckon it would be better to catch the OverflowError in a separate except clause with a more explicit error.

Sorry, first time contributing to pydantic. Does that mean just changing the msg or the type of the error? If the latter, is there a way type codes are chosen?

Comment thread pydantic/validators.py
except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
except (TypeError, ValueError):
raise errors.IntegerError()
except OverflowError:
raise errors.IntegerOverflowError()

I think this is meant.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then, like 144

-        return float(v)
+        rv = float(v)
+        if rv in (float("inf"), float("-inf")):
+            raise errors.FloatOverflowError()
+        if rv != rv:
+            raise errors.FloatNanError()
+        return rv

P.S. the weird line of code from https://stackoverflow.com/a/44154660/705086

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

floats are a separate case. I would say that float('inf') is a valid float, arguably so is float('nan').

Copy link
Copy Markdown
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, just a few things to fix and more test cases to add.

Comment thread pydantic/validators.py
except (TypeError, ValueError, OverflowError):
raise errors.IntegerError()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

floats are a separate case. I would say that float('inf') is a valid float, arguably so is float('nan').

Comment thread tests/test_validators.py
Model(a=2.2250738585072011e308)
assert exc_info.value.errors() == [
{'loc': ('a',), 'msg': 'value is not a valid integer', 'type': 'type_error.integer'}
]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the case of passing float('inf') etc.

@samuelcolvin
Copy link
Copy Markdown
Member

please update.

@github-actions github-actions Bot added the awaiting author revision awaiting changes from the PR author label Aug 4, 2022
@github-actions github-actions Bot assigned ojii and unassigned samuelcolvin and PrettyWood Aug 4, 2022
@ojii
Copy link
Copy Markdown
Contributor Author

ojii commented Aug 5, 2022

This is looking good, just a few things to fix and more test cases to add.

@samuelcolvin i've added more test cases, do you think the error code/msg should be handled different? If so, how are new messages/codes decided?

@samuelcolvin
Copy link
Copy Markdown
Member

samuelcolvin commented Aug 5, 2022

At this stage, best if you can roughly follow the code from pydantic-core to minimize the incompatibility with v2.

@samuelcolvin
Copy link
Copy Markdown
Member

@ojii
Copy link
Copy Markdown
Contributor Author

ojii commented Aug 5, 2022

Since the structure of the error dict is different between v1 and v2 anyway, does it make sense trying to make it v2-like? I feel making it fit in with the rest of v1 makes more sense (eg in v1, error codes seem to always be prefixed with type_error. or value_error. but in v2 not?)

@samuelcolvin
Copy link
Copy Markdown
Member

Yes, that's what I meant by roughly follow, e.g. use value_error.int_nan?

@ojii
Copy link
Copy Markdown
Contributor Author

ojii commented Aug 5, 2022

Yes, that's what I meant by roughly follow, e.g. use value_error.int_nan?

When I change it to that, interestingly enough float('nan') gets caught in the already existing exceptions (ValueError). This proposed change (catching OverflowError) only catches float('inf') or other huge floats, so int_nan is probably not the right thing here.

Copy link
Copy Markdown
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, let's just stick with this then.

Comment thread changes/3112-ojii.md Outdated
@samuelcolvin samuelcolvin enabled auto-merge (squash) August 5, 2022 11:08
@samuelcolvin
Copy link
Copy Markdown
Member

Good catch, thanks so much for the PR.

@samuelcolvin samuelcolvin merged commit 88ade4b into pydantic:master Aug 5, 2022
@ojii ojii deleted the int-overflow branch August 6, 2022 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting author revision awaiting changes from the PR author

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants