Skip to content

fix: skip infinite values during serialization#99

Merged
alespour merged 10 commits intomainfrom
fix/do-not-serialize-infinite-values
Jul 8, 2024
Merged

fix: skip infinite values during serialization#99
alespour merged 10 commits intomainfrom
fix/do-not-serialize-infinite-values

Conversation

@alespour
Copy link
Contributor

@alespour alespour commented Jul 3, 2024 •
edited
Loading

Fixes https://github.com/influxdata/EAR/issues/5299

Proposed Changes

np.inf and -np.inf values in a Pandas data frame are skipped during during serialization into line protocol.

Other fixes:

  • replace polars.DataFrame.apply (deprecated, and does not exists in latest version) with polars.DataFrame.map_rows
  • replace pandas data frame replace() with apply(lambda: ) to avoid future problems
dataframe_serializer.py:250: FutureWarning: A value is trying to be set on a copy of a DataFrame or Series through chained assignment using an inplace method.

The behavior will change in pandas 3.0. This inplace method will never work because the intermediate object on which we are setting values always behaves as a copy.

For example, when doing 'df[col].method(value, inplace=True)', try using 'df.method({col: value}, inplace=True)' or df[col] = df[col].method(value) instead, to perform the operation inplace on the original object.

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • Tests pass
  • Commit messages are conventional

@codecov
Copy link

codecov bot commented Jul 3, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 53.26%. Comparing base (1fe1460) to head (146b987).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main      #99      +/-   ##
==========================================
+ Coverage   52.84%   53.26%   +0.41%     
==========================================
  Files          33       33              
  Lines        2036     2037       +1     
==========================================
+ Hits         1076     1085       +9     
+ Misses        960      952       -8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alespour alespour marked this pull request as ready for review July 3, 2024 13:53
@alespour alespour requested a review from bednar July 4, 2024 08:36
bednar
bednar approved these changes Jul 8, 2024
View reviewed changes
Copy link
Member

@bednar bednar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

Please back port this fix also to https://github.com/Influxdata/influxdb-client-python.

@alespour alespour merged commit 66776f1 into main Jul 8, 2024
@alespour alespour deleted the fix/do-not-serialize-infinite-values branch July 8, 2024 10:01
@bednar bednar added this to the 0.7.0 milestone Jul 8, 2024
@bednar bednar mentioned this pull request Aug 12, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bednar bednar bednar approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.7.0

Development

Successfully merging this pull request may close these issues.

2 participants

@alespour @bednar