Skip to content

Merge projection selects too many columns#283

Merged
rjzamora merged 3 commits intodask:mainfrom
phofl:merge_projection
Aug 30, 2023
Merged

Merge projection selects too many columns#283
rjzamora merged 3 commits intodask:mainfrom
phofl:merge_projection

Conversation

@phofl
Copy link
Collaborator

@phofl phofl commented Aug 30, 2023

No description provided.

Copy link
Member

@rjzamora rjzamora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @phofl ! Let's use _convert_to_list (assuming that works)

Comment on lines +206 to +213
if isinstance(self.left_on, list):
left_on = self.left_on
else:
left_on = [self.left_on] if self.left_on is not None else []
if isinstance(self.right_on, list):
right_on = self.right_on
else:
right_on = [self.right_on] if self.right_on is not None else []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if isinstance(self.left_on, list):
left_on = self.left_on
else:
left_on = [self.left_on] if self.left_on is not None else []
if isinstance(self.right_on, list):
right_on = self.right_on
else:
right_on = [self.right_on] if self.right_on is not None else []
left_on = _convert_to_list(self.left_on)
right_on = _convert_to_list(self.right_on)

This seems more readable to me, but you will need to add the following import _convert_to_list from dask_expr._util at the top.

Comment on lines +208 to +209
if left_on is None:
left_on = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why you need this? Tests seem to pass fine without it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is necessary, then we probably want to clean up the dead logic below that assumes left_on and right_on could be None.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry that got a bit too entangled, that's needed for #284 (which also cleans up the dead logic)

@rjzamora rjzamora merged commit ec30959 into dask:main Aug 30, 2023
@phofl phofl deleted the merge_projection branch August 30, 2023 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants