Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: celsiustx/dask
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: q0
Choose a base ref
...
head repository: celsiustx/dask
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: qs
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 12 commits
  • 24 files changed
  • 1 contributor

Commits on May 20, 2021

  1. improve divisions, partition-size checking 

    preserve_partition_sizes: impl

maybe_infer_partition_sizes: impl

new_dd_object: impl

attr: handle_out

attr: _args, __setstate__

partition-sizes: TODOs

partition_sizes: df ctors
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    744f297 View commit details
    Browse the repository at this point in the history

  2. test_io.test_from_dask_array_index (from_pandas) 

    update partition_sizes assertions (from_pandas)

improve `is_scalar_for_elemwise` / _Frame detection

update repartition tests

attr: add _len, idx_starts, idx_ranges

check_partition_sizes test

new_dd_object: from_pandas

from_pandas test: explicit divisions check
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    c7f7ad8 View commit details
    Browse the repository at this point in the history

  3. ctor: _Frame.map 

    test util error msgs

wip: set_index

new_dd_object: DataFrame.__getitem__(str|scalar)

new_dd_object: from_dask_array (is_scalar_for_elemwise)
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    daa29ab View commit details
    Browse the repository at this point in the history

  4. test_concat_partition_sizes 

    merge_indexed_dataframes (merge/concat)

new_dd_object/concat: stack_partitions

new_dd_object: concat_indexed_dataframes

partition_sizes: parquet

new_dd_object: get_partition, _partition

new_dd_object: _LocIndexer._loc_element

new_dd_object: copy

new_dd_object: _repartition_from_boundaries (repartition_npartitions, repartition_size), dd.repartition(pd.DataFrame)

new_dd_object: _rename_dask

attr: use partition_sizes in to_dask_array(lengths=True)

ctor: quantile
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    2947a41 View commit details
    Browse the repository at this point in the history

  5. maybe_infer_partition_sizes/new_dd_object: align 

    maybe_infer_partition_sizes/new_dd_object: dd.repartition

new_dd_object: fillna

new_dd_object: _Frame.{head,tail}

new_dd_object: elemwise

new_dd_object: mode

new_dd_object: Index.head
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    32c04b5 View commit details
    Browse the repository at this point in the history

  6. preserve_partition_sizes: get/set index 

    preserve_partition_sizes: pd.isna

preserve_partition_sizes: to_timedelta

preserve_partition_sizes: to_datetime

preserve_partition_sizes: merge_asof
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    f19919b View commit details
    Browse the repository at this point in the history

  7. preserve_partition_sizes: to_frame 

    stack/concatenate partition_sizes tests

preserve_partition_sizes: from_dask_array test (values, index)

preserve_partition_sizes: .values, accept ndarray as valid divisions
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    0976a11 View commit details
    Browse the repository at this point in the history

  8. preserve_partition_sizes: df.apply 

    preserve_partition_sizes: rename

preserve_partition_sizes: numeric, get_dummies

preserve_partition_sizes: _iloc columns

preserve_partition_sizes: binary methods

(re)set_index partition_sizes

preserve_partition_sizes: reset_index

preserve_partition_sizes: apply

preserve_partition_sizes: to_dask_array tests (to_dask_array, to_frame)
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    b04e38c View commit details
    Browse the repository at this point in the history

  9. map_partitions: allow ndarray in transform_divisions(?) 

    series_from_dask_array / _Frame.assign() impl

test_assign

DataFrame.assign(Array|ndarray) using partition_sizes

test_mixed_dask_array_operations

test_partition_sizes various ops

_validate_chunks

stats, agg, simple methods
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    21a2e21 View commit details
    Browse the repository at this point in the history

  10. add Series.__getitem__ test 

    consolidate {DataFrame,Series}.__getitem__ in _Frame

Array.__getitem__(Series)

preserve delegation of Array ops to _Frame

use partition_sizes in DataFrame.__getitem__(list/ndarray)
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    738e951 View commit details
    Browse the repository at this point in the history

  11. _Frame.iloc impl 

    move DataFrame.iloc into _Frame
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    1b23389 View commit details
    Browse the repository at this point in the history

  12. test_partition_sizes: slice w/ dd.Series 

    repartition in .to_dask_array

repartition_sizes using iloc
    @ryan-williams
    ryan-williams committed May 20, 2021
    Configuration menu
    Copy the full SHA
    5e96967 View commit details
    Browse the repository at this point in the history
Loading