Capture the inner monad state and associate it with the work by adithyaov · Pull Request #583 · composewell/streamly

adithyaov · 2020-07-08T12:17:31Z

adithyaov · 2020-07-09T01:17:53Z

Generating reports for Prelude.Async...
Prelude.Async(time)
Benchmark                                                              default(0)(μs) default(1) - default(0)(%)
---------------------------------------------------------------------- -------------- --------------------------
Prelude.Async/o-1-space/generation/unfoldr                                      33.80                     +15.38
Prelude.Async/o-1-space/concat-foldable/foldMapM (List)                      22137.02                     +14.47
Prelude.Async/o-1-space/monad-outer-product/toNull                           38304.49                     +14.21
Prelude.Async/o-1-space/generation/unfoldrM                                  69829.15                     +11.27
Prelude.Async/o-1-space/concat/concatMapWithId (n of 1) (fromFoldable)       70054.39                     +10.99
Prelude.Async/o-1-space/mapping/mapM                                         73430.42                     +10.59
Prelude.Async/o-1-space/generation/fromFoldableM (List)                      24804.77                     +10.10
Prelude.Async/o-1-space/generation/unfoldrM maxThreads 1                     49302.18                      +8.80
Prelude.Async/o-1-space/mapping/foldrSShared                                 75537.91                      +7.98
Prelude.Async/o-n-space/monad-outer-product/toListSome                        6030.13                      +7.68
Prelude.Async/o-1-space/monad-outer-product/toNullAp                         32765.10                      +7.31
Prelude.Async/o-1-space/mapping/fmap                                         66698.53                      +5.78
Prelude.Async/o-n-space/monad-outer-product/toList                           72758.88                      +4.54
Prelude.Async/o-1-space/concat-foldable/foldMapWithM (<>) (List)             23908.80                      +4.39
Prelude.Async/o-1-space/monad-outer-product/filterSome                       16286.68                      +3.84
Prelude.Async/o-1-space/concat-foldable/foldMapWith (<>) (List)              26758.59                      +3.38
Prelude.Async/o-1-space/mapping/map                                          66793.30                      +3.38
Prelude.Async/o-1-space/monad-outer-product/breakAfterSome                   35006.00                      +3.20
Prelude.Async/o-1-space/monad-outer-product/filterAllIn                      35076.38                      +3.20
Prelude.Async/o-1-space/concat-foldable/foldMapWith (<>) (Stream)            25639.69                      +2.64
Prelude.Async/o-1-space/generation/unfoldrM maxBuffer 1 (x/10 ops)          228034.62                      +2.39
Prelude.Async/o-1-space/monad-outer-product/filterAllOut                     16532.15                      +1.61
Prelude.Async/o-1-space/monad-outer-product/toNull3                          87357.40                      +0.77
Prelude.Async/o-1-space/generation/fromFoldable (List)                        1696.42                      -2.13
Prelude.Async/o-1-space/concat/concatMapWith (n of 1)                       115121.75                      -2.97
Prelude.Async/o-1-space/concat/concatMapWith (sqrt x of sqrt x)              19023.58                      -7.26
Prelude.Async/o-1-space/concat/concatMapWith (1 of n)                        39275.83                      -9.51
Prelude.Async/o-1-space/generation/fromListM                                  1160.58                      -9.85
Prelude.Async/o-1-space/joining/async (2 of n/2)                             19371.43                     -11.28
Prelude.Async/o-1-space/joining/async (4 of n/4)                             19821.76                     -11.81
Prelude.Async/o-1-space/joining/async (2 of (2 of n/4)                       19309.72                     -14.88

Generating reports for Prelude.Ahead...
Prelude.Ahead(time)
Benchmark                                                              default(0)(μs) default(1) - default(0)(%)
---------------------------------------------------------------------- -------------- --------------------------
Prelude.Ahead/o-1-space/concat-foldable/foldMapWith (<>) (Stream)            91868.28                     +11.41
Prelude.Ahead/o-1-space/concat/concatMapWithId (n of 1) (fromFoldable)       92008.56                      +9.91
Prelude.Ahead/o-1-space/mapping/map                                          83334.55                      +9.56
Prelude.Ahead/o-1-space/monad-outer-product/filterAllIn                      78890.17                      +9.44
Prelude.Ahead/o-1-space/generation/fromFoldable                               1538.09                      +9.30
Prelude.Ahead/o-1-space/concat-foldable/foldMapWithM (<>) (List)             90142.67                      +8.40
Prelude.Ahead/o-1-space/concat/concatMapWith (sqrt x of sqrt x)              22146.90                      +8.14
Prelude.Ahead/o-1-space/concat-foldable/foldMapM (List)                      89882.70                      +7.21
Prelude.Ahead/o-1-space/generation/fromFoldableM                             88665.20                      +6.58
Prelude.Ahead/o-1-space/concat/concatMapWith (1 of n)                        40996.65                      +6.22
Prelude.Ahead/o-1-space/concat-foldable/foldMapWith (<>) (List)              92328.50                      +6.17
Prelude.Ahead/o-1-space/monad-outer-product/breakAfterSome                   79063.07                      +4.48
Prelude.Ahead/o-1-space/mapping/mapM                                        115701.14                      +3.00
Prelude.Ahead/o-n-space/monad-outer-product/toListSome                       11677.93                      +2.08
Prelude.Ahead/o-1-space/monad-outer-product/filterSome                       41606.57                      +1.38
Prelude.Ahead/o-1-space/mapping/fmap                                         86582.17                      +1.34
Prelude.Ahead/o-1-space/monad-outer-product/toNull                           85450.30                      +0.93
Prelude.Ahead/o-1-space/generation/unfoldrM maxThreads 1                     66610.47                      +0.48
Prelude.Ahead/o-1-space/generation/unfoldrM maxBuffer 1 (x/10 ops)          252186.63                      -0.28
Prelude.Ahead/o-1-space/monad-outer-product/toNullAp                         86900.67                      -0.47
Prelude.Ahead/o-1-space/monad-outer-product/filterAllOut                     42433.70                      -0.84
Prelude.Ahead/o-1-space/concat/concatMapWith (n of 1)                       180527.20                      -0.96
Prelude.Ahead/o-n-space/monad-outer-product/toList                          124566.95                      -1.27
Prelude.Ahead/o-1-space/generation/unfoldrM                                  88999.21                      -2.10
Prelude.Ahead/o-1-space/monad-outer-product/toNull3                         178483.10                      -9.08
Prelude.Ahead/o-1-space/generation/unfoldr                                      74.51                    -135.24

Generating reports for Prelude.WAsync...
Prelude.WAsync(time)
Benchmark                                                               default(0)(μs) default(1) - default(0)(%)
----------------------------------------------------------------------- -------------- --------------------------
Prelude.WAsync/o-1-space/generation/unfoldrM maxBuffer 1 (x/10 ops)          242020.15                     +25.62
Prelude.WAsync/o-1-space/concat/concatMapWith (1 of n)                       102952.94                     +14.58
Prelude.WAsync/o-1-space/concat/concatMapWith (sqrt x of sqrt x)              53918.40                     +11.25
Prelude.WAsync/o-1-space/generation/fromFoldable                               1697.40                      +8.58
Prelude.WAsync/o-1-space/mapping/map                                          63430.60                      +7.38
Prelude.WAsync/o-n-heap/monad-outer-product/filterAllOut                      24399.86                      +7.27
Prelude.WAsync/o-n-heap/monad-outer-product/filterSome                        23966.93                      +5.78
Prelude.WAsync/o-1-space/concat/concatMapWith (n of 1)                       149104.53                      +4.64
Prelude.WAsync/o-1-space/mapping/mapM                                         92686.90                      +4.01
Prelude.WAsync/o-1-space/generation/unfoldrM                                  63412.85                      +1.76
Prelude.WAsync/o-1-space/generation/unfoldrM maxThreads 1                     52359.32                      +1.48
Prelude.WAsync/o-n-space/monad-outer-product/toListSome                       20672.66                      +0.60
Prelude.WAsync/o-n-heap/monad-outer-product/toNull3                          214489.90                      +0.36
Prelude.WAsync/o-1-space/mapping/fmap                                         68110.23                      -1.63
Prelude.WAsync/o-1-space/concat/concatMapWithId (n of 1) (fromFoldable)       74669.69                      -1.99
Prelude.WAsync/o-n-heap/monad-outer-product/toNullAp                          82401.90                      -2.24
Prelude.WAsync/o-n-heap/monad-outer-product/breakAfterSome                    79206.92                      -2.65
Prelude.WAsync/o-n-space/monad-outer-product/toList                          116881.68                      -3.30
Prelude.WAsync/o-n-heap/monad-outer-product/filterAllIn                       78829.08                      -3.31
Prelude.WAsync/o-n-heap/monad-outer-product/toNull                            89696.38                      -3.32
Prelude.WAsync/o-1-space/concat-foldable/foldMapWith (<>) (Stream)            30900.30                     -12.27
Prelude.WAsync/o-1-space/concat-foldable/foldMapWithM (<>) (List)             30290.92                     -13.36
Prelude.WAsync/o-1-space/concat-foldable/foldMapWith (<>) (List)              34770.91                     -18.53
Prelude.WAsync/o-1-space/generation/fromFoldableM                             31263.53                     -18.59
Prelude.WAsync/o-1-space/concat-foldable/foldMapM (List)                      31851.05                     -21.19
Prelude.WAsync/o-1-space/generation/unfoldr                                      60.86                     -99.46

Generating reports for Prelude.ZipAsync...
Prelude.ZipAsync(time)
Benchmark                                               default(0)(μs) default(1) - default(0)(%)
------------------------------------------------------- -------------- --------------------------
Prelude.ZipAsync/o-1-space/joining/zipAsyncM (2,x/2)           8375.26                      +1.07
Prelude.ZipAsync/o-1-space/joining/zipAsyncAp (2,x/2)         10282.05                      +0.41
Prelude.ZipAsync/o-1-space/joining/fmap zipAsyncly              559.83                      +0.17
Prelude.ZipAsync/o-1-space/joining/zipAsync (2,x/2)            8444.91                      -1.47
Prelude.ZipAsync/o-1-space/monad-outer-product/toNullAp          77.06                      -1.70

harendra-kumar · 2020-08-20T22:24:08Z

src/Streamly/Internal/Data/SVar.hs

             , Ord
             )

+type StreamWR t m a = Either (RunInIO m, t m a) (t m a)


Instead of using an Either type we can just use the tuple (RunInIO m, t m a), for the Right case of the Either the RunInIO state will be the initial state i.e. svarMrun.

Yeah, that was the initial plan, that way we can remove svarMrun from the SVar as well.

harendra-kumar · 2020-08-20T22:28:16Z

src/Streamly/Internal/Data/Stream/Ahead.hs

+                -- Use mrun here instead of the fold
+                else do
+                    res <- liftIO $ runin (runStreamWithYieldLimit True seqNo r)
+                    restoreM res


Instead of using runin, restoreM here we can use it inside runStreamWithYieldLimit. If we remove the Either type these two cases (Right/Left) will just become the second case.

I take that back, runStreamWithYieldLimit is recursive, so its better done here.

harendra-kumar · 2020-08-20T22:47:40Z

src/Streamly/Internal/Data/Stream/Async.hs

-        else liftIO $ do
+        else do
+            runInIO <- captureMonadState
            -- XXX we also need to save the monadic state here


We can remove this comment now.

remove this comment.

harendra-kumar · 2020-08-22T21:02:08Z

src/Streamly/Internal/Data/Stream/Ahead.hs

                  liftIO (incrementYieldLimit sv)
                  nextHeap seqNo
+            -- Shouldn't we be using mrun here?
+            -- Use mrun in loopHeap instead


We can remove these comments.

harendra-kumar · 2020-08-22T21:03:04Z

src/Streamly/Internal/Data/Stream/Async.hs

-        else liftIO $ do
+        else do
+            runInIO <- captureMonadState
            -- XXX we also need to save the monadic state here


remove this comment.

harendra-kumar · 2020-08-22T21:04:25Z

src/Streamly/Internal/Data/SVar.hs

             , Ord
             )

+type StreamWR t m a = (RunInIO m, t m a)


I like to use the tuple directly instead of using a type indirection especially if the type is not very complicated. It saves one indirection for the reader of the code.

adithyaov · 2020-08-27T18:54:51Z

@harendra-kumar Please review & merge this once the CI passes.

harendra-kumar

Update the changelog, describing the bug fix
Can you move the test commit before the fix commits so that I can verify that the test fails before the fix and passes after the fix.

- Issue#369 - Preservation of monadic state across threads

harendra-kumar · 2020-08-28T22:31:57Z

I ran the benchmarks, there aren't significant changes. Merging this.

adithyaov changed the title ~~Capture the inner monad state and associate it with the work~~ WIP: Capture the inner monad state and associate it with the work Jul 8, 2020

adithyaov changed the title ~~WIP: Capture the inner monad state and associate it with the work~~ Capture the inner monad state and associate it with the work Jul 8, 2020

adithyaov force-pushed the state-bug branch from 8f345c8 to aa5052b Compare July 8, 2020 15:45

adithyaov linked an issue Jul 9, 2020 that may be closed by this pull request

Capture and save state of the thread when suspending #369

Closed

adithyaov changed the title ~~Capture the inner monad state and associate it with the work~~ Capture the inner monad state and associate it with the work for Async Jul 9, 2020

adithyaov requested a review from harendra-kumar July 9, 2020 01:15

adithyaov force-pushed the state-bug branch from aa5052b to d7cf150 Compare July 9, 2020 06:30

adithyaov changed the title ~~Capture the inner monad state and associate it with the work for Async~~ Capture the inner monad state and associate it with the work Jul 9, 2020

harendra-kumar requested changes Aug 20, 2020

View reviewed changes

adithyaov force-pushed the state-bug branch 2 times, most recently from 8e3b603 to 743faac Compare August 22, 2020 10:09

harendra-kumar requested changes Aug 22, 2020

View reviewed changes

adithyaov force-pushed the state-bug branch 2 times, most recently from c398f27 to 99f2dab Compare August 27, 2020 18:54

harendra-kumar requested changes Aug 28, 2020

View reviewed changes

adithyaov added 4 commits August 28, 2020 13:52

Add test case for checking monadic state transfer for async & ahead

de9b4ce

Capture inner monad state and associate it with the work for async

04f4b4c

Capture inner monad state and associate it with the work for ahead

7e0b636

Record the Bug Fix in the Changelog.md

f30b484

- Issue#369 - Preservation of monadic state across threads

adithyaov force-pushed the state-bug branch from 99f2dab to f30b484 Compare August 28, 2020 08:34

harendra-kumar merged commit 086c80a into master Aug 28, 2020

adithyaov deleted the state-bug branch January 11, 2021 22:07

Conversation

adithyaov commented Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adithyaov commented Jul 9, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adithyaov commented Aug 27, 2020

Uh oh!

harendra-kumar left a comment

Choose a reason for hiding this comment

Uh oh!

harendra-kumar commented Aug 28, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

adithyaov commented Jul 8, 2020 •

edited

Loading