faster traversable sorting with the help of vector-algorithms by mikeplus64 · Pull Request #2 · treeowl/sort-traversable

mikeplus64 · 2017-05-01T10:41:29Z

Seems to be much faster across the board.

I originally used a simpler implementation which just used Foldable's length to get the length of the target vector, which was only faster for very large (100,000+) lists and Seq.

Benchmark results:

sort-traversable-0.1.0.0: build (lib + bench)
Preprocessing library sort-traversable-0.1.0.0...
[4 of 4] Compiling Data.Traversable.Sort.Vector ( src/Data/Traversable/Sort/Vector.hs, .stack-work/dist/x86_64-linux-nopie/Cabal-1.24.2.0/build/Data/Traversable/Sort/Vector.o )
Preprocessing benchmark 'bench' for sort-traversable-0.1.0.0...
[1 of 1] Compiling Main             ( benchmarks/Main.hs, .stack-work/dist/x86_64-linux-nopie/Cabal-1.24.2.0/build/bench/bench-tmp/Main.o ) [Data.Traversable.Sort.Vector changed]
Linking .stack-work/dist/x86_64-linux-nopie/Cabal-1.24.2.0/build/bench/bench ...
sort-traversable-0.1.0.0: copy/register
Installing library in
/home/mike/Code/sort-traversable/.stack-work/install/x86_64-linux-nopie/lts-8.12/8.0.2/lib/x86_64-linux-ghc-8.0.2/sort-traversable-0.1.0.0-7SAZscPywY33FxrJIhy6Rm
Registering sort-traversable-0.1.0.0...
sort-traversable-0.1.0.0: benchmarks
Running 1 benchmarks...
Benchmark bench: RUNNING...
1111000      
1112000      
100000       
benchmarking 1000/list/Data.List
time                 202.7 μs   (202.2 μs .. 203.3 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 204.9 μs   (203.9 μs .. 206.5 μs)
std dev              4.126 μs   (3.402 μs .. 4.719 μs)
variance introduced by outliers: 14% (moderately inflated)
             
benchmarking 1000/list/HSTrav
time                 175.4 μs   (174.4 μs .. 177.2 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 174.8 μs   (174.5 μs .. 175.6 μs)
std dev              1.412 μs   (763.7 ns .. 2.670 μs)
             
benchmarking 1000/list/HSTrav vector
time                 72.19 μs   (72.14 μs .. 72.26 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 72.50 μs   (72.29 μs .. 72.92 μs)
std dev              906.1 ns   (547.7 ns .. 1.348 μs)
             
benchmarking 1000/sequence/sort
time                 321.1 μs   (315.1 μs .. 329.2 μs)
                     0.996 R²   (0.992 R² .. 1.000 R²)
mean                 319.0 μs   (315.9 μs .. 326.1 μs)
std dev              14.58 μs   (6.849 μs .. 27.69 μs)
variance introduced by outliers: 42% (moderately inflated)
             
benchmarking 1000/sequence/unstableSort
time                 236.4 μs   (233.9 μs .. 239.2 μs)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 234.3 μs   (233.5 μs .. 235.8 μs)
std dev              3.575 μs   (2.515 μs .. 4.765 μs)
             
benchmarking 1000/sequence/HSTrav
time                 236.0 μs   (232.9 μs .. 241.2 μs)
                     0.998 R²   (0.996 R² .. 1.000 R²)
mean                 234.1 μs   (233.2 μs .. 237.9 μs)
std dev              5.340 μs   (1.931 μs .. 11.10 μs)
variance introduced by outliers: 16% (moderately inflated)
             
benchmarking 1000/sequence/HSTrav vector
time                 180.0 μs   (178.5 μs .. 181.6 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 179.2 μs   (178.8 μs .. 180.0 μs)
std dev              1.865 μs   (805.0 ns .. 2.952 μs)
             
benchmarking 10000/list/Data.List
time                 6.563 ms   (6.245 ms .. 6.833 ms)
                     0.983 R²   (0.971 R² .. 0.991 R²)
mean                 6.468 ms   (6.208 ms .. 6.786 ms)
std dev              849.1 μs   (620.9 μs .. 1.270 ms)
variance introduced by outliers: 72% (severely inflated)
             
benchmarking 10000/list/HSTrav
time                 4.501 ms   (4.351 ms .. 4.660 ms)
                     0.983 R²   (0.972 R² .. 0.991 R²)
mean                 4.536 ms   (4.372 ms .. 4.769 ms)
std dev              601.0 μs   (451.5 μs .. 848.3 μs)
variance introduced by outliers: 76% (severely inflated)
             
benchmarking 10000/list/HSTrav vector
time                 1.225 ms   (1.222 ms .. 1.229 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.224 ms   (1.223 ms .. 1.226 ms)
std dev              5.302 μs   (3.177 μs .. 8.435 μs)
             
benchmarking 10000/sequence/sort
time                 10.69 ms   (10.26 ms .. 11.06 ms)
                     0.987 R²   (0.977 R² .. 0.994 R²)
mean                 10.64 ms   (10.21 ms .. 11.06 ms)
std dev              1.167 ms   (840.8 μs .. 1.598 ms)
variance introduced by outliers: 57% (severely inflated)
             
benchmarking 10000/sequence/unstableSort
time                 5.738 ms   (5.479 ms .. 5.987 ms)
                     0.982 R²   (0.971 R² .. 0.989 R²)
mean                 5.677 ms   (5.454 ms .. 5.964 ms)
std dev              747.2 μs   (571.8 μs .. 1.126 ms)
variance introduced by outliers: 73% (severely inflated)
             
benchmarking 10000/sequence/HSTrav
time                 5.581 ms   (5.373 ms .. 5.788 ms)
                     0.984 R²   (0.972 R² .. 0.992 R²)
mean                 5.523 ms   (5.306 ms .. 5.807 ms)
std dev              742.7 μs   (562.4 μs .. 1.077 ms)
variance introduced by outliers: 73% (severely inflated)
             
benchmarking 10000/sequence/HSTrav vector
time                 4.015 ms   (3.887 ms .. 4.141 ms)
                     0.985 R²   (0.974 R² .. 0.992 R²)
mean                 4.018 ms   (3.859 ms .. 4.277 ms)
std dev              640.2 μs   (487.8 μs .. 947.8 μs)
variance introduced by outliers: 81% (severely inflated)
             
benchmarking 100000/list/Data.List
time                 129.4 ms   (124.6 ms .. 132.2 ms)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 131.5 ms   (130.4 ms .. 133.1 ms)
std dev              1.762 ms   (1.203 ms .. 2.288 ms)
variance introduced by outliers: 11% (moderately inflated)
             
benchmarking 100000/list/HSTrav
time                 100.0 ms   (96.93 ms .. 102.7 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 98.96 ms   (97.74 ms .. 100.4 ms)
std dev              1.967 ms   (1.379 ms .. 2.986 ms)
             
benchmarking 100000/list/HSTrav vector
time                 28.87 ms   (27.37 ms .. 30.50 ms)
                     0.989 R²   (0.981 R² .. 0.996 R²)
mean                 27.58 ms   (26.53 ms .. 28.59 ms)
std dev              2.258 ms   (1.683 ms .. 2.969 ms)
variance introduced by outliers: 32% (moderately inflated)
             
benchmarking 100000/sequence/sort
time                 152.2 ms   (142.1 ms .. 158.5 ms)
                     0.997 R²   (0.986 R² .. 1.000 R²)
mean                 150.6 ms   (146.8 ms .. 153.8 ms)
std dev              4.658 ms   (3.081 ms .. 5.871 ms)
variance introduced by outliers: 12% (moderately inflated)
             
benchmarking 100000/sequence/unstableSort
time                 102.6 ms   (100.1 ms .. 105.5 ms)
                     0.999 R²   (0.996 R² .. 1.000 R²)
mean                 102.9 ms   (101.9 ms .. 104.2 ms)
std dev              1.859 ms   (1.454 ms .. 2.462 ms)
             
benchmarking 100000/sequence/HSTrav
time                 100.7 ms   (99.24 ms .. 102.1 ms)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 99.05 ms   (97.97 ms .. 100.1 ms)
std dev              1.637 ms   (1.203 ms .. 2.258 ms)
             
benchmarking 100000/sequence/HSTrav vector
time                 56.77 ms   (54.75 ms .. 58.47 ms)
                     0.998 R²   (0.995 R² .. 0.999 R²)
mean                 57.77 ms   (56.55 ms .. 59.39 ms)
std dev              2.332 ms   (1.296 ms .. 4.005 ms)
             
benchmarking 1000000/list/Data.List
time                 2.059 s    (NaN s .. 2.096 s)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 2.067 s    (2.057 s .. 2.075 s)
std dev              11.68 ms   (0.0 s .. 13.16 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/list/HSTrav
time                 1.842 s    (1.588 s .. 2.030 s)
                     0.997 R²   (0.996 R² .. 1.000 R²)
mean                 1.901 s    (1.840 s .. 1.935 s)
std dev              53.65 ms   (0.0 s .. 57.65 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/list/HSTrav vector
time                 428.4 ms   (349.9 ms .. 488.2 ms)
                     0.996 R²   (0.987 R² .. 1.000 R²)
mean                 408.5 ms   (393.4 ms .. 416.4 ms)
std dev              13.16 ms   (0.0 s .. 13.68 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/sequence/sort
time                 2.561 s    (2.497 s .. 2.605 s)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 2.570 s    (2.560 s .. 2.577 s)
std dev              10.57 ms   (0.0 s .. 12.16 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/sequence/unstableSort
time                 2.025 s    (2.004 s .. 2.059 s)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 2.025 s    (2.022 s .. 2.028 s)
std dev              5.290 ms   (0.0 s .. 5.419 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/sequence/HSTrav
time                 1.875 s    (1.801 s .. 1.946 s)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.863 s    (1.849 s .. 1.873 s)
std dev              15.85 ms   (0.0 s .. 17.79 ms)
variance introduced by outliers: 19% (moderately inflated)
             
benchmarking 1000000/sequence/HSTrav vector
time                 785.0 ms   (703.6 ms .. 922.4 ms)
                     0.997 R²   (0.991 R² .. 1.000 R²)
mean                 756.1 ms   (735.3 ms .. 771.7 ms)
std dev              23.80 ms   (0.0 s .. 27.01 ms)
variance introduced by outliers: 19% (moderately inflated)
             
Benchmark bench: FINISH
Completed 2 action(s).

treeowl · 2019-10-17T19:18:18Z

I'm sorry for not getting to this for so long. I've left several comments.

treeowl · 2019-10-17T19:00:20Z

src/Data/Traversable/Sort/Vector.hs

+
+{-# INLINE indexed #-}
+indexed :: forall f a s. (Ord a, Foldable f) => f a -> ST s (VM.STVector s a)
+indexed x = do


I don't think this implementation is the right approach. A left fold is too weird here, and you're building up a big closure chain. Why not just use the conversion Data.Vector offers? Alternatively, you could get really fancy and do a little local sorting along the way, but that would mean digging into the sorting algorithm to avoid duplicating work. On the plus side, that would really do a good job of justifying the addition of this module to what's otherwise a proof of concept package.

treeowl · 2019-10-17T19:06:34Z

src/Data/Traversable/Sort/Vector.hs

+import qualified Data.Vector.Algorithms.Intro as Intro
+import qualified Data.Vector.Mutable          as VM
+
+{-# INLINE sortTraversableBy #-}


This name is confusing. This is not actually a sorting function at all, and the Ord constraint isn't needed.

faster traversable sorting with the help of vector-algorithms

8976596

treeowl reviewed Oct 17, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

faster traversable sorting with the help of vector-algorithms#2

faster traversable sorting with the help of vector-algorithms#2
mikeplus64 wants to merge 1 commit intotreeowl:masterfrom
mikeplus64:master

mikeplus64 commented May 1, 2017

Uh oh!

treeowl commented Oct 17, 2019

Uh oh!

treeowl Oct 17, 2019

Uh oh!

treeowl Oct 17, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mikeplus64 commented May 1, 2017

Uh oh!

treeowl commented Oct 17, 2019

Uh oh!

treeowl Oct 17, 2019

Choose a reason for hiding this comment

Uh oh!

treeowl Oct 17, 2019

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants