Eratic partree benchmark runtime

[toddaaro]

I had some irregular results while running the benchmark script. The machine has a quad core i7 Nehalem. The benchmark itself is running in a Vmware virtual machine with 8 cores ( 1 per CPU thread ). Host OS is Win7 x64.

The partree benchmark usually takes under one second to run, but in this particular instance took over 40 seconds.

# TestName Variant NumThreads   MinTime MedianTime MaxTime                                                                                       
#  

# Tue Oct 4 12:25:17 PDT 2011                                                                                                                    
# Linux ubuntu 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux                                       
# Determined machine to have 8 hardware threads.                                                                                                 
# The Glorious Glasgow Haskell Compilation System, version 7.0.3                                                                                 
#                                                                                                                                                
# Running each test for 1 trials.                                                                                                                
#  ... with default compiler options:  -O2 -rtsopts                                                                                              
#  ... with default runtime options:  -qa                                                                                                        
# Using the following settings from the benchmarking environment:                                                                                
# BENCHLIST=  THREADSETTINGS=8  TRIALS=  SHORTRUN=1  KEEPGOING=  GHC=  GHC_FLAGS=  GHC_RTS=                                                      
# *** Config [0 ..], testing with command/args: parfib.exe                                                                                       
parfib.exe 8 0.00 0.00 0.00                                                                                                                      



# *** Config [1 ..], testing with command/args: parfib.exe                                                                                       
parfib.exe 8 0.00 0.00 0.00                                                                                                                      



# *** Config [2 ..], testing with command/args: blackscholes.exe                                                                                 
blackscholes.exe 8 0.21 0.21 0.21                                                                                                                



# *** Config [3 ..], testing with command/args: cholesky.exe                                                                                     
cholesky.exe 8                                                                                                                                   



# *** Config [4 ..], testing with command/args: nbody.exe                                                                                        
nbody.exe 8 0.03 0.03 0.03                                                                                                                       



# *** Config [5 ..], testing with command/args: mandel.exe                                                                                       
mandel.exe 8 0.10 0.10 0.10                                                                                                                      



# *** Config [6 ..], testing with command/args: coins.exe                                                                                        
coins.exe 8 0.94 0.94 0.94                                                                                                                       



# *** Config [7 ..], testing with command/args: queens.exe                                                                                       
queens.exe 8 0.22 0.22 0.22                                                                                                                      



# *** Config [8 ..], testing with command/args: partree/partree.exe                                                                              
partree/partree.exe 8 40.33 40.33 40.33                                                                                                          



# *** Config [9 ..], testing with command/args: matmult/matmult.exe                                                                              
matmult/matmult.exe 8 0.28 0.28 0.28                                                                                                             



# *** Config [10 ..], testing with command/args: sumeuler/sumeuler.exe                                                                           
sumeuler/sumeuler.exe 8 0.93 0.93 0.93           

[simonmar]

The Strategies version of partree also gives erratic results, FWIW, but I've never seen this degree of variability.