s/BenchmarkOutcome.st
author Jan Vrany <jan.vrany@fit.cvut.cz>
Wed, 03 Dec 2014 23:53:25 +0000
changeset 262 8d2849dd3227
parent 142 c69d1eb92d91
child 285 0cf54ee76de5
permissions -rw-r--r--
Added support for (user-defined) benchmark instruments. * Introduced `BenchmarkMeasurementInstrument`, an abstraction to allow pluggin in a different aspects to measure - execution time, number scavenges, number fo full GC's... * Introduced `BenchmarkMeasurement`, a value object that holds a result of measurement done by a single `BenchmarkMeasurementInstrument`. * Code refactored so now `BenchmarkOutcome` now keeps a set of `BenchmarkMeasure`s * Measurements from each instrument are in reports. JSON report keep 'times' array for backward compatibility. * Each platform may define it's set of intruments that is used by default and __ALWAYS__. They should be non-intrusive. * Added command line option `-i` / `--instrument` to specify user-defined instrument to use. *
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     1
"{ Package: 'jv:calipel/s' }"
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     2
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     3
Object subclass:#BenchmarkOutcome
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
     4
	instanceVariableNames:'measurements benchmark parameters'
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     5
	classVariableNames:''
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     6
	poolDictionaries:''
12
3a7ebd3da52f Class categories renamed from CalipeL/S to CalipeL-S...
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 6
diff changeset
     7
	category:'CalipeL-S-Core'
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     8
!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     9
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    10
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    11
!BenchmarkOutcome class methodsFor:'instance creation'!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    12
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    13
benchmark: benchmark parameters: parameters measurements: measurements
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    14
    ^ self new benchmark: benchmark parameters: parameters measurements: measurements
29
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    15
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    16
    "Created: / 24-11-2014 / 06:56:04 / Jan Vrany <jan.vrany@fit.cvut.cz>"
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    17
! !
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    18
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    19
!BenchmarkOutcome methodsFor:'accessing'!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    20
29
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    21
benchmark
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    22
    ^ benchmark
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    23
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    24
    "Created: / 11-06-2013 / 23:19:53 / Jan Vrany <jan.vrany@fit.cvut.cz>"
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    25
!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    26
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    27
best
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    28
    "Return the best run, i.e., run with minimal time"
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    29
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    30
    ^ measurements inject: measurements anyOne into:[ :best :each |
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    31
        | bestTime eachTime |
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    32
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    33
        bestTime := best detect:[:m | BenchmarkMeasurementInstrument isExecuttionTimeInstrument: m instrument ] ifNone:[ nil ].
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    34
        eachTime := best detect:[:m | BenchmarkMeasurementInstrument isExecuttionTimeInstrument: m instrument ] ifNone:[ nil ].
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    35
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    36
        bestTime notNil ifTrue:[ 
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    37
            eachTime notNil ifTrue:[ 
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    38
                eachTime value < bestTime value ifTrue:[ each ] ifFalse:[ best ]
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    39
            ] ifFalse:[
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    40
                best
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    41
            ]
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    42
        ] ifFalse:[ 
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    43
            eachTime notNil ifTrue:[ each ] ifFalse:[ best ]
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    44
        ].
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    45
    ].
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    46
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    47
    "Created: / 25-11-2014 / 01:18:25 / Jan Vrany <jan.vrany@fit.cvut.cz>"
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    48
    "Modified: / 02-12-2014 / 23:28:15 / Jan Vrany <jan.vrany@fit.cvut.cz>"
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    49
!
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    50
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    51
measurements
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    52
    ^ measurements
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    53
!
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    54
142
c69d1eb92d91 Include parameters and their values in JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 29
diff changeset
    55
parameters
c69d1eb92d91 Include parameters and their values in JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 29
diff changeset
    56
    ^ parameters
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    57
!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    58
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    59
time
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    60
    ^ (self times select:[:t | t notNil]) min
6
25b264cec44e Added parameter to BenchmarkResult to run each benchmark multiple times.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 5
diff changeset
    61
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    62
    "Modified (format): / 25-11-2014 / 01:23:40 / Jan Vrany <jan.vrany@fit.cvut.cz>"
6
25b264cec44e Added parameter to BenchmarkResult to run each benchmark multiple times.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 5
diff changeset
    63
!
25b264cec44e Added parameter to BenchmarkResult to run each benchmark multiple times.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 5
diff changeset
    64
25b264cec44e Added parameter to BenchmarkResult to run each benchmark multiple times.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 5
diff changeset
    65
times
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    66
    ^ measurements collect:[ :run | 
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    67
        | timeMeasurement |
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    68
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    69
        timeMeasurement := run detect:[:each | BenchmarkMeasurementInstrument isExecuttionTimeInstrument: each instrument ] ifNone:[ nil ].
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    70
        timeMeasurement value.
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    71
    ]
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    72
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    73
    "Modified: / 02-12-2014 / 23:27:48 / Jan Vrany <jan.vrany@fit.cvut.cz>"
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    74
! !
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    75
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    76
!BenchmarkOutcome methodsFor:'initialization'!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    77
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    78
benchmark: benchmarkArg parameters: parametersArg measurements: measurementsArg 
29
00d2eaa41853 Initial version of JSON report.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 12
diff changeset
    79
    benchmark := benchmarkArg.
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    80
    measurements := measurementsArg.
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    81
    parameters := parametersArg.
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    82
262
8d2849dd3227 Added support for (user-defined) benchmark instruments.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 142
diff changeset
    83
    "Created: / 24-11-2014 / 06:56:59 / Jan Vrany <jan.vrany@fit.cvut.cz>"
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    84
! !
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    85
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    86
!BenchmarkOutcome class methodsFor:'documentation'!
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    87
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    88
version_HG
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    89
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    90
    ^ '$Changeset: <not expanded> $'
5
8669edf62d9b Fixes to make it running on Squeak/Pharo.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 0
diff changeset
    91
!
8669edf62d9b Fixes to make it running on Squeak/Pharo.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 0
diff changeset
    92
8669edf62d9b Fixes to make it running on Squeak/Pharo.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 0
diff changeset
    93
version_MC
8669edf62d9b Fixes to make it running on Squeak/Pharo.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 0
diff changeset
    94
    ^ 'CalipeL_S-Core-JanVrany.7 5c300a20-c9d7-11e2-a959-606720e43e2c 2013-05-31T10:49:17 JanVrany'
0
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    95
! !
c86831d0f7e7 Initial version of CalipeL/S.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    96