"{ Package: 'jv:calipel/s/benchmarks/stx' }"
"{ NameSpace: Smalltalk }"
Object subclass:#BenchmarkSlopstone
instanceVariableNames:'testParams testBlocks'
classVariableNames:''
poolDictionaries:''
category:'CalipeL-S-Benchmarks-St/X'
!
!BenchmarkSlopstone class methodsFor:'documentation'!
documentation
"
this is a low-level benchmark.
The results are biased towards a 33Mhz/486, running ParcPlace VisualWorks v1.0.
This configuration has a rating of 1 for all tests.
claus:
Since it tests highly optimizable code (adding constants, for example)
which can be (and is) easily optimized away, the numbers given are
not too valuable to compare different smalltalk systems.
(it may be useful, to compare different machines running the same
smalltalk, though).
See also: readme method
[author:]
Bruce Samuelson
[start with:]
Benchmarks::SlopstoneBenchmark new runBenchmark
"
!
readme
"INTRODUCTION
Slopstone: Smalltalk Low level OPeration Stones
Portable Low Level Benchmarks for ST80 and ST/V (using 16-bit SmallIntegers)
Placed in public domain January 1993 (c) Bruce Samuelson
Permission is given to place this in public Smalltalk archives
Use monospaced fonts if possible to view the methods in this class.
(1) Collect garbage if supported (2) do 'SlopstoneBenchmark new runBenchmark'.
Results are printed in the Transcript window.
Post results for your machines to comp.lang.smalltalk or
mail them to bruce@ling.uta.edu or bruce@utafll.uta.edu.
DISCUSSION
This readme method would normally be in the class comment for ST80. ST/V-DOS
doesn't support class comments.
The benchmarks test strictly low level operations. They do not test higher
level operations such as forming sets, sorting, or streaming, nor do they test
applications. They also do not test user interface operations because of the
non-portability of this area of Smalltalk and its sensitivity to the
performance of the video subsystem. The tests are cpu bound. They do not
access files and should not cause disk paging.
The benchmarks use loop counts of 16000 because SmallIntegers cannot exceed
16383 for ST/V-DOS. 16-bit implementions would perform worse with large loop
counts. The benchmarks are also suitable for testing 32-bit versions of
Smalltalk.
DEFINITION OF REFERENCE MACHINE (ONE SLOPSTONE)
The following machine is the one on which I developed these benchmarks. By
convention it is defined to operate at one slopstone. It's a mid range
performer for current ParcPlace versions of Smalltalk.
Hardware: Amax 486DX/33 (includes internal floating point processor and
internal 8K cache), 256K external cache, 16MB RAM.
Software: ParcPlace VisualWorks 1.0, Windows 3.1, DOS 5.0 (plain vanilla
setup).
COMPARISON TO XEROX DORADO
For reference, the machine runs at 649% of a Dorado on ParcPlace benchmarks
for ST80 4.1. Its fast video card helps on these PPS benchmarks. I didn't run
them for VisualWorks 1.0. It would be somewhat slower because there are vastly
more classes.
SlopstoneBenchmark new runBenchmark
EXAMPLE RESULTS FOR REFERENCE MACHINE
1000s time 1000s of
itera- sec- iterations slop-
tions onds per sec stones explanation
3808 0.577 6600 1.0 add integers
544 2.262 240 1.0 add floats
960 1.088 882 1.0 access strings
320 0.908 352 1.0 create objects
160 1.49 107 1.0 copy objects
480 1.129 425 1.0 perform selectors
896 1.237 724 1.0 evaluate blocks
640 1.151 555 1.0 harmonic mean"
!
results
"
Benchmarks::SlopstoneBenchmark new runBenchmark
Notice:
the addInt, addFloat, perform and block benchmarks
are optimized by the system (the JIT) and are therefore
too short to be measured reasonably on most systems
(time below 10ms)
P6/400 linux/egcs/elf
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.119 8067 9.14652 access strings
320 0.08 4000 11.3636 create objects
160 0.06 2667 24.9221 copy objects
480 0.001 480000 1129.41 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.00664 96275 173.513 harmonic mean
P6/266 linux/gcc/elf
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.137 7007 7.94478 access strings
320 0.114 2807 7.97448 create objects
160 0.106 1509 14.1069 copy objects
480 0.001 480000 1129.41 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.00773 82698 149.043 harmonic mean
alpha/433 osf1/cc
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.152 6316 7.16076 access strings
320 0.131 2443 6.93963 create objects
160 0.102 1569 14.6601 copy objects
480 0.001 480000 1129.41 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.00796 80317 144.753 harmonic mean
Ultra/250 solaris2.6
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.154 6234 7.06776 access strings
320 0.17 1882 5.34759 create objects
160 0.113 1416 13.233 copy objects
480 0.001 480000 1129.41 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.0084 76116 137.181 harmonic mean
P5/200 linux/gcc/elf (slow memory interface - notebook)
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.315 3048 3.45535 access strings
320 0.277 1155 3.28192 create objects
160 0.206 777 7.25887 copy objects
480 0.002 240000 564.706 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.0120056 53276 96.0165 harmonic mean
P5/133 linux/gcc/a.out
3808 0.002 1904000 288.485 add integers
544 0.002 272000 1133.33 add floats
960 0.36 2667 3.02343 access strings
320 0.293 1092 3.1027 create objects
160 0.241 664 6.20468 copy objects
480 0.002 240000 564.706 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.0153779 41593 74.9606 harmonic mean
alpha/233 osf1/cc
3808 0.001 3808000 576.97 add integers
544 0.001 544000 2266.67 add floats
960 0.57 1684 1.90954 access strings
320 0.979 327 0.928591 create objects
160 0.71 225 2.10609 copy objects
480 0.001 480000 1129.41 perform selectors
896 0.001 896000 1237.57 create & evaluate blocks
640 0.0169151 37813 68.1485 harmonic mean
ss10/40 solaris 2.5
3808 0.006 634667 96.1616 add integers
544 0.005 108800 453.333 add floats
320 0.918 349 0.990295 create objects
160 0.694 231 2.15465 copy objects
480 0.006 80000 188.235 perform selectors
896 0.005 179200 247.514 create & evaluate blocks
640 0.0480143 13321 24.0082 harmonic mean
"
! !
!BenchmarkSlopstone methodsFor:'benchmarking'!
execute
| n nTests iters times speeds stones scale printA printB printC param
count speed0 expln block time iter speed stone harMean hm power |
Transcript cr; cr; show: 'Starting benchmarks...'.
n := 16000. "Number of times each test block will be evaluated."
n > 16383 ifTrue: [self halt: 'Count exceeded max small int for ST/V-DOS.'].
nTests := testParams size.
nTests = testBlocks size ifFalse: [self halt: 'Inconsistent test count.'].
iters := OrderedCollection new.
times := OrderedCollection new.
speeds := OrderedCollection new.
stones := OrderedCollection new.
scale := 1000. "So iterations can be reported as 1000s of iterations"
"The following blocks are restricted to two args by ST/V-DOS."
printA :=
[:iter1 :time1 |
Transcript cr.
Transcript nextPutAll: (iter1 / scale) rounded printString.
Transcript nextPutAll: ' '.
Transcript nextPutAll: time1 printString.
Transcript nextPutAll: ' '].
printB :=
[:speed1 :slop1 |
Transcript nextPutAll: (speed1 / scale) rounded printString.
Transcript nextPutAll: ' '.
Transcript nextPutAll: slop1 printString.
Transcript nextPutAll: ' '].
printC :=
[:expln1 |
Transcript show: expln1].
Transcript show: '
1000s time 1000s of
itera- sec- iterations slop-
tions onds per sec stones explanation
'.
1 to: nTests do:
[:i |
param := testParams at: i.
count := param at: 1. "repetitions of a test inside its block"
speed0 := (param at: 2) * scale. "iters/sec for a one-slopstone machine"
expln := param at: 3.
block := testBlocks at: i.
time := Time millisecondsToRun: [n timesRepeat: block].
time := (time max: 1) / 1000.0. "time is now in seconds"
iter := count * n.
speed := iter / time.
stone := speed / speed0.
iters add: iter.
times add: time.
speeds add: speed.
stones add: stone.
printA value: iter value: time.
printB value: speed value: stone.
printC value: expln.].
harMean :=
[:numbers |
hm := 1.
power := 1 / nTests.
numbers do: [:number | hm := hm * (number raisedTo: power)].
hm].
Transcript cr.
printA value: (harMean value: iters) value: (harMean value: times).
printB value: (harMean value: speeds) value: (harMean value: stones).
printC value: 'harmonic mean'.
Transcript cr; cr; show: 'Benchmarks complete.'; cr
!
extendedSetup
self setup.
testParams
add: #(238 6600 'add integer vars *');
add: #( 34 240 'add float vars *');
add: #( 56 724 'create blocks *');
add: #( 56 724 'evaluate blocks *');
add: #( 56 724 'create self blocks *');
add: #( 56 724 'evaluate self blocks *');
add: #( 56 724 'create & evaluate self blocks *');
add: #( 56 724 'create full blocks *');
add: #( 56 724 'evaluate full blocks *');
add: #( 56 724 'create & evaluate full blocks *');
yourself.
self setup4
!
runBenchmark
"SlopstoneBenchmark new runBenchmark"
self setup.
self execute
!
runBenchmark2
"SlopstoneBenchmark new runBenchmark2"
self extendedSetup.
self execute
!
setup
self setup1.
self setup2.
self setup3.
!
setup1
"Numbers in testParams represent the following:
Column 1 number of internal repetitions for each test inside its block
Column 2 thousands of iterations per second for a one-slopstone machine."
testParams := OrderedCollection new.
testParams
add: #(238 6600 'add integers');
add: #( 34 240 'add floats');
add: #( 60 882 'access strings');
add: #( 20 352 'create objects');
add: #( 10 107 'copy objects');
add: #( 30 425 'perform selectors');
add: #( 56 724 'create & evaluate blocks');
yourself.
testBlocks := OrderedCollection new.
"In the integer addition test, I originally had 340 internal repetitions.
This caused the Digitalk compiler to blow up. The ParcPlace compiler compiled
it ok but interestingly addition performed at slightly more than 50% of the
performance with an internal count of 238. Perhaps something magical happens
at 256."
!
setup2
testBlocks
add: [1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+
1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1];
add: [1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+
1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0]
!
setup3
testBlocks
add: ['a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.
'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1];
add: [Object new. Object new. Object new. Object new. Object new.
Object new. Object new. Object new. Object new. Object new.
Object new. Object new. Object new. Object new. Object new.
Object new. Object new. Object new. Object new. Object new];
add: [Object new copy copy copy copy copy copy copy copy copy copy];
add: [0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.
0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself];
add: [[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value.
[] value. [] value. [] value. [] value. [] value. [] value. [] value]
!
setup4
|b v|
testBlocks
add: [b := 1.
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b];
add: [b := 1.0.
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+
b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b];
add: [[]. []. [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] .
[] . [] . [] . [] . [] . [] . [] ];
add: [b := [].
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value];
add: [[self]. [self]. [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] .
[self] . [self] . [self] . [self] . [self] . [self] . [self] ];
add: [b := [self].
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value];
add: [[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.
[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value];
add: [[ v := 0]. [v := 0]. [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .
[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] ];
add: [b := [v := 0].
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value.
b value. b value. b value. b value. b value. b value. b value];
add: [[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.
[v := 0] value. [self] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value]
! !
!BenchmarkSlopstone class methodsFor:'documentation'!
version_HG
^ '$Changeset: <not expanded> $'
! !