"{ Package: 'jv:calipel/s/benchmarks/stx' }"

Object subclass:#BenchmarkSlopstone

	instanceVariableNames:'testParams testBlocks'

	classVariableNames:''

	poolDictionaries:''

	category:'CalipeL-S-Benchmarks-St/X'

!

!BenchmarkSlopstone class methodsFor:'documentation'!

documentation

"

    this is a low-level benchmark.

    The results are biased towards a 33Mhz/486, running ParcPlace VisualWorks v1.0.

    This configuration has a rating of 1 for all tests.

    claus:

      Since it tests highly optimizable code (adding constants, for example)

      which can be (and is) easily optimized away, the numbers given are

      not too valuable to compare different smalltalk systems.

      (it may be useful, to compare different machines running the same

       smalltalk, though).

    See also: readme method

    [author:]

        Bruce Samuelson

    [start with:]

        Benchmarks::SlopstoneBenchmark new runBenchmark

"

!

readme

"INTRODUCTION

Slopstone: Smalltalk Low level OPeration Stones

Portable Low Level Benchmarks for ST80 and ST/V (using 16-bit SmallIntegers)

Placed in public domain January 1993  (c) Bruce Samuelson

Permission is given to place this in public Smalltalk archives

Use monospaced fonts if possible to view the methods in this class.

(1) Collect garbage if supported (2) do 'SlopstoneBenchmark new runBenchmark'.

Results are printed in the Transcript window.

Post results for your machines to comp.lang.smalltalk or

mail them to bruce@ling.uta.edu or bruce@utafll.uta.edu.

DISCUSSION

This readme method would normally be in the class comment for ST80. ST/V-DOS

doesn't support class comments.

The benchmarks test strictly low level operations. They do not test higher

level operations such as forming sets, sorting, or streaming, nor do they test

applications. They also do not test user interface operations because of the

non-portability of this area of Smalltalk and its sensitivity to the

performance of the video subsystem. The tests are cpu bound. They do not

access files and should not cause disk paging.

The benchmarks use loop counts of 16000 because SmallIntegers cannot exceed

16383 for ST/V-DOS. 16-bit implementions would perform worse with large loop

counts. The benchmarks are also suitable for testing 32-bit versions of

Smalltalk.

DEFINITION OF REFERENCE MACHINE (ONE SLOPSTONE)

The following machine is the one on which I developed these benchmarks. By

convention it is defined to operate at one slopstone. It's a mid range

performer for current ParcPlace versions of Smalltalk.

Hardware: Amax 486DX/33 (includes internal floating point processor and

internal 8K cache), 256K external cache, 16MB RAM.

Software: ParcPlace VisualWorks 1.0, Windows 3.1, DOS 5.0 (plain vanilla

setup).

COMPARISON TO XEROX DORADO

For reference, the machine runs at 649% of a Dorado on ParcPlace benchmarks

for ST80 4.1. Its fast video card helps on these PPS benchmarks. I didn't run

them for VisualWorks 1.0. It would be somewhat slower because there are vastly

more classes.

SlopstoneBenchmark new runBenchmark

EXAMPLE RESULTS FOR REFERENCE MACHINE

1000s    time    1000s of

itera-   sec-    iterations   slop-

tions    onds    per sec      stones   explanation

3808     0.577   6600         1.0      add integers

 544     2.262    240         1.0      add floats

 960     1.088    882         1.0      access strings

 320     0.908    352         1.0      create objects

 160     1.49     107         1.0      copy objects

 480     1.129    425         1.0      perform selectors

 896     1.237    724         1.0      evaluate blocks

 640     1.151    555         1.0      harmonic mean"

!

results

"

        Benchmarks::SlopstoneBenchmark new runBenchmark

        Notice:

            the addInt, addFloat, perform and block benchmarks

            are optimized by the system (the JIT) and are therefore

            too short to be measured reasonably on most systems

            (time below 10ms)

        P6/400 linux/egcs/elf       

               3808     0.001     3808000     576.97     add integers

                544     0.001      544000    2266.67     add floats

                960     0.119        8067       9.14652  access strings

                320     0.08         4000      11.3636   create objects

                160     0.06         2667      24.9221   copy objects

                480     0.001      480000    1129.41     perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.00664     96275     173.513    harmonic mean

        P6/266 linux/gcc/elf       

               3808     0.001     3808000     576.97     add integers

                544     0.001      544000    2266.67     add floats

                960     0.137        7007       7.94478  access strings

                320     0.114        2807       7.97448  create objects

                160     0.106        1509      14.1069   copy objects

                480     0.001      480000    1129.41     perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.00773     82698     149.043    harmonic mean

        alpha/433 osf1/cc

                3808    0.001     3808000     576.97      add integers

                544     0.001      544000    2266.67      add floats

                960     0.152        6316       7.16076   access strings

                320     0.131        2443       6.93963   create objects

                160     0.102        1569      14.6601    copy objects

                480     0.001      480000    1129.41      perform selectors

                896     0.001      896000    1237.57      create & evaluate blocks

                640     0.00796     80317     144.753     harmonic mean

        Ultra/250 solaris2.6

                3808    0.001     3808000     576.97     add integers

                544     0.001      544000    2266.67     add floats

                960     0.154        6234       7.06776  access strings

                320     0.17         1882       5.34759  create objects

                160     0.113        1416      13.233    copy objects

                480     0.001      480000    1129.41     perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.0084      76116     137.181    harmonic mean

        P5/200 linux/gcc/elf (slow memory interface - notebook)      

                3808    0.001     3808000     576.97     add integers

                544     0.001      544000    2266.67     add floats

                960     0.315        3048       3.45535  access strings

                320     0.277        1155       3.28192  create objects

                160     0.206         777       7.25887  copy objects

                480     0.002      240000     564.706    perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.0120056   53276      96.0165   harmonic mean

        P5/133 linux/gcc/a.out      

                3808    0.002     1904000     288.485    add integers

                544     0.002      272000    1133.33     add floats

                960     0.36         2667       3.02343  access strings

                320     0.293        1092       3.1027   create objects

                160     0.241         664       6.20468  copy objects

                480     0.002      240000     564.706    perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.0153779     41593     74.9606  harmonic mean

        alpha/233 osf1/cc

                3808    0.001     3808000     576.97     add integers

                544     0.001      544000    2266.67     add floats

                960     0.57         1684       1.90954  access strings

                320     0.979         327       0.928591 create objects

                160     0.71          225       2.10609  copy objects

                480     0.001      480000    1129.41     perform selectors

                896     0.001      896000    1237.57     create & evaluate blocks

                640     0.0169151   37813      68.1485   harmonic mean

        ss10/40 solaris 2.5

                3808    0.006      634667      96.1616    add integers

                544     0.005      108800     453.333     add floats

                320     0.918         349       0.990295  create objects

                160     0.694         231       2.15465   copy objects

                480     0.006       80000     188.235     perform selectors

                896     0.005      179200     247.514     create & evaluate blocks

                640     0.0480143   13321      24.0082    harmonic mean

"

! !

!BenchmarkSlopstone methodsFor:'benchmarking'!

execute

| n nTests iters times speeds stones scale printA printB printC param

count speed0 expln block time iter speed stone harMean hm power |

Transcript cr; cr; show: 'Starting benchmarks...'.

n := 16000. "Number of times each test block will be evaluated."

n > 16383 ifTrue: [self halt: 'Count exceeded max small int for ST/V-DOS.'].

nTests := testParams size.

nTests  = testBlocks size ifFalse: [self halt: 'Inconsistent test count.'].

iters  := OrderedCollection new.

times  := OrderedCollection new.

speeds := OrderedCollection new.

stones := OrderedCollection new.

scale  := 1000. "So iterations can be reported as 1000s of iterations"

"The following blocks are restricted to two args by ST/V-DOS."

printA :=

  [:iter1 :time1 |

  Transcript cr.

  Transcript nextPutAll: (iter1 / scale) rounded printString.

  Transcript nextPutAll: '     '.

  Transcript nextPutAll: time1 printString.

  Transcript nextPutAll: '     '].

printB :=

  [:speed1 :slop1 |

  Transcript nextPutAll: (speed1 / scale) rounded printString.

  Transcript nextPutAll: '     '.

  Transcript nextPutAll: slop1 printString.

  Transcript nextPutAll: '     '].

printC :=

  [:expln1 |

  Transcript show: expln1].

Transcript show: '

1000s    time    1000s of

itera-   sec-    iterations   slop-

tions    onds    per sec      stones   explanation

'.

1 to: nTests do:

  [:i |

  param  := testParams at: i.

  count  := param at: 1.           "repetitions of a test inside its block"

  speed0 := (param at: 2) * scale. "iters/sec for a one-slopstone machine"

  expln  := param at: 3.

  block  := testBlocks at: i.

  time   := Time millisecondsToRun: [n timesRepeat: block].

  time   := (time max: 1) / 1000.0. "time is now in seconds"

  iter   := count * n.

  speed  := iter / time.

  stone  := speed / speed0.

  iters  add: iter.

  times  add: time.

  speeds add: speed.

  stones add: stone.

  printA value: iter value: time.

  printB value: speed value: stone.

  printC value: expln.].

harMean :=

  [:numbers |

  hm := 1.

  power := 1 / nTests.

  numbers do: [:number | hm := hm * (number raisedTo: power)].

  hm].

Transcript cr.

printA value: (harMean value: iters) value: (harMean value: times).

printB value: (harMean value: speeds) value: (harMean value: stones).

printC value: 'harmonic mean'.

Transcript cr; cr; show: 'Benchmarks complete.'; cr

!

extendedSetup

    self setup.

    testParams

      add: #(238 6600 'add integer vars *');

      add: #( 34  240 'add float vars *');

      add: #( 56  724 'create blocks *');

      add: #( 56  724 'evaluate blocks *');

      add: #( 56  724 'create self blocks *');

      add: #( 56  724 'evaluate self blocks *');

      add: #( 56  724 'create & evaluate self blocks *');

      add: #( 56  724 'create full blocks *');

      add: #( 56  724 'evaluate full blocks *');

      add: #( 56  724 'create & evaluate full blocks *');

      yourself.

    self setup4

!

runBenchmark

       "SlopstoneBenchmark new runBenchmark"

	self setup.

	self execute

!

runBenchmark2

       "SlopstoneBenchmark new runBenchmark2"

	self extendedSetup.

	self execute

!

setup

    self setup1.

    self setup2.

    self setup3.

!

setup1

"Numbers in testParams represent the following:

Column 1   number of internal repetitions for each test inside its block

Column 2   thousands of iterations per second for a one-slopstone machine."

testParams := OrderedCollection new.

testParams

  add: #(238 6600 'add integers');

  add: #( 34  240 'add floats');

  add: #( 60  882 'access strings');

  add: #( 20  352 'create objects');

  add: #( 10  107 'copy objects');

  add: #( 30  425 'perform selectors');

  add: #( 56  724 'create & evaluate blocks');

  yourself.

testBlocks := OrderedCollection new.

"In the integer addition test, I originally had 340 internal repetitions.

This caused the Digitalk compiler to blow up. The ParcPlace compiler compiled

it ok but interestingly addition performed at slightly more than 50% of the

performance with an internal count of 238. Perhaps something magical happens

at 256."

!

setup2

testBlocks

  add: [1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+

	1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1+1];

  add: [1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+

	1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0+1.0]

!

setup3

testBlocks

  add: ['a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1.

	'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1. 'a' at: 1];

  add: [Object new. Object new. Object new. Object new. Object new.

	Object new. Object new. Object new. Object new. Object new.

	Object new. Object new. Object new. Object new. Object new.

	Object new. Object new. Object new. Object new. Object new];

  add: [Object new copy copy copy copy copy copy copy copy copy copy];

  add: [0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself.

	0 perform: #yourself. 0 perform: #yourself. 0 perform: #yourself];

  add: [[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value.

	[] value. [] value. [] value. [] value. [] value. [] value. [] value]

!

setup4

  |b v|

testBlocks

  add: [b := 1.

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b];

  add: [b := 1.0.

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+

	b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b+b];

  add: [[]. []. [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] .

	[] . [] . [] . [] . [] . [] . [] ];

  add: [b := [].

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value];

  add: [[self]. [self]. [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] .

	[self] . [self] . [self] . [self] . [self] . [self] . [self] ];

  add: [b := [self].

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value];

  add: [[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value.

	[self] value. [self] value. [self] value. [self] value. [self] value. [self] value. [self] value];

  add: [[ v := 0]. [v := 0]. [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] .

	[v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] . [v := 0] ];

  add: [b := [v := 0].

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value.

	b value. b value. b value. b value. b value. b value. b value];

  add: [[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value.

	[v := 0] value. [self] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value. [v := 0] value]

! !