author | Claus Gittinger <cg@exept.de> |
Sat, 27 Oct 2018 08:43:11 +0200 | |
branch | cvs_MAIN |
changeset 3859 | 0c49a3597caa |
parent 3404 | 2127b5057d60 |
child 3507 | 041e88b3db01 |
permissions | -rw-r--r-- |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
1 |
#!/usr/bin/ruby |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
2 |
DOCUMENTATION = <<DOCEND |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
3 |
A simple script to run all tests on all supported JDKs. Use this to check |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
4 |
changes before pushing to upstream repository! |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
5 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
6 |
== Parameters == |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
7 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
8 |
Following parameters are currently supported: |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
9 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
10 |
* STX_LIBJAVA_RELEASE ..... which Java releases to test. Defaults to |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
11 |
'JDK6, JDK7, zulu7' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
12 |
* STX_LIBJAVA_ENABLE_JIT .. test with Java JIT on ("1") or off ("0"). |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
13 |
Defaults to off ("0") |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
14 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
15 |
This script takes parameter values, compute all possible combinations |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
16 |
and run tests on each combination, writing results to an output directory. |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
17 |
Parameter values are specified as a list of comma-separated values. |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
18 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
19 |
== Results == |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
20 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
21 |
After all tests finish, a short summay is written to the standard output. |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
22 |
In addition, this summary, test report in JUnit xml format as well and |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
23 |
tests' stdout is written to a new directory named "results-YYYY-MM-DD-HH-MM-SS" |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
24 |
where YYYY-MM-DD-HH-MM-SS is current date time. |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
25 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
26 |
== Examples == |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
27 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
28 |
Run all standard tests (to be run before push!): |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
29 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
30 |
./run-tests.rb |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
31 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
32 |
Run basic and Mauve test suites on JDK7 in both interpreted and JIT modes: |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
33 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
34 |
./run-tests.rb -D "STX_LIBJAVA_RELEASE=JDK7" -D "STX_LIBJAVA_ENABLE_JIT=1,0"\ |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
35 |
-p stx:libjava -p stx:libjava/tests/mauve |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
36 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
37 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
38 |
DOCEND |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
39 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
40 |
|
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
41 |
require 'rbconfig' |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
42 |
require 'fileutils' |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
43 |
require 'optparse' |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
44 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
45 |
PACKAGES = [ |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
46 |
'stx:libjava', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
47 |
'stx:libjava/tests/mauve', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
48 |
'stx:libjava/experiments', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
49 |
'stx:libjava/tools' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
50 |
] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
51 |
PARAM_NAMES=[ 'STX_LIBJAVA_ENABLE_JIT', 'STX_LIBJAVA_RELEASE' ] |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
52 |
PARAMS = { |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
53 |
'STX_LIBJAVA_ENABLE_JIT' => [ |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
54 |
#'1', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
55 |
'0' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
56 |
], |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
57 |
'STX_LIBJAVA_RELEASE' => [ |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
58 |
'JDK6', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
59 |
'JDK7', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
60 |
'zulu7' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
61 |
] |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
62 |
} |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
63 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
64 |
REPORT_DIR="results-#{Time.now.strftime("%Y-%m-%d-%H-%M-%S")}" |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
65 |
|
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
66 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
67 |
def win32? |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
68 |
return (RbConfig::CONFIG['host_os'] =~ /mingw32/) != nil |
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
69 |
end |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
70 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
71 |
def unix? |
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
72 |
if win32_wine? |
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
73 |
return false |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
74 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
75 |
return (RbConfig::CONFIG['host_os'] =~ /linux|solaris/) != nil |
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
76 |
end |
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
77 |
|
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
78 |
if win32? |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
79 |
REPORT_RUNNER = File.join(File.dirname(__FILE__), '..', '..', 'goodies', 'builder', 'reports' , 'report-runner.bat') |
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
80 |
else |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
81 |
REPORT_RUNNER = File.join(File.dirname(__FILE__), '..', '..', 'goodies', 'builder', 'reports' , 'report-runner.sh') |
3020
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
82 |
end |
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
83 |
|
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
84 |
|
e7c286435bfe
Fixed tests'run-all.rb to run under Windows as well.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3000
diff
changeset
|
85 |
|
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
86 |
def run_package(package, params = {}) |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
87 |
ident = [] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
88 |
setup = 'Stdin close' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
89 |
params.keys.sort.each do | name | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
90 |
value = params[name] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
91 |
ident.push("#{name}=#{value}") |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
92 |
ENV[name] = value |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
93 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
94 |
if ident.size > 0 then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
95 |
logf = File.join(REPORT_DIR, "#{package.tr(':/', '__')}-#{ident.join('-')}-Test.out") |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
96 |
ident = "#{ident.join('-')}" |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
97 |
cmd = %W[#{REPORT_RUNNER} -S #{setup} -D #{REPORT_DIR} -i #{ident} -r Builder::TestReport -p #{package}] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
98 |
#cmd = %W[#{REPORT_RUNNER} -S #{setup} -D #{REPORT_DIR} -i #{ident} -r Builder::TestReport ] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
99 |
else |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
100 |
logf = File.join(REPORT_DIR, "#{package.tr(':/', '__')}-Test.out") |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
101 |
ident = '' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
102 |
cmd = %W[#{REPORT_RUNNER} -S #{setup} -D #{REPORT_DIR} -r Builder::TestReport -p #{package}] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
103 |
end |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
104 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
105 |
puts "Running #{package} {#{ident}}" |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
106 |
puts cmd.join() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
107 |
pattern = /SUMMARY: (?<run>\d+) run, (?<passed>\d+) passed, (?<skipped>\d+) skipped, (?<failed>\d+) failed, (?<error>\d+) error/ |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
108 |
result = { |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
109 |
'run' => '?', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
110 |
'passed' => '?', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
111 |
'skipped' => '?', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
112 |
'failed' => '?', |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
113 |
'error' => '?' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
114 |
} |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
115 |
IO.popen(cmd + [ :err => [:child, :out]]) do | out | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
116 |
File.open(logf, "w") do | log | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
117 |
out.each do | line | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
118 |
log.puts line |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
119 |
puts line |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
120 |
match = pattern.match (line) |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
121 |
if match then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
122 |
result = { |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
123 |
'run' => match['run'], |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
124 |
'passed' => match['passed'], |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
125 |
'skipped' => match['skipped'], |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
126 |
'failed' => match['failed'], |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
127 |
'error' => match['error'] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
128 |
} |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
129 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
130 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
131 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
132 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
133 |
result['ident'] = ident |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
134 |
result['package'] = package |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
135 |
result['params'] = params |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
136 |
return result |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
137 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
138 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
139 |
def combine(params, bound = {}, &block) |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
140 |
if bound.size == params.size then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
141 |
yield bound |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
142 |
else |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
143 |
params_to_combine = params.keys.sort.select { | p | not bound.key? p } |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
144 |
param = params_to_combine.to_a().first() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
145 |
values = params[param] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
146 |
values.each do | value | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
147 |
combined = bound.clone() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
148 |
combined[param] = value |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
149 |
combine(params, combined, &block) |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
150 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
151 |
end |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
152 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
153 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
154 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
155 |
def write_results_txt_to_file(filename, results) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
156 |
File.open(filename, "a+") do |file| |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
157 |
write_results_txt(file, results) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
158 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
159 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
160 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
161 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
162 |
def write_results_txt(file, results) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
163 |
file.write("\n") |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
164 |
file.write(Time.now.to_s) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
165 |
file.write("\n") |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
166 |
params = PARAMS.keys.sort |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
167 |
values = [[ "Package"] + params + ['result', 'run', 'passed', 'skipped' , 'failed' , 'error']] |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
168 |
results.each do | result | |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
169 |
param_values = params.collect { | p | result['params'][p] } |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
170 |
result_text = '?' |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
171 |
if result['run'] != '0' then |
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
172 |
if result['error'] != '0' then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
173 |
result_text = 'ERROR' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
174 |
elsif result['failed'] != '0' then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
175 |
result_text = 'FAILED' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
176 |
else |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
177 |
result_text = 'passed' |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
178 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
179 |
end |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
180 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
181 |
row = [ result['package'] ] + param_values + [ result_text, result['run'], result['passed'], result['skipped'], result['failed'], result['error'] ] |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
182 |
values.push(row) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
183 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
184 |
max_lengths = values[0].map { |val| val.length } |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
185 |
values.each do |row| |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
186 |
row.each_with_index do |elem, index| |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
187 |
elem_size = elem.size |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
188 |
max_lengths[index] = elem_size if elem_size > max_lengths[index] |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
189 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
190 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
191 |
values.each do |val| |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
192 |
format = max_lengths.map { |length| "%#{length}s" }.join(" " * 3) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
193 |
file.write(format % val) |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
194 |
file.write("\n") |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
195 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
196 |
file.write("--\n") |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
197 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
198 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
199 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
200 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
201 |
def main() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
202 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
203 |
FileUtils.mkdir_p REPORT_DIR |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
204 |
results = [] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
205 |
packages = [] |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
206 |
params = {} |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
207 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
208 |
optparse = OptionParser.new do | opts | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
209 |
opts.banner = "Usage: run-tests.rb [-D NAME=VALUE [-D...]] [-p PACKAGE [-p ...]]" |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
210 |
opts.on('-D', '--define NAME=VALUES', "Define a parameter NAME with VALUES") do | define | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
211 |
name , value = define.split("=") |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
212 |
params[name] = value.split(",").collect { | e | e.strip } |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
213 |
end |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
214 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
215 |
opts.on('-p', '--package PACKAGE', "Run tests for PACKAGE") do | package | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
216 |
packages << package |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
217 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
218 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
219 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
220 |
opts.on(nil, '--help', "Prints this message") do |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
221 |
puts DOCUMENTATION |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
222 |
puts optparse.help() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
223 |
exit 0 |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
224 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
225 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
226 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
227 |
optparse.parse! |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
228 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
229 |
# Validate parameters |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
230 |
params.each do | name, values | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
231 |
if not PARAM_NAMES.include? name then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
232 |
puts "ERROR: Invalid parameter name: #{name}" |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
233 |
puts DOCUMENTATION |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
234 |
puts optparse.help() |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
235 |
exit 0 |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
236 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
237 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
238 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
239 |
PARAMS.each do | name, values | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
240 |
if not params.has_key? name then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
241 |
params[name] = values |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
242 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
243 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
244 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
245 |
if packages.size == 0 then |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
246 |
packages = PACKAGES |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
247 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
248 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
249 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
250 |
combine(params) do | bound | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
251 |
packages.each do | package | |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
252 |
results.push(run_package(package, bound)) |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
253 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
254 |
end |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
255 |
|
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
256 |
write_results_txt_to_file(File.join(REPORT_DIR, 'results.txt'), results) |
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
257 |
write_results_txt(STDOUT, results) |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
258 |
end |
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
259 |
|
3404
2127b5057d60
testing: improved run-all.rb script to run tests with different configurations.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
3324
diff
changeset
|
260 |
main() |
3000
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
261 |
|
08d798b4ef80
Added run-all.rb scrip to run all tests on several different configurations
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
262 |