| author | Jan Vrany <jan.vrany@fit.cvut.cz> |
| Fri, 24 Jul 2015 15:37:23 +0100 | |
| changeset 503 | ff58cd9f1f3c |
| parent 484 | e829f3860745 |
| parent 502 | 1e45d3c96ec5 |
| child 516 | 3b81c9e53352 |
| permissions | -rw-r--r-- |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
1 |
"{ Package: 'stx:goodies/petitparser/compiler' }"
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
2 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
3 |
"{ NameSpace: Smalltalk }"
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
4 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
5 |
PPCRewritingVisitor subclass:#PPCTokenizingVisitor |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
6 |
instanceVariableNames:'tokens' |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
7 |
classVariableNames:'' |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
8 |
poolDictionaries:'' |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
9 |
category:'PetitCompiler-Visitors' |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
10 |
! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
11 |
|
|
484
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
12 |
|
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
13 |
!PPCTokenizingVisitor methodsFor:'hooks'! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
14 |
|
|
459
4751c407bb40
Merged with PetitCompiler-JanKurs.20150510144201, PetitCompiler-Tests-JanKurs.20150510144201, PetitCompiler-Extras-Tests-JanKurs.20150510144201, PetitCompiler-Benchmarks-JanKurs.20150510144201
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
452
diff
changeset
|
15 |
afterAccept: node retval: parserNode |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
16 |
self isRoot ifTrue: [ |
|
459
4751c407bb40
Merged with PetitCompiler-JanKurs.20150510144201, PetitCompiler-Tests-JanKurs.20150510144201, PetitCompiler-Extras-Tests-JanKurs.20150510144201, PetitCompiler-Benchmarks-JanKurs.20150510144201
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
452
diff
changeset
|
17 |
| tokenizerNode whitespaceNode | |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
18 |
self change. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
19 |
tokens addLast: self eofToken. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
20 |
tokens do: [ :token | token unmarkForInline ]. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
21 |
|
|
502
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
22 |
whitespaceNode := tokens detect: [ :e | e isTrimmingTokenNode ] ifNone:[nil]. |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
23 |
whitespaceNode notNil ifTrue:[ |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
24 |
whitespaceNode := whitespaceNode whitespace copy |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
25 |
unmarkForInline; |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
26 |
name: 'consumeWhitespace'; |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
27 |
yourself |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
28 |
] ifFalse:[ |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
29 |
whitespaceNode := (PPCNilNode new) |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
30 |
name: 'consumeWhitespace'; |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
31 |
yourself |
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
32 |
]. |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
33 |
tokenizerNode := PPCTokenChoiceNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
34 |
children: tokens asArray; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
35 |
name: 'nextToken'; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
36 |
yourself. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
37 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
38 |
^ PPCTokenizingParserNode new |
|
459
4751c407bb40
Merged with PetitCompiler-JanKurs.20150510144201, PetitCompiler-Tests-JanKurs.20150510144201, PetitCompiler-Extras-Tests-JanKurs.20150510144201, PetitCompiler-Benchmarks-JanKurs.20150510144201
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
452
diff
changeset
|
39 |
parser: parserNode; |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
40 |
tokenizer: tokenizerNode; |
|
459
4751c407bb40
Merged with PetitCompiler-JanKurs.20150510144201, PetitCompiler-Tests-JanKurs.20150510144201, PetitCompiler-Extras-Tests-JanKurs.20150510144201, PetitCompiler-Benchmarks-JanKurs.20150510144201
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
452
diff
changeset
|
41 |
whitespace: whitespaceNode; |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
42 |
name: #'mainParser'; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
43 |
yourself |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
44 |
]. |
|
459
4751c407bb40
Merged with PetitCompiler-JanKurs.20150510144201, PetitCompiler-Tests-JanKurs.20150510144201, PetitCompiler-Extras-Tests-JanKurs.20150510144201, PetitCompiler-Benchmarks-JanKurs.20150510144201
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
452
diff
changeset
|
45 |
^ parserNode |
|
502
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
46 |
|
|
1e45d3c96ec5
Updated to PetitCompiler-JanVrany.135, PetitCompiler-Tests-JanKurs.93, PetitCompiler-Extras-Tests-JanVrany.16, PetitCompiler-Benchmarks-JanKurs.12
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
459
diff
changeset
|
47 |
"Modified: / 12-05-2015 / 01:37:57 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
|
452
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
48 |
! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
49 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
50 |
eofToken |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
51 |
| ws | |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
52 |
ws := PPCStarNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
53 |
child: (PPCMessagePredicateNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
54 |
message: #isSeparator; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
55 |
yourself); |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
56 |
yourself. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
57 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
58 |
^ PPCTrimmingTokenNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
59 |
child: PPCEndOfFileNode new; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
60 |
whitespace: ws; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
61 |
tokenClass: PPToken; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
62 |
yourself. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
63 |
! ! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
64 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
65 |
!PPCTokenizingVisitor methodsFor:'initialization'! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
66 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
67 |
initialize |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
68 |
super initialize. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
69 |
tokens := OrderedCollection new. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
70 |
! ! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
71 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
72 |
!PPCTokenizingVisitor methodsFor:'testing'! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
73 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
74 |
isRoot |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
75 |
^ openSet size = 1 |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
76 |
! ! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
77 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
78 |
!PPCTokenizingVisitor methodsFor:'tokens'! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
79 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
80 |
addToken: token |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
81 |
(tokens contains: [:e | e == token] ) ifFalse: [ |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
82 |
tokens addLast: token |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
83 |
] |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
84 |
! ! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
85 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
86 |
!PPCTokenizingVisitor methodsFor:'visiting'! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
87 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
88 |
visitActionNode: node |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
89 |
(node hasProperty: #trimmingToken) ifTrue: [ |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
90 |
self change. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
91 |
self addToken: node. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
92 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
93 |
^ PPCTokenConsumeNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
94 |
child: node; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
95 |
yourself |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
96 |
]. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
97 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
98 |
^ super visitActionNode: node |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
99 |
! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
100 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
101 |
visitTokenNode: node |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
102 |
self change. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
103 |
self addToken: node. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
104 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
105 |
^ PPCTokenConsumeNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
106 |
child: node; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
107 |
yourself. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
108 |
! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
109 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
110 |
visitTrimmingTokenNode: node |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
111 |
self change. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
112 |
self addToken: node. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
113 |
|
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
114 |
^ PPCTokenConsumeNode new |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
115 |
child: node; |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
116 |
yourself. |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
117 |
! ! |
|
9f4558b3be66
Updated to PetitCompiler-JanKurs.111, PetitCompiler-Tests-JanKurs.51, PetitCompiler-Benchmarks-JanKurs.7, added PetitCompiler-Extras-Tests-JanKurs.4
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
118 |
|
|
484
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
119 |
!PPCTokenizingVisitor class methodsFor:'documentation'! |
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
120 |
|
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
121 |
version_HG |
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
122 |
|
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
123 |
^ '$Changeset: <not expanded> $' |
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
124 |
! ! |
|
e829f3860745
Do not create intermediate collection when parsing sequence if not necesary.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
461
diff
changeset
|
125 |