author | Claus Gittinger <cg@exept.de> |
Thu, 10 Aug 2017 10:05:49 +0200 | |
branch | cvs_MAIN |
changeset 1041 | 4413bae59f15 |
parent 976 | f67ddc355b4d |
permissions | -rw-r--r-- |
252
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
1 |
" |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
2 |
stx:goodies/smallsense - A productivity plugin for Smalltalk/X IDE |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
3 |
Copyright (C) 2013-2014 Jan Vrany |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
4 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
5 |
This library is free software; you can redistribute it and/or |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
6 |
modify it under the terms of the GNU Lesser General Public |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
7 |
License as published by the Free Software Foundation; either |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
8 |
version 2.1 of the License. |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
9 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
10 |
This library is distributed in the hope that it will be useful, |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
11 |
but WITHOUT ANY WARRANTY; without even the implied warranty of |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
12 |
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
13 |
Lesser General Public License for more details. |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
14 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
15 |
You should have received a copy of the GNU Lesser General Public |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
16 |
License along with this library; if not, write to the Free Software |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
17 |
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
18 |
" |
249
8bc64027b189
Package renamed to stx:goodies/smallsense
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
238
diff
changeset
|
19 |
"{ Package: 'stx:goodies/smallsense' }" |
199
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
20 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
21 |
"{ NameSpace: SmallSense }" |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
22 |
|
202
5c8610dad14c
TokenExpression* renamed to TokenPattern*
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
200
diff
changeset
|
23 |
Regex::RxMatcher subclass:#TokenPatternMatcher |
199
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
24 |
instanceVariableNames:'' |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
25 |
classVariableNames:'' |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
26 |
poolDictionaries:'' |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
27 |
category:'SmallSense-Utils-Matcher' |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
28 |
! |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
29 |
|
202
5c8610dad14c
TokenExpression* renamed to TokenPattern*
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
200
diff
changeset
|
30 |
!TokenPatternMatcher class methodsFor:'documentation'! |
199
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
31 |
|
252
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
32 |
copyright |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
33 |
" |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
34 |
stx:goodies/smallsense - A productivity plugin for Smalltalk/X IDE |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
35 |
Copyright (C) 2013-2014 Jan Vrany |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
36 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
37 |
This library is free software; you can redistribute it and/or |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
38 |
modify it under the terms of the GNU Lesser General Public |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
39 |
License as published by the Free Software Foundation; either |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
40 |
version 2.1 of the License. |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
41 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
42 |
This library is distributed in the hope that it will be useful, |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
43 |
but WITHOUT ANY WARRANTY; without even the implied warranty of |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
44 |
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
45 |
Lesser General Public License for more details. |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
46 |
|
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
47 |
You should have received a copy of the GNU Lesser General Public |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
48 |
License along with this library; if not, write to the Free Software |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
49 |
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
50 |
" |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
51 |
! |
feba6ee5c814
Added copyright notice and license information (LGPL2)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
249
diff
changeset
|
52 |
|
199
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
53 |
documentation |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
54 |
" |
211
8d5259c39445
Added support for CARET virtual token generated at cursor position (when provided)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
203
diff
changeset
|
55 |
A custom regex-like matcher to match token streams. |
199
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
56 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
57 |
[author:] |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
58 |
Jan Vrany <jan.vrany@fit.cvut.cz> |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
59 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
60 |
[instance variables:] |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
61 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
62 |
[class variables:] |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
63 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
64 |
[see also:] |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
65 |
|
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
66 |
" |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
67 |
! ! |
c2d7421cb317
Initial work on token stream matcher
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff
changeset
|
68 |
|
202
5c8610dad14c
TokenExpression* renamed to TokenPattern*
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
200
diff
changeset
|
69 |
!TokenPatternMatcher methodsFor:'accessing'! |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
70 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
71 |
subexpression: subIndex |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
72 |
"returns the matches for a parenthized subexpression. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
73 |
notice that non-matching subexpressions deliver an empty matchString; |
976 | 74 |
also be careful with nested parenthesis. |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
75 |
With index==1, you get the whole matchString" |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
76 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
77 |
| originalPosition start end reply | |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
78 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
79 |
originalPosition := self position. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
80 |
start := self subBeginning: subIndex. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
81 |
end := self subEnd: subIndex. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
82 |
(start isNil or: [end isNil]) ifTrue: [^'']. |
238
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
83 |
reply := (Array new: end first - start first) writeStream. |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
84 |
self position: start. |
238
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
85 |
start first to: end first - 1 do: [:ignored | reply nextPut: stream next]. |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
86 |
self position: originalPosition. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
87 |
^reply contents |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
88 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
89 |
" |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
90 |
|matcher| |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
91 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
92 |
matcher := Regex::RxMatcher new |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
93 |
initializeFromString:'(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[ ]+(:isDigit::isDigit:?)[ ]*,[ ]*19(:isDigit::isDigit:)' |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
94 |
ignoreCase:false. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
95 |
(matcher matches:'Aug 6, 1996') ifTrue:[ |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
96 |
matcher subexpression:2 |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
97 |
] ifFalse:[ |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
98 |
self error. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
99 |
]. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
100 |
" |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
101 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
102 |
"Created: / 06-05-2014 / 15:46:37 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
238
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
103 |
"Modified: / 13-06-2014 / 17:33:58 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
104 |
! ! |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
105 |
|
202
5c8610dad14c
TokenExpression* renamed to TokenPattern*
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
200
diff
changeset
|
106 |
!TokenPatternMatcher methodsFor:'double dispatch'! |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
107 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
108 |
syntaxToken: tokenNode |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
109 |
| type value | |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
110 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
111 |
type := tokenNode type. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
112 |
value := tokenNode value. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
113 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
114 |
^ Regex::RxmPredicate new predicate: |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
115 |
[:token | |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
116 |
(token isSymbol or:[token isCharacter]) ifTrue:[ |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
117 |
(type == token) and:[ value isNil or:[value == token ] ] |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
118 |
] ifFalse:[ |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
119 |
(type == token type) and:[ value isNil or:[value = token value] ] |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
120 |
]. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
121 |
]. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
122 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
123 |
"Created: / 06-05-2014 / 14:38:32 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
124 |
"Modified: / 06-05-2014 / 15:59:07 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
203
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
125 |
! |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
126 |
|
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
127 |
syntaxTokenSet: tokenSetNode |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
128 |
| tokens| |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
129 |
|
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
130 |
tokens := tokenSetNode tokens. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
131 |
|
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
132 |
^ Regex::RxmPredicate new predicate: |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
133 |
[:token | |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
134 |
| matches | |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
135 |
matches := tokens anySatisfy:[:tokenNode | |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
136 |
| type value | |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
137 |
|
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
138 |
type := tokenNode type. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
139 |
value := tokenNode value. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
140 |
(token isSymbol or:[token isCharacter]) ifTrue:[ |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
141 |
(type = token) and:[ value isNil or:[value == token ] ] |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
142 |
] ifFalse:[ |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
143 |
(type = token type) and:[ value isNil or:[value = token value] ] |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
144 |
]. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
145 |
]. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
146 |
tokenSetNode negated ifTrue:[ |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
147 |
matches := matches not. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
148 |
]. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
149 |
matches. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
150 |
]. |
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
151 |
|
c70b7351eda6
Changed TokenPatternParser to parse from string rather from literal array.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
202
diff
changeset
|
152 |
"Created: / 09-05-2014 / 16:22:42 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
153 |
! ! |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
154 |
|
202
5c8610dad14c
TokenExpression* renamed to TokenPattern*
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
200
diff
changeset
|
155 |
!TokenPatternMatcher methodsFor:'initialize-release'! |
200
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
156 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
157 |
initialize: syntaxTreeRoot ignoreCase: aBoolean |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
158 |
"Compile myself for the regex with the specified syntax tree. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
159 |
See comment and `building' protocol in this class and |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
160 |
#dispatchTo: methods in syntax tree components for details |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
161 |
on double-dispatch building. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
162 |
The argument is supposedly a RxsRegex." |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
163 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
164 |
ignoreCase := aBoolean. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
165 |
self buildFrom: syntaxTreeRoot. |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
166 |
"/ startOptimizer := RxMatchOptimizer new initialize: syntaxTreeRoot ignoreCase: aBoolean |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
167 |
|
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
168 |
"Created: / 06-05-2014 / 14:39:36 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
169 |
! ! |
12d6a2f82d95
Fixed basic token expression matching.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
199
diff
changeset
|
170 |
|
238
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
171 |
!TokenPatternMatcher methodsFor:'streaming'! |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
172 |
|
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
173 |
position |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
174 |
^ Array with: stream position with: stream sourceStream position |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
175 |
|
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
176 |
"Created: / 13-06-2014 / 16:52:37 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
177 |
! |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
178 |
|
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
179 |
position: positionsPair |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
180 |
[ |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
181 |
stream position:positionsPair first. |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
182 |
] on: Stream positionErrorSignal do:[ |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
183 |
"/ No that much tokens in backlog, reset backlog |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
184 |
"/ and set source stream's position |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
185 |
stream position: positionsPair first sourcePosition: positionsPair second. |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
186 |
] |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
187 |
|
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
188 |
"Created: / 13-06-2014 / 16:53:07 / Jan Vrany <jan.vrany@fit.cvut.cz>" |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
189 |
! ! |
d5a32e41181f
Fixes for token matcher: correctly (re)set position backwards.
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
211
diff
changeset
|
190 |
|
858 | 191 |
!TokenPatternMatcher class methodsFor:'documentation'! |
192 |
||
193 |
version_CVS |
|
194 |
||
195 |
^ '$Header$' |
|
196 |
! ! |
|
197 |