parsers/smalltalk/PPSmalltalkTokenParser.st
author Jan Vrany <jan.vrany@fit.cvut.cz>
Mon, 23 Nov 2015 11:14:30 +0100
changeset 551 00ebb1b85f53
parent 417 3c0a91182e65
permissions -rw-r--r--
Fixed CI scripts on Windows For an unknown reason, unzip on Windows reports status code 50 (presumably "the disk is (or was) full during extraction.") even if there's plenty of space. To workaround this, simply ignore status code 50 on Windows. Sigh.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
385
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     1
"{ Package: 'stx:goodies/petitparser/parsers/smalltalk' }"
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     2
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     3
PPTokenParser subclass:#PPSmalltalkTokenParser
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     4
	instanceVariableNames:''
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     5
	classVariableNames:'SeparatorPredicate'
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     6
	poolDictionaries:''
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     7
	category:'PetitSmalltalk-Core'
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     8
!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
     9
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    10
PPSmalltalkTokenParser comment:'A parser that knows how to skip comments and whitespace in Smalltalk and how to instantiate tokens.'
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    11
!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    12
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    13
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    14
!PPSmalltalkTokenParser class methodsFor:'initialization'!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    15
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    16
initialize
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    17
	SeparatorPredicate := PPCharSetPredicate on: [ :char | char isSeparator ]
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    18
! !
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    19
417
3c0a91182e65 Smalltalk grammar updated to allow for Smalltalk/X EOL comments
Jan Vrany <jan.vrany@fit.cvut.cz>
parents: 385
diff changeset
    20
385
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    21
!PPSmalltalkTokenParser methodsFor:'parsing'!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    22
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    23
parseComments: anArray on: aPPContext
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    24
	| start comments |
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    25
	comments := anArray.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    26
	[ [ aPPContext atEnd not and: [ SeparatorPredicate value: aPPContext uncheckedPeek ] ]
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    27
		whileTrue: [ aPPContext next ].
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    28
	 aPPContext atEnd not and: [ aPPContext uncheckedPeek = $" ] ] whileTrue: [
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    29
		aPPContext next.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    30
		start := aPPContext position.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    31
		aPPContext upTo: $".
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    32
		comments := comments copyWith: (start to: aPPContext position) ].
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    33
	^ comments
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    34
!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    35
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    36
parseOn: aPPContext
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    37
	| memento comments token |
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    38
	memento := aPPContext remember.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    39
	comments := self
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    40
		parseComments: #()
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    41
		on: aPPContext.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    42
	token := super parseOn: aPPContext.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    43
	token isPetitFailure ifTrue: [
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    44
		aPPContext restore: memento.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    45
		^ token ].
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    46
	comments := self
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    47
		parseComments: comments
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    48
		on: aPPContext.
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    49
	^ token comments: comments
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    50
! !
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    51
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    52
!PPSmalltalkTokenParser methodsFor:'private'!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    53
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    54
defaultTokenClass
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    55
	^ PPSmalltalkToken
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    56
! !
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    57
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    58
!PPSmalltalkTokenParser class methodsFor:'documentation'!
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    59
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    60
version_HG
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    61
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    62
    ^ '$Changeset: <not expanded> $'
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    63
! !
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    64
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    65
44a36ed4e484 Commited a Smalltalk parser (MC package PetitSmalltalk)
Jan Vrany <jan.vrany@fit.cvut.cz>
parents:
diff changeset
    66
PPSmalltalkTokenParser initialize!