# HG changeset patch # User Jan Vrany # Date 1440426854 -3600 # Node ID f6f68d32de73515067e398b774dba6ed9b8f1466 # Parent b5316ef152748f8645edc7a1d9bc57b0a4cff8ff Merged in PetitCompiler-JanVrany.170, PetitCompiler-Tests-JanKurs.116, PetitCompiler-Extras-Tests-JanKurs.29, PetitCompiler-Benchmarks-JanKurs.19 Name: PetitCompiler-JanVrany.170 Author: JanVrany Time: 24-08-2015, 03:19:51.340 PM UUID: c20a744f-3b41-4aaa-bb8a-71ce74a2a952 Name: PetitCompiler-Tests-JanKurs.116 Author: JanKurs Time: 24-08-2015, 11:37:54.332 AM UUID: 549e0927-358a-4a1b-8270-050ccfcb4217 Name: PetitCompiler-Extras-Tests-JanKurs.29 Author: JanKurs Time: 24-08-2015, 11:36:52.503 AM UUID: ea1dbb67-f884-4237-8f34-adb0677c0954 Name: PetitCompiler-Benchmarks-JanKurs.19 Author: JanKurs Time: 24-08-2015, 11:48:47.045 AM UUID: 1c342fdb-8ddd-4104-9c47-a8f589c51694 diff -r b5316ef15274 -r f6f68d32de73 compiler/Make.proto --- a/compiler/Make.proto Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/Make.proto Mon Aug 24 15:34:14 2015 +0100 @@ -34,7 +34,7 @@ # add the path(es) here:, # ********** OPTIONAL: MODIFY the next lines *** # LOCALINCLUDES=-Ifoo -Ibar -LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/analyzer -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser -I$(INCLUDE_TOP)/stx/libbasic -I$(INCLUDE_TOP)/stx/libbasic2 -I$(INCLUDE_TOP)/stx/libview +LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/analyzer -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser -I$(INCLUDE_TOP)/stx/libbasic -I$(INCLUDE_TOP)/stx/libbasic2 # if you need any additional defines for embedded C code, @@ -104,13 +104,8 @@ cd ../../../libbasic && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" cd ../../refactoryBrowser/parser && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" cd ../../../libbasic2 && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" - cd ../../../libview && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" cd ../ && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" - cd ../../../libview2 && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" cd ../parsers/smalltalk && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" - cd ../../sunit && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" - cd ../tests && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" - cd ../parsers/java && $(MAKE) "CFLAGS_LOCAL=$(GLOBALDEFINES)" @@ -143,20 +138,17 @@ $(OUTDIR)PPCASTUtilities.$(O) PPCASTUtilities.$(H): PPCASTUtilities.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCArguments.$(O) PPCArguments.$(H): PPCArguments.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCBridge.$(O) PPCBridge.$(H): PPCBridge.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCClass.$(O) PPCClass.$(H): PPCClass.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCClassBuilder.$(O) PPCClassBuilder.$(H): PPCClassBuilder.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeBlock.$(O) PPCCodeBlock.$(H): PPCCodeBlock.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGen.$(O) PPCCodeGen.$(H): PPCCodeGen.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilationError.$(O) PPCCompilationError.$(H): PPCCompilationError.st $(INCLUDE_TOP)/stx/libbasic/Error.$(H) $(INCLUDE_TOP)/stx/libbasic/Exception.$(H) $(INCLUDE_TOP)/stx/libbasic/GenericException.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilationWarning.$(O) PPCCompilationWarning.$(H): PPCCompilationWarning.st $(INCLUDE_TOP)/stx/libbasic/GenericException.$(H) $(INCLUDE_TOP)/stx/libbasic/Notification.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/UserNotification.$(H) $(INCLUDE_TOP)/stx/libbasic/Warning.$(H) $(STCHDR) $(OUTDIR)PPCCompiledMethod.$(O) PPCCompiledMethod.$(H): PPCCompiledMethod.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompiler.$(O) PPCCompiler.$(H): PPCCompiler.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenErrorStrategy.$(O) PPCCompilerTokenErrorStrategy.$(H): PPCCompilerTokenErrorStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenRememberStrategy.$(O) PPCCompilerTokenRememberStrategy.$(H): PPCCompilerTokenRememberStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenizingErrorStrategy.$(O) PPCCompilerTokenizingErrorStrategy.$(H): PPCCompilerTokenizingErrorStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenizingRememberStrategy.$(O) PPCCompilerTokenizingRememberStrategy.$(H): PPCCompilerTokenizingRememberStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCConfiguration.$(O) PPCConfiguration.$(H): PPCConfiguration.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCContext.$(O) PPCContext.$(H): PPCContext.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(STCHDR) $(OUTDIR)PPCContextMemento.$(O) PPCContextMemento.$(H): PPCContextMemento.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCDistinctScanner.$(O) PPCDistinctScanner.$(H): PPCDistinctScanner.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCGuard.$(O) PPCGuard.$(H): PPCGuard.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCIdGenerator.$(O) PPCIdGenerator.$(H): PPCIdGenerator.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCMethod.$(O) PPCMethod.$(H): PPCMethod.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -165,6 +157,7 @@ $(OUTDIR)PPCPluggableConfiguration.$(O) PPCPluggableConfiguration.$(H): PPCPluggableConfiguration.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCScanner.$(O) PPCScanner.$(H): PPCScanner.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCScannerCodeGenerator.$(O) PPCScannerCodeGenerator.$(H): PPCScannerCodeGenerator.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCScannerError.$(O) PPCScannerError.$(H): PPCScannerError.st $(INCLUDE_TOP)/stx/libbasic/Error.$(H) $(INCLUDE_TOP)/stx/libbasic/Exception.$(H) $(INCLUDE_TOP)/stx/libbasic/GenericException.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCScannerResultStrategy.$(O) PPCScannerResultStrategy.$(H): PPCScannerResultStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenGuard.$(O) PPCTokenGuard.$(H): PPCTokenGuard.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCompiledParser.$(O) PPCompiledParser.$(H): PPCompiledParser.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -176,6 +169,7 @@ $(OUTDIR)PEGFsaDeterminizator.$(O) PEGFsaDeterminizator.$(H): PEGFsaDeterminizator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaAbstractDeterminizator.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaEpsilonTransition.$(O) PEGFsaEpsilonTransition.$(H): PEGFsaEpsilonTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaGenerator.$(O) PEGFsaGenerator.$(H): PEGFsaGenerator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PEGFsaParserState.$(O) PEGFsaParserState.$(H): PEGFsaParserState.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaState.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaPredicateTransition.$(O) PEGFsaPredicateTransition.$(H): PEGFsaPredicateTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaSequenceDeterminizator.$(O) PEGFsaSequenceDeterminizator.$(H): PEGFsaSequenceDeterminizator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaAbstractDeterminizator.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaUncopiableState.$(O) PEGFsaUncopiableState.$(H): PEGFsaUncopiableState.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaState.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -188,22 +182,25 @@ $(OUTDIR)PPCDistinctResultStrategy.$(O) PPCDistinctResultStrategy.$(H): PPCDistinctResultStrategy.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfFileNode.$(O) PPCEndOfFileNode.$(H): PPCEndOfFileNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCFSACodeGen.$(O) PPCFSACodeGen.$(H): PPCFSACodeGen.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCodeGen.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCFSAVisitor.$(O) PPCFSAVisitor.$(H): PPCFSAVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInlinedMethod.$(O) PPCInlinedMethod.$(H): PPCInlinedMethod.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCMethod.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitor.$(O) PPCInliningVisitor.$(H): PPCInliningVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCListNode.$(O) PPCListNode.$(H): PPCListNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNilNode.$(O) PPCNilNode.$(H): PPCNilNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCNoResultStrategy.$(O) PPCNoResultStrategy.$(H): PPCNoResultStrategy.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableNode.$(O) PPCPluggableNode.$(H): PPCPluggableNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCProfilingContext.$(O) PPCProfilingContext.$(H): PPCProfilingContext.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCContext.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(STCHDR) $(OUTDIR)PPCRewritingVisitor.$(O) PPCRewritingVisitor.$(H): PPCRewritingVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenCodeGenerator.$(O) PPCTokenCodeGenerator.$(H): PPCTokenCodeGenerator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCodeGen.$(O) PPCTokenizingCodeGen.$(H): PPCTokenizingCodeGen.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCodeGen.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCTokenizingCompiler.$(O) PPCTokenizingCompiler.$(H): PPCTokenizingCompiler.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCompiler.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingConfiguration.$(O) PPCTokenizingConfiguration.$(H): PPCTokenizingConfiguration.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCConfiguration.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalCodeGen.$(O) PPCUniversalCodeGen.$(H): PPCUniversalCodeGen.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCodeGen.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalConfiguration.$(O) PPCUniversalConfiguration.$(H): PPCUniversalConfiguration.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCConfiguration.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalResultStrategy.$(O) PPCUniversalResultStrategy.$(H): PPCUniversalResultStrategy.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUnknownNode.$(O) PPCUnknownNode.$(H): PPCUnknownNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPTokenizingCompiledParser.$(O) PPTokenizingCompiledParser.$(H): PPTokenizingCompiledParser.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCompiledParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaEOFTransition.$(O) PEGFsaEOFTransition.$(H): PEGFsaEOFTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaPredicateTransition.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PEGFsaParserTransition.$(O) PEGFsaParserTransition.$(H): PEGFsaParserTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaPredicateTransition.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PEGFsaTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractActionNode.$(O) PPCAbstractActionNode.$(H): PPCAbstractActionNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAndNode.$(O) PPCAndNode.$(H): PPCAndNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCharSetPredicateNode.$(O) PPCCharSetPredicateNode.$(H): PPCCharSetPredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -240,6 +237,7 @@ $(OUTDIR)PPCTokenizingParserNode.$(O) PPCTokenizingParserNode.$(H): PPCTokenizingParserNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitor.$(O) PPCTokenizingVisitor.$(H): PPCTokenizingVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCRewritingVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTrimmingTokenNode.$(O) PPCTrimmingTokenNode.$(H): PPCTrimmingTokenNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalCodeGenerator.$(O) PPCUniversalCodeGenerator.$(H): PPCUniversalCodeGenerator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCodeGenerator.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCActionNode.$(O) PPCActionNode.$(H): PPCActionNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractActionNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCDeterministicChoiceNode.$(O) PPCDeterministicChoiceNode.$(H): PPCDeterministicChoiceNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCChoiceNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizingSequenceNode.$(O) PPCRecognizingSequenceNode.$(H): PPCRecognizingSequenceNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCSequenceNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -253,7 +251,7 @@ $(OUTDIR)PPCMappedActionNode.$(O) PPCMappedActionNode.$(H): PPCMappedActionNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractActionNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCActionNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarMessagePredicateNode.$(O) PPCTokenStarMessagePredicateNode.$(H): PPCTokenStarMessagePredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarSeparatorNode.$(O) PPCTokenStarSeparatorNode.$(H): PPCTokenStarSeparatorNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCTokenStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPActionParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPAndParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCharSetPredicate.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPChoiceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPContext.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEndOfFileParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEndOfInputParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEpsilonParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFailure.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFlattenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPListParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPNotParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPOptionalParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPluggableParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPToken.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTrimmingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java/PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBLiteralNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBLiteralValueNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBProgramNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBStatementNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBValueNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Character.$(H) $(INCLUDE_TOP)/stx/libbasic/Magnitude.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(INCLUDE_TOP)/stx/libbasic/UndefinedObject.$(H) $(STCHDR) +$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPActionParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPAndParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCharSetPredicate.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPChoiceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPContext.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEndOfFileParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEndOfInputParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEpsilonParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFailure.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFlattenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPListParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPNotParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPOptionalParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPluggableParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPToken.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTrimmingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBLiteralNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBLiteralValueNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBProgramNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBStatementNode.$(H) $(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser/RBValueNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Character.$(H) $(INCLUDE_TOP)/stx/libbasic/Magnitude.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(INCLUDE_TOP)/stx/libbasic/UndefinedObject.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r b5316ef15274 -r f6f68d32de73 compiler/Make.spec --- a/compiler/Make.spec Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/Make.spec Mon Aug 24 15:34:14 2015 +0100 @@ -64,20 +64,17 @@ PPCASTUtilities \ PPCArguments \ PPCBridge \ + PPCClass \ PPCClassBuilder \ PPCCodeBlock \ PPCCodeGen \ PPCCompilationError \ PPCCompilationWarning \ PPCCompiledMethod \ - PPCCompiler \ - PPCCompilerTokenErrorStrategy \ - PPCCompilerTokenRememberStrategy \ - PPCCompilerTokenizingErrorStrategy \ - PPCCompilerTokenizingRememberStrategy \ PPCConfiguration \ PPCContext \ PPCContextMemento \ + PPCDistinctScanner \ PPCGuard \ PPCIdGenerator \ PPCMethod \ @@ -86,6 +83,7 @@ PPCPluggableConfiguration \ PPCScanner \ PPCScannerCodeGenerator \ + PPCScannerError \ PPCScannerResultStrategy \ PPCTokenGuard \ PPCompiledParser \ @@ -97,6 +95,7 @@ PEGFsaDeterminizator \ PEGFsaEpsilonTransition \ PEGFsaGenerator \ + PEGFsaParserState \ PEGFsaPredicateTransition \ PEGFsaSequenceDeterminizator \ PEGFsaUncopiableState \ @@ -109,22 +108,25 @@ PPCDistinctResultStrategy \ PPCEndOfFileNode \ PPCFSACodeGen \ + PPCFSAVisitor \ PPCInlinedMethod \ PPCInliningVisitor \ PPCListNode \ PPCNilNode \ + PPCNoResultStrategy \ PPCPluggableNode \ PPCProfilingContext \ PPCRewritingVisitor \ PPCTokenCodeGenerator \ PPCTokenizingCodeGen \ - PPCTokenizingCompiler \ PPCTokenizingConfiguration \ + PPCUniversalCodeGen \ PPCUniversalConfiguration \ PPCUniversalResultStrategy \ PPCUnknownNode \ PPTokenizingCompiledParser \ PEGFsaEOFTransition \ + PEGFsaParserTransition \ PPCAbstractActionNode \ PPCAndNode \ PPCCharSetPredicateNode \ @@ -161,6 +163,7 @@ PPCTokenizingParserNode \ PPCTokenizingVisitor \ PPCTrimmingTokenNode \ + PPCUniversalCodeGenerator \ PPCActionNode \ PPCDeterministicChoiceNode \ PPCRecognizingSequenceNode \ @@ -192,20 +195,17 @@ $(OUTDIR_SLASH)PPCASTUtilities.$(O) \ $(OUTDIR_SLASH)PPCArguments.$(O) \ $(OUTDIR_SLASH)PPCBridge.$(O) \ + $(OUTDIR_SLASH)PPCClass.$(O) \ $(OUTDIR_SLASH)PPCClassBuilder.$(O) \ $(OUTDIR_SLASH)PPCCodeBlock.$(O) \ $(OUTDIR_SLASH)PPCCodeGen.$(O) \ $(OUTDIR_SLASH)PPCCompilationError.$(O) \ $(OUTDIR_SLASH)PPCCompilationWarning.$(O) \ $(OUTDIR_SLASH)PPCCompiledMethod.$(O) \ - $(OUTDIR_SLASH)PPCCompiler.$(O) \ - $(OUTDIR_SLASH)PPCCompilerTokenErrorStrategy.$(O) \ - $(OUTDIR_SLASH)PPCCompilerTokenRememberStrategy.$(O) \ - $(OUTDIR_SLASH)PPCCompilerTokenizingErrorStrategy.$(O) \ - $(OUTDIR_SLASH)PPCCompilerTokenizingRememberStrategy.$(O) \ $(OUTDIR_SLASH)PPCConfiguration.$(O) \ $(OUTDIR_SLASH)PPCContext.$(O) \ $(OUTDIR_SLASH)PPCContextMemento.$(O) \ + $(OUTDIR_SLASH)PPCDistinctScanner.$(O) \ $(OUTDIR_SLASH)PPCGuard.$(O) \ $(OUTDIR_SLASH)PPCIdGenerator.$(O) \ $(OUTDIR_SLASH)PPCMethod.$(O) \ @@ -214,6 +214,7 @@ $(OUTDIR_SLASH)PPCPluggableConfiguration.$(O) \ $(OUTDIR_SLASH)PPCScanner.$(O) \ $(OUTDIR_SLASH)PPCScannerCodeGenerator.$(O) \ + $(OUTDIR_SLASH)PPCScannerError.$(O) \ $(OUTDIR_SLASH)PPCScannerResultStrategy.$(O) \ $(OUTDIR_SLASH)PPCTokenGuard.$(O) \ $(OUTDIR_SLASH)PPCompiledParser.$(O) \ @@ -225,6 +226,7 @@ $(OUTDIR_SLASH)PEGFsaDeterminizator.$(O) \ $(OUTDIR_SLASH)PEGFsaEpsilonTransition.$(O) \ $(OUTDIR_SLASH)PEGFsaGenerator.$(O) \ + $(OUTDIR_SLASH)PEGFsaParserState.$(O) \ $(OUTDIR_SLASH)PEGFsaPredicateTransition.$(O) \ $(OUTDIR_SLASH)PEGFsaSequenceDeterminizator.$(O) \ $(OUTDIR_SLASH)PEGFsaUncopiableState.$(O) \ @@ -237,22 +239,25 @@ $(OUTDIR_SLASH)PPCDistinctResultStrategy.$(O) \ $(OUTDIR_SLASH)PPCEndOfFileNode.$(O) \ $(OUTDIR_SLASH)PPCFSACodeGen.$(O) \ + $(OUTDIR_SLASH)PPCFSAVisitor.$(O) \ $(OUTDIR_SLASH)PPCInlinedMethod.$(O) \ $(OUTDIR_SLASH)PPCInliningVisitor.$(O) \ $(OUTDIR_SLASH)PPCListNode.$(O) \ $(OUTDIR_SLASH)PPCNilNode.$(O) \ + $(OUTDIR_SLASH)PPCNoResultStrategy.$(O) \ $(OUTDIR_SLASH)PPCPluggableNode.$(O) \ $(OUTDIR_SLASH)PPCProfilingContext.$(O) \ $(OUTDIR_SLASH)PPCRewritingVisitor.$(O) \ $(OUTDIR_SLASH)PPCTokenCodeGenerator.$(O) \ $(OUTDIR_SLASH)PPCTokenizingCodeGen.$(O) \ - $(OUTDIR_SLASH)PPCTokenizingCompiler.$(O) \ $(OUTDIR_SLASH)PPCTokenizingConfiguration.$(O) \ + $(OUTDIR_SLASH)PPCUniversalCodeGen.$(O) \ $(OUTDIR_SLASH)PPCUniversalConfiguration.$(O) \ $(OUTDIR_SLASH)PPCUniversalResultStrategy.$(O) \ $(OUTDIR_SLASH)PPCUnknownNode.$(O) \ $(OUTDIR_SLASH)PPTokenizingCompiledParser.$(O) \ $(OUTDIR_SLASH)PEGFsaEOFTransition.$(O) \ + $(OUTDIR_SLASH)PEGFsaParserTransition.$(O) \ $(OUTDIR_SLASH)PPCAbstractActionNode.$(O) \ $(OUTDIR_SLASH)PPCAndNode.$(O) \ $(OUTDIR_SLASH)PPCCharSetPredicateNode.$(O) \ @@ -289,6 +294,7 @@ $(OUTDIR_SLASH)PPCTokenizingParserNode.$(O) \ $(OUTDIR_SLASH)PPCTokenizingVisitor.$(O) \ $(OUTDIR_SLASH)PPCTrimmingTokenNode.$(O) \ + $(OUTDIR_SLASH)PPCUniversalCodeGenerator.$(O) \ $(OUTDIR_SLASH)PPCActionNode.$(O) \ $(OUTDIR_SLASH)PPCDeterministicChoiceNode.$(O) \ $(OUTDIR_SLASH)PPCRecognizingSequenceNode.$(O) \ diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsa.st --- a/compiler/PEGFsa.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsa.st Mon Aug 24 15:34:14 2015 +0100 @@ -64,8 +64,8 @@ ^ startState ! -stateNamed: name - ^ states detect: [ :e | e name = name ] +stateNamed: stateName + ^ states detect: [ :e | e name = stateName ] ! states @@ -203,6 +203,10 @@ ^ false ! +hasNoRetvals + ^ self finalStates isEmpty +! + is: state furtherThan: anotherState ^ (distances at: state) >= (distances at: anotherState) @@ -294,6 +298,7 @@ topologicalOrder | collection | collection := OrderedCollection new. + self statesReachableFrom: startState openSet: collection. ^ collection ! ! @@ -321,7 +326,7 @@ ! hash - ^ states hash bitXor: (startState bitXor: name) + ^ states hash bitXor: (startState hash bitXor: name hash) ! isIsomorphicTo: anotherFsa @@ -360,7 +365,9 @@ ]. states := map values asIdentitySet. - startState := map at: startState. + startState isNil ifFalse: [ + startState := map at: startState. + ]. states do: [ :s | s transitions do: [:t | @@ -369,59 +376,6 @@ ] ! ! -!PEGFsa methodsFor:'gt'! - -gtGraphViewIn: composite - - composite roassal2 - title: 'Graph'; - initializeView: [ RTMondrian new ]; - painting: [ :view | - self viewGraphOn: view. - ]. -! - -gtStringViewIn: composite - - - composite text - title: 'Textual Representation'; - display: [ :fsa | fsa asString ] -! - -viewGraphOn: b - b shape circle size: 50. - b shape color: Color gray muchLighter muchLighter. - b shape withText: #gtName. - b nodes: (self nonFinalStates). - - b shape circle size: 50. - b shape color: Color gray muchLighter. - b shape withText: #gtName. - b nodes: (self finalStates). - - b shape arrowedLine. - b edges - connectToAll: [ :state | - state transitions select: [:t | (self isBackTransition:t) not] - thenCollect: #destination ] - labelled: [ :t | (self transitionFrom: t key to: t value) gtName ]. - - b shape arrowedLine. - b shape color: Color red. - b edges - connectToAll: [ :state | - state transitions select: [:t | (self isBackTransition: t) ] - thenCollect: #destination ] - labelled: [ :t | (self transitionFrom: t key to: t value) gtName ]. - - - b layout horizontalTree . - b layout layout horizontalGap: 30. - - ^ b -! ! - !PEGFsa methodsFor:'ids'! defaultName @@ -433,7 +387,7 @@ ! prefix - ^ nil + ^ 'scan' ! suffix @@ -496,10 +450,6 @@ fromState addTransition: transition ! -addTransitionFrom: fromState to: toState onPredicate: block - self addTransitionFrom: fromState to: toState onPredicate: block priority: 0 -! - addTransitionFrom: fromState to: toState onPredicate: block priority: priority | transition | transition := PEGFsaPredicateTransition new @@ -560,6 +510,12 @@ ^ PEGFsaMinimizator new minimize: self ! +removeFinals + self finalStates do: [ :s | + s final: false + ] +! + removePriorities self states select: [ :s| s hasPriority ] thenDo: [ :s | s priority: 0 @@ -775,6 +731,7 @@ checkConsistency self assert: (states includes: startState). + states do: [ :s | s transitions do: [ :t | self assert: (states includes: t destination). ] ]. diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaCharacterTransition.st --- a/compiler/PEGFsaCharacterTransition.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsaCharacterTransition.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PEGFsaTransition subclass:#PEGFsaCharacterTransition - instanceVariableNames:'' + instanceVariableNames:'characterSet' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-FSA' diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaGenerator.st --- a/compiler/PEGFsaGenerator.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsaGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -262,6 +262,18 @@ ^ self visitPredicateNode: node ! +visitNilNode: node + "not much to do here..." + | startState | + startState := PEGFsaState new. + + ^ PEGFsa new + addState: startState; + startState: startState; + finalState: startState; + yourself +! + visitNode: node self error: 'node not supported' ! @@ -373,6 +385,10 @@ ^ self visit: node child ! +visitTokenWhitespaceNode: node + ^ self visit: node child +! + visitTrimmingTokenCharacterNode: node "I do not care about trimming (so far), it should be handled by TokenCodeGenerator" ^ self visit: node child @@ -381,5 +397,23 @@ visitTrimmingTokenNode: node "I do not care about trimming (so far), it should be handled by TokenCodeGenerator" ^ self visit: node child +! + +visitUnknownNode: node + | state fsa | + + state := PEGFsaParserState new. + state name: 'Unknown Parser'. + state parser: node parser. + + fsa := PEGFsa new + addState: state; + + startState: state; + finalState: state; + + yourself. + + ^ fsa ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaParserState.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PEGFsaParserState.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,21 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PEGFsaState subclass:#PEGFsaParserState + instanceVariableNames:'parser' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-FSA' +! + +!PEGFsaParserState methodsFor:'accessing'! + +parser + ^ parser +! + +parser: anObject + parser := anObject +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaParserTransition.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PEGFsaParserTransition.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,33 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PEGFsaPredicateTransition subclass:#PEGFsaParserTransition + instanceVariableNames:'parser' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-FSA' +! + +!PEGFsaParserTransition methodsFor:'accessing'! + +parser + ^ parser +! + +parser: anObject + self halt: 'delete me!!'. + parser := anObject +! ! + +!PEGFsaParserTransition methodsFor:'as yet unclassified'! + +acceptsCodePoint: codePoint + self halt: 'should not be sent...'. + ^ true +! + +isParser + ^ true +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaPredicateTransition.st --- a/compiler/PEGFsaPredicateTransition.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsaPredicateTransition.st Mon Aug 24 15:34:14 2015 +0100 @@ -65,6 +65,11 @@ ^ false ! +isParser + self halt: 'delete me!!'. + ^ false +! + isPredicateTransition ^ true ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaState.st --- a/compiler/PEGFsaState.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsaState.st Mon Aug 24 15:34:14 2015 +0100 @@ -60,20 +60,6 @@ ^ self isFinal and: [ self info isFsaFailure ] ! -multivalue - - ^ self isMultivalue - - "Modified: / 17-08-2015 / 12:03:10 / Jan Vrany " -! - -multivalue: anObject - self flag: 'JK: Obsolete?'. - "multivalue := anObject" - - "Modified: / 17-08-2015 / 12:03:39 / Jan Vrany " -! - name ^ name ! @@ -126,6 +112,11 @@ ^ infos values ! +transition + self assert: transitions size == 1. + ^ transitions anyOne +! + transitions ^ transitions ! ! @@ -558,12 +549,9 @@ ! isFailure - self error: 'Obsolete?'. - " - ^ self isFinal and: [ retval class == PEGFsaFailure ] - " + ^ self isFinal" and: [ retval class == PEGFsaFailure ]" - "Modified: / 17-08-2015 / 12:01:54 / Jan Vrany " + "Modified: / 24-08-2015 / 15:31:52 / Jan Vrany " ! isFinal @@ -611,7 +599,6 @@ state isFinal ifFalse: [ ^ newState retval: self retval ]. (self priority = state priority) ifTrue: [ - newState multivalue: true. ^ newState retval: { self retval . state retval }. ]. diff -r b5316ef15274 -r f6f68d32de73 compiler/PEGFsaTransition.st --- a/compiler/PEGFsaTransition.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PEGFsaTransition.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" Object subclass:#PEGFsaTransition - instanceVariableNames:'destination priority characterSet' + instanceVariableNames:'destination priority' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-FSA' @@ -19,14 +19,6 @@ !PEGFsaTransition methodsFor:'accessing'! -characterSet - ^ characterSet -! - -characterSet: anObject - characterSet := anObject -! - destination ^ destination ! @@ -88,7 +80,6 @@ postCopy super postCopy. - characterSet := characterSet copy. ! ! !PEGFsaTransition methodsFor:'gt'! @@ -109,10 +100,6 @@ !PEGFsaTransition methodsFor:'modifications'! -addCharacter: character - characterSet at: character codePoint put: true -! - decreasePriority self decreasePriorityBy: 1 ! @@ -178,7 +165,7 @@ !PEGFsaTransition methodsFor:'testing'! accepts: character - ^ characterSet at: character codePoint + ^ self subclassResponsibility ! isCharacterTransition diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCArguments.st --- a/compiler/PPCArguments.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCArguments.st Mon Aug 24 15:34:14 2015 +0100 @@ -32,6 +32,7 @@ ! codeGenerator + self error: 'deprecated?'. ^ self at: #codeGenerator ifAbsent: PPCCodeGenerator ! @@ -95,6 +96,14 @@ self set: #parserName to: value. ! +parserSuperclass + ^ self at: #parserSuperclass ifAbsent: PPTokenizingCompiledParser +! + +parserSuperclass: value + self set: #parserSuperclass to: value. +! + profile ^ self at: #profile ifAbsent: false ! @@ -119,6 +128,14 @@ self set: #scannerName to: value. ! +scannerSuperclass + ^ self at: #scannerSuperclass ifAbsent: PPCDistinctScanner +! + +scannerSuperclass: value + self set: #scannerSuperclass to: value. +! + specialize ^ self at: #specialize ifAbsent: true ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCClass.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCClass.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,298 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Object subclass:#PPCClass + instanceVariableNames:'methodDictionary currentMethod constants idGen arguments + methodStack returnVariable properties' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Compiler-Codegen' +! + +!PPCClass methodsFor:'accessing'! + +arguments: args + arguments := args +! + +constants + ^ constants +! + +currentMethod + ^ currentMethod +! + +currentNonInlineMethod + ^ methodStack + detect:[:m | m isInline not ] + ifNone:[ self error: 'No non-inlined method'] + + "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " +! + +currentReturnVariable + ^ currentMethod returnVariable +! + +idGen + ^ idGen +! + +idGen: anObject + idGen := anObject +! + +ids + ^ idGen ids +! + +methodDictionary + ^ methodDictionary +! + +name + ^ self propertyAt: #name +! + +name: value + ^ self propertyAt: #name put: value +! + +superclass + ^ self propertyAt: #superclass +! + +superclass: value + ^ self propertyAt: #superclass put: value +! ! + +!PPCClass methodsFor:'accessing-properties'! + +hasProperty: aKey + "Test if the property aKey is present." + + ^ properties notNil and: [ properties includesKey: aKey ] +! + +properties + ^ properties +! + +properties: aDictionary + properties := aDictionary +! + +propertyAt: aKey + ^ self propertyAt: aKey ifAbsent: [ nil ] +! + +propertyAt: aKey ifAbsent: aBlock + "Answer the property value associated with aKey or, if aKey isn't found, answer the result of evaluating aBlock." + + ^ properties isNil + ifTrue: [ aBlock value ] + ifFalse: [ properties at: aKey ifAbsent: aBlock ] +! + +propertyAt: aKey ifAbsentPut: aBlock + "Answer the property associated with aKey or, if aKey isn't found store the result of evaluating aBlock as new value." + + ^ self propertyAt: aKey ifAbsent: [ self propertyAt: aKey put: aBlock value ] +! + +propertyAt: aKey put: anObject + "Set the property at aKey to be anObject. If aKey is not found, create a new entry for aKey and set is value to anObject. Answer anObject." + + ^ (properties ifNil: [ properties := Dictionary new: 1 ]) + at: aKey put: anObject +! ! + +!PPCClass methodsFor:'constants'! + +addConstant: value as: name + (constants includesKey: name) ifTrue:[ + (constants at: name) ~= value ifTrue:[ + self error:'Duplicate constant!!'. + ]. + ^ self. + ]. + constants at: name put: value + + "Modified: / 29-05-2015 / 07:22:39 / Jan Vrany " +! ! + +!PPCClass methodsFor:'ids'! + +asSelector: string + "e.g. '234znak 43 ) 2' asLegalSelector = #v234znak432" + + | toUse | + + toUse := string select: [:char | char isAlphaNumeric or: [ char = $_ ] ]. + (toUse isEmpty or: [ toUse first isLetter not ]) + ifTrue: [ toUse := 'v', toUse ]. + toUse first isUppercase ifFalse:[ + toUse := toUse copy. + toUse at: 1 put: toUse first asLowercase + ]. + ^toUse + + "Modified: / 10-05-2015 / 07:29:57 / Jan Vrany " +! + +idFor: anObject + ^ idGen idFor: anObject +! + +idFor: anObject defaultName: defaultName + ^ idGen idFor: anObject defaultName: defaultName +! + +numberIdFor: object + ^ idGen numericIdFor: object +! ! + +!PPCClass methodsFor:'initialization'! + +initialize + super initialize. + + methodStack := Stack new. + methodDictionary := IdentityDictionary new. + constants := Dictionary new. + idGen := PPCIdGenerator new. +! ! + +!PPCClass methodsFor:'method cache'! + +cachedMethod: id + ^ methodDictionary at: id ifAbsent: [ nil ] +! + +cachedMethod: id ifPresent: aBlock + ^ methodDictionary at: id ifPresent: aBlock +! + +store: method as: id + self assert: (method isKindOf: PPCMethod). + methodDictionary at: id put: method. +! ! + +!PPCClass methodsFor:'support'! + +parsedValueOf: aBlock to: aString + | tmpVarirable method | + + self assert:aBlock isBlock. + self assert:aString isNil not. + tmpVarirable := returnVariable. + returnVariable := aString. + method := [ + aBlock value + ] ensure:[ returnVariable := tmpVarirable ]. + + self assert: (method isMethod). + ^ method +! + +pop + | retval | + retval := methodStack pop. + currentMethod := methodStack isEmpty + ifTrue: [ nil ] + ifFalse: [ methodStack top ]. + ^ retval + + "Modified: / 21-11-2014 / 12:27:25 / Jan Vrany " +! + +push + methodStack push: currentMethod. + (methodStack size > 500 )ifTrue: [ self error: 'unless it is very complex grammar, there is an error somewhere' ] + + "Modified: / 21-11-2014 / 12:27:18 / Jan Vrany " +! + +returnVariable + self error: 'Should never be called and accessed outside this class'. + ^ returnVariable +! + +startInline + | indentationLevel | + indentationLevel := currentMethod indentationLevel. + + currentMethod := PPCInlinedMethod new. + currentMethod returnVariable: returnVariable. + currentMethod indentationLevel: indentationLevel. + self push. + + "Modified: / 01-06-2015 / 21:48:35 / Jan Vrany " +! + +startInline: id + | indentationLevel | + (methodDictionary includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. + indentationLevel := currentMethod indentationLevel. + + currentMethod := PPCInlinedMethod new. + currentMethod id: id. + currentMethod returnVariable: returnVariable. + currentMethod indentationLevel: indentationLevel. + self push. + + "Modified: / 01-06-2015 / 21:48:35 / Jan Vrany " +! + +startMethod: id category: category + (methodDictionary includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. + + currentMethod := PPCMethod new. + currentMethod id: id. + currentMethod category: category. + + self push. + self store: currentMethod as: id. + + "Modified: / 01-06-2015 / 21:19:41 / Jan Vrany " +! + +stopInline + ^ self pop. + + "Modified: / 01-06-2015 / 21:37:59 / Jan Vrany " +! + +stopMethod + self store: currentMethod as: currentMethod methodName. + ^ self pop. +! ! + +!PPCClass methodsFor:'variables'! + +allocateReturnVariable + ^ self allocateReturnVariableNamed: 'retval' + + "Created: / 23-04-2015 / 18:03:40 / Jan Vrany " + "Modified: / 15-06-2015 / 17:52:56 / Jan Vrany " +! + +allocateReturnVariableNamed: name + "Allocate (or return previously allocated one) temporary variable used for + storing a parser's return value (the parsed object)" + ^ currentMethod allocateReturnVariableNamed: name + + "Created: / 15-06-2015 / 18:04:48 / Jan Vrany " +! + +allocateTemporaryVariableNamed: preferredName + "Allocate a new variable with (preferably) given name. + Returns a real variable name that should be used." + + ^ self currentNonInlineMethod allocateTemporaryVariableNamed: preferredName + + "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCodeGen.st --- a/compiler/PPCCodeGen.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCCodeGen.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,8 +3,7 @@ "{ NameSpace: Smalltalk }" Object subclass:#PPCCodeGen - instanceVariableNames:'compilerStack compiledParser methodCache currentMethod constants - returnVariable arguments idGen' + instanceVariableNames:'clazz arguments' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Compiler-Codegen' @@ -32,195 +31,176 @@ arguments := args ! -constants - ^ constants +clazz + ^ clazz +! + +clazz: aPPCClass + clazz := aPPCClass ! currentMethod - ^ currentMethod -! - -currentNonInlineMethod - ^ compilerStack - detect:[:m | m isInline not ] - ifNone:[ self error: 'No non-inlined method'] - - "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " -! - -currentReturnVariable - ^ currentMethod returnVariable + ^ clazz currentMethod ! idGen - ^ idGen + ^ clazz idGen ! -idGen: anObject - idGen := anObject +idGen: idGenerator + ^ clazz idGen: idGenerator ! ids - ^ idGen ids + ^ clazz idGen ids ! methodCategory ^ 'generated' -! - -methodDictionary - ^ methodCache -! - -methodFor: object - | id | - id := self idFor: object. - ^ methodCache at: id ifAbsent: [ nil ] ! ! -!PPCCodeGen methodsFor:'code generation'! - -add: string - currentMethod add: string. -! +!PPCCodeGen methodsFor:'caching'! -addConstant: value as: name - (constants includesKey: name) ifTrue:[ - (constants at: name) ~= value ifTrue:[ - self error:'Duplicate constant!!'. - ]. - ^ self. - ]. - constants at: name put: value - - "Modified: / 29-05-2015 / 07:22:39 / Jan Vrany " -! - -addOnLine: string - currentMethod addOnLine: string. -! - -addVariable: name - ^ self currentNonInlineMethod addVariable: name - - "Modified: / 23-04-2015 / 17:34:02 / Jan Vrany " +cacheMethod: method as: id + ^ clazz store: method as: id ! -call: anotherMethod - currentMethod add: anotherMethod call. -! - -callOnLine: anotherMethod - currentMethod addOnLine: anotherMethod call. -! - -dedent - currentMethod dedent -! - -indent - currentMethod indent -! - -nl - currentMethod nl +cachedMethod: id + ^ clazz cachedMethod: id ! -smartRemember: parser to: variableName - parser isContextFree ifTrue: [ - self codeAssign: 'context lwRemember.' - to: variableName. - ] ifFalse: [ - self codeAssign: 'context remember.' - to: variableName. - ] -! - -smartRestore: parser from: mementoName - parser isContextFree ifTrue: [ - self add: 'context lwRestore: ', mementoName, '.'. - ] ifFalse: [ - self add: 'context restore: ', mementoName, '.'. - ] +cachedMethod: id ifPresent: aBlock + ^ clazz cachedMethod: id ifPresent: aBlock ! ! -!PPCCodeGen methodsFor:'coding'! +!PPCCodeGen methodsFor:'code'! -code:aStringOrBlockOrRBParseNode - currentMethod code: aStringOrBlockOrRBParseNode +code: aStringOrBlockOrRBParseNode + clazz currentMethod code: aStringOrBlockOrRBParseNode "Created: / 01-06-2015 / 23:49:11 / Jan Vrany " ! codeAssert: aCode - self add: 'self assert: (', aCode, ').'. -! - -codeAssign: code to: variable - self assert: variable isNil not. - - "TODO JK: Hack alert, whatever is magic constant!!" - (variable == #whatever) ifFalse: [ - "Do not assign, if somebody does not care!!" - self add: variable ,' := ', code. - ] -! - -codeAssignParsedValueOf:aBlock to:aString - | tmpVarirable method | - - self assert:aBlock isBlock. - self assert:aString isNil not. - tmpVarirable := returnVariable. - returnVariable := aString. - method := [ - aBlock value - ] ensure:[ returnVariable := tmpVarirable ]. - self assert: (method isKindOf: PPCMethod). - method isInline ifTrue:[ - self callOnLine:method - ] ifFalse:[ - self codeEvaluateAndAssign:(method call) to:aString. - ] - - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " + self code: 'self assert: (', aCode, ').'. ! codeBlock: contents - currentMethod codeBlock: contents + clazz currentMethod codeBlock: contents "Created: / 01-06-2015 / 22:35:32 / Jan Vrany " ! codeCall: aMethod self assert: (aMethod isKindOf: PPCMethod). - self add: aMethod call. + self code: aMethod call. ! -codeClearError - self add: 'self clearError.'. -! - -codeComment: string - currentMethod add: '"', string, '"'. +codeCallOnLine: aMethod + self assert: (aMethod isKindOf: PPCMethod). + self codeOnLine: aMethod call. ! codeDot - self addOnLine:'.'. + self codeOnLine: '.'. "Created: / 16-06-2015 / 06:09:07 / Jan Vrany " ! -codeError - self add: 'self error: ''message notspecified''.'. +codeNl + self code: ''. +! + +codeOnLine:aStringOrBlockOrRBParseNode + clazz currentMethod codeOnLine: aStringOrBlockOrRBParseNode + + "Created: / 01-06-2015 / 23:49:11 / Jan Vrany " +! + +codeReturn + clazz currentMethod isInline ifTrue: [ + "If inlined, the return variable already holds the value" + ] ifFalse: [ + arguments profile ifTrue:[ + self codeProfileStop. + ]. + self code: '^ ', clazz currentMethod returnVariable + ]. + + "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " + "Modified: / 01-06-2015 / 21:49:04 / Jan Vrany " +! + +codeReturn: code + " - returns whatever is in code OR + - assigns whatever is in code into the returnVariable" + clazz currentMethod isInline ifTrue:[ + self codeEvaluateAndAssign: code to: clazz currentMethod returnVariable. + ] ifFalse: [ + arguments profile ifTrue:[ + self codeProfileStop. + ]. + self code: '^ '. + self codeOnLine: code + ] + + "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " + "Modified: / 01-06-2015 / 21:48:51 / Jan Vrany " ! -codeError: errorMessage - self add: 'self error: ''', errorMessage, '''.' +codeReturnParsedValueOf: aBlock + | method | + + method := clazz parsedValueOf: aBlock to: clazz currentReturnVariable. + + method isInline ifTrue:[ + self codeCallOnLine: method. + self codeReturn: clazz currentReturnVariable. + ] ifFalse:[ + self codeReturn: method call. + + ] + + "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " +! ! + +!PPCCodeGen methodsFor:'code assignment'! + +codeAssign: stringOrBlock to: variable + self assert: variable isNil not. + + stringOrBlock isString ifTrue: [ + ^ self codeAssignString: stringOrBlock to: variable + ]. + + (stringOrBlock isKindOf: BlockClosure) ifTrue: [ + ^ self codeAssignParsedValueOf: stringOrBlock to: variable + ]. + + self error: 'unknown argument'. ! -codeError: errorMessage at: position - self add: 'self error: ''', errorMessage, ''' at: ', position asString, '.' +codeAssignParsedValueOf:aBlock to: variable + | method | + method := clazz parsedValueOf: aBlock to: variable . + + method isInline ifTrue:[ + self codeCallOnLine:method + ] ifFalse:[ + self codeAssignString: (method call) to: variable. + ] + + "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " +! + +codeAssignString: string to: variable + self assert: variable isNil not. + + "TODO JK: Hack alert, whatever is magic constant!!" + (variable == #whatever) ifFalse: [ + "Do not assign, if somebody does not care!!" + self code: variable ,' := ', string. + ] ! codeEvaluate: selector argument: argument on: variable @@ -229,65 +209,110 @@ "TODO JK: Hack alert, whatever is magic constant!!" (variable == #whatever) ifFalse: [ "Do not assign, if somebody does not care!!" - self add: variable, ' ', selector,' ', argument. + self code: variable, ' ', selector,' ', argument. ] ifTrue: [ "In case argument has a side effect" - self add: argument + self code: argument + ] +! + +codeEvaluateAndAssign: stringOrBlock to: variable + "Contrary to codeAssign:to: I always put code onto the stream" + stringOrBlock isString ifTrue: [ + self codeEvaluateAndAssignString: stringOrBlock to: variable + ] ifFalse: [ + self assert: (stringOrBlock isKindOf: BlockClosure). + self codeEvaluateAndAssignParsedValueOf: stringOrBlock to: variable ] ! -codeEvaluateAndAssign: argument to: variable +codeEvaluateAndAssignParsedValueOf: aBlock to: variable + | method | + method := clazz parsedValueOf: aBlock to: variable . + + + method isInline ifFalse: [ + self codeEvaluateAndAssignString: method call to: variable. + ] ifTrue: [ + "if inlined, the variable is already filled in, just call it" + self code: method call + ] +! + +codeEvaluateAndAssignString: string to: variable + "Contrary to codeAssign:to: I always put code onto the stream" + self assert: string isString. self assert: variable isNil not. "TODO JK: Hack alert, whatever is magic constant!!" (variable == #whatever) ifFalse: [ - "Do not assign, if somebody does not care!!" - self add: variable ,' := ', argument. + self codeAssignString: string to: variable ] ifTrue: [ - "In case an argument has a side effect" - self add: argument. + "In case code has a side effect" + self code: string. ] +! ! + +!PPCCodeGen methodsFor:'code debugging'! + +codeComment: string + self code: '"', string, '"'. ! codeHalt - self add: 'self halt. ' + self code: 'self halt. ' ! codeHaltIfShiftPressed arguments debug ifTrue: [ ((Smalltalk respondsTo: #isSmalltalkX) and:[Smalltalk isSmalltalkX]) ifFalse:[ - self add: 'Halt ifShiftPressed.' + self code: 'Halt ifShiftPressed.' ] ] "Modified: / 10-05-2015 / 07:39:47 / Jan Vrany " ! -codeIf: condition then: then - self codeIf: condition then: then else: nil +codeProfileStart + self code: 'context methodInvoked: #', clazz currentMethod methodName, '.' + + "Created: / 01-06-2015 / 21:17:19 / Jan Vrany " +! - "Created: / 16-06-2015 / 06:07:06 / Jan Vrany " +codeProfileStop + self code: 'context methodFinished: #', clazz currentMethod methodName, '.' + + "Created: / 01-06-2015 / 21:19:11 / Jan Vrany " +! + +codeTranscriptShow: text + (arguments profile) ifTrue: [ + self code: 'Transcript show: ', text storeString, '; cr.'. + ] ! -codeIf: condition then: then else: else - currentMethod - add: '('; - codeOnLine: condition; - addOnLine: ')'. - then notNil ifTrue:[ - currentMethod - addOnLine:' ifTrue: '; - codeBlock: then. - ]. - else notNil ifTrue:[ - currentMethod - addOnLine:' ifFalse: '; - codeBlock: else. - ]. - self codeDot. +profileTokenRead: tokenName + arguments profile ifTrue: [ + self code: 'context tokenRead: ', tokenName storeString, '.' + ] +! ! + +!PPCCodeGen methodsFor:'code error handling'! - "Created: / 01-06-2015 / 22:43:15 / Jan Vrany " - "Modified: / 16-06-2015 / 06:09:33 / Jan Vrany " +codeClearError + self code: 'error := false.'. +! + +codeError + self code: 'self error: ''message notspecified''.'. +! + +codeError: errorMessage + self code: 'self error: ''', errorMessage, '''.' +! + +codeError: errorMessage at: position + self code: 'self error: ''', errorMessage, ''' at: ', position asString, '.' ! codeIfErrorThen: then @@ -300,343 +325,156 @@ ^ self codeIf: 'error' then: then else: else "Created: / 16-06-2015 / 06:05:56 / Jan Vrany " +! ! + +!PPCCodeGen methodsFor:'code primitives'! + +add: string + self error: 'deprecated?'. + clazz currentMethod add: string. ! -codeNextToken - self add: 'self nextToken.' - - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 23-04-2015 / 20:51:41 / Jan Vrany " -! - -codeOnLIne:aStringOrBlockOrRBParseNode - currentMethod codeOnLine: aStringOrBlockOrRBParseNode - - "Created: / 01-06-2015 / 23:49:11 / Jan Vrany " +addConstant: value as: name + clazz addConstant: value as: name ! -codeParsedValueOf: aBlock - | tmpVarirable method | - - self assert: aBlock isBlock. - tmpVarirable := returnVariable. - returnVariable := #whatever. - method := [ - aBlock value - ] ensure:[ returnVariable := tmpVarirable ]. - self assert: returnVariable == tmpVarirable. - self assert: (method isKindOf: PPCMethod). - - self codeCall: method. +addOnLine: string + self error: 'deprecated'. + clazz currentMethod addOnLine: string. ! -codeProfileStart - self add: 'context methodInvoked: #', currentMethod methodName, '.' +addVariable: name + ^ clazz currentNonInlineMethod addVariable: name - "Created: / 01-06-2015 / 21:17:19 / Jan Vrany " -! - -codeProfileStop - self add: 'context methodFinished: #', currentMethod methodName, '.' - - "Created: / 01-06-2015 / 21:19:11 / Jan Vrany " + "Modified: / 23-04-2015 / 17:34:02 / Jan Vrany " ! -codeReturn - currentMethod isInline ifTrue: [ - "If inlined, the return variable already holds the value" - ] ifFalse: [ - arguments profile ifTrue:[ - self codeProfileStop. - ]. - self add: '^ ', currentMethod returnVariable - ]. +call: anotherMethod + self error: 'deprecated?'. + clazz currentMethod add: anotherMethod call. +! - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 01-06-2015 / 21:49:04 / Jan Vrany " +callOnLine: anotherMethod + self error: 'deprecated?'. + clazz currentMethod addOnLine: anotherMethod call. ! -codeReturn: code - " - returns whatever is in code OR - - assigns whatever is in code into the returnVariable" - currentMethod isInline ifTrue:[ - self codeEvaluateAndAssign: code to: currentMethod returnVariable. - ] ifFalse: [ - arguments profile ifTrue:[ - self codeProfileStop. - ]. - self add: '^ ', code - ] - - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 01-06-2015 / 21:48:51 / Jan Vrany " +dedent + clazz currentMethod dedent ! -codeReturnParsedValueOf: aBlock - | tmpVarirable method | +indent + clazz currentMethod indent +! ! - self assert:aBlock isBlock. - tmpVarirable := returnVariable. - method := aBlock value. - self assert: returnVariable == tmpVarirable. - self assert: (method isKindOf: PPCMethod). - method isInline ifTrue:[ - self callOnLine:method. - self codeReturn: returnVariable. - ] ifFalse:[ - self codeReturn: method call. - - ] +!PPCCodeGen methodsFor:'code structures'! - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " +codeIf: condition then: then + self codeIf: condition then: then else: nil + + "Created: / 16-06-2015 / 06:07:06 / Jan Vrany " ! -codeStoreValueOf: aBlock intoVariable: aString - | tmpVarirable method | - self assert: aBlock isBlock. - self assert: aString isNil not. - - tmpVarirable := returnVariable. - returnVariable := aString. - method := [ - aBlock value - ] ensure: [ - returnVariable := tmpVarirable +codeIf: condition then: then else: else + self + code: '('; + codeOnLine: condition; + codeOnLine: ')'. + then notNil ifTrue:[ + self + codeOnLine:' ifTrue: '; + codeBlock: then. ]. - - method isInline ifTrue: [ - self callOnLine: method - ] ifFalse: [ - self codeEvaluateAndAssign: (method call) to: aString. - ] - - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " -! + else notNil ifTrue:[ + self + codeOnLine:' ifFalse: '; + codeBlock: else. + ]. + self codeDot. -codeTokenGuard: node ifFalse: codeBlock - | guard id | - guard := PPCTokenGuard on: node. - (guard makesSense) ifTrue: [ - id := self idFor: guard firstToken. - - self add: 'self ', id asString, ' ifFalse: ['. - self indent. - codeBlock value. - self dedent. - self add: '].'. - ] -! - -codeTranscriptShow: text - (arguments profile) ifTrue: [ - self add: 'Transcript show: ', text storeString, '; cr.'. - ] + "Created: / 01-06-2015 / 22:43:15 / Jan Vrany " + "Modified: / 16-06-2015 / 06:09:33 / Jan Vrany " ! ! !PPCCodeGen methodsFor:'ids'! -asSelector: string - "e.g. '234znak 43 ) 2' asLegalSelector = #v234znak432" - - | toUse | - - toUse := string select: [:char | char isAlphaNumeric or: [ char = $_ ] ]. - (toUse isEmpty or: [ toUse first isLetter not ]) - ifTrue: [ toUse := 'v', toUse ]. - toUse first isUppercase ifFalse:[ - toUse := toUse copy. - toUse at: 1 put: toUse first asLowercase - ]. - ^toUse - - "Modified: / 10-05-2015 / 07:29:57 / Jan Vrany " -! - idFor: anObject - ^ idGen idFor: anObject + ^ clazz idFor: anObject ! idFor: anObject defaultName: defaultName - ^ idGen idFor: anObject defaultName: defaultName -! - -idFor: object prefixed: prefix - ^ self idFor: object prefixed: prefix suffixed: '' -! - -idFor: object prefixed: prefix suffixed: suffix - self error: 'Should no longer be used'. - " - | name id | - ^ idCache at: object ifAbsentPut: [ - ((object canHavePPCId) and: [object name isNotNil]) ifTrue: [ - ""Do not use prefix, if there is a name"" - name := self asSelector: (object name asString). - id := (name, suffix) asSymbol. - - ""Make sure, that the generated ID is uniqe!!"" - (idCache includes: id) ifTrue: [ - (id, '_', idCache size asString) asSymbol - ] ifFalse: [ - id - ] - ] ifFalse: [ - (prefix, '_', (idCache size asString), suffix) asSymbol - ] - ] - " - - "Modified: / 17-08-2015 / 12:00:28 / Jan Vrany " -! - -idFor: object suffixed: suffix - self assert: (object isKindOf: PPCNode) description: 'Shold use PPCNode for ids'. - ^ self idFor: object prefixed: object prefix suffixed: suffix effect: #none + ^ clazz idFor: anObject defaultName: defaultName ! numberIdFor: object - ^ idGen numericIdFor: object + ^ clazz numberIdFor: object ! ! !PPCCodeGen methodsFor:'initialization'! -copy: parser - self halt: 'deprecated?'. - ^ parser transform: [ :p | p copy ]. -! - initialize super initialize. - compilerStack := Stack new. - methodCache := IdentityDictionary new. - constants := Dictionary new. - idGen := PPCIdGenerator new. -! ! - -!PPCCodeGen methodsFor:'profiling'! - -profileTokenRead: tokenName - arguments profile ifTrue: [ - self add: 'context tokenRead: ', tokenName storeString, '.' - ] + clazz := PPCClass new. ! ! !PPCCodeGen methodsFor:'support'! -cache: id as: value - methodCache at: id put: value. -! - -cachedValue: id - ^ methodCache at: id ifAbsent: [ nil ] -! - -cachedValue: id ifPresent: block - ^ methodCache at: id ifPresent: block -! +startInline + ^ clazz startInline -checkCache: id - | method | - self flag: 'deprecated?'. - - "Check if method is hand written" - method := compiledParser ifNotNil: [ compiledParser compiledMethodAt: id ifAbsent: [ nil ] ]. - method ifNotNil: [ ^ PPCCompiledMethod new id: id; yourself ]. - - ^ self cachedValue: id -! - -pop - | retval | - retval := compilerStack pop. - currentMethod := compilerStack isEmpty - ifTrue: [ nil ] - ifFalse: [ compilerStack top ]. - ^ retval - - "Modified: / 21-11-2014 / 12:27:25 / Jan Vrany " -! - -push - compilerStack push: currentMethod. - (compilerStack size > 500 )ifTrue: [ self error: 'unless it is very complex grammar, there is an error somewhere' ] - - "Modified: / 21-11-2014 / 12:27:18 / Jan Vrany " + "Modified: / 01-06-2015 / 21:48:35 / Jan Vrany " ! startInline: id - | indentationLevel | - (methodCache includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. - indentationLevel := currentMethod indentationLevel. - - currentMethod := PPCInlinedMethod new. - currentMethod id: id. - currentMethod returnVariable: returnVariable. - currentMethod indentationLevel: indentationLevel. - self push. + ^ clazz startInline: id "Modified: / 01-06-2015 / 21:48:35 / Jan Vrany " ! startMethod: id - (methodCache includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. - - currentMethod := PPCMethod new. - currentMethod id: id. - currentMethod category: self methodCategory. + clazz startMethod: id category: self methodCategory. arguments profile ifTrue:[ self codeProfileStart. ]. - self push. - - self cache: id as: currentMethod. - - "Modified: / 01-06-2015 / 21:19:41 / Jan Vrany " ! stopInline - ^ self pop. + ^ clazz stopInline "Modified: / 01-06-2015 / 21:37:59 / Jan Vrany " ! stopMethod - self cache: currentMethod methodName as: currentMethod. - "arguments profile ifTrue: [ Transcript show: currentMethod code; cr. ]." - ^ self pop. + ^ clazz stopInline "Modified: / 01-06-2015 / 21:38:05 / Jan Vrany " -! - -top - ^ compilerStack top ! ! !PPCCodeGen methodsFor:'variables'! allocateReturnVariable - ^ self allocateReturnVariableNamed: 'retval' + ^ clazz allocateReturnVariableNamed: 'retval' "Created: / 23-04-2015 / 18:03:40 / Jan Vrany " "Modified: / 15-06-2015 / 17:52:56 / Jan Vrany " ! allocateReturnVariableNamed: name - "Allocate (or return previously allocated one) temporary variable used for - storing a parser's return value (the parsed object)" - ^ currentMethod allocateReturnVariableNamed: name - - "Created: / 15-06-2015 / 18:04:48 / Jan Vrany " + ^ clazz allocateReturnVariableNamed: name ! allocateTemporaryVariableNamed: preferredName "Allocate a new variable with (preferably) given name. Returns a real variable name that should be used." - ^ self currentNonInlineMethod allocateTemporaryVariableNamed: preferredName + ^ clazz allocateTemporaryVariableNamed: preferredName "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " +! + +currentReturnVariable + ^ clazz currentReturnVariable ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCodeGenerator.st --- a/compiler/PPCCodeGenerator.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCCodeGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,10 +3,10 @@ "{ NameSpace: Smalltalk }" PPCNodeVisitor subclass:#PPCCodeGenerator - instanceVariableNames:'compiler' + instanceVariableNames:'codeGen' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Visitors' + category:'PetitCompiler-Visitors-CodeGenerators' ! !PPCCodeGenerator class methodsFor:'as yet unclassified'! @@ -25,8 +25,21 @@ !PPCCodeGenerator methodsFor:'accessing'! -compiler: aPPCCompiler - compiler := aPPCCompiler +arguments: args + super arguments: args. + codeGen arguments: args. +! + +clazz: aPPCClass + codeGen clazz: aPPCClass +! + +codeGen + ^ codeGen +! + +codeGen: anObject + codeGen := anObject ! guards @@ -42,36 +55,36 @@ children := choiceNode children. useGuards ifTrue:[ self addGuard: (children at: choiceChildNodeIndex) ifTrue: [ - compiler add: 'self clearError.'. - compiler - codeAssignParsedValueOf:[ self visit:(children at: choiceChildNodeIndex) ] + codeGen codeClearError. + codeGen + codeEvaluateAndAssign:[ self visit:(children at: choiceChildNodeIndex) ] to: resultVar. - compiler add: 'error ifFalse: [ '. - compiler codeReturn: resultVar. - compiler add: ' ].'. + codeGen codeIf: 'error' then: nil else: [ + codeGen codeReturn: resultVar. + ]. ] ifFalse:[ - compiler add: 'error := true.'. + codeGen code: 'error := true.'. ]. - compiler add: 'error ifTrue:[ '. - choiceChildNodeIndex < children size ifTrue:[ - self generateChoiceChildOf: choiceNode atIndex: choiceChildNodeIndex + 1 useGuards: useGuards storeResultInto: resultVar. - ] ifFalse:[ - compiler codeError: 'no choice suitable'. + codeGen codeIf: 'error' then: [ + choiceChildNodeIndex < children size ifTrue:[ + self generateChoiceChildOf: choiceNode atIndex: choiceChildNodeIndex + 1 useGuards: useGuards storeResultInto: resultVar. + ] ifFalse:[ + codeGen codeError: 'no choice suitable'. + ]. ]. - compiler addOnLine: '].'. ] ifFalse:[ choiceChildNodeIndex <= children size ifTrue:[ - compiler add: 'self clearError.'. - compiler - codeAssignParsedValueOf:[ self visit:(children at: choiceChildNodeIndex) ] + codeGen codeClearError. + codeGen + codeEvaluateAndAssignParsedValueOf:[ self visit:(children at: choiceChildNodeIndex) ] to: resultVar. - compiler add: 'error ifFalse: [ '. - compiler codeReturn: resultVar. - compiler add: ' ].'. + codeGen codeIf: 'error' then: nil else: [ + codeGen codeReturn: resultVar. + ]. self generateChoiceChildOf: choiceNode atIndex: choiceChildNodeIndex + 1 useGuards: useGuards storeResultInto: resultVar. ] ifFalse:[ - compiler codeError: 'no choice suitable'. + codeGen codeError: 'no choice suitable'. ]. ]. @@ -83,21 +96,21 @@ child := sequenceNode children at: sequenceNodeChildIndex. childValueVar := elementVars at: sequenceNodeChildIndex. - compiler codeAssignParsedValueOf: [ self visit:child ] + codeGen codeEvaluateAndAssign: [ self visit:child ] to: childValueVar. child acceptsEpsilon ifFalse: [ - compiler codeIfErrorThen: [ + codeGen codeIfErrorThen: [ "Handle error in the first element in a special way, because one does not need to do backtracking if the first element fails." (sequenceNodeChildIndex == 1) ifTrue: [ - compiler codeReturn: 'failure' + codeGen codeReturn: 'failure' ] ifFalse: [ - compiler smartRestore: sequenceNode from: mementoVar. - compiler codeReturn: 'failure.'. + codeGen restore: sequenceNode from: mementoVar. + codeGen codeReturn: 'failure.'. ] ] else:[ sequenceNode returnParsedObjectsAsCollection ifTrue:[ - compiler add: self retvalVar , ' at: ', sequenceNodeChildIndex asString, ' put: ', childValueVar, '.'. + codeGen code: self retvalVar , ' at: ', sequenceNodeChildIndex asString, ' put: ', childValueVar, '.'. ]. (sequenceNodeChildIndex < sequenceNode children size) ifTrue:[ self generateSequenceChildOf: sequenceNode atIndex: sequenceNodeChildIndex + 1 useMememntoVar: mementoVar storeResultInto: elementVars. @@ -106,7 +119,7 @@ ] ifTrue:[ sequenceNode returnParsedObjectsAsCollection ifTrue:[ - compiler add: self retvalVar , ' at: ', sequenceNodeChildIndex asString, ' put: ', childValueVar, '.'. + codeGen code: self retvalVar , ' at: ', sequenceNodeChildIndex asString, ' put: ', childValueVar, '.'. ]. (sequenceNodeChildIndex < sequenceNode children size) ifTrue:[ self generateSequenceChildOf: sequenceNode atIndex: sequenceNodeChildIndex + 1 useMememntoVar: mementoVar storeResultInto: elementVars. @@ -115,59 +128,6 @@ ] ! ! -!PPCCodeGenerator methodsFor:'guards'! - -addGuard: node ifTrue: trueBlock ifFalse: falseBlock - | guard id | - (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. - id := compiler idFor: node. - -" falseBlock isNil ifFalse: [ - compiler add: 'context atEnd'. - compiler addOnLine: ' ifTrue: ['. - compiler indent. - falseBlock value. - compiler dedent. - compiler addOnLine: '].'. - ]." - - guard id: (compiler idFor: guard defaultName: #guard). - guard compileGuard: compiler. - - trueBlock isNil ifFalse: [ - compiler addOnLine: ' ifTrue: ['. - compiler indent. - trueBlock value. - compiler dedent. - falseBlock isNil ifTrue: [ compiler addOnLine: '].' ] - ifFalse: [ compiler add: ']'. ] - ]. - falseBlock isNil ifFalse: [ - compiler addOnLine: ' ifFalse: ['. - compiler indent. - falseBlock value. - compiler dedent. - compiler addOnLine: '].'. - ]. - ^ true -! - -addGuardTrimming: node - | guard firsts id | - (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. - - id := compiler idFor: node. - firsts := node firstSetWithTokens. - - - (firsts allSatisfy: [ :e | e isTrimmingTokenNode ]) ifTrue: [ - "If we start with trimming, we should invoke the whitespace parser" - self compileTokenWhitespace: firsts anyOne. - ^ true - ]. - ^ false -! ! - !PPCCodeGenerator methodsFor:'hooks'! afterAccept: node retval: retval @@ -186,7 +146,7 @@ ! openDetected: node - ^ compiler checkCache: (compiler idFor: node) + ^ codeGen cachedMethod: (codeGen idFor: node) ! ! !PPCCodeGenerator methodsFor:'private'! @@ -230,14 +190,14 @@ method isNil ifTrue:[ PPCCompilationError new signalWith: 'oops, no method found (internal error)!!'. ]. - source := method source. + source := method sourceCode. source isNil ifTrue:[ PPCCompilationError new signalWith: 'unavailable source for method ', method printString ,'!!'. ]. "Following actually copies the method to the target class, though the APU is not nice. This has to be cleaned up" - (compiler cachedValue: node selector) isNil ifTrue:[ - compiler cache: node selector as: (PPCMethod new id: node selector; source: source; yourself). + (codeGen cachedMethod: node selector) isNil ifTrue:[ + codeGen cacheMethod: (PPCMethod new id: node selector; source: source; yourself) as: node selector. "Now compile self-sends of the just copied method" self copySelfSentMethodsOf: method parseTree inClass: aClass ]. @@ -266,67 +226,65 @@ !PPCCodeGenerator methodsFor:'support'! compileTokenWhitespace: node - compiler add: 'context atWs ifFalse: ['. - compiler indent. - compiler - codeAssignParsedValueOf:[ self visit:node whitespace ] + codeGen codeIf: 'context atWs' then: nil else: [ + codeGen + codeEvaluateAndAssign:[ self visit:node whitespace ] to:#whatever. - compiler add: 'context setWs.'. - compiler dedent. - compiler add: '].'. + codeGen code: 'context setWs.'. + ] ! notCharSetPredicateBody: node | classificationId classification | self error: 'deprecated.'. classification := node extendClassification: node predicate classification. - classificationId := (compiler idFor: classification defaultName: #classification). - compiler addConstant: classification as: classificationId. + classificationId := (codeGen idFor: classification defaultName: #classification). + codeGen addConstant: classification as: classificationId. - compiler addOnLine: '(', classificationId, ' at: context peek asInteger)'. - compiler indent. - compiler add: ' ifTrue: [ self error: '' predicate not expected'' ]'. - compiler add: ' ifFalse: [ nil ].'. - compiler dedent. + codeGen addOnLine: '(', classificationId, ' at: context peek asInteger)'. + codeGen indent. + codeGen add: ' ifTrue: [ self error: '' predicate not expected'' ]'. + codeGen add: ' ifFalse: [ nil ].'. + codeGen dedent. ! notMessagePredicateBody: node self error: 'deprecated'. - compiler addOnLine: '(context peek ', node message, ')'. - compiler indent. - compiler add: ' ifTrue: [ self error: '' predicate not expected'' ]'. - compiler add: ' ifFalse: [ nil ].'. - compiler dedent. + codeGen addOnLine: '(context peek ', node message, ')'. + codeGen indent. + codeGen add: ' ifTrue: [ self error: '' predicate not expected'' ]'. + codeGen add: ' ifFalse: [ nil ].'. + codeGen dedent. ! predicateBody: node | tmpId | self error:'deprecated'. - tmpId := (compiler idFor: node predicate prefixed: #predicate). - compiler addConstant: node predicate as: tmpId. + tmpId := (codeGen idFor: node predicate prefixed: #predicate). + codeGen addConstant: node predicate as: tmpId. - compiler addOnLine: '(context atEnd not and: [ ', tmpId , ' value: context uncheckedPeek])'. - compiler indent. - compiler add: 'ifFalse: [ self error: ''predicate not found'' ]'. - compiler add: 'ifTrue: [ context next ].'. - compiler dedent. + codeGen addOnLine: '(context atEnd not and: [ ', tmpId , ' value: context uncheckedPeek])'. + codeGen indent. + codeGen add: 'ifFalse: [ self error: ''predicate not found'' ]'. + codeGen add: 'ifTrue: [ context next ].'. + codeGen dedent. ! retvalVar - ^ compiler currentReturnVariable + ^ codeGen currentReturnVariable "Modified: / 15-06-2015 / 18:20:37 / Jan Vrany " ! startMethodForNode:node node isMarkedForInline ifTrue:[ - compiler startInline: (compiler idFor: node). - compiler codeComment: 'BEGIN inlined code of ' , node printString. - compiler indent. + codeGen startInline: (codeGen idFor: node). + codeGen codeComment: 'BEGIN inlined code of ' , node printString. + codeGen indent. ] ifFalse:[ - compiler startMethod: (compiler idFor: node). - compiler codeComment: 'GENERATED by ' , node printString. - compiler allocateReturnVariable. + codeGen startMethod: (codeGen idFor: node). + codeGen codeComment: 'GENERATED by ' , node printString. + codeGen allocateReturnVariable. ]. "Created: / 23-04-2015 / 15:51:06 / Jan Vrany " @@ -335,12 +293,12 @@ ! stopMethodForNode:aPPCNode - ^ aPPCNode isMarkedForInline ifTrue:[ - compiler dedent. - compiler add: '"END inlined code of ' , aPPCNode printString , '"'. - compiler stopInline. + ^ codeGen currentMethod isInline ifTrue:[ + codeGen dedent. + codeGen code: '"END inlined code of ' , aPPCNode printString , '"'. + codeGen stopInline. ] ifFalse:[ - compiler stopMethod + codeGen stopMethod ]. "Created: / 23-04-2015 / 15:51:09 / Jan Vrany " @@ -354,11 +312,11 @@ ! cachedDetected: node - ^ compiler checkCache: (compiler idFor: node) + ^ codeGen clazz cachedMethod: (codeGen idFor: node) ! isCached: node - ^ (compiler checkCache: (compiler idFor: node)) isNil not + ^ (codeGen cachedMethod: (codeGen idFor: node)) isNil not ! ! !PPCCodeGenerator methodsFor:'visiting'! @@ -440,7 +398,7 @@ So if the method is not inline, make last statement a return. if the method is inline, make it assignment to retvalVar." blockBody statements notEmpty ifTrue:["Care for empty blocks - [:t | ] !!" - compiler currentMethod isInline ifTrue:[ + codeGen currentMethod isInline ifTrue:[ | assignment | assignment := RBAssignmentNode variable: (RBVariableNode named: self retvalVar) value: blockBody statements last. @@ -453,11 +411,11 @@ ]. ]. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:self retvalVar. - compiler codeIfErrorThen: [ - compiler codeReturn: 'failure'. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. + codeGen codeIfErrorThen: [ + codeGen codeReturn: 'failure'. ] else: [ - compiler code: blockBody. + codeGen code: blockBody. ] "Modified: / 27-07-2015 / 15:49:15 / Jan Vrany " @@ -466,22 +424,22 @@ visitAndNode: node | mementoVar | - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node child to: mementoVar. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen remember: node child to: mementoVar. - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler smartRestore: node child from: mementoVar. + codeGen restore: node child from: mementoVar. - compiler codeReturn. + codeGen codeReturn. "Modified: / 23-04-2015 / 15:59:07 / Jan Vrany " ! visitAnyNode: node - compiler codeReturn: 'context next ifNil: [ error := true. ].'. + codeGen codeReturn: 'context next ifNil: [ error := true. ].'. "Modified: / 23-04-2015 / 20:52:15 / Jan Vrany " ! @@ -490,17 +448,14 @@ | classification classificationId | classification := node extendClassification: node predicate classification. - classificationId := compiler idFor: classification defaultName: #classification. - compiler addConstant: classification as: classificationId. + classificationId := codeGen idFor: classification defaultName: #classification. + codeGen addConstant: classification as: classificationId. - compiler add: '(', classificationId, ' at: context peek asInteger)'. - compiler indent. - compiler add: 'ifFalse: ['. - compiler codeError: 'predicate not found'. - compiler add: '] ifTrue: [ '. - compiler codeReturn: 'context next'. - compiler add: '].'. - compiler dedent. + codeGen codeIf: '(', classificationId, ' at: context peek asInteger)' then: [ + codeGen codeReturn: 'context next'. + ] else: [ + codeGen codeError: 'predicate not found'. + ] ! visitCharacterNode: node @@ -508,22 +463,15 @@ node character ppcPrintable ifTrue: [ chid := node character storeString ] ifFalse: [ - chid := compiler idFor: node character defaultName: #char. - compiler addConstant: (Character value: node character asInteger) as: chid . + chid := codeGen idFor: node character defaultName: #char. + codeGen addConstant: (Character value: node character asInteger) as: chid . ]. - compiler add: '(context peek == ', chid, ')'. - compiler indent. - compiler add: 'ifFalse: ['. - compiler indent. - compiler codeError: node character asInteger asString, ' expected'. - compiler dedent. - compiler add: '] ifTrue: [ '. - compiler indent. - compiler codeReturn: 'context next'. - compiler dedent. - compiler add: '].'. - compiler dedent. + codeGen codeIf: '(context peek == ', chid, ')' then: [ + codeGen codeReturn: 'context next'. + ] else: [ + codeGen codeError: node character asInteger asString, ' expected'. + ]. ! visitChild: child of: node @@ -540,7 +488,7 @@ visitChoiceNode: node | whitespaceConsumed useGuards resultVar | - resultVar := compiler allocateTemporaryVariableNamed: 'element'. + resultVar := codeGen allocateTemporaryVariableNamed: 'element'. whitespaceConsumed := self addGuardTrimming: node. useGuards := whitespaceConsumed. self generateChoiceChildOf: node atIndex: 1 useGuards: useGuards storeResultInto: resultVar @@ -550,43 +498,41 @@ ! visitEndOfFileNode: node - compiler codeReturn: 'context atEnd ifTrue: [ #EOF ] ifFalse: [ self error: ''EOF expected!!'' ].'. + codeGen codeReturn: 'context atEnd ifTrue: [ #EOF ] ifFalse: [ self error: ''EOF expected!!'' ].'. ! visitEndOfInputNode: node - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler codeIf: 'context atEnd' - then: [ compiler codeReturn ] - else: [ compiler codeError: 'End of input expected' ]. + codeGen codeIf: 'context atEnd' + then: [ codeGen codeReturn ] + else: [ codeGen codeError: 'End of input expected' ]. "Modified: / 26-05-2015 / 19:03:09 / Jan Vrany " ! visitForwardNode: node - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler codeReturn. + codeGen codeReturn. ! visitLiteralNode: node | positionVar encodedLiteral | encodedLiteral := node encodeQuotes: node literal. - positionVar := compiler allocateTemporaryVariableNamed: 'position'. + positionVar := codeGen allocateTemporaryVariableNamed: 'position'. - compiler codeAssign: 'context position.' to: positionVar. - compiler add: '((context next: ', node literal size asString, ') = #''', encodedLiteral, ''') ifTrue: ['. - compiler codeReturn: '#''', encodedLiteral, ''' '. - compiler add: '] ifFalse: ['. - compiler indent. - compiler add: 'context position: ', positionVar, '.'. - compiler codeError: encodedLiteral, ' expected' at: positionVar. - compiler dedent. - compiler add: '].'. + codeGen codeAssign: 'context position.' to: positionVar. + codeGen codeIf: '((context next: ', node literal size asString, ') = #''', encodedLiteral, ''')' then: [ + codeGen codeReturn: '#''', encodedLiteral, ''' '. + ] else: [ + codeGen code: 'context position: ', positionVar, '.'. + codeGen codeError: encodedLiteral, ' expected' at: positionVar. + ]. ! visitMappedActionNode: node @@ -600,7 +546,7 @@ "Block return value is return value of last statement. So if the method is not inline, make last statement a return. if the method is inline, make it assignment to retvalVar." - compiler currentMethod isInline ifTrue:[ + codeGen currentMethod isInline ifTrue:[ | assignment | assignment := RBAssignmentNode variable: (RBVariableNode named: self retvalVar) value: blockBody statements last. @@ -633,22 +579,22 @@ ]. ]. - compiler codeAssignParsedValueOf: [ self visit: child ] to: self retvalVar. - compiler codeIf: 'error' then: [ - compiler codeReturn: 'failure'. + codeGen codeEvaluateAndAssign: [ self visit: child ] to: self retvalVar. + codeGen codeIf: 'error' then: [ + codeGen codeReturn: 'failure'. ] else: [ "If the child is sequence and not inlined, extract nodes from returned collection into used-to-be block variables" (child isSequenceNode and:[ child returnParsedObjectsAsCollection ]) ifTrue:[ blockNode arguments withIndexDo:[ :arg :idx | node child isMarkedForInline ifFalse:[ - compiler allocateTemporaryVariableNamed: arg name. - compiler codeAssign: (self retvalVar , ' at: ', idx printString) to: arg name. + codeGen allocateTemporaryVariableNamed: arg name. + codeGen codeAssign: (self retvalVar , ' at: ', idx printString) to: arg name. ]. - compiler addOnLine: '.'; nl. + codeGen codeOnLine: '.'; codeNl. ]. ]. - compiler code: blockBody. + codeGen code: blockBody. ] "Created: / 02-06-2015 / 17:28:55 / Jan Vrany " @@ -656,34 +602,30 @@ ! visitMessagePredicateNode: node - compiler add: '(context peek ', node message, ') ifFalse: ['. - compiler codeError: 'predicate not found'. - compiler add: '] ifTrue: [ '. - compiler codeReturn: ' context next'. - compiler add: '].'. - + codeGen codeIf: '(context peek ', node message, ')' then: [. + codeGen codeReturn: ' context next'. + ] else: [ + codeGen codeError: 'predicate not found'. + ] "Modified: / 23-04-2015 / 18:39:03 / Jan Vrany " ! visitNilNode: node - compiler codeReturn: 'nil.'. + codeGen codeReturn: 'nil.'. ! visitNotCharSetPredicateNode: node | classificationId classification | classification := node extendClassification: node predicate classification. - classificationId := (compiler idFor: classification defaultName: #classification). - compiler addConstant: classification as: classificationId. + classificationId := (codeGen idFor: classification defaultName: #classification). + codeGen addConstant: classification as: classificationId. - compiler addOnLine: '(', classificationId, ' at: context peek asInteger)'. - compiler indent. - compiler add: ' ifTrue: ['. - compiler codeError: 'predicate not expected'. - compiler add: '] ifFalse: ['. - compiler codeReturn: 'nil'. - compiler add: '].'. - compiler dedent. + codeGen codeIf: '(', classificationId, ' at: context peek asInteger)' then: [ + codeGen codeError: 'predicate not expected'. + ] else: [ + codeGen codeReturn: 'nil'. + ] ! visitNotCharacterNode: node @@ -691,22 +633,15 @@ node character ppcPrintable ifTrue: [ chid := node character storeString ] ifFalse: [ - chid := compiler idFor: node character defaultName: #char. - compiler addConstant: (Character value: node character asInteger) as: chid . + chid := codeGen idFor: node character defaultName: #char. + codeGen addConstant: (Character value: node character asInteger) as: chid . ]. - compiler add: '(context peek == ', chid, ')'. - compiler indent. - compiler add: 'ifTrue: ['. - compiler indent. - compiler codeError: node character asInteger asString, ' not expected'. - compiler dedent. - compiler add: '] ifFalse: [ '. - compiler indent. - compiler codeReturn: 'nil.'. - compiler dedent. - compiler add: '].'. - compiler dedent. + codeGen codeIf: '(context peek == ', chid, ')' then: [ + codeGen codeError: node character asInteger asString, ' not expected'. + ] else: [ + codeGen codeReturn: 'nil.'. + ]. ! visitNotLiteralNode: node @@ -714,113 +649,102 @@ encodedLiteral := node encodeQuotes: node literal. size := node literal size asString. - compiler add: '((context peek: ', size, ') =#''', encodedLiteral, ''')'. - compiler indent. - compiler add: 'ifTrue: ['. - compiler codeError: encodedLiteral, ' not expected'. - compiler add: '] ifFalse: [ '. - compiler codeReturn: 'nil' . - compiler add: '].'. - compiler dedent. + codeGen codeIf: '((context peek: ', size, ') =#''', encodedLiteral, ''')' then: [ + codeGen codeError: encodedLiteral, ' not expected'. + ] else: [ + codeGen codeReturn: 'nil' . + ] ! visitNotMessagePredicateNode: node - compiler addOnLine: '(context peek ', node message, ')'. - compiler indent. - compiler add: ' ifTrue: [ '. - compiler codeError: 'predicate not expected'. - compiler add: '] ifFalse: ['. - compiler codeReturn: 'nil'. - compiler add: ' ].'. - compiler dedent. + codeGen codeIf: '(context peek ', node message, ')' then: [ + codeGen codeError: 'predicate not expected'. + ] else: [ + codeGen codeReturn: 'nil'. + ] ! visitNotNode: node | mementoVar | - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node child to: mementoVar. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen remember: node child to: mementoVar. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:#whatever. - compiler smartRestore: node child from: mementoVar. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:#whatever. + codeGen restore: node child from: mementoVar. - compiler add: '^ error ifFalse: [ self error ] ifTrue: [ self clearError. nil ]'. + codeGen code: '^ error ifFalse: [ self error ] ifTrue: [ self clearError. nil ]'. "Modified: / 05-05-2015 / 14:29:52 / Jan Vrany " ! visitOptionalNode: node - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler add: 'error ifTrue: [ '. - compiler indent. - compiler add: 'self clearError. '. - compiler codeAssign: 'nil.' to: self retvalVar. - compiler dedent. - compiler add: '].'. - compiler codeReturn. + codeGen codeIf: 'error' then: [ + codeGen codeClearError. + codeGen codeAssign: 'nil.' to: self retvalVar. + ]. + codeGen codeReturn. ! visitPluggableNode: node | blockId | - blockId := compiler idFor: node block defaultName: #pluggableBlock. + blockId := codeGen idFor: node block defaultName: #pluggableBlock. - compiler addConstant: node block as: blockId. - compiler codeReturn: blockId, ' value: context.'. + codeGen addConstant: node block as: blockId. + codeGen codeReturn: blockId, ' value: context.'. ! visitPlusNode: node | elementVar | - elementVar := compiler allocateTemporaryVariableNamed: 'element'. + elementVar := codeGen allocateTemporaryVariableNamed: 'element'. " self tokenGuards ifTrue: [ compiler codeTokenGuard: node ifFalse: [ compiler codeError: 'at least one occurence expected' ]. ]. " - compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. + codeGen codeAssign: 'OrderedCollection new.' to: self retvalVar. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:elementVar. - compiler add: 'error ifTrue: ['. - compiler codeError: 'at least one occurence expected'. - compiler add: '] ifFalse: ['. - compiler indent. + codeGen codeIf: 'error' then: [ + codeGen codeError: 'at least one occurence expected'. + ] else: [ (self retvalVar ~~ #whatever) ifTrue:[ - compiler add: self retvalVar , ' add: ',elementVar , '.'. + codeGen code: self retvalVar , ' add: ',elementVar , '.'. ]. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. - compiler add: '[ error ] whileFalse: ['. - compiler indent. + codeGen codeEvaluateAndAssignParsedValueOf:[ self visit:node child ] to:elementVar. + codeGen code: '[ error ] whileFalse: ['. + codeGen indent. (self retvalVar ~~ #whatever) ifTrue:[ - compiler add: self retvalVar , ' add: ',elementVar , '.'. + codeGen code: self retvalVar , ' add: ',elementVar , '.'. ]. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. - compiler dedent. - compiler add: '].'. - compiler add: 'self clearError.'. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:elementVar. + codeGen dedent. + codeGen code: '].'. + codeGen code: 'self clearError.'. + (self retvalVar ~~ #whatever) ifTrue:[ - compiler codeReturn: self retvalVar , ' asArray.'. + codeGen codeReturn: self retvalVar , ' asArray.'. ]. - compiler dedent. - compiler add: '].'. - + ]. "Modified: / 26-05-2015 / 19:04:27 / Jan Vrany " ! visitPredicateNode: node | pid | - pid := (compiler idFor: node predicate defaultName: #predicate). + pid := (codeGen idFor: node predicate defaultName: #predicate). - compiler addConstant: node predicate as: pid. + codeGen addConstant: node predicate as: pid. - compiler add: '(context atEnd not and: [ ', pid , ' value: context uncheckedPeek])'. - compiler indent. - compiler add: 'ifFalse: ['. - compiler codeError: 'predicate not found'. - compiler add: '] ifTrue: [ ', self retvalVar ,' := context next ].'. - compiler dedent. - compiler codeReturn. + codeGen codeIf: '(context atEnd not and: [ ', pid , ' value: context uncheckedPeek])' then: [ + codeGen code: self retvalVar ,' := context next.'. + ] else: [ + codeGen codeError: 'predicate not found'. + ]. + codeGen codeReturn. "Modified: / 23-04-2015 / 21:48:11 / Jan Vrany " ! @@ -831,26 +755,24 @@ canBacktrack := (node children allButFirst allSatisfy: [:e | e acceptsEpsilon ]) not. canBacktrack ifTrue: [ - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node to: mementoVar. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen remember: node to: mementoVar. ]. - compiler - codeAssignParsedValueOf:[ self visit:(node children at:1) ] + codeGen + codeEvaluateAndAssign:[ self visit:(node children at:1) ] to:#whatever. - compiler add: 'error ifTrue: [ ^ failure ].'. + codeGen code: 'error ifTrue: [ ^ failure ].'. 2 to: (node children size) do: [ :idx | |child| child := node children at: idx. - compiler codeAssignParsedValueOf:[ self visit:child ] to:#whatever. + codeGen codeEvaluateAndAssignParsedValueOf:[ self visit:child ] to:#whatever. child acceptsEpsilon ifFalse: [ - compiler add: 'error ifTrue: [ '. - compiler indent. - compiler smartRestore: node from: mementoVar. - compiler add: ' ^ failure .'. - compiler dedent. - compiler add: '].'. + codeGen codeIf: 'error' then: [ + codeGen restore: node from: mementoVar. + codeGen code: ' ^ failure .'. + ] ]. ]. ! @@ -861,7 +783,7 @@ elementVars := node preferredChildrenVariableNames. elementVars do:[:e | - compiler allocateTemporaryVariableNamed: e. + codeGen allocateTemporaryVariableNamed: e. ]. canBacktrack := (node children allButFirst allSatisfy: [:e | e acceptsEpsilon ]) not. @@ -870,15 +792,15 @@ self addGuard: node ifTrue: nil ifFalse: [ compiler addOnLine: ' ^ self error' ]. " canBacktrack ifTrue: [ - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node to: mementoVar. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen remember: node to: mementoVar. ]. node returnParsedObjectsAsCollection ifTrue:[ - compiler codeAssign: 'Array new: ', node children size asString, '.' to: self retvalVar. + codeGen codeAssign: 'Array new: ', node children size asString, '.' to: self retvalVar. ]. self generateSequenceChildOf: node atIndex: 1 useMememntoVar: mementoVar storeResultInto: elementVars. - compiler codeReturn + codeGen codeReturn "Modified (comment): / 16-06-2015 / 06:38:02 / Jan Vrany " ! @@ -887,11 +809,11 @@ | retvalVar sizeVar | retvalVar := self retvalVar. - sizeVar := compiler allocateTemporaryVariableNamed: 'size'. - compiler add: sizeVar , ' := context size - context position.'. - compiler add: retvalVar,' := Array new: ',sizeVar,'.'. - compiler add: '(1 to: ',sizeVar,') do: [ :e | ',retvalVar,' at: e put: context next ].'. - compiler codeReturn. + sizeVar := codeGen allocateTemporaryVariableNamed: 'size'. + codeGen code: sizeVar , ' := context size - context position.'. + codeGen code: retvalVar,' := Array new: ',sizeVar,'.'. + codeGen code: '(1 to: ',sizeVar,') do: [ :e | ',retvalVar,' at: e put: context next ].'. + codeGen codeReturn. "Modified: / 15-06-2015 / 18:53:58 / Jan Vrany " ! @@ -901,67 +823,63 @@ classification := node extendClassification: node predicate classification. - classificationId := compiler idFor: classification defaultName: #classification. - compiler addConstant: classification as: classificationId. + classificationId := codeGen idFor: classification defaultName: #classification. + codeGen addConstant: classification as: classificationId. - compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. - compiler add: '[ ', classificationId, ' at: context peek asInteger ] whileTrue: ['. - compiler indent. - compiler codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. - compiler dedent. - compiler add: '].'. - compiler codeAssign: self retvalVar, ' asArray.' to: self retvalVar. - compiler codeReturn. + codeGen codeAssign: 'OrderedCollection new.' to: self retvalVar. + codeGen code: '[ ', classificationId, ' at: context peek asInteger ] whileTrue: ['. + codeGen indent. + codeGen codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. + codeGen dedent. + codeGen code: '].'. + codeGen codeAssign: self retvalVar, ' asArray.' to: self retvalVar. + codeGen codeReturn. ! visitStarMessagePredicateNode: node - compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. - compiler add: '[ context peek ', node message, ' ] whileTrue: ['. - compiler indent. - compiler codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. - compiler dedent. - compiler add: '].'. - compiler codeAssign: self retvalVar, ' asArray.' to: self retvalVar. - compiler codeReturn. + codeGen codeAssign: 'OrderedCollection new.' to: self retvalVar. + codeGen code: '[ context peek ', node message, ' ] whileTrue: ['. + codeGen indent. + codeGen codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. + codeGen dedent. + codeGen code: '].'. + codeGen codeAssign: self retvalVar, ' asArray.' to: self retvalVar. + codeGen codeReturn. ! visitStarNode: node | elementVar | - elementVar := compiler allocateTemporaryVariableNamed: 'element'. + elementVar := codeGen allocateTemporaryVariableNamed: 'element'. + codeGen codeAssign: 'OrderedCollection new.' to: self retvalVar. - self addGuard: node child ifTrue: nil ifFalse: [ compiler codeReturn: '#()' ]. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:elementVar. + codeGen codeIf: 'error' then: [ + codeGen codeClearError. + codeGen codeReturn: self retvalVar, ' asArray.'. + ]. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. - compiler codeIf: 'error' - then: [ - compiler codeClearError. - compiler codeReturn: '#()'. - ] else: [ - compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. - ]. - - compiler add: '[ error ] whileFalse: ['. - compiler indent. - compiler add: self retvalVar, ' add: ', elementVar, '.'. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. - compiler dedent. - compiler add: '].'. - compiler codeClearError. - compiler codeReturn: self retvalVar, ' asArray.'. + codeGen code: '[ error ] whileFalse: ['. + codeGen indent. + codeGen code: self retvalVar, ' add: ', elementVar, '.'. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:elementVar. + codeGen dedent. + codeGen code: '].'. + codeGen codeClearError. + codeGen codeReturn: self retvalVar, ' asArray.'. ! visitSymbolActionNode: node | elementVar | - elementVar := compiler allocateTemporaryVariableNamed: 'element'. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:elementVar. - compiler add: 'error ifFalse: [ '. - compiler codeReturn: elementVar, ' ', node block asString, '.'. - compiler add: '] ifTrue: ['. - compiler codeReturn: 'failure'. - compiler add: ']'. + elementVar := codeGen allocateTemporaryVariableNamed: 'element'. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:elementVar. + codeGen codeIf: 'error' then: [ + codeGen codeReturn: 'failure' + ] else: [ + codeGen codeReturn: elementVar, ' ', node block asString, '.'. + ] ! visitTokenActionNode: node @@ -971,75 +889,68 @@ the input value. " - compiler add: '^ '. - compiler callOnLine: (node child compileWith: compiler). + codeGen add: '^ '. + codeGen callOnLine: (node child compileWith: codeGen). ! visitTokenNode: node | startVar endVar | - startVar := compiler allocateTemporaryVariableNamed: 'start'. - endVar := compiler allocateTemporaryVariableNamed: 'end'. + startVar := codeGen allocateTemporaryVariableNamed: 'start'. + endVar := codeGen allocateTemporaryVariableNamed: 'end'. - compiler profileTokenRead: (compiler idFor: node). + codeGen profileTokenRead: (codeGen idFor: node). - compiler codeAssign: 'context position + 1.' to: startVar. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:#whatever. - compiler add: 'error ifFalse: [ '. - compiler indent. - compiler codeAssign: 'context position.' to: endVar. - - compiler codeReturn: node tokenClass asString, ' on: (context collection) - start: ', startVar, ' - stop: ', endVar, ' - value: nil.'. - compiler dedent. - compiler add: '].'. + codeGen codeAssign: 'context position + 1.' to: startVar. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:#whatever. + codeGen codeIf: 'error' then: nil else: [ + codeGen codeAssign: 'context position.' to: endVar. + codeGen codeReturn: node tokenClass asString, ' on: (context collection) + start: ', startVar, ' + stop: ', endVar, ' + value: nil.'. + ] ! visitTokenStarMessagePredicateNode: node - compiler add: '[ context peek ', node message,' ] whileTrue: ['. - compiler indent. - compiler add: 'context next'. - compiler indent. - compiler dedent. - compiler add: '].'. + codeGen code: '[ context peek ', node message,' ] whileTrue: ['. + codeGen indent. + codeGen code: 'context next'. + codeGen indent. + codeGen dedent. + codeGen code: '].'. ! visitTokenStarSeparatorNode: node - compiler add: 'context skipSeparators.'. + codeGen code: 'context skipSeparators.'. ! visitTokenWhitespaceNode: node - compiler codeAssignParsedValueOf:[ self visit:node child ] to:#whatever. - compiler codeReturn. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:#whatever. + codeGen codeReturn. ! visitTrimNode: node | mementoVar | "TODO: This ignores the TrimmingParser trimmer object!!" - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node child to: mementoVar. - compiler add: 'context skipSeparators.'. + codeGen remember: node child to: mementoVar. + codeGen code: 'context skipSeparators.'. - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler add: 'error ifTrue: [ '. - compiler indent. - compiler smartRestore: node child from: mementoVar. - compiler codeReturn. - compiler dedent. - compiler add: '] ifFalse: [' . - compiler indent. - compiler add: 'context skipSeparators.'. - compiler codeReturn. - compiler dedent. - compiler add: '].'. + codeGen codeIf: 'error' then: [ + codeGen restore: node child from: mementoVar. + codeGen codeReturn. + ] else: [ + codeGen code: 'context skipSeparators.'. + codeGen codeReturn. + ] ! visitTrimmingTokenCharacterNode: node @@ -1049,50 +960,47 @@ visitTrimmingTokenNode: node | id guard startVar endVar | - startVar := compiler allocateTemporaryVariableNamed: 'start'. - endVar := compiler allocateTemporaryVariableNamed: 'end'. + startVar := codeGen allocateTemporaryVariableNamed: 'start'. + endVar := codeGen allocateTemporaryVariableNamed: 'end'. - id := compiler idFor: node. - compiler profileTokenRead: id. + id := codeGen idFor: node. + codeGen profileTokenRead: id. self compileTokenWhitespace: node. (arguments guards and: [(guard := PPCGuard on: node) makesSense]) ifTrue: [ guard id: id, '_guard'. - compiler add: 'context atEnd ifTrue: [ self error ].'. - guard compileGuard: compiler. - compiler addOnLine: 'ifFalse: [ self error ].'. - compiler add: 'error ifFalse: ['. - compiler indent. + codeGen code: 'context atEnd ifTrue: [ self error ].'. + guard compileGuard: codeGen. + codeGen codeOnLine: 'ifFalse: [ self error ].'. + codeGen code: 'error ifFalse: ['. + codeGen indent. ]. - compiler codeAssign: 'context position + 1.' to: startVar. - compiler codeAssignParsedValueOf:[ self visit:node child ] to:#whatever. + codeGen codeAssign: 'context position + 1.' to: startVar. + codeGen codeEvaluateAndAssign:[ self visit:node child ] to:#whatever. (arguments guards and: [(guard := PPCGuard on: node) makesSense]) ifTrue: [ - compiler dedent. - compiler add: '].'. + codeGen dedent. + codeGen code: '].'. ]. - compiler add: 'error ifFalse: [ '. - compiler indent. - compiler codeAssign: 'context position.' to: endVar. - -" self compileSecondWhitespace: compiler." - self compileTokenWhitespace: node. + codeGen codeIf: 'error' then: nil else: [ + codeGen codeAssign: 'context position.' to: endVar. + " self compileSecondWhitespace: compiler." + self compileTokenWhitespace: node. - compiler codeReturn: node tokenClass asString, ' on: (context collection) - start: ', startVar, ' - stop: ', endVar, ' - value: nil'. - compiler dedent. - compiler add: '].' + codeGen codeReturn: node tokenClass asString, ' on: (context collection) + start: ', startVar, ' + stop: ', endVar, ' + value: nil'. + ] ! visitUnknownNode: node | compiledChild compiledParser id | - id := compiler idFor: node. + id := codeGen idFor: node. compiledParser := node parser copy. "Compile all the children and call compiled version of them instead of the original one" @@ -1101,15 +1009,14 @@ compiledParser replace: child with: compiledChild bridge. ]. - compiler addConstant: compiledParser as: id. + codeGen addConstant: compiledParser as: id. - compiler codeClearError. - compiler add: '(', self retvalVar, ' := ', id, ' parseOn: context) isPetitFailure'. - compiler indent. - compiler add: ' ifTrue: [self error: ', self retvalVar, ' message at: ', self retvalVar, ' position ].'. - compiler dedent. - compiler add: 'error := ', self retvalVar, ' isPetitFailure.'. - compiler codeReturn. + codeGen codeClearError. + codeGen codeIf: '(', self retvalVar, ' := ', id, ' parseOn: context) isPetitFailure' then: [ + codeGen codeError: 'self error: ', self retvalVar at: self retvalVar, ' position .'. + ]. + codeGen code: 'error := ', self retvalVar, ' isPetitFailure.'. + codeGen codeReturn. "Modified: / 15-06-2015 / 17:59:23 / Jan Vrany " ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCompiler.st --- a/compiler/PPCCompiler.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,691 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -Object subclass:#PPCCompiler - instanceVariableNames:'compilerStack compiledParser cache currentMethod ids constants - compiledParserName compiledParserSuperclass returnVariable - arguments' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler' -! - - -!PPCCompiler class methodsFor:'instance creation'! - -new - "return an initialized instance" - - ^ self on: PPCArguments default -! - -on: aPPCArguments - "return an initialized instance" - - ^ self basicNew - arguments: aPPCArguments; - initializeForCompiledClassName: aPPCArguments parserName -! ! - -!PPCCompiler methodsFor:'accessing'! - -arguments: args - arguments := args -! - -compiledParser - ^ compiledParser -! - -compiledParserSuperclass - ^ compiledParserSuperclass ifNil: [ PPCompiledParser ] -! - -currentMethod - ^ currentMethod -! - -currentNonInlineMethod - ^ compilerStack - detect:[:m | m isInline not ] - ifNone:[ self error: 'No non-inlined method'] - - "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " -! - -currentReturnVariable - ^ currentMethod returnVariable -! - -ids - ^ ids -! ! - -!PPCCompiler methodsFor:'cleaning'! - -clean: class -" Transcript show: ('Cleaning time: ', - [ -" self cleanGeneratedMethods: class. - self cleanInstVars: class. - self cleanConstants: class. -" ] timeToRun asMilliSeconds asString, 'ms'); cr. " -! - -cleanConstants: class - class constants removeAll. -! - -cleanGeneratedMethods: class - ((Smalltalk respondsTo:#isSmalltalkX) and:[ Smalltalk isSmalltalkX ]) ifTrue:[ - class methodsDo: [ :mthd | - (mthd category beginsWith: 'generated') ifTrue:[ - class removeSelector: mthd selector. - ] - ] - ] ifFalse: [ - (class allProtocolsUpTo: class) do: [ :protocol | - (protocol beginsWith: 'generated') ifTrue: [ - class removeProtocol: protocol. - ] - ] - ] -! - -cleanInstVars: class - class class instanceVariableNames: ''. -! - -cleanParsers: class - class parsers removeAll. -! ! - -!PPCCompiler methodsFor:'code generation'! - -add: string - currentMethod add: string. -! - -addComment: string - currentMethod add: '"', string, '"'. -! - -addConstant: value as: name - (constants includesKey: name) ifTrue:[ - (constants at: name) ~= value ifTrue:[ - self error:'Duplicate constant!!'. - ]. - ^ self. - ]. - constants at: name put: value - - "Modified: / 29-05-2015 / 07:22:39 / Jan Vrany " -! - -addOnLine: string - currentMethod addOnLine: string. -! - -addVariable: name - ^ self currentNonInlineMethod addVariable: name - - "Modified: / 23-04-2015 / 17:34:02 / Jan Vrany " -! - -call: anotherMethod - currentMethod add: anotherMethod call. -! - -callOnLine: anotherMethod - currentMethod addOnLine: anotherMethod call. -! - -codeComment: string - currentMethod add: '"', string, '"'. -! - -dedent - currentMethod dedent -! - -indent - currentMethod indent -! - -nl - currentMethod nl -! - -smartRemember: parser to: variableName - parser isContextFree ifTrue: [ - self codeAssign: 'context lwRemember.' - to: variableName. - ] ifFalse: [ - self codeAssign: 'context remember.' - to: variableName. - ] -! - -smartRestore: parser from: mementoName - parser isContextFree ifTrue: [ - self add: 'context lwRestore: ', mementoName, '.'. - ] ifFalse: [ - self add: 'context restore: ', mementoName, '.'. - ] -! ! - -!PPCCompiler methodsFor:'code generation - coding'! - -code:aStringOrBlockOrRBParseNode - currentMethod code: aStringOrBlockOrRBParseNode - - "Created: / 01-06-2015 / 23:49:11 / Jan Vrany " -! - -codeAssign: code to: variable - self assert: variable isNil not. - - "TODO JK: Hack alert, whatever is magic constant!!" - (variable == #whatever) ifFalse: [ - "Do not assign, if somebody does not care!!" - self add: variable ,' := ', code. - ] -! - -codeAssignParsedValueOf:aBlock to:aString - | tmpVarirable method | - - self assert:aBlock isBlock. - self assert:aString isNil not. - tmpVarirable := returnVariable. - returnVariable := aString. - method := [ - aBlock value - ] ensure:[ returnVariable := tmpVarirable ]. - self assert: (method isKindOf: PPCMethod). - method isInline ifTrue:[ - self callOnLine:method - ] ifFalse:[ - self codeEvaluateAndAssign:(method call) to:aString. - ] - - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " -! - -codeBlock: contents - currentMethod codeBlock: contents - - "Created: / 01-06-2015 / 22:35:32 / Jan Vrany " -! - -codeClearError - self add: 'self clearError.'. -! - -codeDot - self addOnLine:'.'. - - "Created: / 16-06-2015 / 06:09:07 / Jan Vrany " -! - -codeError - self add: 'self error: ''message notspecified''.'. -! - -codeError: errorMessage - self add: 'self error: ''', errorMessage, '''.' -! - -codeError: errorMessage at: position - self add: 'self error: ''', errorMessage, ''' at: ', position asString, '.' -! - -codeEvaluate: selector argument: argument on: variable - self assert: variable isNil not. - - "TODO JK: Hack alert, whatever is magic constant!!" - (variable == #whatever) ifFalse: [ - "Do not assign, if somebody does not care!!" - self add: variable, ' ', selector,' ', argument. - ] ifTrue: [ - "In case argument has a side effect" - self add: argument - ] -! - -codeEvaluateAndAssign: argument to: variable - self assert: variable isNil not. - - "TODO JK: Hack alert, whatever is magic constant!!" - (variable == #whatever) ifFalse: [ - "Do not assign, if somebody does not care!!" - self add: variable ,' := ', argument. - ] ifTrue: [ - "In case an argument has a side effect" - self add: argument. - ] -! - -codeHalt - self add: 'self halt. ' -! - -codeHaltIfShiftPressed - arguments debug ifTrue: [ - ((Smalltalk respondsTo: #isSmalltalkX) and:[Smalltalk isSmalltalkX]) ifFalse:[ - self add: 'Halt ifShiftPressed.' - ] - ] - - "Modified: / 10-05-2015 / 07:39:47 / Jan Vrany " -! - -codeIf: condition then: then - self codeIf: condition then: then else: nil - - "Created: / 16-06-2015 / 06:07:06 / Jan Vrany " -! - -codeIf: condition then: then else: else - currentMethod - add: '('; - code: condition; - addOnLine: ')'. - then notNil ifTrue:[ - currentMethod - addOnLine:' ifTrue:'; - codeBlock: then. - ]. - else notNil ifTrue:[ - currentMethod - addOnLine:' ifFalse:'; - codeBlock: else. - ]. - self codeDot. - - "Created: / 01-06-2015 / 22:43:15 / Jan Vrany " - "Modified: / 16-06-2015 / 06:09:33 / Jan Vrany " -! - -codeIfErrorThen: then - ^ self codeIf: 'error' then: then else: nil - - "Created: / 16-06-2015 / 06:06:44 / Jan Vrany " -! - -codeIfErrorThen: then else: else - ^ self codeIf: 'error' then: then else: else - - "Created: / 16-06-2015 / 06:05:56 / Jan Vrany " -! - -codeNextToken - self add: 'self nextToken.' - - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 23-04-2015 / 20:51:41 / Jan Vrany " -! - -codeProfileStart - self add: 'context methodInvoked: #', currentMethod methodName, '.' - - "Created: / 01-06-2015 / 21:17:19 / Jan Vrany " -! - -codeProfileStop - self add: 'context methodFinished: #', currentMethod methodName, '.' - - "Created: / 01-06-2015 / 21:19:11 / Jan Vrany " -! - -codeReturn - currentMethod isInline ifTrue: [ - "If inlined, the return variable already holds the value" - ] ifFalse: [ - arguments profile ifTrue:[ - self codeProfileStop. - ]. - self add: '^ ', currentMethod returnVariable - ]. - - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 01-06-2015 / 21:49:04 / Jan Vrany " -! - -codeReturn: code - " - returns whatever is in code OR - - assigns whatever is in code into the returnVariable" - currentMethod isInline ifTrue:[ - self codeEvaluateAndAssign: code to: currentMethod returnVariable. - ] ifFalse: [ - arguments profile ifTrue:[ - self codeProfileStop. - ]. - self add: '^ ', code - ] - - "Created: / 23-04-2015 / 18:01:05 / Jan Vrany " - "Modified: / 01-06-2015 / 21:48:51 / Jan Vrany " -! - -codeReturnParsedValueOf:aBlock - | tmpVarirable method | - - self assert:aBlock isBlock. - tmpVarirable := returnVariable. - method := aBlock value. - self assert: returnVariable == tmpVarirable. - self assert: (method isKindOf: PPCMethod). - method isInline ifTrue:[ - self callOnLine:method. - self codeReturn: returnVariable. - ] ifFalse:[ - self codeReturn: method call. - - ] - - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " -! - -codeStoreValueOf: aBlock intoVariable: aString - | tmpVarirable method | - self assert: aBlock isBlock. - self assert: aString isNil not. - - tmpVarirable := returnVariable. - returnVariable := aString. - method := [ - aBlock value - ] ensure: [ - returnVariable := tmpVarirable - ]. - - method isInline ifTrue: [ - self callOnLine: method - ] ifFalse: [ - self codeEvaluateAndAssign: (method call) to: aString. - ] - - "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " -! - -codeTokenGuard: node ifFalse: codeBlock - | guard id | - guard := PPCTokenGuard on: node. - (guard makesSense) ifTrue: [ - id := self idFor: guard firstToken. - - self add: 'self ', id asString, ' ifFalse: ['. - self indent. - codeBlock value. - self dedent. - self add: '].'. - ] -! - -codeTranscriptShow: text - (arguments profile) ifTrue: [ - self add: 'Transcript show: ', text storeString, '; cr.'. - ] -! ! - -!PPCCompiler methodsFor:'code generation - ids'! - -asSelector: string - "e.g. '234znak 43 ) 2' asLegalSelector = #v234znak432" - - | toUse | - - toUse := string select: [:char | char isAlphaNumeric or: [ char = $_ ] ]. - (toUse isEmpty or: [ toUse first isLetter not ]) - ifTrue: [ toUse := 'v', toUse ]. - toUse first isUppercase ifFalse:[ - toUse := toUse copy. - toUse at: 1 put: toUse first asLowercase - ]. - ^toUse - - "Modified: / 10-05-2015 / 07:29:57 / Jan Vrany " -! - -idFor: object - self assert: (object isKindOf: PPCNode). - ^ self idFor: object prefixed: object prefix suffixed: object suffix effect: #none -! - -idFor: object prefixed: prefix - ^ self idFor: object prefixed: prefix effect: #none -! - -idFor: object prefixed: prefix effect: effect - ^ self idFor: object prefixed: prefix suffixed: '' effect: effect. -! - -idFor: object prefixed: prefix suffixed: suffix effect: effect - | name id | - ^ ids at: object ifAbsentPut: [ - ((object isKindOf: PPCNode) and: [object name isNotNil]) ifTrue: [ - "Do not use prefix, if there is a name" - name := self asSelector: (object name asString). - id := (name, suffix) asSymbol. - - "Make sure, that the generated ID is uniqe!!" - (ids includes: id) ifTrue: [ - (id, '_', ids size asString) asSymbol - ] ifFalse: [ - id - ] - ] ifFalse: [ - (prefix, '_', (ids size asString), suffix) asSymbol - ] - ] -! - -idFor: object suffixed: suffix - self assert: (object isKindOf: PPCNode) description: 'Shold use PPCNode for ids'. - ^ self idFor: object prefixed: object prefix suffixed: suffix effect: #none -! ! - -!PPCCompiler methodsFor:'code generation - profiling'! - -profileTokenRead: tokenName - arguments profile ifTrue: [ - self add: 'context tokenRead: ', tokenName storeString, '.' - ] -! ! - -!PPCCompiler methodsFor:'code generation - support'! - -cache: id as: value - cache at: id put: value. -! - -cachedValue: id - ^ cache at: id ifAbsent: [ nil ] -! - -checkCache: id - | method | - "Check if method is hand written" - method := compiledParser ifNotNil: [ compiledParser compiledMethodAt: id ifAbsent: [ nil ] ]. - method ifNotNil: [ ^ PPCCompiledMethod new id: id; yourself ]. - - ^ self cachedValue: id -! - -pop - | retval | - retval := compilerStack pop. - currentMethod := compilerStack isEmpty - ifTrue: [ nil ] - ifFalse: [ compilerStack top ]. - ^ retval - - "Modified: / 21-11-2014 / 12:27:25 / Jan Vrany " -! - -push - compilerStack push: currentMethod. - (compilerStack size > 500 )ifTrue: [ self error: 'unless it is very complex grammar, there is an error somewhere' ] - - "Modified: / 21-11-2014 / 12:27:18 / Jan Vrany " -! - -startInline: id - | indentationLevel | - (cache includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. - indentationLevel := currentMethod indentationLevel. - - currentMethod := PPCInlinedMethod new. - currentMethod id: id. - currentMethod returnVariable: returnVariable. - currentMethod indentationLevel: indentationLevel. - self push. - - "Modified: / 01-06-2015 / 21:48:35 / Jan Vrany " -! - -startMethod: id - (cache includesKey: id) ifTrue: [ self error: 'OOOUPS!!' ]. - - currentMethod := PPCMethod new. - currentMethod id: id. - arguments profile ifTrue:[ - self codeProfileStart. - ]. - self push. - - self cache: id as: currentMethod. - - "Modified: / 01-06-2015 / 21:19:41 / Jan Vrany " -! - -stopInline - ^ self pop. - - "Modified: / 01-06-2015 / 21:37:59 / Jan Vrany " -! - -stopMethod - self cache: currentMethod methodName as: currentMethod. - - "arguments profile ifTrue: [ Transcript show: currentMethod code; cr. ]." - ^ self pop. - - "Modified: / 01-06-2015 / 21:38:05 / Jan Vrany " -! - -top - ^ compilerStack top -! ! - -!PPCCompiler methodsFor:'code generation - variables'! - -allocateReturnVariable - ^ self allocateReturnVariableNamed: 'retval' - - "Created: / 23-04-2015 / 18:03:40 / Jan Vrany " - "Modified: / 15-06-2015 / 17:52:56 / Jan Vrany " -! - -allocateReturnVariableNamed: name - "Allocate (or return previously allocated one) temporary variable used for - storing a parser's return value (the parsed object)" - ^ currentMethod allocateReturnVariableNamed: name - - "Created: / 15-06-2015 / 18:04:48 / Jan Vrany " -! - -allocateTemporaryVariableNamed: preferredName - "Allocate a new variable with (preferably) given name. - Returns a real variable name that should be used." - - ^ self currentNonInlineMethod allocateTemporaryVariableNamed: preferredName - - "Created: / 23-04-2015 / 17:33:31 / Jan Vrany " -! ! - -!PPCCompiler methodsFor:'compiling'! - -compileParser - self installVariables. - self installMethods. - self installClassConstants. - - ^ compiledParser -! - -copy: parser - ^ parser transform: [ :p | p copy ]. -! - -installClassConstants - constants keysAndValuesDo: [ :key :value | - compiledParser constants at: key put: value - ] -! - -installMethods - cache keysAndValuesDo: [ :key :method | - compiledParser compileSilently: method source classified: method category. - ] - - "Modified: / 24-07-2015 / 19:45:17 / Jan Vrany " -! - -installVariables - | varString | - varString := constants keys inject: '' into: [:r :e | r, ' ', e ]. - - (self compiledParserSuperclass) - subclass: compiledParserName - instanceVariableNames: varString - classVariableNames: '' - poolDictionaries: '' - category: 'PetitCompiler-Generated'. - - compiledParser := Smalltalk at: compiledParserName. -! ! - -!PPCCompiler methodsFor:'initialization'! - -initializeForCompiledClassName: aString - - self initialize. - compilerStack := Stack new. - cache := IdentityDictionary new. - constants := Dictionary new. - ids := IdentityDictionary new. - - - compiledParserName := aString asSymbol. - - ((Smalltalk respondsTo:#isSmalltalkX) and:[ Smalltalk isSmalltalkX ]) ifTrue:[ - | rPackageOrganizer | - rPackageOrganizer := Smalltalk at: #RPackageOrganizer. - rPackageOrganizer notNil ifTrue:[ - rPackageOrganizer default registerPackageNamed: 'PetitCompiler-Generated'. - ]. - ] ifFalse: [ - RPackageOrganizer default registerPackageNamed: 'PetitCompiler-Generated'. - ]. - - Smalltalk at: compiledParserName ifPresent: [ :class | - compiledParser := class. - self clean: compiledParser. - ]. - - - Transcript cr; show: 'intialized for: ', aString; cr. - - "Modified: / 26-05-2015 / 17:09:17 / Jan Vrany " -! ! - -!PPCCompiler class methodsFor:'documentation'! - -version_HG - - ^ '$Changeset: $' -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCompilerTokenErrorStrategy.st --- a/compiler/PPCCompilerTokenErrorStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,37 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -Object subclass:#PPCCompilerTokenErrorStrategy - instanceVariableNames:'compiler' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler-Codegen-Straregies' -! - -!PPCCompilerTokenErrorStrategy class methodsFor:'as yet unclassified'! - -on: aPPCCompiler - ^ self new - compiler: aPPCCompiler; - yourself -! ! - -!PPCCompilerTokenErrorStrategy methodsFor:'as yet unclassified'! - -codeClearError - compiler add: 'self clearError.' -! - -codeError - compiler add: 'error := true.' -! - -codeError: message - compiler add: 'error := true.' -! - -compiler: aPPCCompiler - compiler := aPPCCompiler -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCompilerTokenRememberStrategy.st --- a/compiler/PPCCompilerTokenRememberStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,46 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -Object subclass:#PPCCompilerTokenRememberStrategy - instanceVariableNames:'compiler' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler-Codegen-Straregies' -! - -!PPCCompilerTokenRememberStrategy class methodsFor:'instance creation'! - -on: aPPCCompiler - ^ self new - compiler: aPPCCompiler; - yourself -! ! - -!PPCCompilerTokenRememberStrategy methodsFor:'accessing'! - -compiler: aPPCCompiler - compiler := aPPCCompiler -! ! - -!PPCCompilerTokenRememberStrategy methodsFor:'as yet unclassified'! - -smartRemember: parser to: variableName - self error: 'deprecated?'. - parser isContextFree ifTrue: [ - compiler codeAssign: 'context lwRemember.' - to: variableName. - ] ifFalse: [ - compiler codeAssign: 'context remember.' - to: variableName. - ] -! - -smartRestore: parser from: mementoName - parser isContextFree ifTrue: [ - compiler add: 'context lwRestore: ', mementoName, '.'. - ] ifFalse: [ - compiler add: 'context restore: ', mementoName, '.'. - ]. -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCompilerTokenizingErrorStrategy.st --- a/compiler/PPCCompilerTokenizingErrorStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,37 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -Object subclass:#PPCCompilerTokenizingErrorStrategy - instanceVariableNames:'compiler' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler-Codegen-Straregies' -! - -!PPCCompilerTokenizingErrorStrategy class methodsFor:'as yet unclassified'! - -on: aPPCCompiler - ^ self new - compiler: aPPCCompiler; - yourself -! ! - -!PPCCompilerTokenizingErrorStrategy methodsFor:'as yet unclassified'! - -codeClearError - compiler add: 'error := false.'. -! - -codeError - compiler add: 'self error: ''error message not specified'' at: context position.'. -! - -codeError: message - compiler add: 'self error: ''', message, ''' at: context position.'. -! - -compiler: aPPCCompiler - compiler := aPPCCompiler -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCCompilerTokenizingRememberStrategy.st --- a/compiler/PPCCompilerTokenizingRememberStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,82 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -Object subclass:#PPCCompilerTokenizingRememberStrategy - instanceVariableNames:'compiler' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler-Codegen-Straregies' -! - - -!PPCCompilerTokenizingRememberStrategy class methodsFor:'instance creation'! - -on: aPPCCompiler - ^ self new - compiler: aPPCCompiler; - yourself -! ! - -!PPCCompilerTokenizingRememberStrategy methodsFor:'accessing'! - -compiler: aPPCCompiler - compiler := aPPCCompiler -! ! - -!PPCCompilerTokenizingRememberStrategy methodsFor:'as yet unclassified'! - -smartRemember: parser to: variableName - compiler allocateTemporaryVariableNamed: '__position'. - compiler allocateTemporaryVariableNamed: '__tokenType'. - compiler allocateTemporaryVariableNamed: '__tokenValue'. - - compiler codeAssign: 'context position.' to: '__position'. - compiler codeAssign: 'currentTokenType.' to: '__tokenType'. - compiler codeAssign: 'currentTokenValue.' to: '__tokenValue'. - -false ifFalse: [ - parser isContextFree ifTrue: [ -" compiler codeAssign: '{ context lwRemember. currentTokenType . currentTokenValue }.' " - compiler codeAssign: 'scanner position.' - to: variableName. - ] ifFalse: [ - compiler codeAssign: '{ context remember. currentTokenType . currentTokenValue }.' - to: variableName. - ] -] -! - -smartRestore: parser from: mementoName - compiler add: 'context lwRestore: __position.'. - compiler codeAssign: '__tokenType.' to: 'currentTokenType'. - compiler codeAssign: '__tokenValue.' to: 'currentTokenValue'. - - -false ifTrue: [ - parser isContextFree ifTrue: [ -" compiler add: 'context lwRestore: (', mementoName, ' at: 1).'." - compiler add: 'context lwRestore: ', mementoName, '.'. - ] ifFalse: [ - compiler add: 'context restore: (', mementoName, ' at: 1).'. - ]. - - compiler codeAssign: 'nil.' to: 'currentTokenType'. -" - compiler codeAssign: '(', mementoName, ' at: 2).' to: 'currentTokenType'. - compiler codeAssign: '(', mementoName, ' at: 3).' to: 'currentTokenValue'. -" -" - compiler code: 'scanner backtrack.'. -" - -] -! ! - -!PPCCompilerTokenizingRememberStrategy class methodsFor:'documentation'! - -version_HG - - ^ '$Changeset: $' -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCConfiguration.st --- a/compiler/PPCConfiguration.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCConfiguration.st Mon Aug 24 15:34:14 2015 +0100 @@ -32,8 +32,8 @@ !PPCConfiguration methodsFor:'accessing'! arguments - arguments isNil ifTrue: [ arguments := self defaultArguments ]. - ^ arguments + arguments isNil ifTrue: [ arguments := self defaultArguments ]. + ^ arguments ! arguments: args @@ -46,7 +46,8 @@ input: whatever ir := whatever. - self remember: #input. + + self remember: (self copyTree: ir) as: #input ! ir @@ -106,10 +107,6 @@ !PPCConfiguration methodsFor:'compiling'! -buildClass: compiler - self subclassResponsibility -! - compile: whatever | time | self input: whatever. @@ -127,33 +124,33 @@ !PPCConfiguration methodsFor:'debugging'! copy: somethingTransformable + self deprecated: 'copy on your own, or whatever, but dont use me'. + ^ somethingTransformable transform: [ :e | e copy ] +! + +copyTree: somethingTransformable ^ somethingTransformable transform: [ :e | e copy ] ! remember: key + self deprecated: 'use remember:as:'. + self arguments debug ifTrue: [ history add: key -> (self copy: ir). ] -! ! - -!PPCConfiguration methodsFor:'hooks'! - -codeCompiler - ^ PPCCodeGen on: arguments ! -codeCompilerOn: args - ^ PPCCodeGen on: args -! - -codeGeneratorVisitorOn: compiler - ^ arguments codeGenerator on: compiler +remember: value as: key + self arguments debug ifTrue: [ + history add: key -> value. + ] ! ! !PPCConfiguration methodsFor:'initialization'! initialize - history := OrderedCollection new + history := OrderedCollection new. + arguments := PPCArguments default. ! ! !PPCConfiguration methodsFor:'phases'! @@ -177,7 +174,8 @@ ir := PPCRecognizerComponentDetector new arguments: arguments; visit: ir. - self remember: #recognizingComponents + + self remember: (self copyTree: ir) as: #recognizingComponents ! createTokens @@ -186,24 +184,8 @@ ir := PPCTokenDetector new arguments: arguments; visit: ir. - self remember: #createTokens -! -generate - | compiler rootMethod compiledParser | - arguments generate ifFalse: [ ^ self ]. - - compiler := self codeCompiler. - - rootMethod := (self codeGeneratorVisitorOn: compiler) - arguments: arguments; - visit: ir. - - compiledParser := self buildClass: compiler. - compiledParser startSymbol: rootMethod methodName. - compiledParser := compiledParser new. - - ir := compiledParser. + self remember: (self copyTree: ir) as: #createTokens ! inline @@ -212,7 +194,8 @@ ir := PPCInliningVisitor new arguments: arguments; visit: ir. - self remember: #inline. + + self remember: (self copyTree: ir) as: #inline. ! merge @@ -222,7 +205,8 @@ ir := PPCMergingVisitor new arguments: arguments; visit: ir. - self remember: #merge + + self remember: (self copyTree: ir) as: #merge ! specialize @@ -237,13 +221,15 @@ ir := (PPCSpecializingVisitor new arguments: arguments; visit: ir). - self remember: #specialize + + self remember: (self copyTree: ir) as: #specialize ! toPPCIr "Creates a PPCNodes from a PPParser" ir := ir asCompilerTree. - self remember: #ppcNodes + + self remember: (self copyTree: ir) as: #ppcNodes ! ! !PPCConfiguration methodsFor:'reporting'! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCDistinctResultStrategy.st --- a/compiler/PPCDistinctResultStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCDistinctResultStrategy.st Mon Aug 24 15:34:14 2015 +0100 @@ -11,10 +11,6 @@ !PPCDistinctResultStrategy methodsFor:'as yet unclassified'! -recordFailure: retval - ^ self recordFailure: retval offset: 0 -! - recordFailure: retval offset: offset offset == 0 ifTrue: [ codeGen codeRecordDistinctMatch: nil. @@ -23,10 +19,6 @@ codeGen codeRecordDistinctMatch: nil offset: offset ! -recordMatch: retval - ^ self recordMatch: retval offset: 0 -! - recordMatch: retval offset: offset offset == 0 ifTrue: [ codeGen codeRecordDistinctMatch: retval. @@ -40,6 +32,24 @@ ^ codeGen code: 'self resetDistinct.'. ! +returnFailure: retval offset: offset + (offset == 0) ifFalse: [ + codeGen codeIf: 'currentChar isNil' then: nil else: [ + codeGen code: ('context skip: -', offset asString); codeDot. + ] + ]. + + codeGen codeReturn: [ codeGen codeRecordDistinctFailure: nil ]. +! + +returnMatch: retval offset: offset + (offset == 0) ifFalse: [ + codeGen code: 'self back.'. + ]. + + codeGen codeReturn: [ codeGen codeRecordDistinctMatch: retval ]. +! + returnResult: state codeGen codeNl. codeGen codeReturnDistinct. diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCDistinctScanner.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCDistinctScanner.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,135 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Object subclass:#PPCDistinctScanner + instanceVariableNames:'position match matchPosition currentChar context returnBlock' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Scanner' +! + +!PPCDistinctScanner methodsFor:'accessing-private'! + +getMatch + ^ match +! + +position + "returns the start position before the scan method..." + ^ position +! + +position: anObject + position := matchPosition := anObject +! + +setMatch: value + match := value +! + +stream + ^ context +! + +stream: anObject + context := anObject +! ! + +!PPCDistinctScanner methodsFor:'initialization'! + +initialize + super initialize. + match := nil. + position := 0. + matchPosition := 0. +! + +resetDistinct +! ! + +!PPCDistinctScanner methodsFor:'memoization'! + +remember + ^ position +! + +restore: restorePosition + context position: (matchPosition := position := restorePosition). + match := nil +! ! + +!PPCDistinctScanner methodsFor:'results'! + +polyResult + | dictionary | + dictionary := IdentityDictionary new. + match isNil ifFalse: [ + dictionary at: match put: matchPosition. + ]. + + ^ dictionary +! + +recordDistinctFailure + self flag: 'Potential bug? What if there is something after the failure? Might that happen for distinct parser?'. + + match := nil. + matchPosition := position. + context position: position. + ^ false +! + +recordDistinctMatch: matchValue + match := matchValue. + matchPosition := context position. + ^ true +! + +recordDistinctMatch: matchValue offset: offset + match := matchValue. + + currentChar isNil ifFalse: [ + matchPosition := context position - offset. + ] ifTrue: [ + matchPosition := context position. + ]. + ^ true +! + +result + ^ match +! + +resultPosition + ^ matchPosition +! + +returnDistinct +" + match isNil ifTrue: [ + self assert: matchPosition == position + ]. +" + + context position: matchPosition. + ^ match isNotNil +! ! + +!PPCDistinctScanner methodsFor:'scanning'! + +back + currentChar isNil ifFalse: [ + context skip: -1 + ] +! + +peekBetween: start and: stop + (currentChar == nil) ifTrue: [ ^ false ]. + ^ (start <= currentChar codePoint) and: [ currentChar codePoint <= stop ] +! + +step + currentChar := context next +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCFSACodeGen.st --- a/compiler/PPCFSACodeGen.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCFSACodeGen.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PPCCodeGen subclass:#PPCFSACodeGen - instanceVariableNames:'fsa backlinkStates compiler' + instanceVariableNames:'' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Scanner' @@ -18,7 +18,7 @@ !PPCFSACodeGen methodsFor:'coding'! codeAbsoluteReturn: code - self add: '^ ', code + self code: '^ ', code ! codeAssertPeek: t @@ -26,44 +26,43 @@ self assert: (t isKindOf: PEGFsaTransition). (t isPredicateTransition and: [t isEOF]) ifTrue: [ - self addOnLine: 'currentChar isNil'. + self codeOnLine: 'currentChar isNil'. ^ self ]. - (t isPredicateTransition) ifTrue: [ - self addOnLine: t predicate asString, ' value: currentChar codePoint'. + self codeOnLine: t predicate asString, ' value: currentChar codePoint'. ^ self ]. (t isAny) ifTrue: [ - self addOnLine: 'true'. + self codeOnLine: 'true'. ^ self ]. (t isSingleCharacter) ifTrue: [ - self addOnLine: 'currentChar == ', t character storeString. + self codeOnLine: 'currentChar == ', t character storeString. ^ self ]. (t isNotSingleCharacter) ifTrue: [ - self addOnLine: 'currentChar ~~ ', t notCharacter storeString. + self codeOnLine: 'currentChar ~~ ', t notCharacter storeString. ^ self ]. (t isLetter) ifTrue: [ - self addOnLine: 'currentChar isLetter'. + self codeOnLine: 'currentChar isLetter'. ^ self ]. (t isWord) ifTrue: [ - self addOnLine: 'currentChar isAlphaNumeric'. + self codeOnLine: 'currentChar isAlphaNumeric'. ^ self ]. (t isDigit) ifTrue: [ - self addOnLine: 'currentChar isDigit'. + self codeOnLine: 'currentChar isDigit'. ^ self ]. @@ -71,16 +70,16 @@ | begin end | begin := t beginOfRange. end := t endOfRange. - self addOnLine: 'self peekBetween: ', begin asString, ' and: ', end asString. + self codeOnLine: 'self peekBetween: ', begin asString, ' and: ', end asString. ^ self ]. - id := idGen cachedSuchThat: [ :e | e = t characterSet ] + id := self idGen cachedSuchThat: [ :e | e = t characterSet ] ifNone: [ self idFor: t characterSet defaultName: 'characterSet' ]. self addConstant: t characterSet as: id. - self addOnLine: '(currentChar isNotNil) and: [', id, ' at: currentChar codePoint ]'. + self codeOnLine: '(currentChar isNotNil) and: [', id, ' at: currentChar codePoint ]'. ! codeAssertPeek: transition ifFalse: falseBlock @@ -93,13 +92,13 @@ ! codeAssertPeek: t ifTrue: block - self addOnLine: '('. + self codeOnLine: '('. self codeAssertPeek: t. - self addOnLine: ') ifTrue: ['. + self codeOnLine: ') ifTrue: ['. self indent. self code: block. self dedent. - self add: ']'. + self code: ']'. ! codeAssertPeek: transition orReturn: priority @@ -130,19 +129,15 @@ codeEndBlockWhileTrue self dedent. - self add: '] whileTrue.'. + self code: '] whileTrue.'. ! codeIfFalse - self addOnLine: ' ifFalse: ['. + self codeOnLine: ' ifFalse: ['. ! codeNextChar - self add: 'self step.' -! - -codeNl - self add: ''. + self code: 'self step.' ! codeNlAssertPeek: characterSet @@ -161,8 +156,12 @@ self add: '^ self returnPriority: ', priority asString, '.' ! +codeRecordDistinctFailure: retval offset: value + self add: 'self recordDistinctFailure: ', retval storeString, ' offset: ', value storeString, '.' +! + codeRecordDistinctMatch: retval offset: value - self add: 'self recordDistinctMatch: ', retval storeString, ' offset: ', value storeString, '.' + self code: 'self recordDistinctMatch: ', retval storeString, ' offset: ', value storeString, '.' ! codeRecordMatch: state priority: priority @@ -173,32 +172,47 @@ self add: 'self recordMatch: ', state storeString, ' priority: ', priority asString, '.' ! +codeReturn: code + arguments profile ifTrue:[ + self codeProfileStop. + ]. + self code: '^ '. + self codeOnLine: code +! + codeReturnResult self addOnLine: '^ self return.' ! codeReturnResult: priority + self error: 'deprecated?'. priority isNil ifTrue: [ ^ self codeReturnResult ]. - + self addOnLine: '^ self returnPriority: ', priority asString, '.' ! codeStartBlock - self add: '['. + self code: '['. self indent. ! ! !PPCFSACodeGen methodsFor:'coding - results'! +codeRecordDistinctFailure: retval + self assert: retval isNil. + + self code: 'self recordDistinctFailure.' +! + codeRecordDistinctMatch: retval - self add: 'self recordDistinctMatch: ', retval storeString, '.' + self code: 'self recordDistinctMatch: ', retval storeString, '.' ! codeRecordFailure: index self assert: index isInteger. - self add: 'self recordFailure: ', index asString, '.' + self code: 'self recordFailure: ', index asString, '.' ! codeRecordMatch: retval @@ -206,23 +220,20 @@ ! codeRecordMatch: retval offset: offset - self add: 'self recordMatch: ', retval storeString, ' offset: ', offset storeString, '.' + self code: 'self recordMatch: ', retval storeString, ' offset: ', offset storeString, '.' ! codeReturn - self addOnLine: '^ self' + self codeOnLine: '^ self' ! codeReturnDistinct - self addOnLine: '^ self returnDistinct.' + self codeOnLine: '^ self returnDistinct.' ! ! !PPCFSACodeGen methodsFor:'intitialization'! initialize super initialize. - - compiler := PPCCodeGen new. - backlinkStates := IdentityDictionary new. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCFSAVisitor.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCFSAVisitor.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,113 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCNodeVisitor subclass:#PPCFSAVisitor + instanceVariableNames:'fsaCache idGen' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Visitors' +! + +!PPCFSAVisitor methodsFor:'accessing'! + +idGen: anObject + idGen := anObject +! ! + +!PPCFSAVisitor methodsFor:'as yet unclassified'! + +unorderedChoiceFromFollowSet: followSet + | followFsas | + + ^ fsaCache at: followSet ifAbsentPut: [ + followFsas := followSet collect: [ :followNode | + followNode asFsa + name: (idGen idFor: followNode); + retval: (idGen idFor: followNode); + yourself + ]. + self unorderedChoiceFromFsas: followFsas. + ] + +! + +unorderedChoiceFromFsas: fsas + | result startState | + result := PEGFsa new. + startState := PEGFsaState new. + + result addState: startState. + result startState: startState. + + fsas do: [ :fsa | + result adopt: fsa. + result addTransitionFrom: startState to: fsa startState. + ]. + + result determinizeStandard. + ^ result +! + +visitToken: tokenNode + | anFsa | + + anFsa := tokenNode asFsa determinize. + anFsa name: (idGen idFor: tokenNode). + anFsa retval: (idGen idFor: tokenNode). + + tokenNode fsa: anFsa. + ^ tokenNode +! + +visitTokenConsumeNode: node + | epsilon anFsa followSet | + followSet := node followSetWithTokens. + + epsilon := followSet anySatisfy: [ :e | e acceptsEpsilon ]. + followSet := followSet reject: [ :e | e acceptsEpsilon ]. + epsilon ifTrue: [ followSet add: PPCEndOfFileNode instance ]. + + anFsa := self unorderedChoiceFromFollowSet: followSet. + anFsa name: 'nextToken_', (idGen idFor: node). + + node nextFsa: anFsa. +! + +visitTokenNode: node + ^ self visitToken: node +! + +visitTokenizingParserNode: node + "TODO JK: hack alert, change the handling of WS!!" + self visitWhitespace: node whitespace. + + self visit: node tokens. + self visit: node parser. + ^ node +! + +visitTrimmingTokenNode: node + ^ self visitToken: node +! + +visitWhitespace: node + "JK HACK: treat ws as token -> create FSA for whitespace" + | retval | + retval := self visitToken: node. + "we don't care about the finals of whitespace" + node fsa removeFinals. + ^ retval +! ! + +!PPCFSAVisitor methodsFor:'initialization'! + +initialize + super initialize. + + "for the given set of nodes, remember the unordered choice fsa + see `unorderedChoiceFromFollowSet:` + " + fsaCache := Dictionary new. +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCGuard.st --- a/compiler/PPCGuard.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCGuard.st Mon Aug 24 15:34:14 2015 +0100 @@ -61,7 +61,7 @@ ! compileAny: compiler - compiler add: '(context atEnd not)'. + compiler code: '(context atEnd not)'. ! compileCharacter: compiler @@ -69,11 +69,11 @@ classification keysAndValuesDo: [ :index :value | value ifTrue: [ (index > 32 and: [ index < 127 ]) ifTrue: [ - compiler add: '(context peek == ', (Character value: index) storeString, ')' + compiler code: '(context peek == ', (Character value: index) storeString, ')' ] ifFalse: [ id := compiler idFor: (Character value: index) defaultName: #character. compiler addConstant: (Character value: index) as: id. - compiler add: '(context peek = ', id, ')'. + compiler code: '(context peek = ', id, ')'. ] ] ]. ! @@ -84,7 +84,7 @@ ! compileMessage: compiler - compiler add: '(context peek ', message, ')' + compiler code: '(context peek ', message, ')' ! testAny @@ -112,7 +112,7 @@ array := ((classification asOrderedCollection) addLast: false; yourself) asArray. compiler addConstant: array as: id. - compiler add: '(', id, ' at: context peek asInteger)'. + compiler code: '(', id, ' at: context peek asInteger)'. ! compileGuard: compiler diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCInlinedMethod.st --- a/compiler/PPCInlinedMethod.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCInlinedMethod.st Mon Aug 24 15:34:14 2015 +0100 @@ -22,13 +22,6 @@ ^ self code ! -code - self error: 'deprecated?'. - ^ (String streamContents:[:s |" buffer codeOn:s "]) trimRight - - "Modified (format): / 01-06-2015 / 21:44:56 / Jan Vrany " -! - isInline ^ true ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCInliningVisitor.st --- a/compiler/PPCInliningVisitor.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCInliningVisitor.st Mon Aug 24 15:34:14 2015 +0100 @@ -106,8 +106,7 @@ "super visitTokenConsumeNode: node." node name isNil ifTrue: [ - self flag: 'temporarily disabled'. - "self markForInline: node" + self markForInline: node ]. ^ node diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCMethod.st --- a/compiler/PPCMethod.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCMethod.st Mon Aug 24 15:34:14 2015 +0100 @@ -21,12 +21,8 @@ !PPCMethod methodsFor:'accessing'! body - self error: 'Should no longer be used'. - " - ^ buffer contents - " - - "Modified: / 17-08-2015 / 11:58:18 / Jan Vrany " + self halt: '???'. +" ^ buffer contents" ! bridge @@ -73,24 +69,6 @@ ^ selector ! -profile - self error: 'Should no longer be used'. - " - ^ profile - " - - "Modified: / 17-08-2015 / 11:58:40 / Jan Vrany " -! - -profile: aBoolean - self error: 'Should no longer be used'. - " - profile := aBoolean - " - - "Modified: / 17-08-2015 / 11:58:46 / Jan Vrany " -! - source ^ source isString ifTrue:[ source @@ -162,13 +140,13 @@ innerBlock := PPCCodeBlock new. innerBlock indentationLevel: outerBlock indentationLevel + 1. [ - outerBlock addOnLine:'['. + outerBlock addOnLine: '['. source := innerBlock. self codeOnLine: contents. ] ensure:[ outerBlock - code: (String streamContents:[:s | innerBlock sourceOn:s]); - add:']'. + codeOnLine: (String streamContents:[:s | innerBlock sourceOn:s]); + add: ']'. source := outerBlock. ] @@ -207,15 +185,13 @@ !PPCMethod methodsFor:'code generation - variables'! addVariable: name - self error: 'Should no longer be used' - " - (variables includes: name) ifTrue:[ + self halt: 'deprecated'. +" (variables includes: name) ifTrue:[ self error:'Duplicate variable name, must rename'. ]. variables add: name. - " - - "Modified: / 17-08-2015 / 11:56:34 / Jan Vrany " +" + "Modified: / 23-04-2015 / 12:29:58 / Jan Vrany " ! allocateReturnVariable @@ -267,15 +243,6 @@ "Created: / 23-04-2015 / 18:23:47 / Jan Vrany " "Modified: / 15-06-2015 / 18:14:02 / Jan Vrany " -! - -variables - self error: 'Should no longer be used'. - " - ^ ' | ', (variables inject: '' into: [ :s :e | s, ' ', e]), ' |' - " - - "Modified: / 17-08-2015 / 11:54:58 / Jan Vrany " ! ! !PPCMethod methodsFor:'initialization'! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCNoResultStrategy.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCNoResultStrategy.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,38 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCScannerResultStrategy subclass:#PPCNoResultStrategy + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Scanner' +! + +!PPCNoResultStrategy methodsFor:'as yet unclassified'! + +recordFailure: retval offset: offset + "nothing to do" +! + +recordMatch: retval offset: offset + "nothing to do" +! + +reset + "nothing to do" +! + +returnFailure: retval offset: offset + "nothing to do" +! + +returnMatch: retval offset: offset + codeGen code: 'stream position: stream position - 1.'. + codeGen codeReturn +! + +returnResult: state + codeGen codeReturn +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCNode.st --- a/compiler/PPCNode.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCNode.st Mon Aug 24 15:34:14 2015 +0100 @@ -102,6 +102,32 @@ "Created: / 23-04-2015 / 15:39:39 / Jan Vrany " ! ! +!PPCNode methodsFor:'accessing-fsa'! + +fsa + ^ self propertyAt: #fsa +! + +fsa: aPEGFsa + self propertyAt: #fsa put: aPEGFsa +! + +hasFsa + ^ self hasProperty: #fsa +! + +hasNextFsa + ^ self hasProperty: #nextFsa +! + +nextFsa + ^ self propertyAt: #nextFsa +! + +nextFsa: aPEGFsa + self propertyAt: #nextFsa put: aPEGFsa +! ! + !PPCNode methodsFor:'accessing-properties'! hasProperty: aKey diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCRecognizerComponentDetector.st --- a/compiler/PPCRecognizerComponentDetector.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCRecognizerComponentDetector.st Mon Aug 24 15:34:14 2015 +0100 @@ -48,10 +48,13 @@ ! visitTokenizingParserNode: node - | newWhitespace | - self change. + | | + + "Do not visit whitespace, it is tokenizer's job" +" self change. newWhitespace := self visitWithRecognizingComponentVisitor: node whitespace. node replace: node whitespace with: newWhitespace. +" "Do not visit tokens, they will be handled by the scanner:" "self visit: node tokens." diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCScanner.st --- a/compiler/PPCScanner.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCScanner.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,8 +3,8 @@ "{ NameSpace: Smalltalk }" Object subclass:#PPCScanner - instanceVariableNames:'match matchPosition matches tokens stream currentChar - maxSymbolNumber position' + instanceVariableNames:'match matchPosition matches tokens currentChar maxSymbolNumber + position context' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Scanner' @@ -33,11 +33,11 @@ ! stream - ^ stream + ^ context ! stream: anObject - stream := anObject + context := anObject ! ! !PPCScanner methodsFor:'initialization'! @@ -55,7 +55,7 @@ reset matchPosition := nil. "This flag says that multimode run the last time" - position := stream position. + position := context position. " matches := Array new: maxSymbolNumber." ! @@ -76,7 +76,7 @@ matchPosition := -1. "this is a flag that the distnict mode was running" " matches := nil." - position := stream position. + position := context position. ! ! @@ -170,18 +170,32 @@ !PPCScanner methodsFor:'results - distinct'! +recordDistinctFailure + match := nil. + matchPosition := position. + ^ false +! + +recordDistinctFailure: matchValue + match := matchValue. + matchPosition := context position. + ^ false +! + recordDistinctMatch: matchValue match := matchValue. - matchPosition := stream position. + matchPosition := context position. + ^ true ! recordDistinctMatch: matchValue offset: offset match := matchValue. currentChar isNil ifFalse: [ - matchPosition := stream position - offset. + matchPosition := context position - offset. ] ifTrue: [ - matchPosition := stream position. - ] + matchPosition := context position. + ]. + ^ true ! returnDistinct @@ -199,14 +213,14 @@ ! recordMatch: index - matches at: index put: stream position. + matches at: index put: context position. ! recordMatch: index offset: offset currentChar isNil ifFalse: [ - matches at: index put: stream position - offset. + matches at: index put: context position - offset. ] ifTrue: [ - matches at: index put: stream position. + matches at: index put: context position. ]. ! @@ -217,12 +231,14 @@ !PPCScanner methodsFor:'scanning'! -next - self error: 'deprecated?'. - stream next +back + currentChar isNil ifFalse: [ + context skip: -1 + ] ! peek + self flag: 'inline this, do not call peek!!'. ^ currentChar ! @@ -232,7 +248,7 @@ ! step - currentChar := stream next + currentChar := context next ! ! !PPCScanner methodsFor:'testing'! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCScannerCodeGenerator.st --- a/compiler/PPCScannerCodeGenerator.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCScannerCodeGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,8 +3,8 @@ "{ NameSpace: Smalltalk }" Object subclass:#PPCScannerCodeGenerator - instanceVariableNames:'codeGen fsa backlinkStates backlinkTransitions arguments openSet - incommingTransitions methodCache id resultStrategy fsaCache' + instanceVariableNames:'codeGen fsa arguments incommingTransitions resultStrategy + fsaCache' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Scanner' @@ -17,7 +17,8 @@ ! arguments: anObject - arguments := anObject + arguments := anObject. + codeGen arguments: anObject. ! codeGen @@ -30,26 +31,24 @@ !PPCScannerCodeGenerator methodsFor:'analysis'! -analyzeBacklinks - backlinkTransitions := fsa backTransitions. - backlinkStates := IdentityDictionary new. - - backlinkTransitions do: [ :t | - (self backlinksTo: (t destination)) add: t. - ]. -! - analyzeDistinctRetvals - (fsa hasDistinctRetvals) ifTrue: [ - resultStrategy := PPCDistinctResultStrategy new + (fsa hasNoRetvals) ifTrue: [ + ^ resultStrategy := PPCNoResultStrategy new codeGen: codeGen; yourself - ] ifFalse: [ - resultStrategy := PPCUniversalResultStrategy new + ]. + + + (fsa hasDistinctRetvals) ifTrue: [ + ^ resultStrategy := PPCDistinctResultStrategy new codeGen: codeGen; - tokens: fsa retvals asArray; yourself - ] + ]. + + resultStrategy := PPCUniversalResultStrategy new + codeGen: codeGen; + tokens: fsa retvals asArray; + yourself ! analyzeTransitions @@ -63,8 +62,8 @@ ]. ! -backlinksTo: state - ^ backlinkStates at: state ifAbsentPut: [ OrderedCollection new ] +clazz: aPPCClass + codeGen clazz: aPPCClass ! containsBacklink: state @@ -83,12 +82,23 @@ ^ incommingTransitions at: state ifAbsentPut: [ IdentitySet new ] ! -isBacklink: transition - ^ backlinkTransitions includes: transition +isSingleTerminatingRetval: state + (state retvals size == 1 and: [ state isFinal ]) ifFalse: [ ^ false ]. + + state transitions isEmpty ifTrue: [ ^ true ]. + ((state transitions size == 1) and: [state destination == state]) ifTrue: [ ^ self startsSimpleLoop: state ]. + + ^ false ! -isBacklinkDestination: state - ^ (self backlinksTo: state) isEmpty not +isSingleTransitionFsa + fsa allTransitions size == 1 ifFalse: [ ^ false ]. + "do not allow loop!!" + fsa startState destination == fsa startState ifTrue: [ ^ false ]. + "so far only single char allowed" + fsa startState transition isCharacterTransition ifFalse: [ ^ false ]. + fsa startState transition isSingleCharacter ifFalse: [ ^ false ]. + ^ true ! startsSimpleLoop: state @@ -106,7 +116,7 @@ !PPCScannerCodeGenerator methodsFor:'caching'! cache: anFsa method: method - fsaCache at: anFsa put: method + ^ fsaCache at: anFsa put: method ! cachedValueForIsomorphicFsa: anFsa @@ -116,37 +126,33 @@ ! isomorphicIsCached: anFsa - ^ fsaCache keys anySatisfy: [ :e | e isIsomorphicTo: anFsa ] + ^ fsaCache keys anySatisfy: [ :e | (e isIsomorphicTo: anFsa) and: [ e name = anFsa name ] ] ! ! !PPCScannerCodeGenerator methodsFor:'code generation'! generate - | method | self assert: fsa isDeterministic. self assert: fsa isWithoutEpsilons. self assert: fsa checkConsistency. (self isomorphicIsCached: fsa) ifTrue: [ + "JK: please not, right now, checks for isomorphism and name + this might be improved in future and name can be 'reused' + " ^ self cachedValueForIsomorphicFsa: fsa ]. - self analyzeBacklinks. self analyzeTransitions. self analyzeDistinctRetvals. - openSet := IdentitySet new. codeGen startMethod: (codeGen idFor: fsa). - codeGen codeComment: (Character codePoint: 13) asString, fsa asString. +" codeGen codeComment: (Character codePoint: 13) asString, fsa asString." resultStrategy reset. self generateFor: fsa startState. - method := codeGen stopMethod. - self cache: fsa method: method. - - ^ method. - + ^ self cache: fsa method: codeGen stopMethod. ! @@ -182,6 +188,16 @@ ! generateFinalFor: state offset: offset + "Handle one retval specially" + (self isSingleTerminatingRetval: state) ifTrue: [ + state isFsaFailure ifTrue: [ + resultStrategy returnFailure: state retval offset: offset. + ] ifFalse: [ + resultStrategy returnMatch: state retval offset: offset. + ]. + ^ self + ]. + state retvalsAndInfosDo: [:retval :info | info isFinal ifTrue: [ info isFsaFailure ifTrue: [ @@ -194,12 +210,26 @@ ! generateFor: state - codeGen cachedValue: (codeGen idFor: state) ifPresent: [ :method | + codeGen cachedMethod: (codeGen idFor: state) ifPresent: [ :method | "if state is already cached, it has multiple incomming links. In such a case, it is compiled as a method, thus return immediatelly" ^ codeGen codeAbsoluteReturn: method call ]. + self flag: 'TODO JK: Hack alert, fix this:'. + (state isKindOf: PEGFsaParserState) ifTrue: [ + | id | + self assert: state transitions isEmpty. + id := codeGen idFor: state parser defaultName: 'parser'. + codeGen addConstant: state parser as: id. + codeGen code: id, ' parseOn: context'. + ^ self + ]. + + (self isSingleTransitionFsa) ifTrue: [ + ^ self generateForSingleTransitionFsa: state. + ]. + (self startsSimpleLoop: state) ifTrue: [ ^ self generateSimpleLoopFor: state ]. @@ -207,20 +237,21 @@ ^ self generateStandardFor: state ! -generateForSingleTransition: t from: state. +generateForSingleTransitionFsa: startState + | transition | + self assert: fsa startState == startState. - (self isJoinPoint: t destination) ifTrue: [ self removeJoinTransition: t ]. + transition := startState transition. - codeGen codeAssertPeek: t ifFalse: [ - resultStrategy returnResult: state - ]. -" (self isBacklink: t) ifTrue: [ - codeGen add: 'true' + transition isSingleCharacter ifTrue: [ + codeGen codeIf: 'context peek == ', transition character storeString then: [ + codeGen code: 'self step'; codeDot. + self generateFinalFor: transition destination. + ]. + codeGen codeReturn: 'false'. ] ifFalse: [ - self generateFor: t destination. + self error: 'should be implemented' ] -" - self generateFor: t destination ! generateForTransition: t from: state @@ -246,8 +277,12 @@ ! generateReturnFor: state - codeGen codeNl. - resultStrategy returnResult: state. +" codeGen codeNl." + (self isSingleTerminatingRetval: state) ifFalse: [ + resultStrategy returnResult: state. + ] ifTrue: [ + "return already generated within the match" + ] ! generateSimpleLoopFor: state @@ -262,7 +297,9 @@ "Last transition did not passed the loop, therefore, we have to record succes with offset -1" self generateFinalFor: state offset: 1. - self generateTransitions: (state transitions reject: [ :t | t == selfTransition ]) for: state. + self generateTransitions: (state transitions reject: [ :t | t == selfTransition ]) + for: state + offset: 1. ! @@ -275,7 +312,8 @@ self generateStopMethod: state. ! -generateStartMethod: state. +generateStartMethod: state + | id | id := codeGen idFor: state. codeGen codeComment: 'START - Generated from state: ', state asString. @@ -297,19 +335,18 @@ codeGen codeComment: 'STOP - Generated from state: ', state asString. ! -generateTransitions: transitions for: state +generateTransitions: transitions for: state offset: offset (transitions size = 0) ifTrue: [ - self generateReturnFor: state. + (offset > 0 and: [ state isFinal not ]) ifTrue: [ + codeGen codeIf: 'currentChar isNil' then: nil else: [ + codeGen codeOnLine: 'context skip: -', offset asString + ]. + ]. ^ self ]. -" (state transitions size = 1) ifTrue: [ - self generateForSingleTransition: state transitions anyOne from: state. - ^ self - ]." - - codeGen codeNl. - transitions do: [ :t | +" codeGen codeNl. +" transitions do: [ :t | self generateForTransition: t from: state ]. @@ -317,8 +354,8 @@ self generateReturnFor: state. codeGen dedent. codeGen codeNl. - transitions size timesRepeat: [ codeGen addOnLine: ']' ]. - codeGen addOnLine: '.'. + transitions size timesRepeat: [ codeGen codeOnLine: ']' ]. + codeGen codeDot. " self closedJoinPoints isEmpty ifFalse: [ @@ -333,7 +370,7 @@ ! generateTransitionsFor: state - ^ self generateTransitions: state transitions for: state + ^ self generateTransitions: state transitions for: state offset: 0 ! setMaxNumericId @@ -365,9 +402,9 @@ builder := PPCClassBuilder new. builder compiledClassName: arguments scannerName. - builder compiledSuperclass: PPCScanner. - builder methodDictionary: codeGen methodDictionary. - builder constants: codeGen constants. + builder compiledSuperclass: arguments scannerSuperclass. + builder methodDictionary: codeGen clazz methodDictionary. + builder constants: codeGen clazz constants. ^ builder compileClass. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCScannerError.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCScannerError.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,17 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Error subclass:#PPCScannerError + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Exceptions' +! + +!PPCScannerError methodsFor:'as yet unclassified'! + +signalWith: message + self signal: message +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCSpecializingVisitor.st --- a/compiler/PPCSpecializingVisitor.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCSpecializingVisitor.st Mon Aug 24 15:34:14 2015 +0100 @@ -149,7 +149,6 @@ ! visitStarNode: node - self visitChildren: node. (node child isKindOf: PPCMessagePredicateNode) ifTrue: [ @@ -187,7 +186,6 @@ ! visitTokenizingParserNode: node - self visitChild: node whitespace of: node. self visitChild: node parser of: node. ^ node ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenCodeGenerator.st --- a/compiler/PPCTokenCodeGenerator.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenCodeGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,27 +3,21 @@ "{ NameSpace: Smalltalk }" PPCNodeVisitor subclass:#PPCTokenCodeGenerator - instanceVariableNames:'compiler scannerGenerator fsaCache' + instanceVariableNames:'codeGen' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Visitors' + category:'PetitCompiler-Visitors-CodeGenerators' ! !PPCTokenCodeGenerator methodsFor:'accessing'! arguments: args super arguments: args. - scannerGenerator arguments: args + codeGen arguments: args. ! -compiler - ^ compiler -! - -compiler: anObject - compiler := anObject. - - scannerGenerator compiler idGen: compiler idGen. +clazz: aPPCClass + codeGen clazz: aPPCClass ! ! !PPCTokenCodeGenerator methodsFor:'code support'! @@ -32,18 +26,19 @@ self assert: node isTokenNode. node isTrimmingTokenNode ifTrue: [ - compiler code: 'self consumeWhitespace.' + codeGen code: 'self scan_consumeWhitespace.' ] ! -createTokenInsance: node id: idCode start: startVar end: endVar - compiler codeTranscriptShow: 'current token type: ', idCode. - compiler codeAssign: idCode, '.' to: 'currentTokenType'. - compiler codeAssign: node tokenClass asString, ' on: (context collection) +createTokenInstance: node id: idCode start: startVar end: endVar + codeGen startInline. + codeGen codeTranscriptShow: 'current token type: ', idCode. + codeGen codeAssign: node tokenClass asString, ' on: (context collection) start: ', startVar, ' stop: ', endVar, ' value: nil.' - to: 'currentTokenValue'. + to: self retvalVar. + ^ codeGen stopInline ! scan: node start: startVar end: endVar @@ -51,72 +46,37 @@ node child name: node name ]. - compiler codeAssign: 'context position + 1.' to: startVar. - compiler add: ((self generateScan: node child) callOn: 'scanner'). -! - -unorderedChoiceFromFollowSet: followSet - | followFsas | - - ^ fsaCache at: followSet ifAbsentPut: [ - followFsas := followSet collect: [ :followNode | - (followNode asFsa) - name: (compiler idFor: followNode); - retval: (compiler idFor: followNode); - yourself - ]. - self unorderedChoiceFromFsas: followFsas. - ] - -! - -unorderedChoiceFromFsas: fsas - | result startState | - result := PEGFsa new. - startState := PEGFsaState new. - - result addState: startState. - result startState: startState. - - fsas do: [ :fsa | - result adopt: fsa. - result addTransitionFrom: startState to: fsa startState. - ]. - - result determinizeStandard. - ^ result + codeGen codeAssign: 'context position + 1.' to: startVar. + codeGen add: ((self generateScan: node child) callOn: 'scanner'). ! ! !PPCTokenCodeGenerator methodsFor:'compiling support'! -compileScanner - ^ scannerGenerator compileScannerClass -! - retvalVar - ^ compiler currentReturnVariable + ^ codeGen currentReturnVariable ! startMethodForNode:node + node isMarkedForInline ifTrue:[ - compiler startInline: (compiler idFor: node). - compiler codeComment: 'BEGIN inlined code of ' , node printString. - compiler indent. + codeGen startInline: (codeGen idFor: node). + codeGen codeComment: 'BEGIN inlined code of ' , node printString. + codeGen indent. ] ifFalse:[ - compiler startMethod: (compiler idFor: node). - compiler currentMethod category: 'generated - tokens'. - compiler codeComment: 'GENERATED by ' , node printString. - compiler allocateReturnVariable. + codeGen startMethod: (codeGen idFor: node). + codeGen currentMethod category: 'generated - tokens'. + codeGen codeComment: 'GENERATED by ' , node printString. + codeGen allocateReturnVariable. ] ! stopMethodForNode:aPPCNode - ^ aPPCNode isMarkedForInline ifTrue:[ - compiler dedent. - compiler add: '"END inlined code of ' , aPPCNode printString , '"'. - compiler stopInline. + ^ codeGen currentMethod isInline ifTrue:[ + codeGen dedent. + codeGen add: '"END inlined code of ' , aPPCNode printString , '"'. + codeGen stopInline. ] ifFalse:[ - compiler stopMethod + codeGen stopMethod ]. ! ! @@ -125,147 +85,75 @@ initialize super initialize. - scannerGenerator := PPCScannerCodeGenerator new. - scannerGenerator arguments: arguments. - - "for the given set of nodes, remember the unordered choice fsa - see `unorderedChoiceFromFollowSet:` - " - fsaCache := Dictionary new. -! ! - -!PPCTokenCodeGenerator methodsFor:'scanning'! - -generateNextScan: node - | epsilon followSet anFsa | - followSet := node followSetWithTokens. - - epsilon := followSet anySatisfy: [ :e | e acceptsEpsilon ]. - followSet := followSet reject: [ :e | e acceptsEpsilon ]. - epsilon ifTrue: [ followSet add: PPCEndOfFileNode instance ]. - - anFsa := self unorderedChoiceFromFollowSet: followSet. - - anFsa name: 'nextToken_', (compiler idFor: node). - node nextFsa: anFsa. - ^ scannerGenerator generate: anFsa. -! - -generateScan: node - | anFsa | - anFsa := node asFsa determinize. - anFsa name: (compiler idFor: node). - anFsa retval: (compiler idFor: node). - - ^ scannerGenerator generate: anFsa. + codeGen := PPCCodeGen new. ! ! !PPCTokenCodeGenerator methodsFor:'visiting'! visitToken: tokenNode - | id startVar endVar numberId | - self startMethodForNode: tokenNode. - - "Tokens cannot be inlined, - - their result is true/false - - the return value is always stored in currentTokenValue - - the current token type is always stored in currentTokenType - " + | scanId id | self assert: tokenNode isMarkedForInline not. - - startVar := compiler allocateTemporaryVariableNamed: 'start'. - endVar := compiler allocateTemporaryVariableNamed: 'end'. - - id := compiler idFor: tokenNode. - numberId := compiler numberIdFor: id. - - compiler add: 'currentTokenType isNil ifFalse: [ ^ currentTokenType == ', id storeString, '].'. + + self startMethodForNode: tokenNode. -" compiler codeComment: 'number for: ', id storeString, ' is: ', numberId storeString. - compiler codeIf: 'scanner match: ', numberId storeString then: [ - compiler codeAssign: '(scanner resultPosition: ', numberId storeString, ').' to: endVar. - self createTokenInsance: tokenNode - id: id storeString - start: '(context position + 1)' - end: endVar. - - compiler code: 'context position: ', endVar, '.'. - - self consumeWhitespace: tokenNode. - compiler codeReturn: 'true'. - ]. - compiler codeIf: 'scanner backtracked not' then: [ - compiler codeReturn: 'false'. - ]. - compiler codeComment: 'No match, no fail, scanner does not know about this...'. -" - compiler profileTokenRead: id. + id := codeGen idFor: tokenNode. + scanId := codeGen idFor: tokenNode fsa. -" self scan: tokenNode start: startVar end: endVar." - " compiler add: 'self assert: scanner isSingleMatch.'." -" compiler codeIf: 'scanner match ' then: [" - + codeGen code: 'match isNil ifFalse: [ ^ match == ', id storeString, '].'. + codeGen profileTokenRead: id. + tokenNode child hasName ifFalse: [ tokenNode child name: tokenNode name ]. - compiler codeAssign: 'context position + 1.' to: startVar. - compiler codeIf: [ compiler code: ((self generateScan: tokenNode child) callOn: 'scanner') ] then: [ - compiler add: 'context position: scanner resultPosition.'. - compiler codeAssign: 'context position.' to: endVar. + codeGen codeIf: 'self ', scanId then: [ self consumeWhitespace: tokenNode. - self createTokenInsance: tokenNode id: id storeString start: startVar end: endVar. - compiler codeReturn: 'true'. + codeGen codeReturn: 'true'. ] else: [ - compiler code: 'scanner backtrackDistinct.'. - compiler code: 'context position: ', startVar, ' - 1.'. - compiler codeReturn: 'false'. + codeGen codeReturn: 'false'. ]. ^ self stopMethodForNode: tokenNode ! visitTokenConsumeNode: node - | id nextScan | + | id nextId | self startMethodForNode: node. - id := (compiler idFor: node child). + id := (codeGen idFor: node child). + nextId := (codeGen idFor: node nextFsa). + + "this will inline scanner consumeXY in the parser" + node markForInline. + - compiler add: 'self ', id asString, ' ifTrue: ['. - compiler indent. + codeGen codeIf: 'self ', id asString then: [ + codeGen codeAssign: [ + self createTokenInstance: node child + id: id asString + start: 'position + 1' + end: 'matchPosition'. + ] to: self retvalVar. - nextScan := self generateNextScan: node. - + + codeGen codeAssign: 'context position' to: 'position'; codeDot. + codeGen codeAssign: 'position' to: 'matchPosition'; codeDot. + codeGen codeAssign: 'nil' to: 'match'; codeDot. + node nextFsa hasDistinctRetvals ifTrue: [ - compiler codeAssign: 'currentTokenValue.' to: self retvalVar. - - compiler add: (nextScan callOn: 'scanner'), '.'. - compiler codeIf: 'scanner match' then: [ - compiler add: 'context position: scanner resultPosition.'. - self createTokenInsance: node child - id: 'scanner result' - start: 'scanner position + 1' - end: 'scanner resultPosition'. + codeGen codeIf: [ codeGen codeOnLine: ('self ', nextId) ] then: [ self consumeWhitespace: node child. - compiler codeReturn. ] else: [ - compiler codeComment: 'Looks like there is an error on its way...'. - compiler code: 'context position: scanner position.'. - compiler codeAssign: 'nil.' to: 'currentTokenType'. - compiler codeReturn. + self flag: 'imo should do something here and not wait...'. + codeGen codeComment: 'Looks like there is an error on its way...'. ] - - ] ifFalse: [ - compiler codeAssign: 'nil.' to: 'currentTokenType'. - compiler codeReturn: 'currentTokenValue'. ]. - compiler dedent. - + codeGen codeReturn. + "Token not found" - compiler add: '] ifFalse: ['. - compiler indent. - compiler codeError: id asString, ' expected'. - compiler dedent. - compiler add: '].'. + ] else: [ +" codeGen code: 'PPCScannerError new signalWith: ''', id asString, ' expected'''." + codeGen codeReturn: 'nil.'. + ]. ^ self stopMethodForNode: node ! @@ -274,34 +162,47 @@ ^ self visitToken: node ! -visitTrimmingTokenCharacterNode: node - | id | - self startMethodForNode:node. +visitTokenizingParserNode: node + "produces token_XY methods" + self visit: node tokens. - "Tokens cannot be inlined, - - their result is true/false - - the return value is always stored in currentTokenValue - - the current token type is always stored in currentTokenType + "TODO JK: hack alert, I don't like WS handling, think of something smarter, + perhaps allow for WS unique per token... " + self visitWhitespace: node whitespace. + + "produces tokenConsume_XY methods" + ^ self visit: node parser +! + +visitTrimmingTokenCharacterNode: node + | id | + self halt. self assert: node isMarkedForInline not. + + self startMethodForNode:node. - id := compiler idFor: node. + id := codeGen idFor: node. - compiler add: 'currentTokenType isNil ifFalse: [ ^ currentTokenType == ', id storeString, '].'. - compiler profileTokenRead: id. + codeGen add: 'match isNil ifFalse: [ ^ match == ', id storeString, '].'. + codeGen profileTokenRead: id. - compiler add: '(context peek == ', node child character storeString, ') ifFalse: [ ^ false ].'. - compiler add: 'context next.'. + codeGen add: '(context uncheckedPeek == ', node child character storeString, ') ifFalse: [ ^ false ].'. + codeGen add: 'context next.'. - self createTokenInsance: node id: id storeString start: 'context position' end: 'context position'. + self createTokenInstance: node id: id storeString start: 'context position' end: 'context position'. self consumeWhitespace: node. - compiler codeReturn: 'true'. + codeGen codeReturn: 'true'. ^ self stopMethodForNode: node ! visitTrimmingTokenNode: node ^ self visitToken: node +! + +visitWhitespace: whitespaceNode + self assert: whitespaceNode name = 'consumeWhitespace'. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenConsumeNode.st --- a/compiler/PPCTokenConsumeNode.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenConsumeNode.st Mon Aug 24 15:34:14 2015 +0100 @@ -25,12 +25,15 @@ nextFsa: aPEGFsa self propertyAt: #nextFsa put: aPEGFsa +! + +suffix + ^ nil ! ! !PPCTokenConsumeNode methodsFor:'as yet unclassified'! markForInline - self error: 'current infrastructure does not allow for this!!'. ^ super markForInline ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenDetector.st --- a/compiler/PPCTokenDetector.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenDetector.st Mon Aug 24 15:34:14 2015 +0100 @@ -55,18 +55,6 @@ yourself ]. - (node child isKindOf: PPCTokenConsumeNode) ifTrue: [ - self change. - self halt: 'JK: this can happen???'. - ^ PPCTrimmingTokenNode new - name: node name; - child: node child; - tokenClass: node child child tokenClass; - whitespace: node trimmer; - yourself - ]. - - ^ node ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenWhitespaceNode.st --- a/compiler/PPCTokenWhitespaceNode.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenWhitespaceNode.st Mon Aug 24 15:34:14 2015 +0100 @@ -12,6 +12,6 @@ !PPCTokenWhitespaceNode methodsFor:'as yet unclassified'! accept: visitor - visitor visitTokenWhitespaceNode: self. + ^ visitor visitTokenWhitespaceNode: self. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingCodeGen.st --- a/compiler/PPCTokenizingCodeGen.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenizingCodeGen.st Mon Aug 24 15:34:14 2015 +0100 @@ -11,49 +11,28 @@ !PPCTokenizingCodeGen methodsFor:'code generation'! -codeClearError - errorStrategy codeClearError -! - -codeError - errorStrategy codeError -! - -codeError: message - errorStrategy codeError: message. -! - -smartRemember: parser to: variableName - rememberStrategy smartRemember: parser to: variableName +codeScannerRememberTo: variableName + self codeAssign: 'scanner remember' to: variableName. + self codeDot. ! -smartRestore: parser from: mementoName - rememberStrategy smartRestore: parser from: mementoName -! ! - -!PPCTokenizingCodeGen methodsFor:'hooks'! - -errorStrategy - ^ errorStrategy ifNil: [ PPCCompilerTokenizingErrorStrategy on: self ] +codeScannerRestoreFrom: variableName + self code: 'scanner restore: ', variableName. + self codeDot. + ! -errorStrategy: whatever - errorStrategy := whatever +remember: node to: variableName + ^ self codeScannerRememberTo: variableName ! -rememberStrategy - ^ rememberStrategy ifNil: [ PPCCompilerTokenizingRememberStrategy on: self ] -! - -rememberStrategy: whatever - rememberStrategy := whatever +restore: node from: mementoName + ^ self codeScannerRestoreFrom: mementoName ! ! !PPCTokenizingCodeGen methodsFor:'initialization'! initialize super initialize. - rememberStrategy := PPCCompilerTokenizingRememberStrategy on: self. - errorStrategy := PPCCompilerTokenizingErrorStrategy on: self. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingCodeGenerator.st --- a/compiler/PPCTokenizingCodeGenerator.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenizingCodeGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,139 +3,81 @@ "{ NameSpace: Smalltalk }" PPCCodeGenerator subclass:#PPCTokenizingCodeGenerator - instanceVariableNames:'tokenGenerator' + instanceVariableNames:'' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Visitors' + category:'PetitCompiler-Visitors-CodeGenerators' ! !PPCTokenizingCodeGenerator methodsFor:'accessing'! guards + self flag: 'Hack alert, the codeGenerator needs refactoring so that we can remove this method'. + self halt: 'deprecated'. "When tokenizing, do not use guards" ^ false -! - -tokenGenerator - self error: 'deprecated'. - - tokenGenerator isNil ifTrue: [ - tokenGenerator := (PPCTokenCodeGenerator on: compiler) - arguments: arguments; - yourself. - ]. - ^ tokenGenerator -! - -tokenGenerator: whatever - tokenGenerator := whatever -! - -tokenGuards - "When tokenizing, do not use guards" - ^ arguments guards ! ! -!PPCTokenizingCodeGenerator methodsFor:'guards'! - -addGuard: node ifTrue: trueBlock ifFalse: falseBlock - | guard id | - guard := PPCTokenGuard on: node. - (self guards not or: guard makesSense not) ifTrue: [ ^ false ]. - - id := compiler idFor: guard firstToken. - - compiler add: 'self ', id asString. +!PPCTokenizingCodeGenerator methodsFor:'initialization'! - trueBlock isNil ifFalse: [ - compiler addOnLine: ' ifTrue: ['. - compiler indent. - trueBlock value. - compiler dedent. - falseBlock isNil ifTrue: [ compiler addOnLine: '].' ] - ifFalse: [ compiler add: ']'. ] - ]. - falseBlock isNil ifFalse: [ - compiler addOnLine: ' ifFalse: ['. - compiler indent. - falseBlock value. - compiler dedent. - compiler addOnLine: '].'. - ]. - ^ true -! ! - -!PPCTokenizingCodeGenerator methodsFor:'scanner'! - -compileScanner - compiler addConstant: self tokenGenerator compileScanner as: #scannerClass. +initialize + super initialize. + + codeGen := PPCTokenizingCodeGen new ! ! !PPCTokenizingCodeGenerator methodsFor:'visiting'! visitAndNode: node - | mementoVar currentTokenVar | + | mementoVar | - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - currentTokenVar := compiler allocateTemporaryVariableNamed: 'currentToken'. + mementoVar := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen remember: node child to: mementoVar. - compiler smartRemember: node child to: mementoVar. - compiler codeAssign: '{ currentTokenValue . currentTokenType }.' to: currentTokenVar. - - compiler - codeAssignParsedValueOf:[ self visit:node child ] + codeGen + codeEvaluateAndAssign:[ self visit:node child ] to:self retvalVar. - compiler smartRestore: node child from: mementoVar. - compiler codeAssign: '(', currentTokenVar, ' at: 1).' to: 'currentTokenValue'. - compiler codeAssign: '(', currentTokenVar, ' at: 2).' to: 'currentTokenType'. - - compiler codeReturn. + codeGen restore: node child from: mementoVar. + codeGen codeReturn. ! visitChoiceNode: node " true ifTrue: [ ^ super visitChoiceNode: node ]." - | possibleError | + | possibleError memento | possibleError := true. + memento := codeGen allocateTemporaryVariableNamed: 'memento'. + codeGen codeScannerRememberTo: memento. + node children do: [ :child | | tokenMethodName | - "TODO: JK: fix this in a proper way. Commented for now to make LRPParser cimpilable - with tokenizing" child acceptsEpsilon "false" ifTrue: [ possibleError := false. - compiler codeIf: 'true' then: [ - compiler codeAssignParsedValueOf:[ self visit:child ] to:self retvalVar. - compiler codeReturn - ]. + codeGen codeEvaluateAndAssign:[ self visit:child ] to:self retvalVar. + codeGen codeReturn. + "makes no sense to compile more..." + ^ self ] ifFalse: [ child firstSetWithTokens do: [ :first | "For each child, for each first compile this:" - tokenMethodName := compiler idFor: first. - compiler add: '(self ', tokenMethodName asString, ')'. - compiler addOnLine: ' ifTrue: ['. - compiler indent. - compiler codeAssignParsedValueOf:[ self visit:child ] to:self retvalVar. - compiler add: 'error ifFalse: ['. - compiler indent. - compiler codeReturn: self retvalVar. - compiler dedent. - compiler add: '] ifTrue: ['. - compiler indent. - compiler codeClearError. - compiler codeAssign: 'nil.' to: 'currentTokenType'. - compiler add: 'context position: currentTokenValue start - 1.'. - compiler dedent. - compiler add: ']'. - compiler dedent. - compiler add: '].' + tokenMethodName := codeGen idFor: first. + codeGen codeIf: '(scanner ', tokenMethodName asString, ')' then: [ + codeGen codeEvaluateAndAssignParsedValueOf:[ self visit:child ] to:self retvalVar. + codeGen codeIf: 'error' then: [ + codeGen codeClearError. + codeGen codeScannerRestoreFrom: memento. + ] else: [ + codeGen codeReturn: self retvalVar. + ] + ] ] ] ]. possibleError ifTrue: [ - compiler codeError: 'no choice found'. + codeGen codeError: 'no choice found'. ] "Modified: / 31-07-2015 / 08:07:59 / Jan Vrany " @@ -145,6 +87,7 @@ | dictionary isInlined | dictionary := IdentityDictionary new. +" codeGen currentMethod methodName asSymbol = #'ch' ifTrue: [ self halt ]." isInlined := node isMarkedForInline. node children do: [ :child | @@ -158,46 +101,31 @@ node children do: [ :child | | tokenMethodName | - tokenMethodName := compiler idFor: (dictionary at: child). - compiler add: '(self ', tokenMethodName asString, ')'. - compiler addOnLine: ' ifTrue: ['. - compiler indent. - compiler codeReturnParsedValueOf:[ self visit:child ]. - compiler dedent. + tokenMethodName := codeGen idFor: (dictionary at: child). + codeGen code: '(scanner ', tokenMethodName asString, ')'. + codeGen codeOnLine: ' ifTrue: ['. + codeGen indent. + codeGen codeReturnParsedValueOf:[ self visit:child ]. + codeGen dedent. isInlined ifTrue:[ - compiler add: '] ifFalse: [' + codeGen code: '] ifFalse: [' ] ifFalse:[ - compiler add: '].'. + codeGen code: '].'. ] ]. - compiler codeError: 'no choice found'. + codeGen codeError: 'no choice found'. isInlined ifTrue:[ - node children size timesRepeat: [ compiler addOnLine: ']' ]. - compiler addOnLine: '.'. + node children size timesRepeat: [ codeGen addOnLine: ']' ]. + codeGen codeOnLine: '.'. ] "Modified: / 21-05-2015 / 15:31:26 / Jan Vrany " ! -visitTokenChoiceNode: node - | trimmingToken | - self assert: (node children allSatisfy: [ :e | e isMarkedForInline not ]). - - node children do: [ :child | - self tokenGenerator visit: child - ] - -" - trimmingToken := node children detect: [ :e | e isTrimmingTokenNode ] ifNone: [ nil ]. - trimmingToken isNil ifFalse: [ - compiler codeStoreValueOf: [ self visit: trimmingToken whitespace ] intoVariable: #whatever. - ]. - super visitChoiceNode: node." -! - visitTokenConsumeNode: node - "dont do anything here" - ^ node + | id | + id := codeGen idFor: node. + codeGen codeReturn: 'scanner ', id, ' ifNil: [ error := true ].'. ! visitTokenNode: node @@ -205,13 +133,11 @@ ! visitTokenizingParserNode: node - self visit: node whitespace. - - compiler codeHaltIfShiftPressed. - compiler - codeAssignParsedValueOf:[ self visit:node parser ] + codeGen codeHaltIfShiftPressed. + codeGen + codeEvaluateAndAssign:[ self visit:node parser ] to:self retvalVar. - compiler codeReturn. + codeGen codeReturn. ! visitTrimmingTokenCharacterNode: node diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingCompiler.st --- a/compiler/PPCTokenizingCompiler.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,62 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -PPCCompiler subclass:#PPCTokenizingCompiler - instanceVariableNames:'rememberStrategy errorStrategy' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Compiler' -! - -!PPCTokenizingCompiler methodsFor:'code generation'! - -codeClearError - errorStrategy codeClearError -! - -codeError - errorStrategy codeError -! - -codeError: message - errorStrategy codeError: message. -! - -smartRemember: parser to: variableName - rememberStrategy smartRemember: parser to: variableName -! - -smartRestore: parser from: mementoName - rememberStrategy smartRestore: parser from: mementoName -! ! - -!PPCTokenizingCompiler methodsFor:'hooks'! - -compiledParserSuperclass - ^ compiledParserSuperclass ifNil: [ PPTokenizingCompiledParser ] -! - -errorStrategy - ^ errorStrategy ifNil: [ PPCCompilerTokenizingErrorStrategy on: self ] -! - -errorStrategy: whatever - errorStrategy := whatever -! - -rememberStrategy - ^ rememberStrategy ifNil: [ PPCCompilerTokenizingRememberStrategy on: self ] -! - -rememberStrategy: whatever - rememberStrategy := whatever -! ! - -!PPCTokenizingCompiler methodsFor:'initialization'! - -initialize - super initialize. - rememberStrategy := PPCCompilerTokenizingRememberStrategy on: self -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingConfiguration.st --- a/compiler/PPCTokenizingConfiguration.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenizingConfiguration.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PPCConfiguration subclass:#PPCTokenizingConfiguration - instanceVariableNames:'codeGen' + instanceVariableNames:'scannerClazz parserClazz idGen' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Core' @@ -11,14 +11,18 @@ !PPCTokenizingConfiguration methodsFor:'compiling'! -buildClass: compiler +arguments: args + super arguments: args. +! + +buildClass: clazz | builder | builder := PPCClassBuilder new. - builder compiledClassName: arguments parserName. - builder compiledSuperclass: PPTokenizingCompiledParser. - builder methodDictionary: compiler methodDictionary. - builder constants: compiler constants. + builder compiledClassName: clazz name. + builder compiledSuperclass: clazz superclass. + builder methodDictionary: clazz methodDictionary. + builder constants: clazz constants. ^ builder compileClass. ! @@ -37,45 +41,128 @@ self merge. self check. self cacheFirstFollow. - self generateScanner. "Please note that codeGen is shared between these two phases" - self generate. + self buildParserClazz. + self unmarkConsumeTokensForInline. + self createFSAs. + self buildScannerTokens. + self buildScannerScans. + self generateScanner. + self generateParser. ! ! -!PPCTokenizingConfiguration methodsFor:'hooks'! +!PPCTokenizingConfiguration methodsFor:'initialization'! -codeCompiler - codeGen isNil ifTrue: [ codeGen := PPCTokenizingCodeGen on: arguments ]. - ^ codeGen +fillInClazzes + parserClazz name: arguments parserName. + parserClazz superclass: PPTokenizingCompiledParser. + + scannerClazz name: arguments scannerName. + scannerClazz superclass: arguments scannerSuperclass. + ! -codeCompilerOn: args - ^ PPCTokenizingCompiler on: args -! - -codeGeneratorVisitorOn: compiler - ^ PPCTokenizingCodeGenerator on: compiler +initialize + super initialize. + + parserClazz := PPCClass new. + scannerClazz := PPCClass new. + + idGen := PPCIdGenerator new. + + "The parser and scanner share the same id generator in order + to use same names for tokens. + " + parserClazz idGen: idGen. + scannerClazz idGen: idGen. ! ! !PPCTokenizingConfiguration methodsFor:'phases'! -createLL1Choices - ir := PPCLL1Visitor new +buildParserClazz + | rootMethod | + rootMethod := PPCTokenizingCodeGenerator new + clazz: parserClazz; arguments: arguments; visit: ir. - self remember: #LL1 + + parserClazz propertyAt: #rootMethod put: rootMethod ! -generateScanner - | generator scanner | +buildScannerScans + | fsas generator | + + "TODO JK: Perhpas write separate visitor for this?" + fsas := IdentitySet new. + fsas addAll: (ir allNodes select: [ :node | node hasFsa ] thenCollect: [:node | node fsa]). + fsas addAll: (ir allNodes select: [ :node | node hasNextFsa ] thenCollect: [:node | node nextFsa]). + fsas := fsas reject: [ :fsa | fsa hasDistinctRetvals not ]. + + generator := PPCScannerCodeGenerator new + clazz: scannerClazz; + arguments: arguments; + yourself. + + fsas do: [ :fsa | generator generate: fsa ]. +! + +buildScannerTokens + | generator | generator := PPCTokenCodeGenerator new - compiler: self codeCompiler; + clazz: scannerClazz; arguments: arguments; yourself. generator visit: ir. +! + +createFSAs + ir := PPCFSAVisitor new + idGen: idGen; + visit: ir. + + self remember: (self copyTree: ir) as: #withFSAs +! + +createLL1Choices + self flag: 'This phase needs revisit and update'. - scanner := generator compileScanner. - self codeCompiler addConstant: scanner as: #scannerClass. + ir := PPCLL1Visitor new + arguments: arguments; + visit: ir. + + self remember: (self copyTree: ir) as: #LL1 +! + +generateParser + | parserClass rootMethod | + arguments generate ifFalse: [ ^ self ]. + rootMethod := parserClazz propertyAt: #rootMethod. + + parserClazz name: arguments parserName. + parserClazz superclass: arguments parserSuperclass. + + parserClass := self buildClass: parserClazz. + parserClass startSymbol: rootMethod methodName. + + self remember: parserClass as: #parser. + ir := parserClass new + + +! + +generateScanner + | scanner | + arguments generate ifFalse: [ ^ self ]. + + scannerClazz name: arguments scannerName. + scannerClazz superclass: arguments scannerSuperclass. + + scanner := (self buildClass: scannerClazz). + parserClazz addConstant: scanner as: #scannerClass. + + ir := scanner. + + self remember: scanner as: #scanner ! tokenize @@ -87,6 +174,17 @@ ir := PPCTokenizingVisitor new arguments: arguments; visit: ir. - self remember: #tokenize + + + self remember: (self copyTree: ir) as: #tokenize +! + +unmarkConsumeTokensForInline + "TODO JK: Hack alert, use visitor, or at leas isTokenConsume" + ir allNodesDo: [ :node | + node class == PPCTokenConsumeNode ifTrue: [ + node unmarkForInline + ] + ] ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingParserNode.st --- a/compiler/PPCTokenizingParserNode.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenizingParserNode.st Mon Aug 24 15:34:14 2015 +0100 @@ -28,14 +28,6 @@ children at: 1 put: node ! -tokenizer - ^ children at: 2 -! - -tokenizer: node - ^ children at: 2 put: node -! - tokens ^ children at: 3 ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCTokenizingVisitor.st --- a/compiler/PPCTokenizingVisitor.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCTokenizingVisitor.st Mon Aug 24 15:34:14 2015 +0100 @@ -119,6 +119,7 @@ ^ PPCTokenConsumeNode new child: node; + name: node name; yourself. ! diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCUniversalCodeGen.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCUniversalCodeGen.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,31 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCCodeGen subclass:#PPCUniversalCodeGen + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Compiler-Codegen' +! + +!PPCUniversalCodeGen methodsFor:'code generation'! + +remember: parser to: variableName + parser isContextFree ifTrue: [ + self codeAssign: 'context lwRemember.' + to: variableName. + ] ifFalse: [ + self codeAssign: 'context remember.' + to: variableName. + ] +! + +restore: parser from: mementoName + parser isContextFree ifTrue: [ + self code: 'context lwRestore: ', mementoName, '.'. + ] ifFalse: [ + self code: 'context restore: ', mementoName, '.'. + ] +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCUniversalCodeGenerator.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCUniversalCodeGenerator.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,80 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCCodeGenerator subclass:#PPCUniversalCodeGenerator + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Visitors-CodeGenerators' +! + +!PPCUniversalCodeGenerator methodsFor:'guards'! + +addGuard: node ifTrue: trueBlock ifFalse: falseBlock + | guard id | + (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. + id := codeGen idFor: node. + +" falseBlock isNil ifFalse: [ + compiler add: 'context atEnd'. + compiler addOnLine: ' ifTrue: ['. + compiler indent. + falseBlock value. + compiler dedent. + compiler addOnLine: '].'. + ]." + + guard id: (codeGen idFor: guard defaultName: #guard). + guard compileGuard: codeGen. + + trueBlock isNil ifFalse: [ + codeGen codeOnLine: ' ifTrue: ['. + codeGen indent. + trueBlock value. + codeGen dedent. + falseBlock isNil ifTrue: [ codeGen codeOnLine: '].' ] + ifFalse: [ codeGen code: ']'. ] + ]. + falseBlock isNil ifFalse: [ + codeGen codeOnLine: ' ifFalse: ['. + codeGen indent. + falseBlock value. + codeGen dedent. + codeGen codeOnLine: '].'. + ]. + ^ true +! + +addGuardTrimming: node + | guard firsts id | + (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. + + id := codeGen idFor: node. + firsts := node firstSetWithTokens. + + + (firsts allSatisfy: [ :e | e isTrimmingTokenNode ]) ifTrue: [ + "If we start with trimming, we should invoke the whitespace parser" + self compileTokenWhitespace: firsts anyOne. + ^ true + ]. + ^ false +! ! + +!PPCUniversalCodeGenerator methodsFor:'initialization'! + +initialize + super initialize. + + codeGen := PPCUniversalCodeGen new +! ! + +!PPCUniversalCodeGenerator methodsFor:'visiting'! + +visitStarNode: node + self addGuard: node child ifTrue: nil ifFalse: [ codeGen codeReturn: '#()' ]. + + super visitStarNode: node +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCUniversalConfiguration.st --- a/compiler/PPCUniversalConfiguration.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCUniversalConfiguration.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PPCConfiguration subclass:#PPCUniversalConfiguration - instanceVariableNames:'' + instanceVariableNames:'clazz' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Core' @@ -11,18 +11,26 @@ !PPCUniversalConfiguration methodsFor:'compiling'! -buildClass: compiler +buildClass | builder | + self assert: (clazz isKindOf: PPCClass). + builder := PPCClassBuilder new. builder compiledClassName: arguments parserName. builder compiledSuperclass: PPCompiledParser. - builder methodDictionary: compiler methodDictionary. - builder constants: compiler constants. + builder methodDictionary: clazz methodDictionary. + builder constants: clazz constants. ^ builder compileClass. ! +initialize + super initialize. + + clazz := PPCClass new. +! + invokePhases self toPPCIr. self createTokens. @@ -36,3 +44,28 @@ self generate. ! ! +!PPCUniversalConfiguration methodsFor:'hooks'! + +codeGenerator + ^ PPCUniversalCodeGenerator new + arguments: arguments +! ! + +!PPCUniversalConfiguration methodsFor:'phases'! + +generate + | rootMethod compiledParser | + arguments generate ifFalse: [ ^ self ]. + + rootMethod := self codeGenerator + arguments: arguments; + clazz: clazz; + visit: ir. + + compiledParser := self buildClass. + compiledParser startSymbol: rootMethod methodName. + compiledParser := compiledParser new. + + ir := compiledParser. +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/PPCUniversalResultStrategy.st --- a/compiler/PPCUniversalResultStrategy.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPCUniversalResultStrategy.st Mon Aug 24 15:34:14 2015 +0100 @@ -33,6 +33,7 @@ !PPCUniversalResultStrategy methodsFor:'as yet unclassified'! recordFailure: retval + self error: 'deprecated?'. codeGen codeRecordFailure: (self indexForRetval: retval) ! @@ -41,6 +42,7 @@ ! recordMatch: retval + self error: 'deprecated?'. codeGen codeComment: 'symbol: ', retval storeString. codeGen codeRecordMatch: (self indexForRetval: retval) ! @@ -55,6 +57,11 @@ ^ codeGen code: 'self reset.' ! +returnMatch: retval offset: offset + codeGen codeComment: 'symbol: ', retval storeString. + codeGen codeReturn: [ codeGen codeRecordMatch: (self indexForRetval: retval) offset: offset ] +! + returnResult: state self assert: (state isKindOf: PEGFsaState). codeGen codeReturn. diff -r b5316ef15274 -r f6f68d32de73 compiler/PPTokenizingCompiledParser.st --- a/compiler/PPTokenizingCompiledParser.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/PPTokenizingCompiledParser.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PPCompiledParser subclass:#PPTokenizingCompiledParser - instanceVariableNames:'currentTokenValue currentTokenType scanner' + instanceVariableNames:'scanner' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Parsers' @@ -12,7 +12,14 @@ !PPTokenizingCompiledParser class methodsFor:'as yet unclassified'! acceptsLoggingOfCompilation - ^ self == PPTokenizingCompiledParser + ^ true +" ^ self == PPTokenizingCompiledParser" +! ! + +!PPTokenizingCompiledParser methodsFor:'accessing'! + +scanner + ^ scanner ! ! !PPTokenizingCompiledParser methodsFor:'initialization'! @@ -24,39 +31,10 @@ !PPTokenizingCompiledParser methodsFor:'tokenizing'! -consume: tokenType - self halt: 'deprecated'. - (self perform: tokenType) ifTrue: [ - currentTokenType := nil. - ^ currentTokenValue. - ]. - "self error: 'expected: ', tokenType storeString, ' got ', currentTokenType storeString." - self error. -! - consumeWhitespace "self shouldBeImplemented " ! -currentTokenType - currentTokenType isNil ifTrue: [ self nextToken ]. - ^ currentTokenType -! - -currentTokenTypeIs: tokenType - "if the type is read" - self halt: 'deprecated'. - currentTokenType isNil ifFalse: [ ^ currentTokenType = tokenType ]. - - "if not, try to read the token" - ^ self perform: tokenType. -! - -currentTokenValue - currentTokenType isNil ifTrue: [ self nextToken ]. - ^ currentTokenType -! - nextToken self shouldBeImplemented ! @@ -69,14 +47,18 @@ failure := PPFailure new message: nil; context: context; position: -1. context noteFailure: failure. error := false. - currentTokenType := nil. scanner := (self class classVarNamed: #scannerClass) new. scanner stream: aPPContext. - - self consumeWhitespace. - retval := self perform: startSymbol. -" self consumeWhitespace." + [ + scanner scan_consumeWhitespace. + scanner position: context position. + + retval := self perform: startSymbol. + ] on: PPCScannerError do: [ :e | + "TODO JK: fix reporting of position?" + retval := PPFailure message: e description at: 0. + ]. (retval isPetitFailure) ifTrue: [ aPPContext noteFailure: failure ]. error ifTrue: [ aPPContext noteFailure: failure. retval := failure ]. diff -r b5316ef15274 -r f6f68d32de73 compiler/abbrev.stc --- a/compiler/abbrev.stc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/abbrev.stc Mon Aug 24 15:34:14 2015 +0100 @@ -14,20 +14,17 @@ PPCASTUtilities PPCASTUtilities stx:goodies/petitparser/compiler 'PetitCompiler-Support' 0 PPCArguments PPCArguments stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCBridge PPCBridge stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCClass PPCClass stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCClassBuilder PPCClassBuilder stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCCodeBlock PPCCodeBlock stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCCodeGen PPCCodeGen stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCCompilationError PPCCompilationError stx:goodies/petitparser/compiler 'PetitCompiler-Exceptions' 1 PPCCompilationWarning PPCCompilationWarning stx:goodies/petitparser/compiler 'PetitCompiler-Exceptions' 1 PPCCompiledMethod PPCCompiledMethod stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 -PPCCompiler PPCCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 -PPCCompilerTokenErrorStrategy PPCCompilerTokenErrorStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen-Straregies' 0 -PPCCompilerTokenRememberStrategy PPCCompilerTokenRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen-Straregies' 0 -PPCCompilerTokenizingErrorStrategy PPCCompilerTokenizingErrorStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen-Straregies' 0 -PPCCompilerTokenizingRememberStrategy PPCCompilerTokenizingRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen-Straregies' 0 PPCConfiguration PPCConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCContext PPCContext stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 PPCContextMemento PPCContextMemento stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 +PPCDistinctScanner PPCDistinctScanner stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCGuard PPCGuard stx:goodies/petitparser/compiler 'PetitCompiler-Guards' 0 PPCIdGenerator PPCIdGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCMethod PPCMethod stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 @@ -36,6 +33,7 @@ PPCPluggableConfiguration PPCPluggableConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCScanner PPCScanner stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCScannerCodeGenerator PPCScannerCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 +PPCScannerError PPCScannerError stx:goodies/petitparser/compiler 'PetitCompiler-Exceptions' 1 PPCScannerResultStrategy PPCScannerResultStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCTokenGuard PPCTokenGuard stx:goodies/petitparser/compiler 'PetitCompiler-Guards' 0 PPCompiledParser PPCompiledParser stx:goodies/petitparser/compiler 'PetitCompiler-Parsers' 4 @@ -47,6 +45,7 @@ PEGFsaDeterminizator PEGFsaDeterminizator stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PEGFsaEpsilonTransition PEGFsaEpsilonTransition stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PEGFsaGenerator PEGFsaGenerator stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 +PEGFsaParserState PEGFsaParserState stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PEGFsaPredicateTransition PEGFsaPredicateTransition stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PEGFsaSequenceDeterminizator PEGFsaSequenceDeterminizator stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PEGFsaUncopiableState PEGFsaUncopiableState stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 @@ -54,27 +53,30 @@ PPCAbstractPredicateNode PPCAbstractPredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAnyNode PPCAnyNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCCharacterNode PPCCharacterNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 -PPCCodeGenerator PPCCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 +PPCCodeGenerator PPCCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors-CodeGenerators' 0 PPCDelegateNode PPCDelegateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCDistinctResultStrategy PPCDistinctResultStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCEndOfFileNode PPCEndOfFileNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 1 PPCFSACodeGen PPCFSACodeGen stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 +PPCFSAVisitor PPCFSAVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCInlinedMethod PPCInlinedMethod stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCInliningVisitor PPCInliningVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCListNode PPCListNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNilNode PPCNilNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 +PPCNoResultStrategy PPCNoResultStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCPluggableNode PPCPluggableNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCProfilingContext PPCProfilingContext stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 PPCRewritingVisitor PPCRewritingVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 -PPCTokenCodeGenerator PPCTokenCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 +PPCTokenCodeGenerator PPCTokenCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors-CodeGenerators' 0 PPCTokenizingCodeGen PPCTokenizingCodeGen stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 -PPCTokenizingCompiler PPCTokenizingCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 PPCTokenizingConfiguration PPCTokenizingConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCUniversalCodeGen PPCUniversalCodeGen stx:goodies/petitparser/compiler 'PetitCompiler-Compiler-Codegen' 0 PPCUniversalConfiguration PPCUniversalConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCUniversalResultStrategy PPCUniversalResultStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Scanner' 0 PPCUnknownNode PPCUnknownNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPTokenizingCompiledParser PPTokenizingCompiledParser stx:goodies/petitparser/compiler 'PetitCompiler-Parsers' 4 PEGFsaEOFTransition PEGFsaEOFTransition stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 +PEGFsaParserTransition PEGFsaParserTransition stx:goodies/petitparser/compiler 'PetitCompiler-FSA' 0 PPCAbstractActionNode PPCAbstractActionNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAndNode PPCAndNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCCharSetPredicateNode PPCCharSetPredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 @@ -107,10 +109,11 @@ PPCTokenNode PPCTokenNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTokenVisitor PPCTokenVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCTokenWhitespaceNode PPCTokenWhitespaceNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 -PPCTokenizingCodeGenerator PPCTokenizingCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 +PPCTokenizingCodeGenerator PPCTokenizingCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors-CodeGenerators' 0 PPCTokenizingParserNode PPCTokenizingParserNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTokenizingVisitor PPCTokenizingVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCTrimmingTokenNode PPCTrimmingTokenNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 +PPCUniversalCodeGenerator PPCUniversalCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors-CodeGenerators' 0 PPCActionNode PPCActionNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCDeterministicChoiceNode PPCDeterministicChoiceNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCRecognizingSequenceNode PPCRecognizingSequenceNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 diff -r b5316ef15274 -r f6f68d32de73 compiler/bc.mak --- a/compiler/bc.mak Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/bc.mak Mon Aug 24 15:34:14 2015 +0100 @@ -35,7 +35,7 @@ -LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\analyzer -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser -I$(INCLUDE_TOP)\stx\libbasic -I$(INCLUDE_TOP)\stx\libbasic2 -I$(INCLUDE_TOP)\stx\libview +LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\analyzer -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser -I$(INCLUDE_TOP)\stx\libbasic -I$(INCLUDE_TOP)\stx\libbasic2 LOCALDEFINES= STCLOCALOPT=-package=$(PACKAGE) -I. $(LOCALINCLUDES) -headerDir=. $(STCLOCALOPTIMIZATIONS) $(STCWARNINGS) $(LOCALDEFINES) -varPrefix=$(LIBNAME) @@ -54,13 +54,8 @@ pushd ..\..\..\libbasic & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " pushd ..\..\refactoryBrowser\parser & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " pushd ..\..\..\libbasic2 & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " - pushd ..\..\..\libview & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " pushd .. & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " - pushd ..\..\..\libview2 & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " pushd ..\parsers\smalltalk & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " - pushd ..\..\sunit & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " - pushd ..\tests & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " - pushd ..\parsers\java & $(MAKE_BAT) "CFLAGS_LOCAL=$(GLOBALDEFINES) " @@ -90,20 +85,17 @@ $(OUTDIR)PPCASTUtilities.$(O) PPCASTUtilities.$(H): PPCASTUtilities.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCArguments.$(O) PPCArguments.$(H): PPCArguments.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCBridge.$(O) PPCBridge.$(H): PPCBridge.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCClass.$(O) PPCClass.$(H): PPCClass.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCClassBuilder.$(O) PPCClassBuilder.$(H): PPCClassBuilder.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeBlock.$(O) PPCCodeBlock.$(H): PPCCodeBlock.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGen.$(O) PPCCodeGen.$(H): PPCCodeGen.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilationError.$(O) PPCCompilationError.$(H): PPCCompilationError.st $(INCLUDE_TOP)\stx\libbasic\Error.$(H) $(INCLUDE_TOP)\stx\libbasic\Exception.$(H) $(INCLUDE_TOP)\stx\libbasic\GenericException.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilationWarning.$(O) PPCCompilationWarning.$(H): PPCCompilationWarning.st $(INCLUDE_TOP)\stx\libbasic\GenericException.$(H) $(INCLUDE_TOP)\stx\libbasic\Notification.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\UserNotification.$(H) $(INCLUDE_TOP)\stx\libbasic\Warning.$(H) $(STCHDR) $(OUTDIR)PPCCompiledMethod.$(O) PPCCompiledMethod.$(H): PPCCompiledMethod.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompiler.$(O) PPCCompiler.$(H): PPCCompiler.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenErrorStrategy.$(O) PPCCompilerTokenErrorStrategy.$(H): PPCCompilerTokenErrorStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenRememberStrategy.$(O) PPCCompilerTokenRememberStrategy.$(H): PPCCompilerTokenRememberStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenizingErrorStrategy.$(O) PPCCompilerTokenizingErrorStrategy.$(H): PPCCompilerTokenizingErrorStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTokenizingRememberStrategy.$(O) PPCCompilerTokenizingRememberStrategy.$(H): PPCCompilerTokenizingRememberStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCConfiguration.$(O) PPCConfiguration.$(H): PPCConfiguration.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCContext.$(O) PPCContext.$(H): PPCContext.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(STCHDR) $(OUTDIR)PPCContextMemento.$(O) PPCContextMemento.$(H): PPCContextMemento.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCDistinctScanner.$(O) PPCDistinctScanner.$(H): PPCDistinctScanner.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCGuard.$(O) PPCGuard.$(H): PPCGuard.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCIdGenerator.$(O) PPCIdGenerator.$(H): PPCIdGenerator.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCMethod.$(O) PPCMethod.$(H): PPCMethod.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -112,6 +104,7 @@ $(OUTDIR)PPCPluggableConfiguration.$(O) PPCPluggableConfiguration.$(H): PPCPluggableConfiguration.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCScanner.$(O) PPCScanner.$(H): PPCScanner.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCScannerCodeGenerator.$(O) PPCScannerCodeGenerator.$(H): PPCScannerCodeGenerator.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCScannerError.$(O) PPCScannerError.$(H): PPCScannerError.st $(INCLUDE_TOP)\stx\libbasic\Error.$(H) $(INCLUDE_TOP)\stx\libbasic\Exception.$(H) $(INCLUDE_TOP)\stx\libbasic\GenericException.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCScannerResultStrategy.$(O) PPCScannerResultStrategy.$(H): PPCScannerResultStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenGuard.$(O) PPCTokenGuard.$(H): PPCTokenGuard.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCompiledParser.$(O) PPCompiledParser.$(H): PPCompiledParser.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -123,6 +116,7 @@ $(OUTDIR)PEGFsaDeterminizator.$(O) PEGFsaDeterminizator.$(H): PEGFsaDeterminizator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaAbstractDeterminizator.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaEpsilonTransition.$(O) PEGFsaEpsilonTransition.$(H): PEGFsaEpsilonTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaGenerator.$(O) PEGFsaGenerator.$(H): PEGFsaGenerator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PEGFsaParserState.$(O) PEGFsaParserState.$(H): PEGFsaParserState.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaState.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaPredicateTransition.$(O) PEGFsaPredicateTransition.$(H): PEGFsaPredicateTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaSequenceDeterminizator.$(O) PEGFsaSequenceDeterminizator.$(H): PEGFsaSequenceDeterminizator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaAbstractDeterminizator.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaUncopiableState.$(O) PEGFsaUncopiableState.$(H): PEGFsaUncopiableState.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaState.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -135,22 +129,25 @@ $(OUTDIR)PPCDistinctResultStrategy.$(O) PPCDistinctResultStrategy.$(H): PPCDistinctResultStrategy.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfFileNode.$(O) PPCEndOfFileNode.$(H): PPCEndOfFileNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCFSACodeGen.$(O) PPCFSACodeGen.$(H): PPCFSACodeGen.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCodeGen.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCFSAVisitor.$(O) PPCFSAVisitor.$(H): PPCFSAVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInlinedMethod.$(O) PPCInlinedMethod.$(H): PPCInlinedMethod.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCMethod.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitor.$(O) PPCInliningVisitor.$(H): PPCInliningVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCListNode.$(O) PPCListNode.$(H): PPCListNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNilNode.$(O) PPCNilNode.$(H): PPCNilNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCNoResultStrategy.$(O) PPCNoResultStrategy.$(H): PPCNoResultStrategy.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableNode.$(O) PPCPluggableNode.$(H): PPCPluggableNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCProfilingContext.$(O) PPCProfilingContext.$(H): PPCProfilingContext.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCContext.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(STCHDR) $(OUTDIR)PPCRewritingVisitor.$(O) PPCRewritingVisitor.$(H): PPCRewritingVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenCodeGenerator.$(O) PPCTokenCodeGenerator.$(H): PPCTokenCodeGenerator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCodeGen.$(O) PPCTokenizingCodeGen.$(H): PPCTokenizingCodeGen.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCodeGen.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCTokenizingCompiler.$(O) PPCTokenizingCompiler.$(H): PPCTokenizingCompiler.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCompiler.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingConfiguration.$(O) PPCTokenizingConfiguration.$(H): PPCTokenizingConfiguration.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCConfiguration.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalCodeGen.$(O) PPCUniversalCodeGen.$(H): PPCUniversalCodeGen.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCodeGen.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalConfiguration.$(O) PPCUniversalConfiguration.$(H): PPCUniversalConfiguration.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCConfiguration.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalResultStrategy.$(O) PPCUniversalResultStrategy.$(H): PPCUniversalResultStrategy.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCScannerResultStrategy.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUnknownNode.$(O) PPCUnknownNode.$(H): PPCUnknownNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPTokenizingCompiledParser.$(O) PPTokenizingCompiledParser.$(H): PPTokenizingCompiledParser.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCompiledParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PEGFsaEOFTransition.$(O) PEGFsaEOFTransition.$(H): PEGFsaEOFTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaPredicateTransition.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PEGFsaParserTransition.$(O) PEGFsaParserTransition.$(H): PEGFsaParserTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaPredicateTransition.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PEGFsaTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractActionNode.$(O) PPCAbstractActionNode.$(H): PPCAbstractActionNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAndNode.$(O) PPCAndNode.$(H): PPCAndNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCharSetPredicateNode.$(O) PPCCharSetPredicateNode.$(H): PPCCharSetPredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -187,6 +184,7 @@ $(OUTDIR)PPCTokenizingParserNode.$(O) PPCTokenizingParserNode.$(H): PPCTokenizingParserNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitor.$(O) PPCTokenizingVisitor.$(H): PPCTokenizingVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCRewritingVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTrimmingTokenNode.$(O) PPCTrimmingTokenNode.$(H): PPCTrimmingTokenNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalCodeGenerator.$(O) PPCUniversalCodeGenerator.$(H): PPCUniversalCodeGenerator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCodeGenerator.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCActionNode.$(O) PPCActionNode.$(H): PPCActionNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractActionNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCDeterministicChoiceNode.$(O) PPCDeterministicChoiceNode.$(H): PPCDeterministicChoiceNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCChoiceNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizingSequenceNode.$(O) PPCRecognizingSequenceNode.$(H): PPCRecognizingSequenceNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCSequenceNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -200,7 +198,7 @@ $(OUTDIR)PPCMappedActionNode.$(O) PPCMappedActionNode.$(H): PPCMappedActionNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractActionNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCActionNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarMessagePredicateNode.$(O) PPCTokenStarMessagePredicateNode.$(H): PPCTokenStarMessagePredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarSeparatorNode.$(O) PPCTokenStarSeparatorNode.$(H): PPCTokenStarSeparatorNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCTokenStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPActionParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPAndParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCharSetPredicate.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPChoiceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPContext.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEndOfFileParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEndOfInputParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEpsilonParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFailure.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFlattenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPListParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPNotParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPOptionalParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPluggableParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPToken.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTrimmingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java\PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBLiteralNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBLiteralValueNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBProgramNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBStatementNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBValueNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Character.$(H) $(INCLUDE_TOP)\stx\libbasic\Magnitude.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(INCLUDE_TOP)\stx\libbasic\UndefinedObject.$(H) $(STCHDR) +$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPActionParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPAndParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCharSetPredicate.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPChoiceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPContext.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEndOfFileParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEndOfInputParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEpsilonParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFailure.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFlattenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPListParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPNotParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPOptionalParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPluggableParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPToken.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTrimmingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBLiteralNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBLiteralValueNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBProgramNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBStatementNode.$(H) $(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser\RBValueNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Character.$(H) $(INCLUDE_TOP)\stx\libbasic\Magnitude.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(INCLUDE_TOP)\stx\libbasic\UndefinedObject.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r b5316ef15274 -r f6f68d32de73 compiler/benchmarks/PPCBenchmark.st --- a/compiler/benchmarks/PPCBenchmark.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/benchmarks/PPCBenchmark.st Mon Aug 24 15:34:14 2015 +0100 @@ -250,7 +250,7 @@ !PPCBenchmark methodsFor:'benchmarks - smalltalk'! benchmarkOpalCompiler - | parser time input | + | time | parser := OpalCompiler new. input := sources smalltalkSourcesBig. time := [ input do: [ :source | parser parse: source ]] timeToRun asMilliseconds. @@ -592,8 +592,7 @@ setupSmalltalkGrammarCompiled configuration := PPCConfiguration universal. - configuration arguments parserName: #PPCompiledSmalltalkGrammar. - configuration arguments scannerName: #PPCompiledSmalltalkScanner. + configuration arguments parserName: #PPSmalltalkGrammarC_Unviersal. configuration arguments profile: profile. parser := PPSmalltalkGrammar new compileWithConfiguration: configuration. @@ -610,8 +609,8 @@ setupSmalltalkGrammarTokenized configuration := PPCConfiguration tokenizing. - configuration arguments parserName: #PPTokenizedSmalltalkGrammar. - configuration arguments scannerName: #PPTokenizedSmalltalkScanner. + configuration arguments parserName: #PPSmalltalkGrammarC_Tokenizing. + configuration arguments scannerName: #PPSmalltalkGrammarC_Scanner. configuration arguments profile: profile. parser := PPSmalltalkGrammar new compileWithConfiguration: configuration. diff -r b5316ef15274 -r f6f68d32de73 compiler/extensions.st --- a/compiler/extensions.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/extensions.st Mon Aug 24 15:34:14 2015 +0100 @@ -293,20 +293,6 @@ position := anInteger ! ! -!PPJavaWhitespaceParser methodsFor:'*petitcompiler'! - -= anotherParser - anotherParser == self ifTrue: [ ^ true ]. - anotherParser class = self class ifFalse: [ ^ false ]. - ^ anotherParser name = self name -! ! - -!PPJavaWhitespaceParser methodsFor:'*petitcompiler'! - -hash - ^ self name hash -! ! - !PPLiteralObjectParser methodsFor:'*petitcompiler'! asCompilerNode diff -r b5316ef15274 -r f6f68d32de73 compiler/libInit.cc --- a/compiler/libInit.cc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/libInit.cc Mon Aug 24 15:34:14 2015 +0100 @@ -40,20 +40,17 @@ _PPCASTUtilities_Init(pass,__pRT__,snd); _PPCArguments_Init(pass,__pRT__,snd); _PPCBridge_Init(pass,__pRT__,snd); +_PPCClass_Init(pass,__pRT__,snd); _PPCClassBuilder_Init(pass,__pRT__,snd); _PPCCodeBlock_Init(pass,__pRT__,snd); _PPCCodeGen_Init(pass,__pRT__,snd); _PPCCompilationError_Init(pass,__pRT__,snd); _PPCCompilationWarning_Init(pass,__pRT__,snd); _PPCCompiledMethod_Init(pass,__pRT__,snd); -_PPCCompiler_Init(pass,__pRT__,snd); -_PPCCompilerTokenErrorStrategy_Init(pass,__pRT__,snd); -_PPCCompilerTokenRememberStrategy_Init(pass,__pRT__,snd); -_PPCCompilerTokenizingErrorStrategy_Init(pass,__pRT__,snd); -_PPCCompilerTokenizingRememberStrategy_Init(pass,__pRT__,snd); _PPCConfiguration_Init(pass,__pRT__,snd); _PPCContext_Init(pass,__pRT__,snd); _PPCContextMemento_Init(pass,__pRT__,snd); +_PPCDistinctScanner_Init(pass,__pRT__,snd); _PPCGuard_Init(pass,__pRT__,snd); _PPCIdGenerator_Init(pass,__pRT__,snd); _PPCMethod_Init(pass,__pRT__,snd); @@ -62,6 +59,7 @@ _PPCPluggableConfiguration_Init(pass,__pRT__,snd); _PPCScanner_Init(pass,__pRT__,snd); _PPCScannerCodeGenerator_Init(pass,__pRT__,snd); +_PPCScannerError_Init(pass,__pRT__,snd); _PPCScannerResultStrategy_Init(pass,__pRT__,snd); _PPCTokenGuard_Init(pass,__pRT__,snd); _PPCompiledParser_Init(pass,__pRT__,snd); @@ -73,6 +71,7 @@ _PEGFsaDeterminizator_Init(pass,__pRT__,snd); _PEGFsaEpsilonTransition_Init(pass,__pRT__,snd); _PEGFsaGenerator_Init(pass,__pRT__,snd); +_PEGFsaParserState_Init(pass,__pRT__,snd); _PEGFsaPredicateTransition_Init(pass,__pRT__,snd); _PEGFsaSequenceDeterminizator_Init(pass,__pRT__,snd); _PEGFsaUncopiableState_Init(pass,__pRT__,snd); @@ -85,22 +84,25 @@ _PPCDistinctResultStrategy_Init(pass,__pRT__,snd); _PPCEndOfFileNode_Init(pass,__pRT__,snd); _PPCFSACodeGen_Init(pass,__pRT__,snd); +_PPCFSAVisitor_Init(pass,__pRT__,snd); _PPCInlinedMethod_Init(pass,__pRT__,snd); _PPCInliningVisitor_Init(pass,__pRT__,snd); _PPCListNode_Init(pass,__pRT__,snd); _PPCNilNode_Init(pass,__pRT__,snd); +_PPCNoResultStrategy_Init(pass,__pRT__,snd); _PPCPluggableNode_Init(pass,__pRT__,snd); _PPCProfilingContext_Init(pass,__pRT__,snd); _PPCRewritingVisitor_Init(pass,__pRT__,snd); _PPCTokenCodeGenerator_Init(pass,__pRT__,snd); _PPCTokenizingCodeGen_Init(pass,__pRT__,snd); -_PPCTokenizingCompiler_Init(pass,__pRT__,snd); _PPCTokenizingConfiguration_Init(pass,__pRT__,snd); +_PPCUniversalCodeGen_Init(pass,__pRT__,snd); _PPCUniversalConfiguration_Init(pass,__pRT__,snd); _PPCUniversalResultStrategy_Init(pass,__pRT__,snd); _PPCUnknownNode_Init(pass,__pRT__,snd); _PPTokenizingCompiledParser_Init(pass,__pRT__,snd); _PEGFsaEOFTransition_Init(pass,__pRT__,snd); +_PEGFsaParserTransition_Init(pass,__pRT__,snd); _PPCAbstractActionNode_Init(pass,__pRT__,snd); _PPCAndNode_Init(pass,__pRT__,snd); _PPCCharSetPredicateNode_Init(pass,__pRT__,snd); @@ -137,6 +139,7 @@ _PPCTokenizingParserNode_Init(pass,__pRT__,snd); _PPCTokenizingVisitor_Init(pass,__pRT__,snd); _PPCTrimmingTokenNode_Init(pass,__pRT__,snd); +_PPCUniversalCodeGenerator_Init(pass,__pRT__,snd); _PPCActionNode_Init(pass,__pRT__,snd); _PPCDeterministicChoiceNode_Init(pass,__pRT__,snd); _PPCRecognizingSequenceNode_Init(pass,__pRT__,snd); diff -r b5316ef15274 -r f6f68d32de73 compiler/stx_goodies_petitparser_compiler.st --- a/compiler/stx_goodies_petitparser_compiler.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/stx_goodies_petitparser_compiler.st Mon Aug 24 15:34:14 2015 +0100 @@ -16,15 +16,15 @@ "The last merged version is: " ^ ' - Name: PetitCompiler-JanKurs.20150510144201 - Author: JanKurs - Time: 10-05-2015, 04:42:29.192 PM - UUID: 58a4786b-1182-4904-8b44-a13d3918f244 + Name: PetitCompiler-JanVrany.170 + Author: JanVrany + Time: 24-08-2015, 03:19:51.340 PM + UUID: c20a744f-3b41-4aaa-bb8a-71ce74a2a952 Repository: http://smalltalkhub.com/mc/JanKurs/PetitParser/main ' "Created: / 03-10-2014 / 02:27:21 / Jan Vrany " - "Modified: / 12-05-2015 / 01:19:08 / Jan Vrany " + "Modified: / 24-08-2015 / 15:23:31 / Jan Vrany " ! monticelloName @@ -57,7 +57,6 @@ ^ #( #'stx:goodies/petitparser' "PPActionParser - extended" - #'stx:goodies/petitparser/parsers/java' "PPJavaWhitespaceParser - extended" #'stx:goodies/petitparser/parsers/smalltalk' "PPSmalltalkGrammar - extended" #'stx:goodies/refactoryBrowser/parser' "RBLiteralNode - extended" #'stx:libbasic' "Character - extended" @@ -76,8 +75,8 @@ ^ #( #'stx:goodies/petitparser/analyzer' "PPSentinel - referenced by PPCompiledParser class>>referringParser" + #'stx:goodies/petitparser/parsers/java' "PPJavaToken - referenced by PPParser>>javaToken" #'stx:libbasic2' "IdentityBag - referenced by PEGFsa>>checkTransitionsIdentity" - #'stx:libview' "Color - referenced by PEGFsa>>viewGraphOn:" ) ! @@ -122,20 +121,17 @@ PPCASTUtilities PPCArguments PPCBridge + PPCClass PPCClassBuilder PPCCodeBlock PPCCodeGen PPCCompilationError PPCCompilationWarning PPCCompiledMethod - PPCCompiler - PPCCompilerTokenErrorStrategy - PPCCompilerTokenRememberStrategy - PPCCompilerTokenizingErrorStrategy - PPCCompilerTokenizingRememberStrategy PPCConfiguration PPCContext PPCContextMemento + PPCDistinctScanner PPCGuard PPCIdGenerator PPCMethod @@ -144,6 +140,7 @@ PPCPluggableConfiguration PPCScanner PPCScannerCodeGenerator + PPCScannerError PPCScannerResultStrategy PPCTokenGuard PPCompiledParser @@ -155,6 +152,7 @@ PEGFsaDeterminizator PEGFsaEpsilonTransition PEGFsaGenerator + PEGFsaParserState PEGFsaPredicateTransition PEGFsaSequenceDeterminizator PEGFsaUncopiableState @@ -167,22 +165,25 @@ PPCDistinctResultStrategy PPCEndOfFileNode PPCFSACodeGen + PPCFSAVisitor PPCInlinedMethod PPCInliningVisitor PPCListNode PPCNilNode + PPCNoResultStrategy PPCPluggableNode PPCProfilingContext PPCRewritingVisitor PPCTokenCodeGenerator PPCTokenizingCodeGen - PPCTokenizingCompiler PPCTokenizingConfiguration + PPCUniversalCodeGen PPCUniversalConfiguration PPCUniversalResultStrategy PPCUnknownNode PPTokenizingCompiledParser PEGFsaEOFTransition + PEGFsaParserTransition PPCAbstractActionNode PPCAndNode PPCCharSetPredicateNode @@ -219,6 +220,7 @@ PPCTokenizingParserNode PPCTokenizingVisitor PPCTrimmingTokenNode + PPCUniversalCodeGenerator PPCActionNode PPCDeterministicChoiceNode PPCRecognizingSequenceNode @@ -336,8 +338,6 @@ PPContext atWs PPContext methodInvoked: PPContext setWs - PPJavaWhitespaceParser #'=' - PPJavaWhitespaceParser hash PPParser compileAs: PPParser compileWithConfiguration: PPParser javaToken diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/Make.proto --- a/compiler/tests/Make.proto Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/Make.proto Mon Aug 24 15:34:14 2015 +0100 @@ -34,7 +34,7 @@ # add the path(es) here:, # ********** OPTIONAL: MODIFY the next lines *** # LOCALINCLUDES=-Ifoo -Ibar -LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/tests -I$(INCLUDE_TOP)/stx/goodies/sunit -I$(INCLUDE_TOP)/stx/libbasic +LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/tests -I$(INCLUDE_TOP)/stx/goodies/sunit -I$(INCLUDE_TOP)/stx/libbasic # if you need any additional defines for embedded C code, @@ -142,10 +142,10 @@ $(OUTDIR)PPCASTUtilitiesTests.$(O) PPCASTUtilitiesTests.$(H): PPCASTUtilitiesTests.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCClassBuilderTest.$(O) PPCClassBuilderTest.$(H): PPCClassBuilderTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGeneratorTest.$(O) PPCCodeGeneratorTest.$(H): PPCCodeGeneratorTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTest.$(O) PPCCompilerTest.$(H): PPCCompilerTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCContextMementoTest.$(O) PPCContextMementoTest.$(H): PPCContextMementoTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPContextMementoTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCContextTest.$(O) PPCContextTest.$(H): PPCContextTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPContextTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCopyVisitorTest.$(O) PPCCopyVisitorTest.$(H): PPCCopyVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCDistinctScannerTest.$(O) PPCDistinctScannerTest.$(H): PPCDistinctScannerTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCGuardTest.$(O) PPCGuardTest.$(H): PPCGuardTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCIdGeneratorTest.$(O) PPCIdGeneratorTest.$(H): PPCIdGeneratorTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitorTest.$(O) PPCInliningVisitorTest.$(H): PPCInliningVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -166,6 +166,7 @@ $(OUTDIR)PPCTokenizingCodeGeneratorTest.$(O) PPCTokenizingCodeGeneratorTest.$(H): PPCTokenizingCodeGeneratorTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingTest.$(O) PPCTokenizingTest.$(H): PPCTokenizingTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitorTest.$(O) PPCTokenizingVisitorTest.$(H): PPCTokenizingVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCUnivarsalGuardTest.$(O) PPCUnivarsalGuardTest.$(H): PPCUnivarsalGuardTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalOptimizationTest.$(O) PPCUniversalOptimizationTest.$(H): PPCUniversalOptimizationTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalTest.$(O) PPCUniversalTest.$(H): PPCUniversalTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests.$(O) stx_goodies_petitparser_compiler_tests.$(H): stx_goodies_petitparser_compiler_tests.st $(INCLUDE_TOP)/stx/libbasic/LibraryDefinition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/ProjectDefinition.$(H) $(STCHDR) diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/Make.spec --- a/compiler/tests/Make.spec Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/Make.spec Mon Aug 24 15:34:14 2015 +0100 @@ -66,10 +66,10 @@ PPCASTUtilitiesTests \ PPCClassBuilderTest \ PPCCodeGeneratorTest \ - PPCCompilerTest \ PPCContextMementoTest \ PPCContextTest \ PPCCopyVisitorTest \ + PPCDistinctScannerTest \ PPCGuardTest \ PPCIdGeneratorTest \ PPCInliningVisitorTest \ @@ -90,6 +90,7 @@ PPCTokenizingCodeGeneratorTest \ PPCTokenizingTest \ PPCTokenizingVisitorTest \ + PPCUnivarsalGuardTest \ PPCUniversalOptimizationTest \ PPCUniversalTest \ stx_goodies_petitparser_compiler_tests \ @@ -113,10 +114,10 @@ $(OUTDIR_SLASH)PPCASTUtilitiesTests.$(O) \ $(OUTDIR_SLASH)PPCClassBuilderTest.$(O) \ $(OUTDIR_SLASH)PPCCodeGeneratorTest.$(O) \ - $(OUTDIR_SLASH)PPCCompilerTest.$(O) \ $(OUTDIR_SLASH)PPCContextMementoTest.$(O) \ $(OUTDIR_SLASH)PPCContextTest.$(O) \ $(OUTDIR_SLASH)PPCCopyVisitorTest.$(O) \ + $(OUTDIR_SLASH)PPCDistinctScannerTest.$(O) \ $(OUTDIR_SLASH)PPCGuardTest.$(O) \ $(OUTDIR_SLASH)PPCIdGeneratorTest.$(O) \ $(OUTDIR_SLASH)PPCInliningVisitorTest.$(O) \ @@ -137,6 +138,7 @@ $(OUTDIR_SLASH)PPCTokenizingCodeGeneratorTest.$(O) \ $(OUTDIR_SLASH)PPCTokenizingTest.$(O) \ $(OUTDIR_SLASH)PPCTokenizingVisitorTest.$(O) \ + $(OUTDIR_SLASH)PPCUnivarsalGuardTest.$(O) \ $(OUTDIR_SLASH)PPCUniversalOptimizationTest.$(O) \ $(OUTDIR_SLASH)PPCUniversalTest.$(O) \ $(OUTDIR_SLASH)stx_goodies_petitparser_compiler_tests.$(O) \ diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PEGFsaScannerIntegrationTest.st --- a/compiler/tests/PEGFsaScannerIntegrationTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PEGFsaScannerIntegrationTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -617,7 +617,7 @@ scanner initialize. scanner stream: stream asPetitStream. - scanner nextToken. + scanner scan_nextToken. result := scanner polyResult. @@ -630,7 +630,7 @@ scanner initialize. scanner stream: stream asPetitStream. - scanner nextToken. + scanner scan_nextToken. result := scanner polyResult. @@ -639,10 +639,14 @@ ! generate + | codeGenerator | fsa name: #nextToken. - scanner := ((PPCScannerCodeGenerator new) - generateAndCompile: fsa). + codeGenerator := PPCScannerCodeGenerator new. + codeGenerator arguments scannerSuperclass: PPCScanner. + + scanner := codeGenerator + generateAndCompile: fsa. compiled := true ! @@ -673,7 +677,7 @@ self compile. scanner stream: stream asPetitStream. - scanner nextToken. + scanner scan_nextToken. result := scanner polyResult. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCCodeGeneratorTest.st --- a/compiler/tests/PPCCodeGeneratorTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCCodeGeneratorTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -23,17 +23,10 @@ configuration := PPCPluggableConfiguration on: [ :_self | _self cacheFirstFollow. + _self check. _self generate. ]. configuration arguments: arguments. - - - compiler := PPCCodeGen new. - compiler arguments: arguments. - - visitor := PPCCodeGenerator new. - visitor compiler: compiler. - visitor arguments: arguments. ! tearDown @@ -986,7 +979,22 @@ self assert: parser parse: 'aaa' to: #($a $a $a) end: 3. self assert: parser parse: 'ab' to: #( $a ) end: 1. - self assert: parser parse: 'b' to: #( ) end: 0. + self assert: parser parse: 'b' to: #() end: 0. + + self assert: parser parse: ''. + self assert: result isArray. +! + +testStarNode2 + node := PPCStarNode new + child: ('aa' asParser asCompilerNode); + yourself. + + self compileTree: node. + + self assert: parser parse: 'aaaa' to: #('aa' 'aa') end: 4. + self assert: parser parse: 'aab' to: #( 'aa' ) end: 2. + self assert: parser parse: 'a' to: #() end: 0. ! testSymbolActionNode diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCCompilerTest.st --- a/compiler/tests/PPCCompilerTest.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,123 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPAbstractParserTest subclass:#PPCCompilerTest - instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 - arguments configuration' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-Core' -! - - -!PPCCompilerTest methodsFor:'as yet unclassified'! - -assert: p parse: whatever - ^ result := super assert: p parse: whatever. -! - -context - ^ context := PPCProfilingContext new -! - -setUp - arguments := PPCArguments default - profile: true; - yourself. - - configuration := PPCUniversalConfiguration new - arguments: arguments; - yourself. -! - -tearDown - | parserClass | - - parserClass := (Smalltalk at: #PPGeneratedParser ifAbsent: [nil]). - parserClass notNil ifTrue:[ - parserClass removeFromSystem - ]. -! ! - -!PPCCompilerTest methodsFor:'tests - guard'! - -testChoiceGuard - parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken / $d asParser trimmingToken plus) - compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). - - self assert: parser parse: 'bar'. - self assert: result inputValue = 'bar'. - - self assert: parser parse: ' foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: ' d'. - self assert: result first inputValue = 'd'. - - self assert: parser fail: ''. - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'predicate' ]). - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). - - self assert: parser fail: 'zorg'. - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). -! - -testEmptyChoiceGuard - parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken / $d asParser trimmingToken star) - compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: 'bar'. - self assert: result inputValue = 'bar'. - - self assert: parser parse: ' foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: ' d'. - self assert: result first inputValue = 'd'. - - self assert: parser parse: ''. - - self assert: parser parse: 'zorg' end: 0. -! - -testGuardSmalltlakToken - parser := (#letter asParser, #word asParser star) smalltalkToken compileWithConfiguration: configuration. - - self assert: parser parse: 'bar'. - self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'seq' ]). - - self assert: parser fail: '123'. - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'seq' ]). -! - -testSequenceGuard - parser := ((#any asParser, #any asParser) wrapped, (#any asParser, #any asParser)) compileWithConfiguration: configuration. - - self assert: parser parse: 'fooo' to: #(#($f $o) #($o $o)). - self assert: parser parse: 'fo oo' to: #(#($f $o) #($ $o)) end: 4. - self assert: parser fail: 'fo'. - -! - -testTrimmerGuard - parser := $a asParser trim, $b asParser compileWithConfiguration: configuration. - - self assert: parser parse: 'ab'. - self assert: parser parse: ' ab'. -! ! - -!PPCCompilerTest class methodsFor:'documentation'! - -version_HG - - ^ '$Changeset: $' -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCDistinctScannerTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCDistinctScannerTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,228 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +TestCase subclass:#PPCDistinctScannerTest + instanceVariableNames:'configuration scanner' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Tokenizing' +! + +!PPCDistinctScannerTest methodsFor:'as yet unclassified'! + +aToken + ^ 'a' asParser token + name: #token; + yourself. +! + +barToken + ^ 'bar' asParser token + name: #bar; + yourself. +! + +fooToken + ^ 'foo' asParser token + name: #foo; + yourself. +! + +fooTrimmingToken + ^ 'foo' asParser trimmingToken + name: #foo; + yourself. +! + +idToken + ^ #letter asParser plus token + name: #id; + yourself. +! + +idTrimmingToken + ^ #letter asParser plus trimmingToken + name: #id; + yourself. +! + +overlappingToken + ^ 'a' asParser token / 'a' asParser token +! + +setUp + configuration := PPCConfiguration tokenizing. +! + +testConsumeToken + | parser | + parser := self fooToken, self overlappingToken. + parser compileWithConfiguration: configuration. + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foobaz' asPetitStream. + scanner perform: #'consume_foo'. + + self assert: scanner position = 3. + self assert: scanner resultPosition = 3. + self assert: scanner result isNil. +! + +testConsumeToken2 + | parser | + parser := self fooToken, self barToken. + parser compileWithConfiguration: configuration. + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foobar' asPetitStream. + scanner perform: #'consume_foo'. + + self assert: scanner position = 3. + self assert: scanner resultPosition = 6. + self assert: scanner result = #bar. +! + +testScan + | parser | + parser := self aToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'a' asPetitStream. + scanner perform: #'scan_token'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 1. + self assert: scanner result = #token. +! + +testScan2 + | parser | + parser := self fooToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foo' asPetitStream. + scanner perform: #'scan_foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner result = #foo. +! + +testScan3 + | parser | + parser := self fooToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'bar' asPetitStream. + scanner perform: #'scan_foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 0. + self assert: scanner result isNil. +! + +testScan4 + | parser | + parser := self fooToken, self idToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foothere' asPetitStream. + scanner perform: #'scan_foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner result = #foo. + + scanner perform: #'consume_foo'. + + self assert: scanner position = 3. + self assert: scanner resultPosition = 8. + self assert: scanner result = #id. +! + +testSequence + | parser result | + parser := self fooTrimmingToken, self idTrimmingToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foo there ' asPetitStream. + scanner perform: #'foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner stream position = 4. + self assert: scanner result = #foo. + + result := scanner perform: #'consume_foo'. + + self assert: scanner position = 4. + self assert: scanner resultPosition = 9. + self assert: scanner stream position = 10. + self assert: scanner result = #id. + + self assert: (result isKindOf: PPToken). + self assert: result start = 1. + self assert: result stop = 3. + self assert: result inputValue = 'foo'. +! + +testToken + | parser | + parser := self fooToken, self idTrimmingToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foo there' asPetitStream. + scanner perform: #'foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner stream position = 3. + self assert: scanner result = #foo. +! + +testTrimmingScan + | parser | + parser := self fooTrimmingToken, self idTrimmingToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foo there' asPetitStream. + scanner perform: #'scan_foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner result = #foo. +! + +testTrimmingToken + | parser result | + parser := self fooTrimmingToken, self idTrimmingToken. + parser compileWithConfiguration: configuration. + + scanner := (Smalltalk at: configuration arguments scannerName) new. + + scanner stream: 'foo there' asPetitStream. + result := scanner perform: #'foo'. + + self assert: scanner position = 0. + self assert: scanner resultPosition = 3. + self assert: scanner stream position = 4. + self assert: scanner result = #foo. + + self assert: result. +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCLTokenizingOptimizationTest.st --- a/compiler/tests/PPCLTokenizingOptimizationTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCLTokenizingOptimizationTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -43,7 +43,7 @@ self assert: result type: PPCTokenizingParserNode. self assert: result parser type: PPCTokenConsumeNode. self assert: result parser child type: PPCTrimmingTokenNode. - self assert: result whitespace type: PPCTokenStarSeparatorNode. + self assert: result whitespace type: PPCStarNode. ! testCompileTrimmingToken @@ -53,7 +53,7 @@ self assert: result type: PPCTokenizingParserNode. self assert: result parser type: PPCTokenConsumeNode. self assert: result parser child type: PPCTrimmingTokenNode. - self assert: result whitespace type: PPCTokenStarSeparatorNode. + self assert: result whitespace type: PPCStarNode. self assert: result tokens children size = 1. self assert: (result tokens children anySatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]) diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCMergingVisitorTest.st --- a/compiler/tests/PPCMergingVisitorTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCMergingVisitorTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -12,12 +12,6 @@ !PPCMergingVisitorTest methodsFor:'as yet unclassified'! -javaWsNode - ^ PPCUnknownNode new - parser: PPJavaWhitespaceParser new; - yourself -! - nilNode ^ PPCNilNode new ! @@ -26,18 +20,6 @@ visitor := PPCMergingVisitor new ! -testJavaWS - node1 := self javaWsNode. - node2 := self javaWsNode. - - node := PPCSequenceNode new - children: { node1 . node2 }. - - self assert: (node children first == node children second) not. - node := visitor visit: node. - self assert: (node children first == node children second). -! - testNilNode node1 := self nilNode. node2 := self nilNode. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCMockCompiler.st --- a/compiler/tests/PPCMockCompiler.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCMockCompiler.st Mon Aug 24 15:34:14 2015 +0100 @@ -38,6 +38,10 @@ addConstant: object as: id "TODO" +! + +code: string + lines add: string ! ! !PPCMockCompiler methodsFor:'initialization'! diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCOverlappingTokensTest.st --- a/compiler/tests/PPCOverlappingTokensTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCOverlappingTokensTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -81,11 +81,13 @@ self assert: parser parse: 'foo: bar id:=another'. - self assert: result first size = 1.. + self assert: result first size = 1. self assert: result second inputValue = 'id'. self assert: result third inputValue = ':='. self assert: result last inputValue = 'another'. + true ifTrue: [ ^ self ]. + "skip for simple tokenizer" self assert: context tokenReadCount == 2 description: 'too many token reads?'. ! @@ -132,6 +134,8 @@ self assert: result third = #foo. self assert: result last = #id. + true ifTrue: [ ^ self ]. + "skip for simple tokenizer" self assert: context tokenReadCount == 1 description: 'too many token reads?'. ! @@ -145,6 +149,8 @@ self assert: result third inputValue = 'foo'. self assert: result last inputValue = 'bar'. + true ifTrue: [ ^ self ]. + "skip for simple tokenizer" self assert: context tokenReadCount == 1 description: 'too many token reads?'. ! diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCScannerCodeGeneratorTest.st --- a/compiler/tests/PPCScannerCodeGeneratorTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCScannerCodeGeneratorTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -14,7 +14,7 @@ fail: stream rule: rule scanner initialize. scanner stream: stream asPetitStream. - scanner perform: rule. + scanner perform: ('scan_', rule) asSymbol. result := scanner polyResult. @@ -28,7 +28,7 @@ parse: stream token: token rule: rule position: position scanner initialize. scanner stream: stream asPetitStream. - scanner perform: rule. + scanner perform: ('scan_', rule) asSymbol. result := scanner polyResult. self assert: (result at: token) = position. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCTokenizingCodeGeneratorTest.st --- a/compiler/tests/PPCTokenizingCodeGeneratorTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCTokenizingCodeGeneratorTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -4,7 +4,7 @@ PPAbstractParserTest subclass:#PPCTokenizingCodeGeneratorTest instanceVariableNames:'visitor node result compiler parser context arguments tokenizer - whitespace' + whitespace configuration' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Tests-Visitors' @@ -25,30 +25,11 @@ ]. ! -compileTokenizer: aNode - tokenizer := visitor visit: aNode -! - compileTree: root - - | configuration | - - configuration := PPCPluggableConfiguration on: [ :_self | - _self cacheFirstFollow. - _self generateScanner. - _self generate. - - ]. - configuration arguments: arguments. - configuration base: PPCConfiguration tokenizing. parser := configuration compile: root. ! -compileWs: aNode - whitespace := visitor visit: aNode -! - context ^ context := PPCProfilingContext new ! @@ -60,12 +41,18 @@ self cleanClass. - compiler := PPCTokenizingCodeGen new. - compiler arguments: arguments. - - visitor := PPCTokenizingCodeGenerator new. - visitor compiler: compiler. - visitor arguments: arguments. + configuration := PPCPluggableConfiguration on: [ :_self | + _self cacheFirstFollow. + _self buildParserClazz. + _self unmarkConsumeTokensForInline. + _self createFSAs. + _self buildScannerTokens. + _self buildScannerScans. + _self generateScanner. + _self generateParser. + ] base: PPCConfiguration tokenizing. + + configuration arguments: arguments. ! tearDown @@ -78,38 +65,12 @@ result := super assert: whatever parse: input. ! -assert: whatever recognizesToken: input - whatever startSymbol: #nextToken. - - self assert: whatever parse: input. - self assert: (result isKindOf: PPToken). - - whatever startSymbol: #start -! - -assert: whatever rejectsToken: input - whatever startSymbol: #nextToken. - - self assert: whatever fail: input. - - whatever startSymbol: #start -! - literalNode: literal ^ PPCLiteralNode new literal: literal; yourself ! -tokenNode: child - child markForInline. - - ^ PPCTokenNode new - child: child; - tokenClass: PPToken; - yourself -! - tokenNodeForEOF | eof | eof := PPCEndOfFileNode new @@ -171,9 +132,8 @@ name: 'nextToken'; yourself. - wsNode := PPCTokenStarSeparatorNode new + wsNode := PPCNilNode new name: 'consumeWhitespace'; - child: PPCNilNode new; yourself. node := PPCTokenizingParserNode new @@ -208,9 +168,8 @@ consumeNode := PPCTokenConsumeNode new child: tokenNode; yourself. - wsNode := PPCTokenStarSeparatorNode new + wsNode := PPCNilNode new name: 'consumeWhitespace'; - child: PPCSentinelNode instance; yourself. node := PPCTokenizingParserNode new @@ -231,7 +190,7 @@ ! testTrimmingToken1 - | token tokenConsume tokensNode eof wsNode | + | token tokenConsume tokensNode eof wsNode separatorNode | token := self trimmingTokenNode: (self literalNode: 'foo'). eof := (self tokenNodeForEOF) yourself. @@ -245,9 +204,14 @@ name: 'nextToken'; yourself. - wsNode := PPCTokenStarSeparatorNode new + separatorNode := PPCLiteralNode new + literal: ' '; + name: 'separator'; + yourself. + + wsNode := PPCStarNode new name: 'consumeWhitespace'; - child: PPCNilNode new; + child: separatorNode; yourself. node := PPCTokenizingParserNode new diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCTokenizingTest.st --- a/compiler/tests/PPCTokenizingTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCTokenizingTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -321,6 +321,18 @@ ! +testCompileStar3 + parser := 'a' asParser trimmingToken star, 'b' asParser trimmingToken + compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: parser parse: 'aaab'. + self assert: result size = 2. + self assert: result first size = 3. + + self assert: parser fail: 'ac'. +! + testCompileTokenComplex2 | a b argumentsWith | "based on the PPSmalltlakGrammar>>blockArgumentsWith" @@ -335,7 +347,7 @@ parser := argumentsWith compileWithConfiguration: configuration. self assert: parser parse: '|'. - self assert: parser parse: ']'. + self assert: parser parse: ']' end: 0. ! testCompileTokenComplex3 @@ -454,7 +466,8 @@ self assert: parser parse: 'foofoo'. self assert: result first inputValue = 'foo'. self assert: result second inputValue = 'foo'. - self assert: (parser class methodDictionary includesKey: #fooToken). + self assert: (parser scanner class methodDictionary includesKey: #fooToken). + self assert: (parser scanner class methodDictionary includesKey: #scan_fooToken). ! testWhitespace @@ -473,7 +486,7 @@ self assert: parser parse: ' foo '. self assert: result first inputValue = 'foo'. - self assert: (context invocations select: [:e | e = #consumeWhitespace ]) size = 3. + self assert: (context invocations select: [:e | e = #scan_consumeWhitespace ]) size = 3. ! testWhitespace2 @@ -493,7 +506,7 @@ self assert: result first inputValue = 'foo'. self assert: result second inputValue = 'foo'. - self assert: (context invocations select: [:e | e = #consumeWhitespace ]) size = 4. + self assert: (context invocations select: [:e | e = #scan_consumeWhitespace ]) size = 4. ! testWhitespace3 @@ -514,6 +527,6 @@ self assert: result second inputValue = 'foo'. self assert: result third inputValue = 'foo'. - self assert: (context invocations select: [:e | e = #consumeWhitespace ]) size = 5. + self assert: (context invocations select: [:e | e = #scan_consumeWhitespace ]) size = 5. ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCUnivarsalGuardTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCUnivarsalGuardTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,115 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +PPAbstractParserTest subclass:#PPCUnivarsalGuardTest + instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 + arguments configuration' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Universal' +! + +!PPCUnivarsalGuardTest methodsFor:'as yet unclassified'! + +assert: p parse: whatever + ^ result := super assert: p parse: whatever. +! + +context + ^ context := PPCProfilingContext new +! + +setUp + arguments := PPCArguments default + profile: true; + yourself. + + configuration := PPCUniversalConfiguration new + arguments: arguments; + yourself. +! + +tearDown + | parserClass | + + parserClass := (Smalltalk at: #PPGeneratedParser ifAbsent: [nil]). + parserClass notNil ifTrue:[ + parserClass removeFromSystem + ]. +! ! + +!PPCUnivarsalGuardTest methodsFor:'tests - guard'! + +testChoiceGuard + parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken / $d asParser trimmingToken plus) + compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). + + self assert: parser parse: 'bar'. + self assert: result inputValue = 'bar'. + + self assert: parser parse: ' foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: ' d'. + self assert: result first inputValue = 'd'. + + self assert: parser fail: ''. + self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'predicate' ]). + self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). + + self assert: parser fail: 'zorg'. + self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). +! + +testEmptyChoiceGuard + parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken / $d asParser trimmingToken star) + compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: 'bar'. + self assert: result inputValue = 'bar'. + + self assert: parser parse: ' foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: ' d'. + self assert: result first inputValue = 'd'. + + self assert: parser parse: ''. + + self assert: parser parse: 'zorg' end: 0. +! + +testGuardSmalltlakToken + parser := (#letter asParser, #word asParser star) smalltalkToken compileWithConfiguration: configuration. + + self assert: parser parse: 'bar'. + self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'seq' ]). + + self assert: parser fail: '123'. + self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'seq' ]). +! + +testSequenceGuard + parser := ((#any asParser, #any asParser) wrapped, (#any asParser, #any asParser)) compileWithConfiguration: configuration. + + self assert: parser parse: 'fooo' to: #(#($f $o) #($o $o)). + self assert: parser parse: 'fo oo' to: #(#($f $o) #($ $o)) end: 4. + self assert: parser fail: 'fo'. + +! + +testTrimmerGuard + parser := $a asParser trim, $b asParser compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: parser parse: ' ab'. +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/PPCUniversalTest.st --- a/compiler/tests/PPCUniversalTest.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/PPCUniversalTest.st Mon Aug 24 15:34:14 2015 +0100 @@ -31,7 +31,7 @@ parserClass := (Smalltalk at: arguments parserName ifAbsent: [nil]). parserClass notNil ifTrue:[ - parserClass removeFromSystem + "parserClass removeFromSystem" ]. "Modified: / 24-07-2015 / 19:21:41 / Jan Vrany " @@ -445,7 +445,6 @@ parser := (#letter asParser, (#digit asParser / #letter asParser) star) trimmingToken compileWithConfiguration: configuration. self assert: parser class methodDictionary size = 4. - self assert: parser parse: 'foo'. self assert: result inputValue = 'foo'. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/abbrev.stc --- a/compiler/tests/abbrev.stc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/abbrev.stc Mon Aug 24 15:34:14 2015 +0100 @@ -16,10 +16,10 @@ PPCASTUtilitiesTests PPCASTUtilitiesTests stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Support' 1 PPCClassBuilderTest PPCClassBuilderTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 PPCCodeGeneratorTest PPCCodeGeneratorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 -PPCCompilerTest PPCCompilerTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 PPCContextMementoTest PPCContextMementoTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Context' 1 PPCContextTest PPCContextTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Context' 1 PPCCopyVisitorTest PPCCopyVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 +PPCDistinctScannerTest PPCDistinctScannerTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Tokenizing' 1 PPCGuardTest PPCGuardTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Guards' 1 PPCIdGeneratorTest PPCIdGeneratorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 PPCInliningVisitorTest PPCInliningVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 @@ -40,6 +40,7 @@ PPCTokenizingCodeGeneratorTest PPCTokenizingCodeGeneratorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 PPCTokenizingTest PPCTokenizingTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Tokenizing' 1 PPCTokenizingVisitorTest PPCTokenizingVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 +PPCUnivarsalGuardTest PPCUnivarsalGuardTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Universal' 1 PPCUniversalOptimizationTest PPCUniversalOptimizationTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Universal' 1 PPCUniversalTest PPCUniversalTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Universal' 1 stx_goodies_petitparser_compiler_tests stx_goodies_petitparser_compiler_tests stx:goodies/petitparser/compiler/tests '* Projects & Packages *' 3 diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/bc.mak --- a/compiler/tests/bc.mak Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/bc.mak Mon Aug 24 15:34:14 2015 +0100 @@ -35,7 +35,7 @@ -LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\tests -I$(INCLUDE_TOP)\stx\goodies\sunit -I$(INCLUDE_TOP)\stx\libbasic +LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\tests -I$(INCLUDE_TOP)\stx\goodies\sunit -I$(INCLUDE_TOP)\stx\libbasic LOCALDEFINES= STCLOCALOPT=-package=$(PACKAGE) -I. $(LOCALINCLUDES) -headerDir=. $(STCLOCALOPTIMIZATIONS) $(STCWARNINGS) $(LOCALDEFINES) -varPrefix=$(LIBNAME) @@ -89,10 +89,10 @@ $(OUTDIR)PPCASTUtilitiesTests.$(O) PPCASTUtilitiesTests.$(H): PPCASTUtilitiesTests.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCClassBuilderTest.$(O) PPCClassBuilderTest.$(H): PPCClassBuilderTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGeneratorTest.$(O) PPCCodeGeneratorTest.$(H): PPCCodeGeneratorTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCompilerTest.$(O) PPCCompilerTest.$(H): PPCCompilerTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCContextMementoTest.$(O) PPCContextMementoTest.$(H): PPCContextMementoTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPContextMementoTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCContextTest.$(O) PPCContextTest.$(H): PPCContextTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPContextTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCopyVisitorTest.$(O) PPCCopyVisitorTest.$(H): PPCCopyVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCDistinctScannerTest.$(O) PPCDistinctScannerTest.$(H): PPCDistinctScannerTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCGuardTest.$(O) PPCGuardTest.$(H): PPCGuardTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCIdGeneratorTest.$(O) PPCIdGeneratorTest.$(H): PPCIdGeneratorTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitorTest.$(O) PPCInliningVisitorTest.$(H): PPCInliningVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -113,6 +113,7 @@ $(OUTDIR)PPCTokenizingCodeGeneratorTest.$(O) PPCTokenizingCodeGeneratorTest.$(H): PPCTokenizingCodeGeneratorTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingTest.$(O) PPCTokenizingTest.$(H): PPCTokenizingTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitorTest.$(O) PPCTokenizingVisitorTest.$(H): PPCTokenizingVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCUnivarsalGuardTest.$(O) PPCUnivarsalGuardTest.$(H): PPCUnivarsalGuardTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalOptimizationTest.$(O) PPCUniversalOptimizationTest.$(H): PPCUniversalOptimizationTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalTest.$(O) PPCUniversalTest.$(H): PPCUniversalTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests.$(O) stx_goodies_petitparser_compiler_tests.$(H): stx_goodies_petitparser_compiler_tests.st $(INCLUDE_TOP)\stx\libbasic\LibraryDefinition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\ProjectDefinition.$(H) $(STCHDR) diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/Make.proto --- a/compiler/tests/extras/Make.proto Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/Make.proto Mon Aug 24 15:34:14 2015 +0100 @@ -150,6 +150,7 @@ $(OUTDIR)PPCLRPEpsilonTransition.$(O) PPCLRPEpsilonTransition.$(H): PPCLRPEpsilonTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPContainedElement.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCLRPTimeoutTransition.$(O) PPCLRPTimeoutTransition.$(H): PPCLRPTimeoutTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPContainedElement.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCLRPWildcardTransition.$(O) PPCLRPWildcardTransition.$(H): PPCLRPWildcardTransition.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPContainedElement.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras/PPCLRPTransition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java/PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/Make.spec --- a/compiler/tests/extras/Make.spec Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/Make.spec Mon Aug 24 15:34:14 2015 +0100 @@ -98,6 +98,7 @@ $(OUTDIR_SLASH)PPCLRPEpsilonTransition.$(O) \ $(OUTDIR_SLASH)PPCLRPTimeoutTransition.$(O) \ $(OUTDIR_SLASH)PPCLRPWildcardTransition.$(O) \ + $(OUTDIR_SLASH)extensions.$(O) \ diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/PPCExpressionGrammarTest_Tokenized.st --- a/compiler/tests/extras/PPCExpressionGrammarTest_Tokenized.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/PPCExpressionGrammarTest_Tokenized.st Mon Aug 24 15:34:14 2015 +0100 @@ -9,9 +9,17 @@ category:'PetitCompiler-Extras-Tests-Expressions' ! + !PPCExpressionGrammarTest_Tokenized methodsFor:'accessing'! compilerConfiguration ^ PPCConfiguration tokenizing ! ! +!PPCExpressionGrammarTest_Tokenized class methodsFor:'documentation'! + +version_HG + + ^ '$Changeset: $' +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/PPCJavaTests.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPCJavaTests.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,34 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +TestCase subclass:#PPCJavaTests + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Java' +! + +!PPCJavaTests methodsFor:'as yet unclassified'! + +javaWsNode + ^ PPCUnknownNode new + parser: PPJavaWhitespaceParser new; + yourself +! + +testJavaWS + | node1 node2 node visitor | + node1 := self javaWsNode. + node2 := self javaWsNode. + + node := PPCSequenceNode new + children: { node1 . node2 }. + + visitor := PPCMergingVisitor new. + + self assert: (node children first == node children second) not. + node := visitor visit: node. + self assert: (node children first == node children second). +! ! + diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/PPCLRPParserVerificationTest_Tokenized.st --- a/compiler/tests/extras/PPCLRPParserVerificationTest_Tokenized.st Mon Aug 17 12:13:16 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,17 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" - -"{ NameSpace: Smalltalk }" - -PPCLRPParserVerificationTest subclass:#PPCLRPParserVerificationTest_Tokenized - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Extras-Tests-LRP' -! - -!PPCLRPParserVerificationTest_Tokenized methodsFor:'accessing'! - -compilerConfiguration - ^ PPCConfiguration tokenizing -! ! - diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/PPCSmalltalkParserTests.st --- a/compiler/tests/extras/PPCSmalltalkParserTests.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/PPCSmalltalkParserTests.st Mon Aug 24 15:34:14 2015 +0100 @@ -45,6 +45,12 @@ "Created: / 29-07-2015 / 16:54:01 / Jan Vrany " ! +compiledScannerClassName + "Return the name of the compiled scanner" + + ^ (self petitParserClass name , 'C_Scanner') asSymbol +! + compilerConfiguration "Return configuration to use when compiling parser (as instance of PPCConfiguration)" @@ -90,6 +96,8 @@ configuration := self compilerConfiguration. configuration arguments parserName: self compiledParserClassName. + configuration arguments scannerName: self compiledScannerClassName. + time := Time millisecondsToRun: [ self petitParser compileWithConfiguration: configuration. ]. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/PPCSmalltalkTests.st --- a/compiler/tests/extras/PPCSmalltalkTests.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/PPCSmalltalkTests.st Mon Aug 24 15:34:14 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" TestCase subclass:#PPCSmalltalkTests - instanceVariableNames:'configuration arguments' + instanceVariableNames:'configuration arguments result' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Extras-Tests-Smalltalk' @@ -11,14 +11,9 @@ !PPCSmalltalkTests methodsFor:'as yet unclassified'! -configuration - arguments := PPCArguments default - profile: true; - yourself. - - ^ PPCTokenizingConfiguration new - arguments: arguments; - yourself. +assert: parser parse: input + result := parser parse: input. + self assert: result isPetitFailure not. ! setUp @@ -33,13 +28,25 @@ testSmalltakToken | token1 | - configuration := self configuration. token1 := 'a' asParser smalltalkToken compileWithConfiguration: configuration. self assert: ((token1 parse: 'a') class == PPSmalltalkToken). self assert: (token1 parse: '"comment" a "another comment"') inputValue = 'a' ! +testSmalltakToken2 + | parser compiled | + parser := 'a' asParser smalltalkToken, 'b' asParser smalltalkToken. + compiled := parser compileWithConfiguration: configuration. + + self assert: compiled parse: 'ab'. + self assert: compiled parse: '"comment" a "another comment" b '. + self assert: result size = 2. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'b'. + +! + testSmalltakWhitespace | ws1 ws2 | ws1 := PPSmalltalkWhitespaceParser new. diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/abbrev.stc --- a/compiler/tests/extras/abbrev.stc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/abbrev.stc Mon Aug 24 15:34:14 2015 +0100 @@ -4,6 +4,7 @@ PPCAbstractParserTest PPCAbstractParserTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Support' 1 PPCCompiledJavaVerificationTest PPCCompiledJavaVerificationTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Java' 1 PPCCompositeParserTest PPCCompositeParserTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Support' 1 +PPCJavaTests PPCJavaTests stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Java' 1 PPCLRPNode PPCLRPNode stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 PPCLRPParser PPCLRPParser stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 PPCLRPParserSmokeTest PPCLRPParserSmokeTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 1 @@ -43,7 +44,6 @@ PPCLRPMachine PPCLRPMachine stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 PPCLRPOnEntry PPCLRPOnEntry stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 PPCLRPOnExit PPCLRPOnExit stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 -PPCLRPParserVerificationTest_Tokenized PPCLRPParserVerificationTest_Tokenized stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 1 PPCLRPParserVerificationTest_Universal PPCLRPParserVerificationTest_Universal stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 1 PPCLRPRunning PPCLRPRunning stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 PPCLRPState PPCLRPState stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-LRP' 0 diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/bc.mak --- a/compiler/tests/extras/bc.mak Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/bc.mak Mon Aug 24 15:34:14 2015 +0100 @@ -97,6 +97,7 @@ $(OUTDIR)PPCLRPEpsilonTransition.$(O) PPCLRPEpsilonTransition.$(H): PPCLRPEpsilonTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPContainedElement.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCLRPTimeoutTransition.$(O) PPCLRPTimeoutTransition.$(H): PPCLRPTimeoutTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPContainedElement.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCLRPWildcardTransition.$(O) PPCLRPWildcardTransition.$(H): PPCLRPWildcardTransition.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPContainedElement.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras\PPCLRPTransition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java\PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/extensions.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/extensions.st Mon Aug 24 15:34:14 2015 +0100 @@ -0,0 +1,22 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }"! + +!PPJavaWhitespaceParser methodsFor:'*PetitCompiler-Extras-Tests'! + += anotherParser + anotherParser == self ifTrue: [ ^ true ]. + anotherParser class = self class ifFalse: [ ^ false ]. + ^ anotherParser name = self name +! ! + +!PPJavaWhitespaceParser methodsFor:'*PetitCompiler-Extras-Tests'! + +hash + ^ self name hash +! ! + +!stx_goodies_petitparser_compiler_tests_extras class methodsFor:'documentation'! + +extensionsVersion_HG + + ^ '$Changeset: $' +! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/libInit.cc --- a/compiler/tests/extras/libInit.cc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/libInit.cc Mon Aug 24 15:34:14 2015 +0100 @@ -49,6 +49,6 @@ _PPCLRPTimeoutTransition_Init(pass,__pRT__,snd); _PPCLRPWildcardTransition_Init(pass,__pRT__,snd); - +_stx_137goodies_137petitparser_137compiler_137tests_137extras_extensions_Init(pass,__pRT__,snd); __END_PACKAGE__(); } diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st --- a/compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st Mon Aug 24 15:34:14 2015 +0100 @@ -16,16 +16,16 @@ "The last merged version is: " ^ ' - Name: PetitCompiler-Extras-Tests-JanKurs.20150510144201 + Name: PetitCompiler-Tests-JanKurs.116 Author: JanKurs - Time: 10-05-2015, 04:59:25.308 PM - UUID: ef43bd1a-be60-4e88-b749-8b635622c969 + Time: 24-08-2015, 11:37:54.332 AM + UUID: 549e0927-358a-4a1b-8270-050ccfcb4217 Repository: http://smalltalkhub.com/mc/JanKurs/PetitParser/main ' "Created: / 03-10-2014 / 02:27:21 / Jan Vrany " - "Modified: / 12-05-2015 / 01:20:28 / Jan Vrany " + "Modified: / 24-08-2015 / 15:28:26 / Jan Vrany " ! monticelloName @@ -77,7 +77,7 @@ by searching all classes (and their packages) which are referenced by my classes." ^ #( - #'stx:goodies/petitparser/compiler' "PPCArguments - referenced by PPCSmalltalkTests>>configuration" + #'stx:goodies/petitparser/compiler' "PPCArguments - referenced by PPCSmalltalkTests>>setUp" #'stx:goodies/petitparser/parsers/smalltalk' "PPSmalltalkGrammar - referenced by PPCSmalltalkGrammarTests>>petitParserClass" #'stx:libbasic2' "Random - referenced by PPCResources>>expressionOfSize:stream:" ) @@ -106,6 +106,7 @@ (PPCAbstractParserTest autoload) (PPCCompiledJavaVerificationTest autoload) (PPCCompositeParserTest autoload) + (PPCJavaTests autoload) PPCLRPNode PPCLRPParser (PPCLRPParserSmokeTest autoload) @@ -145,7 +146,6 @@ PPCLRPMachine PPCLRPOnEntry PPCLRPOnExit - (#'PPCLRPParserVerificationTest_Tokenized' autoload) (#'PPCLRPParserVerificationTest_Universal' autoload) PPCLRPRunning PPCLRPState @@ -168,6 +168,8 @@ A correponding method with real names must be present in my concrete subclasses" ^ #( + PPJavaWhitespaceParser #'=' + PPJavaWhitespaceParser hash ) ! ! diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/libInit.cc --- a/compiler/tests/libInit.cc Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/libInit.cc Mon Aug 24 15:34:14 2015 +0100 @@ -42,10 +42,10 @@ _PPCASTUtilitiesTests_Init(pass,__pRT__,snd); _PPCClassBuilderTest_Init(pass,__pRT__,snd); _PPCCodeGeneratorTest_Init(pass,__pRT__,snd); -_PPCCompilerTest_Init(pass,__pRT__,snd); _PPCContextMementoTest_Init(pass,__pRT__,snd); _PPCContextTest_Init(pass,__pRT__,snd); _PPCCopyVisitorTest_Init(pass,__pRT__,snd); +_PPCDistinctScannerTest_Init(pass,__pRT__,snd); _PPCGuardTest_Init(pass,__pRT__,snd); _PPCIdGeneratorTest_Init(pass,__pRT__,snd); _PPCInliningVisitorTest_Init(pass,__pRT__,snd); @@ -66,6 +66,7 @@ _PPCTokenizingCodeGeneratorTest_Init(pass,__pRT__,snd); _PPCTokenizingTest_Init(pass,__pRT__,snd); _PPCTokenizingVisitorTest_Init(pass,__pRT__,snd); +_PPCUnivarsalGuardTest_Init(pass,__pRT__,snd); _PPCUniversalOptimizationTest_Init(pass,__pRT__,snd); _PPCUniversalTest_Init(pass,__pRT__,snd); _stx_137goodies_137petitparser_137compiler_137tests_Init(pass,__pRT__,snd); diff -r b5316ef15274 -r f6f68d32de73 compiler/tests/stx_goodies_petitparser_compiler_tests.st --- a/compiler/tests/stx_goodies_petitparser_compiler_tests.st Mon Aug 17 12:13:16 2015 +0100 +++ b/compiler/tests/stx_goodies_petitparser_compiler_tests.st Mon Aug 24 15:34:14 2015 +0100 @@ -16,15 +16,15 @@ "The last merged version is: " ^ ' - Name: PetitCompiler-Tests-JanKurs.20150510144201 + Name: PetitCompiler-Tests-JanKurs.116 Author: JanKurs - Time: 10-05-2015, 04:32:12.870 PM - UUID: 2a8fd41a-331b-4dcf-a7a3-752a50ce86e7 + Time: 24-08-2015, 11:37:54.332 AM + UUID: 549e0927-358a-4a1b-8270-050ccfcb4217 Repository: http://smalltalkhub.com/mc/JanKurs/PetitParser/main ' "Created: / 03-10-2014 / 02:27:21 / Jan Vrany " - "Modified: / 12-05-2015 / 01:20:05 / Jan Vrany " + "Modified: / 24-08-2015 / 15:27:27 / Jan Vrany " ! monticelloName @@ -75,7 +75,6 @@ ^ #( #'stx:goodies/petitparser' "PPCharSetPredicate - referenced by PEGFsaGeneratorTest>>testCharSetPredicateNode" #'stx:goodies/petitparser/compiler' "FooScanner - referenced by FooScannerTest>>setUp" - #'stx:goodies/petitparser/parsers/java' "PPJavaWhitespaceParser - referenced by PPCMergingVisitorTest>>javaWsNode" ) ! @@ -122,10 +121,10 @@ PPCASTUtilitiesTests PPCClassBuilderTest PPCCodeGeneratorTest - PPCCompilerTest PPCContextMementoTest PPCContextTest PPCCopyVisitorTest + PPCDistinctScannerTest PPCGuardTest PPCIdGeneratorTest PPCInliningVisitorTest @@ -146,6 +145,7 @@ PPCTokenizingCodeGeneratorTest PPCTokenizingTest PPCTokenizingVisitorTest + PPCUnivarsalGuardTest PPCUniversalOptimizationTest PPCUniversalTest #'stx_goodies_petitparser_compiler_tests'