# HG changeset patch # User Jan Vrany # Date 1432213942 -3600 # Node ID f6d77fee98112f0691eb222ec15e7f3291904e1c # Parent 4751c407bb402b73db78a598c2a1e373aa2cbe02 Updated to PetitCompiler-JanKurs.118, PetitCompiler-Tests-JanKurs.46, PetitCompiler-Extras-Tests-JanKurs.11, and PetitCompiler-Benchmarks-JanKurs.11 Name: PetitCompiler-JanKurs.118 Author: JanKurs Time: 13-05-2015, 03:59:01.292 PM UUID: 4a8ccd94-3131-4cc7-9098-528f8e5ea0b5 Name: PetitCompiler-Tests-JanKurs.46 Author: JanKurs Time: 04-05-2015, 04:25:06.162 PM UUID: 9f4cf8b7-876e-4a13-9579-b833f016db66 Name: PetitCompiler-Extras-Tests-JanKurs.11 Author: JanKurs Time: 13-05-2015, 04:27:27.940 PM UUID: e9f30c31-fbd0-4e96-ad2a-868f88d20ea8 Name: PetitCompiler-Benchmarks-JanKurs.11 Author: JanKurs Time: 13-05-2015, 02:21:49.932 PM UUID: 6a23fd1e-a86f-46db-8221-cc41b778d32c diff -r 4751c407bb40 -r f6d77fee9811 compiler/Make.proto --- a/compiler/Make.proto Tue May 12 01:24:03 2015 +0100 +++ b/compiler/Make.proto Thu May 21 14:12:22 2015 +0100 @@ -135,7 +135,9 @@ $(OUTDIR)PPCBridge.$(O) PPCBridge.$(H): PPCBridge.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompiledMethod.$(O) PPCCompiledMethod.$(H): PPCCompiledMethod.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompiler.$(O) PPCCompiler.$(H): PPCCompiler.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCCompilerTokenErrorStrategy.$(O) PPCCompilerTokenErrorStrategy.$(H): PPCCompilerTokenErrorStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilerTokenRememberStrategy.$(O) PPCCompilerTokenRememberStrategy.$(H): PPCCompilerTokenRememberStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCCompilerTokenizingErrorStrategy.$(O) PPCCompilerTokenizingErrorStrategy.$(H): PPCCompilerTokenizingErrorStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilerTokenizingRememberStrategy.$(O) PPCCompilerTokenizingRememberStrategy.$(H): PPCCompilerTokenizingRememberStrategy.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCConfiguration.$(O) PPCConfiguration.$(H): PPCConfiguration.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCContext.$(O) PPCContext.$(H): PPCContext.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(STCHDR) @@ -145,31 +147,31 @@ $(OUTDIR)PPCNode.$(O) PPCNode.$(H): PPCNode.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeVisitor.$(O) PPCNodeVisitor.$(H): PPCNodeVisitor.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableConfiguration.$(O) PPCPluggableConfiguration.$(H): PPCPluggableConfiguration.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenGuard.$(O) PPCTokenGuard.$(H): PPCTokenGuard.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCompiledParser.$(O) PPCompiledParser.$(H): PPCompiledParser.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler.$(O) stx_goodies_petitparser_compiler.$(H): stx_goodies_petitparser_compiler.st $(INCLUDE_TOP)/stx/libbasic/LibraryDefinition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/ProjectDefinition.$(H) $(STCHDR) -$(OUTDIR)PPCAbstractCharacterNode.$(O) PPCAbstractCharacterNode.$(H): PPCAbstractCharacterNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractLiteralNode.$(O) PPCAbstractLiteralNode.$(H): PPCAbstractLiteralNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractPredicateNode.$(O) PPCAbstractPredicateNode.$(H): PPCAbstractPredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAnyNode.$(O) PPCAnyNode.$(H): PPCAnyNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCCharacterNode.$(O) PPCCharacterNode.$(H): PPCCharacterNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGenerator.$(O) PPCCodeGenerator.$(H): PPCCodeGenerator.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCDelegateNode.$(O) PPCDelegateNode.$(H): PPCDelegateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfFileNode.$(O) PPCEndOfFileNode.$(H): PPCEndOfFileNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInlinedMethod.$(O) PPCInlinedMethod.$(H): PPCInlinedMethod.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCMethod.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitor.$(O) PPCInliningVisitor.$(H): PPCInliningVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1Configuration.$(O) PPCLL1Configuration.$(H): PPCLL1Configuration.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCConfiguration.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCListNode.$(O) PPCListNode.$(H): PPCListNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNilNode.$(O) PPCNilNode.$(H): PPCNilNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableNode.$(O) PPCPluggableNode.$(H): PPCPluggableNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCProfilingContext.$(O) PPCProfilingContext.$(H): PPCProfilingContext.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCContext.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(STCHDR) $(OUTDIR)PPCRewritingVisitor.$(O) PPCRewritingVisitor.$(H): PPCRewritingVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCompiler.$(O) PPCTokenizingCompiler.$(H): PPCTokenizingCompiler.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCompiler.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenizingConfiguration.$(O) PPCTokenizingConfiguration.$(H): PPCTokenizingConfiguration.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCConfiguration.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalConfiguration.$(O) PPCUniversalConfiguration.$(H): PPCUniversalConfiguration.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCConfiguration.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCUnknownNode.$(O) PPCUnknownNode.$(H): PPCUnknownNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPTokenizingCompiledParser.$(O) PPTokenizingCompiledParser.$(H): PPTokenizingCompiledParser.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCompiledParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractActionNode.$(O) PPCAbstractActionNode.$(H): PPCAbstractActionNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCAndNode.$(O) PPCAndNode.$(H): PPCAndNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCharSetPredicateNode.$(O) PPCCharSetPredicateNode.$(H): PPCCharSetPredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCCharacterNode.$(O) PPCCharacterNode.$(H): PPCCharacterNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractCharacterNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCChoiceNode.$(O) PPCChoiceNode.$(H): PPCChoiceNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCCopyVisitor.$(O) PPCCopyVisitor.$(H): PPCCopyVisitor.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNodeVisitor.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCRewritingVisitor.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfInputNode.$(O) PPCEndOfInputNode.$(H): PPCEndOfInputNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -180,6 +182,7 @@ $(OUTDIR)PPCMessagePredicateNode.$(O) PPCMessagePredicateNode.$(H): PPCMessagePredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNegateNode.$(O) PPCNegateNode.$(H): PPCNegateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNotCharSetPredicateNode.$(O) PPCNotCharSetPredicateNode.$(H): PPCNotCharSetPredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCNotCharacterNode.$(O) PPCNotCharacterNode.$(H): PPCNotCharacterNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCCharacterNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNotLiteralNode.$(O) PPCNotLiteralNode.$(H): PPCNotLiteralNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractLiteralNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNotMessagePredicateNode.$(O) PPCNotMessagePredicateNode.$(H): PPCNotMessagePredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNotNode.$(O) PPCNotNode.$(H): PPCNotNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) @@ -212,6 +215,7 @@ $(OUTDIR)PPCSymbolActionNode.$(O) PPCSymbolActionNode.$(H): PPCSymbolActionNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCAbstractActionNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenChoiceNode.$(O) PPCTokenChoiceNode.$(H): PPCTokenChoiceNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCChoiceNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTrimNode.$(O) PPCTrimNode.$(H): PPCTrimNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCSequenceNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCTrimmingCharacterTokenNode.$(O) PPCTrimmingCharacterTokenNode.$(H): PPCTrimmingCharacterTokenNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCListNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCTrimmingTokenNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarMessagePredicateNode.$(O) PPCTokenStarMessagePredicateNode.$(H): PPCTokenStarMessagePredicateNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarSeparatorNode.$(O) PPCTokenStarSeparatorNode.$(H): PPCTokenStarSeparatorNode.st $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCDelegateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCStarNode.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/compiler/PPCTokenStarMessagePredicateNode.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPActionParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPAndParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCharSetPredicate.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPChoiceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPContext.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEndOfInputParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPEpsilonParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFailure.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPFlattenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPListParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPNotParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPOptionalParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPluggableParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateObjectParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPPredicateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPRepeatingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPSequenceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPStream.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPToken.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPTrimmingParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java/PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk/PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Character.$(H) $(INCLUDE_TOP)/stx/libbasic/Magnitude.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/PeekableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/PositionableStream.$(H) $(INCLUDE_TOP)/stx/libbasic/ReadStream.$(H) $(INCLUDE_TOP)/stx/libbasic/Stream.$(H) $(INCLUDE_TOP)/stx/libbasic/UndefinedObject.$(H) $(STCHDR) diff -r 4751c407bb40 -r f6d77fee9811 compiler/Make.spec --- a/compiler/Make.spec Tue May 12 01:24:03 2015 +0100 +++ b/compiler/Make.spec Thu May 21 14:12:22 2015 +0100 @@ -55,7 +55,9 @@ PPCBridge \ PPCCompiledMethod \ PPCCompiler \ + PPCCompilerTokenErrorStrategy \ PPCCompilerTokenRememberStrategy \ + PPCCompilerTokenizingErrorStrategy \ PPCCompilerTokenizingRememberStrategy \ PPCConfiguration \ PPCContext \ @@ -65,31 +67,31 @@ PPCNode \ PPCNodeVisitor \ PPCPluggableConfiguration \ + PPCTokenGuard \ PPCompiledParser \ stx_goodies_petitparser_compiler \ - PPCAbstractCharacterNode \ PPCAbstractLiteralNode \ PPCAbstractPredicateNode \ PPCAnyNode \ + PPCCharacterNode \ PPCCodeGenerator \ PPCDelegateNode \ PPCEndOfFileNode \ PPCInlinedMethod \ PPCInliningVisitor \ - PPCLL1Configuration \ PPCListNode \ PPCNilNode \ PPCPluggableNode \ PPCProfilingContext \ PPCRewritingVisitor \ PPCTokenizingCompiler \ + PPCTokenizingConfiguration \ PPCUniversalConfiguration \ PPCUnknownNode \ PPTokenizingCompiledParser \ PPCAbstractActionNode \ PPCAndNode \ PPCCharSetPredicateNode \ - PPCCharacterNode \ PPCChoiceNode \ PPCCopyVisitor \ PPCEndOfInputNode \ @@ -100,6 +102,7 @@ PPCMessagePredicateNode \ PPCNegateNode \ PPCNotCharSetPredicateNode \ + PPCNotCharacterNode \ PPCNotLiteralNode \ PPCNotMessagePredicateNode \ PPCNotNode \ @@ -132,6 +135,7 @@ PPCSymbolActionNode \ PPCTokenChoiceNode \ PPCTrimNode \ + PPCTrimmingCharacterTokenNode \ PPCTokenStarMessagePredicateNode \ PPCTokenStarSeparatorNode \ @@ -143,7 +147,9 @@ $(OUTDIR_SLASH)PPCBridge.$(O) \ $(OUTDIR_SLASH)PPCCompiledMethod.$(O) \ $(OUTDIR_SLASH)PPCCompiler.$(O) \ + $(OUTDIR_SLASH)PPCCompilerTokenErrorStrategy.$(O) \ $(OUTDIR_SLASH)PPCCompilerTokenRememberStrategy.$(O) \ + $(OUTDIR_SLASH)PPCCompilerTokenizingErrorStrategy.$(O) \ $(OUTDIR_SLASH)PPCCompilerTokenizingRememberStrategy.$(O) \ $(OUTDIR_SLASH)PPCConfiguration.$(O) \ $(OUTDIR_SLASH)PPCContext.$(O) \ @@ -153,31 +159,31 @@ $(OUTDIR_SLASH)PPCNode.$(O) \ $(OUTDIR_SLASH)PPCNodeVisitor.$(O) \ $(OUTDIR_SLASH)PPCPluggableConfiguration.$(O) \ + $(OUTDIR_SLASH)PPCTokenGuard.$(O) \ $(OUTDIR_SLASH)PPCompiledParser.$(O) \ $(OUTDIR_SLASH)stx_goodies_petitparser_compiler.$(O) \ - $(OUTDIR_SLASH)PPCAbstractCharacterNode.$(O) \ $(OUTDIR_SLASH)PPCAbstractLiteralNode.$(O) \ $(OUTDIR_SLASH)PPCAbstractPredicateNode.$(O) \ $(OUTDIR_SLASH)PPCAnyNode.$(O) \ + $(OUTDIR_SLASH)PPCCharacterNode.$(O) \ $(OUTDIR_SLASH)PPCCodeGenerator.$(O) \ $(OUTDIR_SLASH)PPCDelegateNode.$(O) \ $(OUTDIR_SLASH)PPCEndOfFileNode.$(O) \ $(OUTDIR_SLASH)PPCInlinedMethod.$(O) \ $(OUTDIR_SLASH)PPCInliningVisitor.$(O) \ - $(OUTDIR_SLASH)PPCLL1Configuration.$(O) \ $(OUTDIR_SLASH)PPCListNode.$(O) \ $(OUTDIR_SLASH)PPCNilNode.$(O) \ $(OUTDIR_SLASH)PPCPluggableNode.$(O) \ $(OUTDIR_SLASH)PPCProfilingContext.$(O) \ $(OUTDIR_SLASH)PPCRewritingVisitor.$(O) \ $(OUTDIR_SLASH)PPCTokenizingCompiler.$(O) \ + $(OUTDIR_SLASH)PPCTokenizingConfiguration.$(O) \ $(OUTDIR_SLASH)PPCUniversalConfiguration.$(O) \ $(OUTDIR_SLASH)PPCUnknownNode.$(O) \ $(OUTDIR_SLASH)PPTokenizingCompiledParser.$(O) \ $(OUTDIR_SLASH)PPCAbstractActionNode.$(O) \ $(OUTDIR_SLASH)PPCAndNode.$(O) \ $(OUTDIR_SLASH)PPCCharSetPredicateNode.$(O) \ - $(OUTDIR_SLASH)PPCCharacterNode.$(O) \ $(OUTDIR_SLASH)PPCChoiceNode.$(O) \ $(OUTDIR_SLASH)PPCCopyVisitor.$(O) \ $(OUTDIR_SLASH)PPCEndOfInputNode.$(O) \ @@ -188,6 +194,7 @@ $(OUTDIR_SLASH)PPCMessagePredicateNode.$(O) \ $(OUTDIR_SLASH)PPCNegateNode.$(O) \ $(OUTDIR_SLASH)PPCNotCharSetPredicateNode.$(O) \ + $(OUTDIR_SLASH)PPCNotCharacterNode.$(O) \ $(OUTDIR_SLASH)PPCNotLiteralNode.$(O) \ $(OUTDIR_SLASH)PPCNotMessagePredicateNode.$(O) \ $(OUTDIR_SLASH)PPCNotNode.$(O) \ @@ -220,6 +227,7 @@ $(OUTDIR_SLASH)PPCSymbolActionNode.$(O) \ $(OUTDIR_SLASH)PPCTokenChoiceNode.$(O) \ $(OUTDIR_SLASH)PPCTrimNode.$(O) \ + $(OUTDIR_SLASH)PPCTrimmingCharacterTokenNode.$(O) \ $(OUTDIR_SLASH)PPCTokenStarMessagePredicateNode.$(O) \ $(OUTDIR_SLASH)PPCTokenStarSeparatorNode.$(O) \ $(OUTDIR_SLASH)extensions.$(O) \ diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCAbstractCharacterNode.st --- a/compiler/PPCAbstractCharacterNode.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,91 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -PPCNode subclass:#PPCAbstractCharacterNode - instanceVariableNames:'character' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Nodes' -! - - -!PPCAbstractCharacterNode methodsFor:'accessing'! - -character - ^ character -! - -character: char - character := char -! - -prefix - ^ #char -! ! - -!PPCAbstractCharacterNode methodsFor:'analysis'! - -acceptsEpsilon - ^ false -! - -firstCharSet - ^ PPCharSetPredicate on: [:e | e = character ] -! - -recognizedSentencesPrim - ^ Array with: character asString -! ! - -!PPCAbstractCharacterNode methodsFor:'comparison'! - -= anotherNode - super = anotherNode ifFalse: [ ^ false ]. - ^ character = anotherNode character. -! - -hash - ^ super hash bitXor: character hash -! ! - -!PPCAbstractCharacterNode methodsFor:'compiling'! - -body: compiler - | id | - - character ppcPrintable ifTrue: [ - id := character storeString - ] ifFalse: [ - id := compiler idFor: character prefixed: #char. - compiler addConstant: (Character value: character asInteger) as: id . - ]. - - compiler add: '(context peek == ', id, ')'. - compiler indent. - compiler add: 'ifFalse: [ self error: ''', character asInteger asString, ' expected'' at: context position ] '. - compiler add: 'ifTrue: [ context next ].'. - compiler dedent. -! ! - -!PPCAbstractCharacterNode methodsFor:'printing'! - -printNameOn: aStream - super printNameOn: aStream. - - character = $" ifTrue: [ - "this is hack to allow for printing '' in comments..." - aStream nextPutAll: ', '; nextPutAll: '$'''''. - ^ self - ]. - - aStream nextPutAll: ', '; print: character -! ! - -!PPCAbstractCharacterNode class methodsFor:'documentation'! - -version_HG - - ^ '$Changeset: $' -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCharacterNode.st --- a/compiler/PPCCharacterNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCCharacterNode.st Thu May 21 14:12:22 2015 +0100 @@ -2,22 +2,64 @@ "{ NameSpace: Smalltalk }" -PPCAbstractCharacterNode subclass:#PPCCharacterNode - instanceVariableNames:'' +PPCNode subclass:#PPCCharacterNode + instanceVariableNames:'character' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Nodes' ! -!PPCCharacterNode methodsFor:'as yet unclassified'! +!PPCCharacterNode methodsFor:'accessing'! + +character + ^ character +! + +character: char + character := char +! -start: compiler id: id - compiler startMethod: id. - compiler add: '^ '. +prefix + ^ #char +! ! + +!PPCCharacterNode methodsFor:'analysis'! + +acceptsEpsilon + ^ false +! + +firstCharSet + ^ PPCharSetPredicate on: [:e | e = character ] ! -stop: compiler - ^ compiler stopMethod +recognizedSentencesPrim + ^ Array with: character asString +! ! + +!PPCCharacterNode methodsFor:'comparison'! + += anotherNode + super = anotherNode ifFalse: [ ^ false ]. + ^ character = anotherNode character. +! + +hash + ^ super hash bitXor: character hash +! ! + +!PPCCharacterNode methodsFor:'printing'! + +printNameOn: aStream + super printNameOn: aStream. + + character = $" ifTrue: [ + "this is hack to allow for printing '' in comments..." + aStream nextPutAll: ', '; nextPutAll: '$'''''. + ^ self + ]. + + aStream nextPutAll: ', not('; print: character; nextPutAll: ')' ! ! !PPCCharacterNode methodsFor:'visiting'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCChoiceNode.st --- a/compiler/PPCChoiceNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCChoiceNode.st Thu May 21 14:12:22 2015 +0100 @@ -9,6 +9,7 @@ category:'PetitCompiler-Nodes' ! + !PPCChoiceNode methodsFor:'accessing'! prefix @@ -50,3 +51,10 @@ ^ visitor visitChoiceNode: self ! ! +!PPCChoiceNode class methodsFor:'documentation'! + +version_HG + + ^ '$Changeset: $' +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCodeGenerator.st --- a/compiler/PPCCodeGenerator.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCCodeGenerator.st Thu May 21 14:12:22 2015 +0100 @@ -33,6 +33,59 @@ ^ arguments guards ! ! +!PPCCodeGenerator methodsFor:'guards'! + +addGuard: node ifTrue: trueBlock ifFalse: falseBlock + | guard id | + (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. + id := compiler idFor: node. + +" falseBlock isNil ifFalse: [ + compiler add: 'context atEnd'. + compiler addOnLine: ' ifTrue: ['. + compiler indent. + falseBlock value. + compiler dedent. + compiler addOnLine: '].'. + ]." + + guard id: (compiler idFor: guard prefixed: #guard). + guard compileGuard: compiler. + + trueBlock isNil ifFalse: [ + compiler addOnLine: ' ifTrue: ['. + compiler indent. + trueBlock value. + compiler dedent. + falseBlock isNil ifTrue: [ compiler addOnLine: '].' ] + ifFalse: [ compiler add: ']'. ] + ]. + falseBlock isNil ifFalse: [ + compiler addOnLine: ' ifFalse: ['. + compiler indent. + falseBlock value. + compiler dedent. + compiler addOnLine: '].'. + ]. + ^ true +! + +addGuardTrimming: node + | guard firsts id | + (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ false]. + + id := compiler idFor: node. + firsts := node firstSetWithTokens. + + + (firsts allSatisfy: [ :e | e isTrimmingTokenNode ]) ifTrue: [ + "If we start with trimming, we should invoke the whitespace parser" + self compileTokenWhitespace: firsts anyOne. + ^ true + ]. + ^ false +! ! + !PPCCodeGenerator methodsFor:'hooks'! afterAccept: node retval: retval @@ -56,36 +109,10 @@ !PPCCodeGenerator methodsFor:'support'! -addGuard: node - | guard firsts id | - (self guards not or: [(guard := PPCGuard on: node) makesSense not]) ifTrue: [ ^ self]. - - id := compiler idFor: node. - firsts := node firstSetWithTokens. - - - (firsts allSatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]) ifTrue: [ - "If we start with trimming, we should invoke the whitespace parser" - self compileTokenWhitespace: firsts anyOne. - - compiler add: 'context atEnd ifTrue: [ ^ self error ].'. - guard id: id, '_guard'. - guard compileGuard: compiler. - compiler addOnLine: 'ifFalse: [ ^ self error ].' - ]. - - (firsts allSatisfy: [ :e | e isTerminal ]) ifTrue: [ - compiler add: 'context atEnd ifTrue: [ ^ self error ].'. - guard id: id, '_guard'. - guard compileGuard: compiler. - compiler addOnLine: 'ifFalse: [ ^ self error ].' - ]. -! - compileTokenWhitespace: node compiler add: 'context atWs ifFalse: ['. compiler indent. - compiler call: (self visit: node whitespace). + compiler codeStoreValueOf: [ self visit: node whitespace ] intoVariable: #whatever. compiler add: 'context setWs.'. compiler dedent. compiler add: '].'. @@ -222,8 +249,9 @@ compiler add: '(', classificationId, ' at: context peek asInteger)'. compiler indent. - compiler add: 'ifFalse: [ self error: ''predicate not found'' ]'. - compiler add: 'ifTrue: [ '. + compiler add: 'ifFalse: ['. + compiler codeError: 'predicate not found'. + compiler add: '] ifTrue: [ '. compiler codeReturn: 'context next'. compiler add: '].'. compiler dedent. @@ -240,9 +268,14 @@ compiler add: '(context peek == ', chid, ')'. compiler indent. - compiler add: 'ifFalse: [ self error: ''', node character asInteger asString, ' expected'' at: context position ] '. - compiler add: 'ifTrue: [ '. + compiler add: 'ifFalse: ['. + compiler indent. + compiler codeError: node character asInteger asString, ' expected'. + compiler dedent. + compiler add: '] ifTrue: [ '. + compiler indent. compiler codeReturn: 'context next'. + compiler dedent. compiler add: '].'. compiler dedent. ! @@ -259,45 +292,31 @@ ! visitChoiceNode: node - | firsts guard whitespaceConsumed elementVar | + | whitespaceConsumed elementVar | "The code is not ready for inlining" self assert: node isMarkedForInline not. - whitespaceConsumed := false. - firsts := node firstSetWithTokens. - elementVar := compiler allocateTemporaryVariableNamed: 'element'. - " - If we want to compile in guard and the choice starts with trimming token, - we should invoke the whitespace parser - " - (self guards and: [ firsts allSatisfy: [ :e | e isTrimmingTokenNode ] ]) ifTrue: [ - self compileTokenWhitespace: firsts anyOne. - whitespaceConsumed := true. - ]. - + whitespaceConsumed := self addGuardTrimming: node. + 1 to: node children size do: [ :idx | |child allowGuard | child := node children at: idx. allowGuard := whitespaceConsumed. - - (allowGuard and: [self guards and: [ (guard := PPCGuard on: child) makesSense ]]) ifTrue: [ - guard id: (compiler idFor: guard prefixed: #guard). - guard compileGuard: compiler. - compiler add: ' ifTrue: [ '. - compiler indent. + + allowGuard ifTrue: [ + self addGuard: child ifTrue: [ compiler add: 'self clearError.'. compiler codeStoreValueOf: [self visit: child] intoVariable: elementVar. - compiler add: 'error ifFalse: [ ^ element ].'. - compiler dedent. - compiler add: ' ].'. - ] ifFalse: [ - compiler add: 'self clearError.'. - compiler codeStoreValueOf: [self visit: child] intoVariable: elementVar. - compiler add: 'error ifFalse: [ ^ element ].'. + compiler add: 'error ifFalse: [ ^ ', elementVar, ' ].'. + ] ifFalse: nil. + ] ifFalse: [ + compiler add: 'self clearError.'. + compiler codeStoreValueOf: [self visit: child] intoVariable: elementVar. + compiler add: 'error ifFalse: [ ^ ', elementVar, ' ].'. ] ]. - compiler add: '^ self error: ''no choice suitable'''. + compiler codeError: 'no choice suitable'. "Modified: / 23-04-2015 / 21:40:23 / Jan Vrany " ! @@ -331,14 +350,16 @@ compiler add: '((context next: ', node literal size asString, ') = #''', encodedLiteral, ''') ifTrue: ['. compiler codeReturn: '#''', encodedLiteral, ''' '. compiler add: '] ifFalse: ['. - compiler add: ' context position: ', positionVar, '.'. - compiler add: ' self error: ''', encodedLiteral, ' expected'' at: position'. + compiler indent. + compiler add: 'context position: ', positionVar, '.'. + compiler codeError: encodedLiteral, ' expected' at: positionVar. + compiler dedent. compiler add: '].'. ! visitMessagePredicateNode: node compiler add: '(context peek ', node message, ') ifFalse: ['. - compiler add: ' self error: ''predicate not found'''. + compiler codeError: 'predicate not found'. compiler add: '] ifTrue: [ '. compiler codeReturn: ' context next'. compiler add: '].'. @@ -359,13 +380,37 @@ compiler addOnLine: '(', classificationId, ' at: context peek asInteger)'. compiler indent. - compiler add: ' ifTrue: [ self error: '' predicate not expected'' ]'. - compiler add: ' ifFalse: ['. + compiler add: ' ifTrue: ['. + compiler codeError: 'predicate not expected'. + compiler add: '] ifFalse: ['. compiler codeReturn: 'nil'. compiler add: '].'. compiler dedent. ! +visitNotCharacterNode: node + | chid | + node character ppcPrintable ifTrue: [ + chid := node character storeString + ] ifFalse: [ + chid := compiler idFor: node character prefixed: #char. + compiler addConstant: (Character value: node character asInteger) as: chid . + ]. + + compiler add: '(context peek == ', chid, ')'. + compiler indent. + compiler add: 'ifTrue: ['. + compiler indent. + compiler codeError: node character asInteger asString, ' not expected'. + compiler dedent. + compiler add: '] ifFalse: [ '. + compiler indent. + compiler codeReturn: 'nil.'. + compiler dedent. + compiler add: '].'. + compiler dedent. +! + visitNotLiteralNode: node | encodedLiteral size | encodedLiteral := node encodeQuotes: node literal. @@ -373,8 +418,9 @@ compiler add: '((context peek: ', size, ') =#''', encodedLiteral, ''')'. compiler indent. - compiler add: 'ifTrue: [ self error: ''', encodedLiteral, ' not expected'' ]'. - compiler add: 'ifFalse: [ '. + compiler add: 'ifTrue: ['. + compiler codeError: encodedLiteral, ' not expected'. + compiler add: '] ifFalse: [ '. compiler codeReturn: 'nil' . compiler add: '].'. compiler dedent. @@ -408,8 +454,10 @@ visitOptionalNode: node compiler codeStoreValueOf: [ self visit: node child ] intoVariable: self retvalVar. compiler add: 'error ifTrue: [ '. - compiler add: ' self clearError. '. + compiler indent. + compiler add: 'self clearError. '. compiler codeAssign: 'nil.' to: self retvalVar. + compiler dedent. compiler add: '].'. compiler codeReturn. ! @@ -423,14 +471,20 @@ ! visitPlusNode: node - | elementVar | + | elementVar | elementVar := compiler allocateTemporaryVariableNamed: 'element'. - + +" self tokenGuards ifTrue: [ + compiler codeTokenGuard: node ifFalse: [ compiler codeError: 'at least one occurence expected' ]. + ]. +" compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. compiler codeStoreValueOf: [ self visit: node child ] intoVariable: elementVar. - compiler add: 'error ifTrue: [ self error: ''at least one occurence expected'' ] ifFalse: ['. + compiler add: 'error ifTrue: ['. + compiler codeError: 'at least one occurence expected'. + compiler add: '] ifFalse: ['. compiler indent. compiler add: self retvalVar , ' add: ',elementVar , '.'. @@ -457,8 +511,9 @@ compiler add: '(context atEnd not and: [ ', pid , ' value: context uncheckedPeek])'. compiler indent. - compiler add: 'ifFalse: [ self error: ''predicate not found'' ]'. - compiler add: 'ifTrue: [ ', self retvalVar ,' := context next ].'. + compiler add: 'ifFalse: ['. + compiler codeError: 'predicate not found'. + compiler add: '] ifTrue: [ ', self retvalVar ,' := context next ].'. compiler dedent. compiler codeReturn. @@ -466,49 +521,66 @@ ! visitRecognizingSequenceNode: node - | mementoVar | + | mementoVar canBacktrack | + + canBacktrack := (node children allButFirst allSatisfy: [:e | e acceptsEpsilon ]) not. - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. - compiler smartRemember: node to: mementoVar. + canBacktrack ifTrue: [ + mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. + compiler smartRemember: node to: mementoVar. + ]. -" self addGuard: compiler." - - compiler codeStoreValueOf: [ self visit: (node children at: 1) ] intoVariable: #whatever. + compiler codeStoreValueOf: [ self visit: (node children at: 1) ] intoVariable: #whatever. compiler add: 'error ifTrue: [ ^ failure ].'. 2 to: (node children size) do: [ :idx | |child| child := node children at: idx. compiler codeStoreValueOf: [ self visit: child ] intoVariable: #whatever. - compiler add: 'error ifTrue: [ '. - compiler indent. - compiler smartRestore: node from: mementoVar. - compiler add: ' ^ failure .'. - compiler dedent. - compiler add: '].'. + + child acceptsEpsilon ifFalse: [ + compiler add: 'error ifTrue: [ '. + compiler indent. + compiler smartRestore: node from: mementoVar. + compiler add: ' ^ failure .'. + compiler dedent. + compiler add: '].'. + ]. ]. ! visitSequenceNode: node - | elementVar mementoVar | + | elementVar mementoVar canBacktrack | elementVar := compiler allocateTemporaryVariableNamed: 'element'. - mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. + canBacktrack := (node children allButFirst allSatisfy: [:e | e acceptsEpsilon ]) not. - compiler smartRemember: node to: mementoVar. +" self addGuardTrimming: node. + self addGuard: node ifTrue: nil ifFalse: [ compiler addOnLine: ' ^ self error' ]. +" + canBacktrack ifTrue: [ + mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. + compiler smartRemember: node to: mementoVar. + ]. + compiler codeAssign: 'Array new: ', node children size asString, '.' to: self retvalVar. - self addGuard: node. - 1 to: (node children size) do: [ :idx | |child| + compiler codeStoreValueOf: [ self visit: (node children at: 1)] intoVariable: elementVar. + compiler add: 'error ifTrue: [ ^ failure ].'. + compiler add: self retvalVar , ' at: 1 put: ', elementVar, '.'. + + 2 to: (node children size) do: [ :idx | |child| child := node children at: idx. compiler codeStoreValueOf: [ self visit: child ] intoVariable: elementVar. - - compiler add: 'error ifTrue: [ '. - compiler indent. - compiler smartRestore: node from: mementoVar. - compiler add: '^ failure.'. - compiler dedent. - compiler add: '].'. + + child acceptsEpsilon ifFalse: [ + compiler add: 'error ifTrue: [ '. + compiler indent. + compiler smartRestore: node from: mementoVar. + compiler add: '^ failure.'. + compiler dedent. + compiler add: '].'. + ]. compiler add: self retvalVar , ' at: ', idx asString, ' put: ',elementVar,'.'. ]. compiler codeReturn @@ -540,10 +612,11 @@ compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. compiler add: '[ ', classificationId, ' at: context peek asInteger ] whileTrue: ['. compiler indent. - compiler add: self retvalVar, ' add: context next.'. + compiler codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. compiler dedent. compiler add: '].'. - compiler codeReturn: 'retval asArray'. + compiler codeAssign: self retvalVar, ' asArray.' to: self retvalVar. + compiler codeReturn. ! visitStarMessagePredicateNode: node @@ -551,10 +624,11 @@ compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. compiler add: '[ context peek ', node message, ' ] whileTrue: ['. compiler indent. - compiler add: self retvalVar, ' add: context next.'. + compiler codeEvaluate: 'add:' argument: 'context next.' on: self retvalVar. compiler dedent. compiler add: '].'. - compiler codeReturn: 'retval asArray'. + compiler codeAssign: self retvalVar, ' asArray.' to: self retvalVar. + compiler codeReturn. ! visitStarNode: node @@ -562,6 +636,8 @@ elementVar := compiler allocateTemporaryVariableNamed: 'element'. + self addGuard: node child ifTrue: nil ifFalse: [ compiler codeReturn: '#()' ]. + compiler codeAssign: 'OrderedCollection new.' to: self retvalVar. compiler codeStoreValueOf: [ self visit: node child ] intoVariable: elementVar. compiler add: '[ error ] whileFalse: ['. @@ -602,6 +678,8 @@ startVar := compiler allocateTemporaryVariableNamed: 'start'. endVar := compiler allocateTemporaryVariableNamed: 'end'. + compiler profileTokenRead: (compiler idFor: node). + compiler codeAssign: 'context position + 1.' to: startVar. compiler codeStoreValueOf: [ self visit: node child ] intoVariable: #whatever. compiler add: 'error ifFalse: [ '. @@ -660,6 +738,10 @@ compiler add: '].'. ! +visitTrimmingTokenCharacterNode: node + ^ self visitTrimmingTokenNode: node +! + visitTrimmingTokenNode: node | id guard startVar endVar | @@ -667,8 +749,8 @@ endVar := compiler allocateTemporaryVariableNamed: 'end'. id := compiler idFor: node. -" (id beginsWith: 'kw') ifTrue: [ self halt. ]." - "self compileFirstWhitespace: compiler." + compiler profileTokenRead: id. + self compileTokenWhitespace: node. (arguments guards and: [(guard := PPCGuard on: node) makesSense]) ifTrue: [ diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCompiler.st --- a/compiler/PPCCompiler.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCCompiler.st Thu May 21 14:12:22 2015 +0100 @@ -8,7 +8,7 @@ arguments' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Core' + category:'PetitCompiler-Compiler' ! @@ -178,9 +178,6 @@ (variable == #whatever) ifFalse: [ "Do not assign, if somebody does not care!!" self add: variable ,' := ', code. - ] ifTrue: [ - "In case code hava a side effect" - self add: code ] ! @@ -188,10 +185,44 @@ self add: 'self clearError.'. ! +codeError + self add: 'self error: ''message notspecified''.'. +! + codeError: errorMessage self add: 'self error: ''', errorMessage, '''.' ! +codeError: errorMessage at: position + self add: 'self error: ''', errorMessage, ''' at: ', position asString, '.' +! + +codeEvaluate: selector argument: argument on: variable + self assert: variable isNil not. + + "TODO JK: Hack alert, whatever is magic constant!!" + (variable == #whatever) ifFalse: [ + "Do not assign, if somebody does not care!!" + self add: variable, ' ', selector,' ', argument. + ] ifTrue: [ + "In case argument has a side effect" + self add: argument + ] +! + +codeEvaluateAndAssign: argument to: variable + self assert: variable isNil not. + + "TODO JK: Hack alert, whatever is magic constant!!" + (variable == #whatever) ifFalse: [ + "Do not assign, if somebody does not care!!" + self add: variable ,' := ', argument. + ] ifTrue: [ + "In case an argument has a side effect" + self add: argument. + ] +! + codeHalt self add: 'self halt. ' ! @@ -228,7 +259,7 @@ " - returns whatever is in code OR - assigns whatever is in code into the returnVariable" currentMethod isInline ifTrue:[ - self codeAssign: code to: currentMethod returnVariable. + self codeEvaluateAndAssign: code to: currentMethod returnVariable. ] ifFalse: [ self add: '^ ', code ] @@ -253,12 +284,26 @@ method isInline ifTrue: [ self callOnLine: method ] ifFalse: [ - self codeAssign: (method call) to: aString. + self codeEvaluateAndAssign: (method call) to: aString. ] "Created: / 23-04-2015 / 18:21:51 / Jan Vrany " ! +codeTokenGuard: node ifFalse: codeBlock + | guard id | + guard := PPCTokenGuard on: node. + (guard makesSense) ifTrue: [ + id := self idFor: guard firstToken. + + self add: 'self ', id asString, ' ifFalse: ['. + self indent. + codeBlock value. + self dedent. + self add: '].'. + ] +! + codeTranscriptShow: text (arguments profile) ifTrue: [ self add: 'Transcript show: ', text storeString, '; cr.'. @@ -302,7 +347,7 @@ ^ ids at: object ifAbsentPut: [ ((object isKindOf: PPCNode) and: [object name isNotNil]) ifTrue: [ "Do not use prefix, if there is a name" - name := self asSelector: object name. + name := self asSelector: (object name asString). id := (name, suffix) asSymbol. "Make sure, that the generated ID is uniqe!!" @@ -322,6 +367,14 @@ ^ self idFor: object prefixed: object prefix suffixed: suffix effect: #none ! ! +!PPCCompiler methodsFor:'code generation - profiling'! + +profileTokenRead: tokenName + arguments profile ifTrue: [ + self add: 'context tokenRead: ', tokenName storeString, '.' + ] +! ! + !PPCCompiler methodsFor:'code generation - support'! cache: id as: value @@ -397,7 +450,7 @@ stopMethod self cache: currentMethod methodName as: currentMethod. - arguments profile ifTrue: [ Transcript show: currentMethod code; cr. ]. + "arguments profile ifTrue: [ Transcript show: currentMethod code; cr. ]." ^ self pop. "Modified: / 01-05-2015 / 14:18:07 / Jan Vrany " diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCompilerTokenErrorStrategy.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCCompilerTokenErrorStrategy.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,37 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Object subclass:#PPCCompilerTokenErrorStrategy + instanceVariableNames:'compiler' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Compiler' +! + +!PPCCompilerTokenErrorStrategy class methodsFor:'as yet unclassified'! + +on: aPPCCompiler + ^ self new + compiler: aPPCCompiler; + yourself +! ! + +!PPCCompilerTokenErrorStrategy methodsFor:'as yet unclassified'! + +codeClearError + compiler add: 'self clearError.' +! + +codeError + compiler add: 'error := true.' +! + +codeError: message + compiler add: 'error := true.' +! + +compiler: aPPCCompiler + compiler := aPPCCompiler +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCompilerTokenRememberStrategy.st --- a/compiler/PPCCompilerTokenRememberStrategy.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCCompilerTokenRememberStrategy.st Thu May 21 14:12:22 2015 +0100 @@ -6,7 +6,7 @@ instanceVariableNames:'compiler' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Core' + category:'PetitCompiler-Compiler' ! !PPCCompilerTokenRememberStrategy class methodsFor:'instance creation'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCompilerTokenizingErrorStrategy.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCCompilerTokenizingErrorStrategy.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,37 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Object subclass:#PPCCompilerTokenizingErrorStrategy + instanceVariableNames:'compiler' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Compiler' +! + +!PPCCompilerTokenizingErrorStrategy class methodsFor:'as yet unclassified'! + +on: aPPCCompiler + ^ self new + compiler: aPPCCompiler; + yourself +! ! + +!PPCCompilerTokenizingErrorStrategy methodsFor:'as yet unclassified'! + +codeClearError + compiler add: 'error := false.'. +! + +codeError + compiler add: 'self error: ''error message not specified'' at: context position.'. +! + +codeError: message + compiler add: 'self error: ''', message, ''' at: context position.'. +! + +compiler: aPPCCompiler + compiler := aPPCCompiler +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCCompilerTokenizingRememberStrategy.st --- a/compiler/PPCCompilerTokenizingRememberStrategy.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCCompilerTokenizingRememberStrategy.st Thu May 21 14:12:22 2015 +0100 @@ -6,7 +6,7 @@ instanceVariableNames:'compiler' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Core' + category:'PetitCompiler-Compiler' ! @@ -29,6 +29,7 @@ smartRemember: parser to: variableName parser isContextFree ifTrue: [ compiler codeAssign: '{ context lwRemember. currentTokenType . currentTokenValue }.' +" compiler codeAssign: 'context lwRemember.' " to: variableName. ] ifFalse: [ compiler codeAssign: '{ context remember. currentTokenType . currentTokenValue }.' @@ -38,12 +39,15 @@ smartRestore: parser from: mementoName parser isContextFree ifTrue: [ - compiler add: 'context lwRestore: ', mementoName, ' first.'. + compiler add: 'context lwRestore: (', mementoName, ' at: 1).'. +" compiler add: 'context lwRestore: ', mementoName, '.'." ] ifFalse: [ - compiler add: 'context restore: ', mementoName, ' first.'. + compiler add: 'context restore: (', mementoName, ' at: 1).'. ]. - compiler codeAssign: mementoName, ' second.' to: 'currentTokenType'. - compiler codeAssign: mementoName, ' third.' to: 'currentTokenValue'. + + compiler codeAssign: '(', mementoName, ' at: 2).' to: 'currentTokenType'. + compiler codeAssign: '(', mementoName, ' at: 3).' to: 'currentTokenValue'. + ! ! !PPCCompilerTokenizingRememberStrategy class methodsFor:'documentation'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCConfiguration.st --- a/compiler/PPCConfiguration.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCConfiguration.st Thu May 21 14:12:22 2015 +0100 @@ -11,10 +11,6 @@ !PPCConfiguration class methodsFor:'as yet unclassified'! -LL1 - ^ PPCLL1Configuration new -! - default ^ self universal ! @@ -25,6 +21,10 @@ yourself ! +tokenizing + ^ PPCTokenizingConfiguration new +! + universal ^ PPCUniversalConfiguration new ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCEndOfFileNode.st --- a/compiler/PPCEndOfFileNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCEndOfFileNode.st Thu May 21 14:12:22 2015 +0100 @@ -18,13 +18,15 @@ !PPCEndOfFileNode methodsFor:'analysis'! acceptsEpsilon - self halt. ^ false ! acceptsEpsilonOpenSet: set - self halt. ^ false +! + +firstCharSet + ^ PPCharSetPredicate on: [:e | true ] ! ! !PPCEndOfFileNode methodsFor:'visiting'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCGuard.st --- a/compiler/PPCGuard.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCGuard.st Thu May 21 14:12:22 2015 +0100 @@ -6,7 +6,7 @@ instanceVariableNames:'classification id message' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Core' + category:'PetitCompiler-Guards' ! !PPCGuard class methodsFor:'as yet unclassified'! @@ -15,6 +15,13 @@ ^ self new initializeFor: aPPCNode; yourself +! + +on: aPPCNode id: id + ^ self new + initializeFor: aPPCNode; + id: id; + yourself ! ! !PPCGuard methodsFor:'accessing'! @@ -62,7 +69,7 @@ classification keysAndValuesDo: [ :index :value | value ifTrue: [ (index > 32 and: [ index < 127 ]) ifTrue: [ - compiler add: '(context peek = ', (Character value: index) storeString, ')' + compiler add: '(context peek == ', (Character value: index) storeString, ')' ] ifFalse: [ id := compiler idFor: (Character value: index) prefixed: #character. compiler addConstant: (Character value: index) as: id. @@ -112,7 +119,6 @@ self assert: self makesSense description: 'No Guard could be compiled'. self assert: id notNil. - self message ifNotNil: [ ^ self compileMessage: compiler ]. self testAny ifTrue: [ ^ self compileAny: compiler ]. self testSingleCharacter ifTrue: [ ^ self compileCharacter: compiler ]. diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCInliningVisitor.st --- a/compiler/PPCInliningVisitor.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCInliningVisitor.st Thu May 21 14:12:22 2015 +0100 @@ -3,7 +3,7 @@ "{ NameSpace: Smalltalk }" PPCNodeVisitor subclass:#PPCInliningVisitor - instanceVariableNames:'acceptedNodes' + instanceVariableNames:'canInline acceptedNodes' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Visitors' @@ -81,6 +81,14 @@ "Modified: / 23-04-2015 / 12:15:49 / Jan Vrany " ! +visitStarCharSetPredicateNode: node + ^ self markForInline: node +! + +visitStarMessagePredicateNode: node + ^ self markForInline: node +! + visitTokenConsumeNode: node super visitTokenConsumeNode: node. node name isNil ifTrue: [ diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCLL1Configuration.st --- a/compiler/PPCLL1Configuration.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,64 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler' }" - -"{ NameSpace: Smalltalk }" - -PPCConfiguration subclass:#PPCLL1Configuration - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Core' -! - -!PPCLL1Configuration methodsFor:'compiling'! - -invokePhases - self toPPCIr. - self createTokens. - self cacheFirstFollow. - self createLL1Choices. - self tokenize. - - "Merge duplicate tokens and recompute first follow" - self merge. - self cacheFirstFollow. - - self specialize. - self createRecognizingComponents. - self specialize. - self inline. - self merge. - self check. - self generate. -! ! - -!PPCLL1Configuration methodsFor:'hooks'! - -codeCompilerOn: args - ^ PPCTokenizingCompiler on: args -! - -codeGeneratorVisitorOn: compiler - ^ PPCTokenizingCodeGenerator on: compiler -! ! - -!PPCLL1Configuration methodsFor:'phases'! - -createLL1Choices - ir := PPCLL1Visitor new - arguments: arguments; - visit: ir. - self remember: #LL1 -! - -tokenize - " - This will try transform the parser into the tokenizing parser - " - arguments tokenize ifFalse: [ ^ self ] . - - ir := PPCTokenizingVisitor new - arguments: arguments; - visit: ir. - self remember: #tokenize -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCMethod.st --- a/compiler/PPCMethod.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCMethod.st Thu May 21 14:12:22 2015 +0100 @@ -33,8 +33,7 @@ ^ category isNil ifTrue: [ category := 'generated' ] ifFalse: [ category ] - - "Modified (format): / 12-05-2015 / 01:21:14 / Jan Vrany " + ! category: value diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCNode.st --- a/compiler/PPCNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCNode.st Thu May 21 14:12:22 2015 +0100 @@ -77,7 +77,7 @@ ! suffix - ^ '' + ^ self isMarkedForInline ifTrue: [ '_inlined' ] ifFalse: [ '' ] ! unmarkForGuard @@ -267,6 +267,7 @@ (anotherNode class = self class) ifFalse: [ ^ false ]. (anotherNode name = name) ifFalse: [ ^ false ]. + (anotherNode isMarkedForInline = self isMarkedForInline) ifFalse: [ ^ false ]. ^ anotherNode children = self children. ! @@ -274,7 +275,7 @@ "TODO JK: IMO not a good hashing function bacause of children size, but at least it is not recursive, which would be worse :) " - ^ self class hash bitXor: (name hash bitXor: self children size hash) + ^ self class hash bitXor: (name hash bitXor: (self isMarkedForInline hash bitXor: (self children size hash))) ! ! !PPCNode methodsFor:'copying'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCNodeVisitor.st --- a/compiler/PPCNodeVisitor.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCNodeVisitor.st Thu May 21 14:12:22 2015 +0100 @@ -184,6 +184,10 @@ ^ self visitNode: node ! +visitNotCharacterNode: node + ^ self visitNode: node +! + visitNotLiteralNode: node ^ self visitNode: node ! @@ -276,6 +280,10 @@ ^ self visitNode: node ! +visitTrimmingTokenCharacterNode: node + ^ self visitNode: node +! + visitTrimmingTokenNode: node ^ self visitNode: node ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCNotCharacterNode.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCNotCharacterNode.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,40 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCCharacterNode subclass:#PPCNotCharacterNode + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Nodes' +! + +!PPCNotCharacterNode methodsFor:'accessing'! + +prefix + ^ #notChar +! ! + +!PPCNotCharacterNode methodsFor:'as yet unclassified'! + +accept: visitor + ^ visitor visitNotCharacterNode: self +! + +firstCharSet + ^ PPCharSetPredicate on: [:e | (e = character) not ] +! + +recognizedSentencesPrim + | retval | + + retval := OrderedCollection new. + "TODO JK: Works only for ASCII :(" + 1 to: 255 do: [ :i | + (character = (Character codePoint: i)) ifFalse: [ + retval add: (Character codePoint: i) asString + ] + ]. + ^ retval +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCProfilingContext.st --- a/compiler/PPCProfilingContext.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCProfilingContext.st Thu May 21 14:12:22 2015 +0100 @@ -3,7 +3,8 @@ "{ NameSpace: Smalltalk }" PPCContext subclass:#PPCProfilingContext - instanceVariableNames:'invocations remembers restores lwRemembers lwRestores totalSize' + instanceVariableNames:'invocations remembers restores lwRemembers lwRestores totalSize + tokenReads' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Context' @@ -36,6 +37,8 @@ lwRemembers := OrderedCollection new. lwRestores := OrderedCollection new. + tokenReads := OrderedCollection new. + totalSize := 0. ! ! @@ -101,6 +104,18 @@ stream: aStream totalSize := totalSize + aStream size. ^ super stream: aStream +! + +tokenRead: tokenName + tokenReads add: tokenName +! + +tokenReadCount + ^ tokenReads size +! + +tokenReads + ^ tokenReads ! ! !PPCProfilingContext methodsFor:'reporting'! @@ -116,6 +131,7 @@ #'lwRestore count' -> self lwRestoreCount. #'remember count' -> self rememberCount. #'restore count' -> self restoreCount. + #'token read count' -> self tokenReadCount. } ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCRecognizingSequenceNode.st --- a/compiler/PPCRecognizingSequenceNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCRecognizingSequenceNode.st Thu May 21 14:12:22 2015 +0100 @@ -12,7 +12,7 @@ !PPCRecognizingSequenceNode methodsFor:'accessing'! suffix - ^ #'_fast' + ^ super suffix, '_fast' ! ! !PPCRecognizingSequenceNode methodsFor:'visiting'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCSpecializingVisitor.st --- a/compiler/PPCSpecializingVisitor.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCSpecializingVisitor.st Thu May 21 14:12:22 2015 +0100 @@ -74,6 +74,14 @@ yourself ]. + (node child isKindOf: PPCCharacterNode) ifTrue: [ + self change. + ^ PPCNotCharacterNode new + name: node name; + character: node child character; + yourself + ]. + ^ node "Modified: / 23-04-2015 / 12:02:15 / Jan Vrany " @@ -171,5 +179,22 @@ ]. ^ node +! + +visitTrimmingTokenNode: node + + self visitChildren: node. + + (node child isKindOf: PPCCharacterNode) ifTrue: [ + self change. + ^ PPCTrimmingCharacterTokenNode new + name: node name; + whitespace: node whitespace; + tokenClass: node tokenClass; + child: node child; + yourself + ]. + + ^ node ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCStarCharSetPredicateNode.st --- a/compiler/PPCStarCharSetPredicateNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCStarCharSetPredicateNode.st Thu May 21 14:12:22 2015 +0100 @@ -44,24 +44,6 @@ ^ super hash bitXor: predicate hash ! ! -!PPCStarCharSetPredicateNode methodsFor:'first follow next'! - -firstSets: aFirstDictionary into: aSet suchThat: aBlock - " - First and follow should be performed on the non-specialized tree, i.e. on a tree - with star -> messageNode. Not on myself. - - The reason for that is, that: - - I am terminal - - I am nullable - - This means, I look like epsilon node for the first follow analysis. And epsilons - are ignored in sequences, thus sequence of StarMessagePredicate, Literal - leads to { Literal } as firstSet and not expected { MessagePredicate, Literal } - " - ^ self error: 'Cannot perform first/follow analysis on myself, sorry for that :(' -! ! - !PPCStarCharSetPredicateNode methodsFor:'visiting'! accept: visitor diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCStarMessagePredicateNode.st --- a/compiler/PPCStarMessagePredicateNode.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCStarMessagePredicateNode.st Thu May 21 14:12:22 2015 +0100 @@ -11,10 +11,6 @@ !PPCStarMessagePredicateNode methodsFor:'accessing'! -children - ^ #() -! - firstCharSet ^ PPCharSetPredicate on: [:char | char perform: message ] ! @@ -33,24 +29,6 @@ ^ #starPredicate ! ! -!PPCStarMessagePredicateNode methodsFor:'analysis'! - -firstSets: aFirstDictionary into: aSet suchThat: aBlock - " - First and follow should be performed on the non-specialized tree, i.e. on a tree - with star -> messageNode. Not on myself. - - The reason for that is, that: - - I am terminal - - I am nullable - - This means, I look like epsilon node for the first follow analysis. And epsilons - are ignored in sequences, thus sequence of StarMessagePredicate, Literal - leads to { Literal } as firstSet and not expected { MessagePredicate, Literal } - " - ^ self error: 'Cannot perform first/follow analysis on myself, sorry for that :(' -! ! - !PPCStarMessagePredicateNode methodsFor:'comparing'! = anotherNode diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTokenCodeGenerator.st --- a/compiler/PPCTokenCodeGenerator.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCTokenCodeGenerator.st Thu May 21 14:12:22 2015 +0100 @@ -16,56 +16,135 @@ return := super afterAccept: node retval: retval. return category: 'generated - tokens'. ^ return +! + +fromTokenMode + compiler rememberStrategy: (PPCCompilerTokenizingRememberStrategy on: compiler). + compiler errorStrategy: (PPCCompilerTokenizingErrorStrategy on: compiler). +! + +toTokenMode + compiler rememberStrategy: (PPCCompilerTokenRememberStrategy on: compiler). + compiler errorStrategy: (PPCCompilerTokenErrorStrategy on: compiler). ! ! !PPCTokenCodeGenerator methodsFor:'visiting'! +visitOptionalNode: node + compiler codeStoreValueOf: [ self visit: node child ] intoVariable: self retvalVar. + compiler codeAssign: 'false.' to: 'error'. + compiler codeReturn. +! + visitTokenNode: node - | id startVar endVar | + | id startVar endVar | + "Tokens cannot be inlined, + - their result is true/false + - the return value is always stored in currentTokenValue + - the current token type is always stored in currentTokenType + " + self assert: node isMarkedForInline not. + startVar := compiler allocateTemporaryVariableNamed: 'start'. endVar := compiler allocateTemporaryVariableNamed: 'end'. id := compiler idFor: node. - compiler rememberStrategy: (PPCCompilerTokenRememberStrategy on: compiler). + self toTokenMode. + + compiler add: 'currentTokenType isNil ifFalse: [ ^ currentTokenType == ', id storeString, '].'. + compiler profileTokenRead: id. + + node allNodes size > 2 ifTrue: [ + self addGuard: node ifTrue: nil ifFalse: [ compiler addOnLine: '^ false' ]. + ]. + compiler codeAssign: 'context position + 1.' to: startVar. compiler codeStoreValueOf: [ self visit: node child ] intoVariable: #whatever. - compiler add: 'error ifFalse: [ '. - compiler indent. - compiler codeAssign: 'context position.' to: endVar. + compiler add: 'error ifTrue: [ ^ error := false ].'. + + compiler codeAssign: 'context position.' to: endVar. + + compiler codeTranscriptShow: 'current token type: ', id storeString. + compiler codeAssign: id storeString, '.' to: 'currentTokenType'. + compiler codeAssign: node tokenClass asString, ' on: (context collection) + start: ', startVar, ' + stop: ', endVar, ' + value: nil.' + to: 'currentTokenValue := ', self retvalVar. - compiler codeTranscriptShow: 'current token type: ', id storeString. - compiler codeAssign: id storeString, '.' to: 'currentTokenType'. - compiler codeAssign: node tokenClass asString, ' on: (context collection) - start: ', startVar, ' - stop: ', endVar, ' - value: nil.' - to: 'currentTokenValue := ', self retvalVar. - compiler codeReturn. - compiler dedent. - compiler add: '].'. - compiler rememberStrategy: (PPCCompilerTokenizingRememberStrategy on: compiler). + + compiler codeClearError. + compiler add: '^ true'. + + self fromTokenMode. +! + +visitTrimmingTokenCharacterNode: node + | id | + + "Tokens cannot be inlined, + - their result is true/false + - the return value is always stored in currentTokenValue + - the current token type is always stored in currentTokenType + " + self assert: node isMarkedForInline not. + + id := compiler idFor: node. + self toTokenMode. + + compiler add: 'currentTokenType isNil ifFalse: [ ^ currentTokenType == ', id storeString, '].'. + compiler profileTokenRead: id. + + self addGuard: node ifTrue: nil ifFalse: [ compiler addOnLine: ' ^ false' ]. + + compiler add: 'context next.'. + + compiler codeTranscriptShow: 'current token type: ', id storeString. + compiler codeAssign: id storeString, '.' to: 'currentTokenType'. + compiler codeAssign: node tokenClass asString, ' on: (context collection) + start: context position + stop: context position + value: nil.' + to: 'currentTokenValue := ', self retvalVar. + + compiler addComment: 'Consume Whitespace:'. + compiler codeStoreValueOf: [ self visit: node whitespace ] intoVariable: #whatever. + compiler nl. + + compiler add: '^ true'. + + self fromTokenMode. ! visitTrimmingTokenNode: node - | id startVar endVar | + | id startVar endVar | + + "Tokens cannot be inlined, + - their result is true/false + - the return value is always stored in currentTokenValue + - the current token type is always stored in currentTokenType + " + self assert: node isMarkedForInline not. startVar := compiler allocateTemporaryVariableNamed: 'start'. endVar := compiler allocateTemporaryVariableNamed: 'end'. id := compiler idFor: node. - compiler rememberStrategy: (PPCCompilerTokenRememberStrategy on: compiler). - + self toTokenMode. - compiler addComment: 'Consume Whitespace:'. - compiler codeStoreValueOf: [ self visit: node whitespace ] intoVariable: #whatever. - compiler nl. + compiler add: 'currentTokenType isNil ifFalse: [ ^ currentTokenType == ', id storeString, '].'. + compiler profileTokenRead: id. + + node allNodes size > 2 ifTrue: [ + self addGuard: node ifTrue: nil ifFalse: [ compiler addOnLine: '^ false' ]. + ]. compiler codeAssign: 'context position + 1.' to: startVar. compiler codeStoreValueOf: [ self visit: node child ] intoVariable: #whatever. - compiler add: 'error ifFalse: [ '. - compiler indent. + compiler add: 'error ifTrue: [ ^ error := false ].'. + compiler codeAssign: 'context position.' to: endVar. compiler addComment: 'Consume Whitespace:'. @@ -80,9 +159,10 @@ stop: ', endVar, ' value: nil.' to: 'currentTokenValue := ', self retvalVar. - compiler codeReturn. - compiler dedent. - compiler add: '].' . - compiler rememberStrategy: (PPCCompilerTokenizingRememberStrategy on: compiler). + + compiler codeClearError. + compiler add: '^ true'. + + self fromTokenMode. ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTokenGuard.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCTokenGuard.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,38 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +Object subclass:#PPCTokenGuard + instanceVariableNames:'node' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Guards' +! + +!PPCTokenGuard class methodsFor:'as yet unclassified'! + +on: node + ^ self new + node: node; + yourself +! ! + +!PPCTokenGuard methodsFor:'as yet unclassified'! + +firstToken + self assert: node firstSetWithTokens size = 1. + ^ node firstSetWithTokens anyOne +! + +makesSense + ^ node firstSetWithTokens size = 1 and: [ node firstSetWithTokens anyOne isTokenNode ] +! + +node + ^ node +! + +node: aPPCNode + node := aPPCNode +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTokenizingCodeGenerator.st --- a/compiler/PPCTokenizingCodeGenerator.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCTokenizingCodeGenerator.st Thu May 21 14:12:22 2015 +0100 @@ -26,43 +26,102 @@ tokenGenerator: whatever tokenGenerator := whatever +! + +tokenGuards + "When tokenizing, do not use guards" + ^ arguments guards +! ! + +!PPCTokenizingCodeGenerator methodsFor:'guards'! + +addGuard: node ifTrue: trueBlock ifFalse: falseBlock + | guard id | + guard := PPCTokenGuard on: node. + (self guards not or: guard makesSense not) ifTrue: [ ^ false ]. + + id := compiler idFor: guard firstToken. + + compiler add: 'self ', id asString. + + trueBlock isNil ifFalse: [ + compiler addOnLine: ' ifTrue: ['. + compiler indent. + trueBlock value. + compiler dedent. + falseBlock isNil ifTrue: [ compiler addOnLine: '].' ] + ifFalse: [ compiler add: ']'. ] + ]. + falseBlock isNil ifFalse: [ + compiler addOnLine: ' ifFalse: ['. + compiler indent. + falseBlock value. + compiler dedent. + compiler addOnLine: '].'. + ]. + ^ true ! ! !PPCTokenizingCodeGenerator methodsFor:'visiting'! +visitAndNode: node + | mementoVar currentTokenVar | + + mementoVar := compiler allocateTemporaryVariableNamed: 'memento'. + currentTokenVar := compiler allocateTemporaryVariableNamed: 'currentToken'. + + compiler smartRemember: node child to: mementoVar. + compiler codeAssign: '{ currentTokenValue . currentTokenType }.' to: currentTokenVar. + + compiler codeStoreValueOf: [ self visit: node child ] intoVariable: self retvalVar. + + compiler smartRestore: node child from: mementoVar. + compiler codeAssign: '(', currentTokenVar, ' at: 1).' to: 'currentTokenValue'. + compiler codeAssign: '(', currentTokenVar, ' at: 2).' to: 'currentTokenType'. + + compiler codeReturn. +! + visitChoiceNode: node " true ifTrue: [ ^ super visitChoiceNode: node ]." - "HACK alert: We are inside token..." + | possibleError | + possibleError := true. - (node firstSetWithTokens contains: [ :e | e isTokenNode not ]) ifTrue: [ ^ super visitChoiceNode: node ]. - node children do: [ :child | | tokenMethodName | - child firstSetWithTokens do: [ :first | - "For each child, for each first compile this:" - tokenMethodName := compiler idFor: first. - compiler add: '(self currentTokenTypeIs: ', tokenMethodName storeString, ')'. - compiler addOnLine: ' ifTrue: ['. - compiler indent. + child acceptsEpsilon ifTrue: [ + possibleError := false. compiler codeStoreValueOf: [ self visit: child ] intoVariable: self retvalVar. - compiler add: 'error ifFalse: ['. - compiler indent. - compiler codeReturn: self retvalVar. - compiler dedent. - compiler add: '] ifTrue: ['. - compiler indent. - compiler codeClearError. - compiler codeAssign: 'nil.' to: 'currentTokenType'. - compiler add: 'context position: currentTokenValue start - 1.'. - compiler dedent. - compiler add: ']'. - compiler dedent. - compiler add: '].' + compiler codeReturn + ] ifFalse: [ + child firstSetWithTokens do: [ :first | + "For each child, for each first compile this:" + + tokenMethodName := compiler idFor: first. + compiler add: '(self ', tokenMethodName asString, ')'. + compiler addOnLine: ' ifTrue: ['. + compiler indent. + compiler codeStoreValueOf: [ self visit: child ] intoVariable: self retvalVar. + compiler add: 'error ifFalse: ['. + compiler indent. + compiler codeReturn: self retvalVar. + compiler dedent. + compiler add: '] ifTrue: ['. + compiler indent. + compiler codeClearError. + compiler codeAssign: 'nil.' to: 'currentTokenType'. + compiler add: 'context position: currentTokenValue start - 1.'. + compiler dedent. + compiler add: ']'. + compiler dedent. + compiler add: '].' + ] ] ]. - - compiler codeError: 'no choice found'. + possibleError ifTrue: [ + compiler codeError: 'no choice found'. + ] "Modified: / 10-05-2015 / 07:37:53 / Jan Vrany " ! @@ -75,23 +134,19 @@ | firstSet | firstSet := child firstSetWithTokens. self assert: firstSet size = 1. - dictionary at: child - put: firstSet anyOne. - + dictionary at: child put: firstSet anyOne. ]. "Tokens are unique" self assert: dictionary values asSet size = node children size. -" currentTokenVar := compiler allocateTemporaryVariableNamed: 'currentToken'. - compiler codeAssign: 'self currentTokenType.' to: currentTokenVar. -" node children do: [ :child | + node children do: [ :child | | tokenMethodName | tokenMethodName := compiler idFor: (dictionary at: child). - compiler add: '(self currentTokenTypeIs: ', tokenMethodName storeString, ')'. + compiler add: '(self ', tokenMethodName asString, ')'. compiler addOnLine: ' ifTrue: ['. compiler indent. - compiler codeStoreValueOf: [ self visit: child ] intoVariable: self retvalVar. - compiler codeReturn: self retvalVar. + compiler codeStoreValueOf: [ self visit: child ] intoVariable: self retvalVar. + compiler codeReturn: self retvalVar. compiler dedent. compiler add: '].' ]. @@ -116,7 +171,22 @@ ! visitTokenConsumeNode: node + | id | + id := (compiler idFor: node child). + compiler add: 'self ', id asString, ' ifTrue: ['. + compiler indent. + compiler codeAssign: 'nil.' to: 'currentTokenType'. + compiler codeReturn: 'currentTokenValue'. + compiler dedent. + compiler add: '] ifFalse: ['. + compiler indent. + compiler codeError: id asString, ' expected'. + compiler dedent. + compiler add: '].'. + +" compiler codeReturn: 'self consume: ', (compiler idFor: node child) storeString, '.' +" ! visitTokenNode: node @@ -127,7 +197,6 @@ self visit: node tokenizer. self visit: node whitespace. -" compiler codeNextToken." compiler codeHaltIfShiftPressed. compiler codeStoreValueOf: [ self visit: node parser ] intoVariable: self retvalVar. compiler codeReturn. diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTokenizingCompiler.st --- a/compiler/PPCTokenizingCompiler.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPCTokenizingCompiler.st Thu May 21 14:12:22 2015 +0100 @@ -3,14 +3,26 @@ "{ NameSpace: Smalltalk }" PPCCompiler subclass:#PPCTokenizingCompiler - instanceVariableNames:'rememberStrategy' + instanceVariableNames:'rememberStrategy errorStrategy' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Core' + category:'PetitCompiler-Compiler' ! !PPCTokenizingCompiler methodsFor:'code generation'! +codeClearError + errorStrategy codeClearError +! + +codeError + errorStrategy codeError +! + +codeError: message + errorStrategy codeError: message. +! + smartRemember: parser to: variableName rememberStrategy smartRemember: parser to: variableName ! @@ -25,6 +37,14 @@ ^ compiledParserSuperclass ifNil: [ PPTokenizingCompiledParser ] ! +errorStrategy + ^ errorStrategy ifNil: [ PPCCompilerTokenizingErrorStrategy on: self ] +! + +errorStrategy: whatever + errorStrategy := whatever +! + rememberStrategy ^ rememberStrategy ifNil: [ PPCCompilerTokenizingRememberStrategy on: self ] ! @@ -40,21 +60,3 @@ rememberStrategy := PPCCompilerTokenizingRememberStrategy on: self ! ! -!PPCTokenizingCompiler methodsFor:'remember strategy'! - -toNormalRememberStrategy - " - When backtracking, currentTokenValue and currentTokenType has - to be restored - " - rememberStrategy := PPCCompilerTokenizingRememberStrategy on: self -! - -toTokenRememberStrategy - " - We dont have to remember currentTokenType and currentTokenValue, - while parsing token - " - rememberStrategy := PPCCompilerTokenRememberStrategy on: self -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTokenizingConfiguration.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCTokenizingConfiguration.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,61 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCConfiguration subclass:#PPCTokenizingConfiguration + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Core' +! + +!PPCTokenizingConfiguration methodsFor:'compiling'! + +invokePhases + self toPPCIr. + self createTokens. + self cacheFirstFollow. + self createLL1Choices. + self tokenize. + self merge. + self specialize. + self createRecognizingComponents. + self specialize. + self inline. + self merge. + self check. + self cacheFirstFollow. + self generate. +! ! + +!PPCTokenizingConfiguration methodsFor:'hooks'! + +codeCompilerOn: args + ^ PPCTokenizingCompiler on: args +! + +codeGeneratorVisitorOn: compiler + ^ PPCTokenizingCodeGenerator on: compiler +! ! + +!PPCTokenizingConfiguration methodsFor:'phases'! + +createLL1Choices + ir := PPCLL1Visitor new + arguments: arguments; + visit: ir. + self remember: #LL1 +! + +tokenize + " + This will try transform the parser into the tokenizing parser + " + arguments tokenize ifFalse: [ ^ self ] . + + ir := PPCTokenizingVisitor new + arguments: arguments; + visit: ir. + self remember: #tokenize +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPCTrimmingCharacterTokenNode.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/PPCTrimmingCharacterTokenNode.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,17 @@ +"{ Package: 'stx:goodies/petitparser/compiler' }" + +"{ NameSpace: Smalltalk }" + +PPCTrimmingTokenNode subclass:#PPCTrimmingCharacterTokenNode + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Nodes' +! + +!PPCTrimmingCharacterTokenNode methodsFor:'as yet unclassified'! + +accept: visitor + ^ visitor visitTrimmingTokenCharacterNode: self +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/PPTokenizingCompiledParser.st --- a/compiler/PPTokenizingCompiledParser.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/PPTokenizingCompiledParser.st Thu May 21 14:12:22 2015 +0100 @@ -12,15 +12,12 @@ !PPTokenizingCompiledParser methodsFor:'tokenizing'! consume: tokenType - (self currentTokenTypeIs: tokenType) ifTrue: [ - | retval | - retval := currentTokenValue. + (self perform: tokenType) ifTrue: [ currentTokenType := nil. - ^ retval - ] ifFalse: [ - "self error: 'expected: ', tokenType storeString, ' got ', currentTokenType storeString." - self error. - ] + ^ currentTokenValue. + ]. + "self error: 'expected: ', tokenType storeString, ' got ', currentTokenType storeString." + self error. ! consumeWhitespace @@ -34,14 +31,11 @@ currentTokenTypeIs: tokenType "if the type is read" + self halt: 'deprecated'. currentTokenType isNil ifFalse: [ ^ currentTokenType = tokenType ]. "if not, try to read the token" - self perform: tokenType. - error ifTrue: [ - ^ error := false. - ]. - ^ true + ^ self perform: tokenType. ! currentTokenValue @@ -65,7 +59,7 @@ self consumeWhitespace. retval := self perform: startSymbol. - self consumeWhitespace. +" self consumeWhitespace." (retval isPetitFailure) ifTrue: [ aPPContext noteFailure: failure ]. error ifTrue: [ aPPContext noteFailure: failure. retval := failure ]. diff -r 4751c407bb40 -r f6d77fee9811 compiler/abbrev.stc --- a/compiler/abbrev.stc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/abbrev.stc Thu May 21 14:12:22 2015 +0100 @@ -4,42 +4,44 @@ PPCArguments PPCArguments stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCBridge PPCBridge stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCCompiledMethod PPCCompiledMethod stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 -PPCCompiler PPCCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 -PPCCompilerTokenRememberStrategy PPCCompilerTokenRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 -PPCCompilerTokenizingRememberStrategy PPCCompilerTokenizingRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCCompiler PPCCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 +PPCCompilerTokenErrorStrategy PPCCompilerTokenErrorStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 +PPCCompilerTokenRememberStrategy PPCCompilerTokenRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 +PPCCompilerTokenizingErrorStrategy PPCCompilerTokenizingErrorStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 +PPCCompilerTokenizingRememberStrategy PPCCompilerTokenizingRememberStrategy stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 PPCConfiguration PPCConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCContext PPCContext stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 PPCContextMemento PPCContextMemento stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 -PPCGuard PPCGuard stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCGuard PPCGuard stx:goodies/petitparser/compiler 'PetitCompiler-Guards' 0 PPCMethod PPCMethod stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCNode PPCNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNodeVisitor PPCNodeVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCPluggableConfiguration PPCPluggableConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCTokenGuard PPCTokenGuard stx:goodies/petitparser/compiler 'PetitCompiler-Guards' 0 PPCompiledParser PPCompiledParser stx:goodies/petitparser/compiler 'PetitCompiler-Parsers' 4 stx_goodies_petitparser_compiler stx_goodies_petitparser_compiler stx:goodies/petitparser/compiler '* Projects & Packages *' 3 -PPCAbstractCharacterNode PPCAbstractCharacterNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAbstractLiteralNode PPCAbstractLiteralNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAbstractPredicateNode PPCAbstractPredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAnyNode PPCAnyNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 +PPCCharacterNode PPCCharacterNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCCodeGenerator PPCCodeGenerator stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCDelegateNode PPCDelegateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCEndOfFileNode PPCEndOfFileNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCInlinedMethod PPCInlinedMethod stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCInliningVisitor PPCInliningVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 -PPCLL1Configuration PPCLL1Configuration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCListNode PPCListNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNilNode PPCNilNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCPluggableNode PPCPluggableNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCProfilingContext PPCProfilingContext stx:goodies/petitparser/compiler 'PetitCompiler-Context' 0 PPCRewritingVisitor PPCRewritingVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 -PPCTokenizingCompiler PPCTokenizingCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 +PPCTokenizingCompiler PPCTokenizingCompiler stx:goodies/petitparser/compiler 'PetitCompiler-Compiler' 0 +PPCTokenizingConfiguration PPCTokenizingConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCUniversalConfiguration PPCUniversalConfiguration stx:goodies/petitparser/compiler 'PetitCompiler-Core' 0 PPCUnknownNode PPCUnknownNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPTokenizingCompiledParser PPTokenizingCompiledParser stx:goodies/petitparser/compiler 'PetitCompiler-Parsers' 4 PPCAbstractActionNode PPCAbstractActionNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCAndNode PPCAndNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCCharSetPredicateNode PPCCharSetPredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 -PPCCharacterNode PPCCharacterNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCChoiceNode PPCChoiceNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCCopyVisitor PPCCopyVisitor stx:goodies/petitparser/compiler 'PetitCompiler-Visitors' 0 PPCEndOfInputNode PPCEndOfInputNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 @@ -50,6 +52,7 @@ PPCMessagePredicateNode PPCMessagePredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNegateNode PPCNegateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNotCharSetPredicateNode PPCNotCharSetPredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 +PPCNotCharacterNode PPCNotCharacterNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNotLiteralNode PPCNotLiteralNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNotMessagePredicateNode PPCNotMessagePredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCNotNode PPCNotNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 @@ -82,5 +85,6 @@ PPCSymbolActionNode PPCSymbolActionNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTokenChoiceNode PPCTokenChoiceNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTrimNode PPCTrimNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 +PPCTrimmingCharacterTokenNode PPCTrimmingCharacterTokenNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTokenStarMessagePredicateNode PPCTokenStarMessagePredicateNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 PPCTokenStarSeparatorNode PPCTokenStarSeparatorNode stx:goodies/petitparser/compiler 'PetitCompiler-Nodes' 0 diff -r 4751c407bb40 -r f6d77fee9811 compiler/bc.mak --- a/compiler/bc.mak Tue May 12 01:24:03 2015 +0100 +++ b/compiler/bc.mak Thu May 21 14:12:22 2015 +0100 @@ -82,7 +82,9 @@ $(OUTDIR)PPCBridge.$(O) PPCBridge.$(H): PPCBridge.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompiledMethod.$(O) PPCCompiledMethod.$(H): PPCCompiledMethod.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompiler.$(O) PPCCompiler.$(H): PPCCompiler.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCCompilerTokenErrorStrategy.$(O) PPCCompilerTokenErrorStrategy.$(H): PPCCompilerTokenErrorStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilerTokenRememberStrategy.$(O) PPCCompilerTokenRememberStrategy.$(H): PPCCompilerTokenRememberStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCCompilerTokenizingErrorStrategy.$(O) PPCCompilerTokenizingErrorStrategy.$(H): PPCCompilerTokenizingErrorStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCompilerTokenizingRememberStrategy.$(O) PPCCompilerTokenizingRememberStrategy.$(H): PPCCompilerTokenizingRememberStrategy.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCConfiguration.$(O) PPCConfiguration.$(H): PPCConfiguration.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCContext.$(O) PPCContext.$(H): PPCContext.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(STCHDR) @@ -92,31 +94,31 @@ $(OUTDIR)PPCNode.$(O) PPCNode.$(H): PPCNode.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeVisitor.$(O) PPCNodeVisitor.$(H): PPCNodeVisitor.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableConfiguration.$(O) PPCPluggableConfiguration.$(H): PPCPluggableConfiguration.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenGuard.$(O) PPCTokenGuard.$(H): PPCTokenGuard.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCompiledParser.$(O) PPCompiledParser.$(H): PPCompiledParser.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler.$(O) stx_goodies_petitparser_compiler.$(H): stx_goodies_petitparser_compiler.st $(INCLUDE_TOP)\stx\libbasic\LibraryDefinition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\ProjectDefinition.$(H) $(STCHDR) -$(OUTDIR)PPCAbstractCharacterNode.$(O) PPCAbstractCharacterNode.$(H): PPCAbstractCharacterNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractLiteralNode.$(O) PPCAbstractLiteralNode.$(H): PPCAbstractLiteralNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractPredicateNode.$(O) PPCAbstractPredicateNode.$(H): PPCAbstractPredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAnyNode.$(O) PPCAnyNode.$(H): PPCAnyNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCCharacterNode.$(O) PPCCharacterNode.$(H): PPCCharacterNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCodeGenerator.$(O) PPCCodeGenerator.$(H): PPCCodeGenerator.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCDelegateNode.$(O) PPCDelegateNode.$(H): PPCDelegateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfFileNode.$(O) PPCEndOfFileNode.$(H): PPCEndOfFileNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInlinedMethod.$(O) PPCInlinedMethod.$(H): PPCInlinedMethod.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCMethod.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitor.$(O) PPCInliningVisitor.$(H): PPCInliningVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1Configuration.$(O) PPCLL1Configuration.$(H): PPCLL1Configuration.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCConfiguration.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCListNode.$(O) PPCListNode.$(H): PPCListNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNilNode.$(O) PPCNilNode.$(H): PPCNilNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCPluggableNode.$(O) PPCPluggableNode.$(H): PPCPluggableNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCProfilingContext.$(O) PPCProfilingContext.$(H): PPCProfilingContext.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCContext.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(STCHDR) $(OUTDIR)PPCRewritingVisitor.$(O) PPCRewritingVisitor.$(H): PPCRewritingVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCompiler.$(O) PPCTokenizingCompiler.$(H): PPCTokenizingCompiler.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCompiler.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenizingConfiguration.$(O) PPCTokenizingConfiguration.$(H): PPCTokenizingConfiguration.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCConfiguration.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUniversalConfiguration.$(O) PPCUniversalConfiguration.$(H): PPCUniversalConfiguration.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCConfiguration.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCUnknownNode.$(O) PPCUnknownNode.$(H): PPCUnknownNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPTokenizingCompiledParser.$(O) PPTokenizingCompiledParser.$(H): PPTokenizingCompiledParser.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCompiledParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAbstractActionNode.$(O) PPCAbstractActionNode.$(H): PPCAbstractActionNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCAndNode.$(O) PPCAndNode.$(H): PPCAndNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCharSetPredicateNode.$(O) PPCCharSetPredicateNode.$(H): PPCCharSetPredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCCharacterNode.$(O) PPCCharacterNode.$(H): PPCCharacterNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractCharacterNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCChoiceNode.$(O) PPCChoiceNode.$(H): PPCChoiceNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCCopyVisitor.$(O) PPCCopyVisitor.$(H): PPCCopyVisitor.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNodeVisitor.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCRewritingVisitor.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCEndOfInputNode.$(O) PPCEndOfInputNode.$(H): PPCEndOfInputNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -127,6 +129,7 @@ $(OUTDIR)PPCMessagePredicateNode.$(O) PPCMessagePredicateNode.$(H): PPCMessagePredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNegateNode.$(O) PPCNegateNode.$(H): PPCNegateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNotCharSetPredicateNode.$(O) PPCNotCharSetPredicateNode.$(H): PPCNotCharSetPredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCNotCharacterNode.$(O) PPCNotCharacterNode.$(H): PPCNotCharacterNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCCharacterNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNotLiteralNode.$(O) PPCNotLiteralNode.$(H): PPCNotLiteralNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractLiteralNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNotMessagePredicateNode.$(O) PPCNotMessagePredicateNode.$(H): PPCNotMessagePredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractPredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNotNode.$(O) PPCNotNode.$(H): PPCNotNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) @@ -159,6 +162,7 @@ $(OUTDIR)PPCSymbolActionNode.$(O) PPCSymbolActionNode.$(H): PPCSymbolActionNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCAbstractActionNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenChoiceNode.$(O) PPCTokenChoiceNode.$(H): PPCTokenChoiceNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCChoiceNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTrimNode.$(O) PPCTrimNode.$(H): PPCTrimNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCSequenceNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCTrimmingCharacterTokenNode.$(O) PPCTrimmingCharacterTokenNode.$(H): PPCTrimmingCharacterTokenNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCListNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCTrimmingTokenNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarMessagePredicateNode.$(O) PPCTokenStarMessagePredicateNode.$(H): PPCTokenStarMessagePredicateNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenStarSeparatorNode.$(O) PPCTokenStarSeparatorNode.$(H): PPCTokenStarSeparatorNode.st $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCDelegateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCStarNode.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\compiler\PPCTokenStarMessagePredicateNode.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)extensions.$(O): extensions.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPActionParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPAndParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCharSetPredicate.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPChoiceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPContext.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEndOfInputParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPEpsilonParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFailure.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPFlattenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPListParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPLiteralSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPNotParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPOptionalParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPluggableParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPossessiveRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateObjectParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPPredicateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPRepeatingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPSequenceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPStream.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPToken.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPTrimmingParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java\PPJavaWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkGrammar.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkTokenParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk\PPSmalltalkWhitespaceParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Character.$(H) $(INCLUDE_TOP)\stx\libbasic\Magnitude.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\PeekableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\PositionableStream.$(H) $(INCLUDE_TOP)\stx\libbasic\ReadStream.$(H) $(INCLUDE_TOP)\stx\libbasic\Stream.$(H) $(INCLUDE_TOP)\stx\libbasic\UndefinedObject.$(H) $(STCHDR) diff -r 4751c407bb40 -r f6d77fee9811 compiler/benchmarks/Make.proto --- a/compiler/benchmarks/Make.proto Tue May 12 01:24:03 2015 +0100 +++ b/compiler/benchmarks/Make.proto Thu May 21 14:12:22 2015 +0100 @@ -34,7 +34,7 @@ # add the path(es) here:, # ********** OPTIONAL: MODIFY the next lines *** # LOCALINCLUDES=-Ifoo -Ibar -LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser -I$(INCLUDE_TOP)/stx/libbasic +LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests/extras -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/refactoryBrowser/parser -I$(INCLUDE_TOP)/stx/libbasic # if you need any additional defines for embedded C code, diff -r 4751c407bb40 -r f6d77fee9811 compiler/benchmarks/PPCBenchmark.st --- a/compiler/benchmarks/PPCBenchmark.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/benchmarks/PPCBenchmark.st Thu May 21 14:12:22 2015 +0100 @@ -4,7 +4,7 @@ Object subclass:#PPCBenchmark instanceVariableNames:'sources report contextClass compile parser context input - configuration' + configuration profile' classVariableNames:'' poolDictionaries:'' category:'PetitCompiler-Benchmarks-Core' @@ -57,7 +57,8 @@ measure: aParser on: anInput name: aString | time result p | - context := self createContext. + self halt: 'deprecated?'. + context := self context. p := compile ifTrue: [ aParser end compile @@ -92,12 +93,16 @@ " ! -reportInput: input time: time name: name +reportInput: anInput time: time name: name | size | - size := input inject: 0 into: [:r :e | r + e size ]. + size := anInput inject: 0 into: [:r :e | r + e size ]. Transcript crShow: 'Size: ', size asString. Transcript crShow: name, ' time: ', time asString. Transcript crShow: 'Time per character: ', (time / size * 1000.0) asString, ' microseconds'. + + (context isKindOf: PPCProfilingContext) ifTrue: [ + context inspect. + ] ! ! !PPCBenchmark methodsFor:'benchmarks'! @@ -130,85 +135,6 @@ time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. self reportInput: input time: time name: 'Java Syntax Compiled'. -! - -benchmarkOpalCompiler - | parser time input | - parser := OpalCompiler new. - input := sources smalltalkSourcesBig. - time := [ input do: [ :source | parser parse: source ]] timeToRun asMilliseconds. - - self reportInput: input time: time name: 'Opal' -! - -benchmarkSmalltalkGrammar - | time | - - self setupSmalltalkGrammar. - - time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. - - self reportInput: input time: time name: 'Smalltalk Grammar'. -! - -benchmarkSmalltalkGrammarCompiled - | time | - - self setupSmalltalkGrammarCompiled. - - time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. - - self reportInput: input time: time name: 'Compiled Smalltalk Grammar'. - -" - size := input inject: 0 into: [:r :e | r + e size ]. - Transcript crShow: 'Compiled Grammar time: ', time asString. - Transcript crShow: 'Time per character: ', (time / size * 1000.0) asString, ' microseconds'. -" -! - -benchmarkSmalltalkGrammarTokenized - | time | - - self setupSmalltalkGrammarTokenized. - - time := [ input do: [ :source | - parser parse: source withContext: context ] - ] timeToRun asMilliSeconds. - - self reportInput: input time: time name: 'Tokenized Smalltalk Grammar'. - -" - size := input inject: 0 into: [:r :e | r + e size ]. - Transcript crShow: 'Compiled Grammar time: ', time asString. - Transcript crShow: 'Time per character: ', (time / size * 1000.0) asString, ' microseconds'. -" -! - -benchmarkSmalltalkParser - | time | - parser := PPSmalltalkParser new. - context := PPContext new. - context initializeFor: parser. - input := sources smalltalkSourcesBig. - - time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. - - self reportInput: input time: time name: 'Smalltalk Parser'. -! - -benchmarkSmalltalkParserCompiled - | time | - - configuration := PPCConfiguration default. - parser := PPSmalltalkParser new compileWithConfiguration: configuration. - context := PPCContext new. - context initializeFor: parser. - input := sources smalltalkSourcesBig. - - time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. - - self reportInput: input time: time name: 'Smalltalk Parser Compiled'. ! ! !PPCBenchmark methodsFor:'benchmarks - expression grammar'! @@ -297,6 +223,87 @@ self measure: parser on: (sources changesSized: 1000*1000) name: #token. ! ! +!PPCBenchmark methodsFor:'benchmarks - smalltalk'! + +benchmarkOpalCompiler + | parser time input | + parser := OpalCompiler new. + input := sources smalltalkSourcesBig. + time := [ input do: [ :source | parser parse: source ]] timeToRun asMilliseconds. + + self reportInput: input time: time name: 'Opal' +! + +benchmarkSmalltalkGrammar + | time | + + self setupSmalltalkGrammar. + + time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. + + self reportInput: input time: time name: 'Smalltalk Grammar'. +! + +benchmarkSmalltalkGrammarCompiled + | time | + + self setupSmalltalkGrammarCompiled. + + time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. + + self reportInput: input time: time name: 'Compiled Smalltalk Grammar'. + +" + size := input inject: 0 into: [:r :e | r + e size ]. + Transcript crShow: 'Compiled Grammar time: ', time asString. + Transcript crShow: 'Time per character: ', (time / size * 1000.0) asString, ' microseconds'. +" +! + +benchmarkSmalltalkGrammarTokenized + | time | + + self setupSmalltalkGrammarTokenized. + + time := [ input do: [ :source | + parser parse: source withContext: context ] + ] timeToRun asMilliSeconds. + + self reportInput: input time: time name: 'Tokenized Smalltalk Grammar'. + +" + size := input inject: 0 into: [:r :e | r + e size ]. + Transcript crShow: 'Compiled Grammar time: ', time asString. + Transcript crShow: 'Time per character: ', (time / size * 1000.0) asString, ' microseconds'. +" +! + +benchmarkSmalltalkParser + | time | + parser := PPSmalltalkParser new. + context := PPContext new. + context initializeFor: parser. + input := sources smalltalkSourcesBig. + + time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. + + self reportInput: input time: time name: 'Smalltalk Parser'. +! + +benchmarkSmalltalkParserCompiled + | time | + + configuration := PPCConfiguration default. + parser := PPSmalltalkParser new compileWithConfiguration: configuration. + context := PPCContext new. + context initializeFor: parser. + input := sources smalltalkSourcesBig. + + time := [ input do: [ :source | parser parse: source withContext: context ]] timeToRun asMilliSeconds. + + self reportInput: input time: time name: 'Smalltalk Parser Compiled'. +! ! + !PPCBenchmark methodsFor:'benchmarks-CalipeL'! benchmarkExpressionGrammarC @@ -381,7 +388,7 @@ !PPCBenchmark methodsFor:'intitialization'! -createContext +context ^ contextClass new ! @@ -389,7 +396,9 @@ super initialize. sources := PPCResources current. contextClass := PPCContext. + compile := false. + profile := false. ! ! !PPCBenchmark methodsFor:'meta'! @@ -424,12 +433,19 @@ ^ #letter asParser, (#letter asParser / #digit asParser) star trim ! ! +!PPCBenchmark methodsFor:'profiling'! + +profile + contextClass := PPCProfilingContext. + profile := true. +! ! + !PPCBenchmark methodsFor:'setup & teardown'! setupExpressionGrammar parser := PPExpressionGrammar new. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources expressionSourcesMedium. ! @@ -439,17 +455,17 @@ configuration := PPCConfiguration universal. configuration arguments name: #PPCompiledExpressionGrammar. parser := PPExpressionGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources expressionSourcesMedium. ! setupExpressionGrammarTokenized - configuration := PPCConfiguration LL1. - configuration arguments name: #PPTokenizedLL1ExpressionGrammar. - parser := PPLL1ExpressionGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + configuration := PPCConfiguration tokenizing. + configuration arguments name: #PPTokenizedExpressionGrammar. + parser := PPExpressionGrammar new compileWithConfiguration: configuration. + context := self context. context initializeFor: parser. input := sources expressionSourcesMedium. ! @@ -457,14 +473,14 @@ setupJavaSyntax parser := PPJavaSyntax new. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources javaSourcesBig. ! setupJavaSyntaxCompiled parser := PPJavaSyntax new compile. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources javaSourcesBig. @@ -478,7 +494,7 @@ setupLL1ExpressionGrammar parser := PPLL1ExpressionGrammar new. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources expressionSourcesBig. ! @@ -488,17 +504,17 @@ configuration := PPCConfiguration universal. configuration arguments name: #PPCompiledLL1ExpressionGrammar. parser := PPLL1ExpressionGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources expressionSourcesBig. ! setupLL1ExpressionGrammarTokenized - configuration := PPCConfiguration universal. + configuration := PPCConfiguration tokenizing. configuration arguments name: #PPTokenizedLL1ExpressionGrammar. parser := PPLL1ExpressionGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources expressionSourcesBig. ! @@ -511,7 +527,7 @@ setupSmalltalkGrammar parser := PPSmalltalkGrammar new. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources smalltalkSourcesBig. ! @@ -520,26 +536,39 @@ configuration := PPCConfiguration universal. configuration arguments name: #PPCompiledSmalltalkGrammar. + configuration arguments profile: profile. + parser := PPSmalltalkGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. - input := sources smalltalkSourcesBig. + + profile ifTrue: [ + input := sources smalltalkSourcesSmall. + ] ifFalse: [ + input := sources smalltalkSourcesBig. + ] ! setupSmalltalkGrammarTokenized - configuration := PPCConfiguration LL1. + configuration := PPCConfiguration tokenizing. configuration arguments name: #PPTokenizedSmalltalkGrammar. + configuration arguments profile: profile. + parser := PPSmalltalkGrammar new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. - input := sources smalltalkSourcesBig. + profile ifTrue: [ + input := sources smalltalkSourcesSmall. + ] ifFalse: [ + input := sources smalltalkSourcesBig. + ] ! setupSmalltalkParser parser := PPSmalltalkParser new. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources smalltalkSourcesBig. ! @@ -548,16 +577,16 @@ configuration := PPCConfiguration universal. parser := PPSmalltalkParser new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources smalltalkSourcesBig. ! setupSmalltalkParserTokenized - configuration := PPCConfiguration LL1. + configuration := PPCConfiguration tokenizing. parser := PPSmalltalkParser new compileWithConfiguration: configuration. - context := PPCContext new. + context := self context. context initializeFor: parser. input := sources smalltalkSourcesBig. ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/benchmarks/bc.mak --- a/compiler/benchmarks/bc.mak Tue May 12 01:24:03 2015 +0100 +++ b/compiler/benchmarks/bc.mak Thu May 21 14:12:22 2015 +0100 @@ -35,7 +35,7 @@ -LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser -I$(INCLUDE_TOP)\stx\libbasic +LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests\extras -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\refactoryBrowser\parser -I$(INCLUDE_TOP)\stx\libbasic LOCALDEFINES= STCLOCALOPT=-package=$(PACKAGE) -I. $(LOCALINCLUDES) -headerDir=. $(STCLOCALOPTIMIZATIONS) $(STCWARNINGS) $(LOCALDEFINES) -varPrefix=$(LIBNAME) diff -r 4751c407bb40 -r f6d77fee9811 compiler/benchmarks/stx_goodies_petitparser_compiler_benchmarks.st --- a/compiler/benchmarks/stx_goodies_petitparser_compiler_benchmarks.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/benchmarks/stx_goodies_petitparser_compiler_benchmarks.st Thu May 21 14:12:22 2015 +0100 @@ -73,7 +73,6 @@ ^ #( #'stx:goodies/petitparser' "PPContext - referenced by PPCBenchmark>>benchmarkSmalltalkParser" #'stx:goodies/petitparser/compiler' "PPCConfiguration - referenced by PPCBenchmark>>benchmarkSmalltalkParserCompiled" - #'stx:goodies/petitparser/compiler/tests' "PPExpressionGrammar - referenced by PPCBenchmark>>setupExpressionGrammar" #'stx:goodies/petitparser/compiler/tests/extras' "PPCResources - referenced by PPCBenchmark>>initialize" #'stx:goodies/petitparser/parsers/java' "PPJavaSyntax - referenced by PPCBenchmark>>benchmarkJavaSyntax" #'stx:goodies/petitparser/parsers/smalltalk' "PPSmalltalkGrammar - referenced by PPCBenchmark>>setupSmalltalkGrammar" diff -r 4751c407bb40 -r f6d77fee9811 compiler/extensions.st --- a/compiler/extensions.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/extensions.st Thu May 21 14:12:22 2015 +0100 @@ -399,9 +399,9 @@ !PPParser methodsFor:'*petitcompiler'! -compileLL1 +compileTokenizing | configuration | - configuration := PPCConfiguration LL1. + configuration := PPCConfiguration tokenizing. ^ self compileWithConfiguration: configuration ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/libInit.cc --- a/compiler/libInit.cc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/libInit.cc Thu May 21 14:12:22 2015 +0100 @@ -31,7 +31,9 @@ _PPCBridge_Init(pass,__pRT__,snd); _PPCCompiledMethod_Init(pass,__pRT__,snd); _PPCCompiler_Init(pass,__pRT__,snd); +_PPCCompilerTokenErrorStrategy_Init(pass,__pRT__,snd); _PPCCompilerTokenRememberStrategy_Init(pass,__pRT__,snd); +_PPCCompilerTokenizingErrorStrategy_Init(pass,__pRT__,snd); _PPCCompilerTokenizingRememberStrategy_Init(pass,__pRT__,snd); _PPCConfiguration_Init(pass,__pRT__,snd); _PPCContext_Init(pass,__pRT__,snd); @@ -41,31 +43,31 @@ _PPCNode_Init(pass,__pRT__,snd); _PPCNodeVisitor_Init(pass,__pRT__,snd); _PPCPluggableConfiguration_Init(pass,__pRT__,snd); +_PPCTokenGuard_Init(pass,__pRT__,snd); _PPCompiledParser_Init(pass,__pRT__,snd); _stx_137goodies_137petitparser_137compiler_Init(pass,__pRT__,snd); -_PPCAbstractCharacterNode_Init(pass,__pRT__,snd); _PPCAbstractLiteralNode_Init(pass,__pRT__,snd); _PPCAbstractPredicateNode_Init(pass,__pRT__,snd); _PPCAnyNode_Init(pass,__pRT__,snd); +_PPCCharacterNode_Init(pass,__pRT__,snd); _PPCCodeGenerator_Init(pass,__pRT__,snd); _PPCDelegateNode_Init(pass,__pRT__,snd); _PPCEndOfFileNode_Init(pass,__pRT__,snd); _PPCInlinedMethod_Init(pass,__pRT__,snd); _PPCInliningVisitor_Init(pass,__pRT__,snd); -_PPCLL1Configuration_Init(pass,__pRT__,snd); _PPCListNode_Init(pass,__pRT__,snd); _PPCNilNode_Init(pass,__pRT__,snd); _PPCPluggableNode_Init(pass,__pRT__,snd); _PPCProfilingContext_Init(pass,__pRT__,snd); _PPCRewritingVisitor_Init(pass,__pRT__,snd); _PPCTokenizingCompiler_Init(pass,__pRT__,snd); +_PPCTokenizingConfiguration_Init(pass,__pRT__,snd); _PPCUniversalConfiguration_Init(pass,__pRT__,snd); _PPCUnknownNode_Init(pass,__pRT__,snd); _PPTokenizingCompiledParser_Init(pass,__pRT__,snd); _PPCAbstractActionNode_Init(pass,__pRT__,snd); _PPCAndNode_Init(pass,__pRT__,snd); _PPCCharSetPredicateNode_Init(pass,__pRT__,snd); -_PPCCharacterNode_Init(pass,__pRT__,snd); _PPCChoiceNode_Init(pass,__pRT__,snd); _PPCCopyVisitor_Init(pass,__pRT__,snd); _PPCEndOfInputNode_Init(pass,__pRT__,snd); @@ -76,6 +78,7 @@ _PPCMessagePredicateNode_Init(pass,__pRT__,snd); _PPCNegateNode_Init(pass,__pRT__,snd); _PPCNotCharSetPredicateNode_Init(pass,__pRT__,snd); +_PPCNotCharacterNode_Init(pass,__pRT__,snd); _PPCNotLiteralNode_Init(pass,__pRT__,snd); _PPCNotMessagePredicateNode_Init(pass,__pRT__,snd); _PPCNotNode_Init(pass,__pRT__,snd); @@ -108,6 +111,7 @@ _PPCSymbolActionNode_Init(pass,__pRT__,snd); _PPCTokenChoiceNode_Init(pass,__pRT__,snd); _PPCTrimNode_Init(pass,__pRT__,snd); +_PPCTrimmingCharacterTokenNode_Init(pass,__pRT__,snd); _PPCTokenStarMessagePredicateNode_Init(pass,__pRT__,snd); _PPCTokenStarSeparatorNode_Init(pass,__pRT__,snd); diff -r 4751c407bb40 -r f6d77fee9811 compiler/stx_goodies_petitparser_compiler.st --- a/compiler/stx_goodies_petitparser_compiler.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/stx_goodies_petitparser_compiler.st Thu May 21 14:12:22 2015 +0100 @@ -111,7 +111,9 @@ PPCBridge PPCCompiledMethod PPCCompiler + PPCCompilerTokenErrorStrategy PPCCompilerTokenRememberStrategy + PPCCompilerTokenizingErrorStrategy PPCCompilerTokenizingRememberStrategy PPCConfiguration PPCContext @@ -121,31 +123,31 @@ PPCNode PPCNodeVisitor PPCPluggableConfiguration + PPCTokenGuard PPCompiledParser #'stx_goodies_petitparser_compiler' - PPCAbstractCharacterNode PPCAbstractLiteralNode PPCAbstractPredicateNode PPCAnyNode + PPCCharacterNode PPCCodeGenerator PPCDelegateNode PPCEndOfFileNode PPCInlinedMethod PPCInliningVisitor - PPCLL1Configuration PPCListNode PPCNilNode PPCPluggableNode PPCProfilingContext PPCRewritingVisitor PPCTokenizingCompiler + PPCTokenizingConfiguration PPCUniversalConfiguration PPCUnknownNode PPTokenizingCompiledParser PPCAbstractActionNode PPCAndNode PPCCharSetPredicateNode - PPCCharacterNode PPCChoiceNode PPCCopyVisitor PPCEndOfInputNode @@ -156,6 +158,7 @@ PPCMessagePredicateNode PPCNegateNode PPCNotCharSetPredicateNode + PPCNotCharacterNode PPCNotLiteralNode PPCNotMessagePredicateNode PPCNotNode @@ -188,6 +191,7 @@ PPCSymbolActionNode PPCTokenChoiceNode PPCTrimNode + PPCTrimmingCharacterTokenNode PPCTokenStarMessagePredicateNode PPCTokenStarSeparatorNode ) @@ -303,7 +307,7 @@ PPEndOfInputParser asCompilerNode PPParser allNodesDo:seen: PPSmalltalkWhitespaceParser hash - PPParser compileLL1 + PPParser compileTokenizing ) ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/Make.proto --- a/compiler/tests/Make.proto Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/Make.proto Thu May 21 14:12:22 2015 +0100 @@ -135,32 +135,23 @@ $(OUTDIR)PPCCopyVisitorTest.$(O) PPCCopyVisitorTest.$(H): PPCCopyVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCGuardTest.$(O) PPCGuardTest.$(H): PPCGuardTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitorTest.$(O) PPCInliningVisitorTest.$(H): PPCInliningVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1OptimizingTest.$(O) PPCLL1OptimizingTest.$(H): PPCLL1OptimizingTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1Test.$(O) PPCLL1Test.$(H): PPCLL1Test.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCLL1VisitorTest.$(O) PPCLL1VisitorTest.$(H): PPCLL1VisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCLTokenizingOptimizationTest.$(O) PPCLTokenizingOptimizationTest.$(H): PPCLTokenizingOptimizationTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCMergingVisitorTest.$(O) PPCMergingVisitorTest.$(H): PPCMergingVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCMockCompiler.$(O) PPCMockCompiler.$(H): PPCMockCompiler.st $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeFirstFollowNextTests.$(O) PPCNodeFirstFollowNextTests.$(H): PPCNodeFirstFollowNextTests.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeTest.$(O) PPCNodeTest.$(H): PPCNodeTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCOptimizeChoicesTest.$(O) PPCOptimizeChoicesTest.$(H): PPCOptimizeChoicesTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCPrototype1OptimizingTest.$(O) PPCPrototype1OptimizingTest.$(H): PPCPrototype1OptimizingTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCPrototype1Test.$(O) PPCPrototype1Test.$(H): PPCPrototype1Test.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizerComponentDetectorTest.$(O) PPCRecognizerComponentDetectorTest.$(H): PPCRecognizerComponentDetectorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizerComponentVisitorTest.$(O) PPCRecognizerComponentVisitorTest.$(H): PPCRecognizerComponentVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCSpecializingVisitorTest.$(O) PPCSpecializingVisitorTest.$(H): PPCSpecializingVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenDetectorTest.$(O) PPCTokenDetectorTest.$(H): PPCTokenDetectorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenGuardTest.$(O) PPCTokenGuardTest.$(H): PPCTokenGuardTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCodeGeneratorTest.$(O) PPCTokenizingCodeGeneratorTest.$(H): PPCTokenizingCodeGeneratorTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenizingTest.$(O) PPCTokenizingTest.$(H): PPCTokenizingTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitorTest.$(O) PPCTokenizingVisitorTest.$(H): PPCTokenizingVisitorTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCompiledExpressionGrammarResource.$(O) PPCompiledExpressionGrammarResource.$(H): PPCompiledExpressionGrammarResource.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestResource.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPCompiledExpressionGrammarTest.$(O) PPCompiledExpressionGrammarTest.$(H): PPCompiledExpressionGrammarTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPCompositeParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPExpressionGrammar.$(O) PPExpressionGrammar.$(H): PPExpressionGrammar.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPExpressionGrammarTest.$(O) PPExpressionGrammarTest.$(H): PPExpressionGrammarTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPCompositeParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPLL1ExpressionGrammar.$(O) PPLL1ExpressionGrammar.$(H): PPLL1ExpressionGrammar.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPLL1ExpressionGrammarTest.$(O) PPLL1ExpressionGrammarTest.$(H): PPLL1ExpressionGrammarTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPCompositeParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedExpressionGrammarResource.$(O) PPTokenizedExpressionGrammarResource.$(H): PPTokenizedExpressionGrammarResource.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestResource.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedExpressionGrammarTest.$(O) PPTokenizedExpressionGrammarTest.$(H): PPTokenizedExpressionGrammarTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPCompositeParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedLL1ExpressionGrammarResource.$(O) PPTokenizedLL1ExpressionGrammarResource.$(H): PPTokenizedLL1ExpressionGrammarResource.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestResource.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedLL1ExpressionGrammarTest.$(O) PPTokenizedLL1ExpressionGrammarTest.$(H): PPTokenizedLL1ExpressionGrammarTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPCompositeParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalOptimizationTest.$(O) PPCUniversalOptimizationTest.$(H): PPCUniversalOptimizationTest.st $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalTest.$(O) PPCUniversalTest.$(H): PPCUniversalTest.st $(INCLUDE_TOP)/stx/goodies/petitparser/tests/PPAbstractParserTest.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestAsserter.$(H) $(INCLUDE_TOP)/stx/goodies/sunit/TestCase.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests.$(O) stx_goodies_petitparser_compiler_tests.$(H): stx_goodies_petitparser_compiler_tests.st $(INCLUDE_TOP)/stx/libbasic/LibraryDefinition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/ProjectDefinition.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/Make.spec --- a/compiler/tests/Make.spec Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/Make.spec Thu May 21 14:12:22 2015 +0100 @@ -58,32 +58,23 @@ PPCCopyVisitorTest \ PPCGuardTest \ PPCInliningVisitorTest \ - PPCLL1OptimizingTest \ - PPCLL1Test \ PPCLL1VisitorTest \ + PPCLTokenizingOptimizationTest \ PPCMergingVisitorTest \ PPCMockCompiler \ PPCNodeFirstFollowNextTests \ PPCNodeTest \ PPCOptimizeChoicesTest \ - PPCPrototype1OptimizingTest \ - PPCPrototype1Test \ PPCRecognizerComponentDetectorTest \ PPCRecognizerComponentVisitorTest \ PPCSpecializingVisitorTest \ PPCTokenDetectorTest \ + PPCTokenGuardTest \ PPCTokenizingCodeGeneratorTest \ + PPCTokenizingTest \ PPCTokenizingVisitorTest \ - PPCompiledExpressionGrammarResource \ - PPCompiledExpressionGrammarTest \ - PPExpressionGrammar \ - PPExpressionGrammarTest \ - PPLL1ExpressionGrammar \ - PPLL1ExpressionGrammarTest \ - PPTokenizedExpressionGrammarResource \ - PPTokenizedExpressionGrammarTest \ - PPTokenizedLL1ExpressionGrammarResource \ - PPTokenizedLL1ExpressionGrammarTest \ + PPCUniversalOptimizationTest \ + PPCUniversalTest \ stx_goodies_petitparser_compiler_tests \ @@ -97,32 +88,23 @@ $(OUTDIR_SLASH)PPCCopyVisitorTest.$(O) \ $(OUTDIR_SLASH)PPCGuardTest.$(O) \ $(OUTDIR_SLASH)PPCInliningVisitorTest.$(O) \ - $(OUTDIR_SLASH)PPCLL1OptimizingTest.$(O) \ - $(OUTDIR_SLASH)PPCLL1Test.$(O) \ $(OUTDIR_SLASH)PPCLL1VisitorTest.$(O) \ + $(OUTDIR_SLASH)PPCLTokenizingOptimizationTest.$(O) \ $(OUTDIR_SLASH)PPCMergingVisitorTest.$(O) \ $(OUTDIR_SLASH)PPCMockCompiler.$(O) \ $(OUTDIR_SLASH)PPCNodeFirstFollowNextTests.$(O) \ $(OUTDIR_SLASH)PPCNodeTest.$(O) \ $(OUTDIR_SLASH)PPCOptimizeChoicesTest.$(O) \ - $(OUTDIR_SLASH)PPCPrototype1OptimizingTest.$(O) \ - $(OUTDIR_SLASH)PPCPrototype1Test.$(O) \ $(OUTDIR_SLASH)PPCRecognizerComponentDetectorTest.$(O) \ $(OUTDIR_SLASH)PPCRecognizerComponentVisitorTest.$(O) \ $(OUTDIR_SLASH)PPCSpecializingVisitorTest.$(O) \ $(OUTDIR_SLASH)PPCTokenDetectorTest.$(O) \ + $(OUTDIR_SLASH)PPCTokenGuardTest.$(O) \ $(OUTDIR_SLASH)PPCTokenizingCodeGeneratorTest.$(O) \ + $(OUTDIR_SLASH)PPCTokenizingTest.$(O) \ $(OUTDIR_SLASH)PPCTokenizingVisitorTest.$(O) \ - $(OUTDIR_SLASH)PPCompiledExpressionGrammarResource.$(O) \ - $(OUTDIR_SLASH)PPCompiledExpressionGrammarTest.$(O) \ - $(OUTDIR_SLASH)PPExpressionGrammar.$(O) \ - $(OUTDIR_SLASH)PPExpressionGrammarTest.$(O) \ - $(OUTDIR_SLASH)PPLL1ExpressionGrammar.$(O) \ - $(OUTDIR_SLASH)PPLL1ExpressionGrammarTest.$(O) \ - $(OUTDIR_SLASH)PPTokenizedExpressionGrammarResource.$(O) \ - $(OUTDIR_SLASH)PPTokenizedExpressionGrammarTest.$(O) \ - $(OUTDIR_SLASH)PPTokenizedLL1ExpressionGrammarResource.$(O) \ - $(OUTDIR_SLASH)PPTokenizedLL1ExpressionGrammarTest.$(O) \ + $(OUTDIR_SLASH)PPCUniversalOptimizationTest.$(O) \ + $(OUTDIR_SLASH)PPCUniversalTest.$(O) \ $(OUTDIR_SLASH)stx_goodies_petitparser_compiler_tests.$(O) \ diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCCodeGeneratorTest.st --- a/compiler/tests/PPCCodeGeneratorTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCCodeGeneratorTest.st Thu May 21 14:12:22 2015 +0100 @@ -519,6 +519,18 @@ self assert: parser parse: '' to: nil end: 0. ! +testNotCharacterNode + node := PPCNotCharacterNode new + character: $: + yourself. + + self compileTree: node. + + self assert: parser parse: 'a' to: nil end: 0. + self assert: parser fail: ':'. + self assert: parser parse: '' to: nil end: 0. +! + testNotLiteralNode | literalNode | literalNode := PPCNotLiteralNode new diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCGuardTest.st --- a/compiler/tests/PPCGuardTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCGuardTest.st Thu May 21 14:12:22 2015 +0100 @@ -6,7 +6,7 @@ instanceVariableNames:'guard compiler' classVariableNames:'' poolDictionaries:'' - category:'PetitCompiler-Tests-Core' + category:'PetitCompiler-Tests-Guards' ! @@ -41,7 +41,7 @@ guard compileGuard: compiler. self assert: compiler lines size = 1. - self assert: compiler lines first = ('(context peek = ', $a storeString ,')'). + self assert: compiler lines first = ('(context peek == ', $a storeString ,')'). ! testCompiling4 @@ -50,7 +50,7 @@ guard compileGuard: compiler. self assert: compiler lines size = 1. - self assert: compiler lines first = ('(context peek = ', $f storeString ,')'). + self assert: compiler lines first = ('(context peek == ', $f storeString ,')'). ! testIdentifierToken diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCLL1OptimizingTest.st --- a/compiler/tests/PPCLL1OptimizingTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,78 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -TestCase subclass:#PPCLL1OptimizingTest - instanceVariableNames:'configuration parser result' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-Core' -! - -!PPCLL1OptimizingTest methodsFor:'as yet unclassified'! - -assert: object type: class - self assert: (object isKindOf: class) -! - -configuration - configuration := PPCLL1Configuration new. - configuration arguments generate: false. - ^ configuration -! - -optimize: aPPParser - ^ self configuration compile: aPPParser. -! - -testCompileToken - parser := 'foo' asParser token. - result := self optimize: parser. - - self assert: result type: PPCTokenizingParserNode. - self assert: result parser type: PPCTokenConsumeNode. - self assert: result parser child type: PPCTokenNode. - self assert: result parser child child literal = 'foo'. -! - -testCompileTrim - parser := 'foo' asParser token trim. - result := self optimize: parser. - - self assert: result type: PPCTokenizingParserNode. - self assert: result parser type: PPCTokenConsumeNode. - self assert: result parser child type: PPCTrimmingTokenNode. - self assert: result parser child whitespace type: PPCTokenStarSeparatorNode. -! - -testCompileTrimmingToken - parser := 'foo' asParser trimmingToken. - result := self optimize: parser. - - self assert: result type: PPCTokenizingParserNode. - self assert: result parser type: PPCTokenConsumeNode. - self assert: result parser child type: PPCTrimmingTokenNode. - self assert: result parser child whitespace type: PPCTokenStarSeparatorNode. - - self assert: result tokenizer children size = 2. - self assert: (result tokenizer children anySatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]) -! - -testCompileTrimmingToken2 - | token | - token := 'foo' asParser trimmingToken. - parser := token wrapped - name: 'fooToken'; - yourself. - result := self optimize: parser. - - self assert: result type: PPCTokenizingParserNode. - self assert: result parser type: PPCTokenConsumeNode. - self assert: result parser name = 'fooToken'. - self assert: result parser child name = 'token_fooToken'. - - self assert: result tokenizer children size = 2. - self assert: (result tokenizer children anySatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]). - self assert: (result tokenizer children anySatisfy: [ :e | e name = 'token_fooToken']). -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCLL1Test.st --- a/compiler/tests/PPCLL1Test.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,250 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPAbstractParserTest subclass:#PPCLL1Test - instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 - arguments configuration' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-Core' -! - -!PPCLL1Test methodsFor:'as yet unclassified'! - -assert: p parse: whatever - ^ result := super assert: p parse: whatever. -! - -cleanClass - | parserClass | - parserClass := (Smalltalk at: arguments name ifAbsent: [nil]). - parserClass notNil ifTrue:[ - self flag: 'uncomment'. -" parserClass removeFromSystem" - ]. -! - -context - ^ context := PPCProfilingContext new -! - -parse: whatever - ^ result := super parse: whatever. -! - -setUp - arguments := PPCArguments default - profile: true; - yourself. - - configuration := PPCLL1Configuration new - arguments: arguments; - yourself. - - self cleanClass. -! - -tearDown - self cleanClass -! - -testChoiceOrder - parser := ( - 'a' asParser token, 'b' asParser token / - 'a' asParser token) - compileWithConfiguration: configuration. - - self assert: parser parse: 'ab'. - self assert: result first inputValue = 'a'. - self assert: result second inputValue = 'b'. - - self assert: parser parse: 'a'. - self assert: result inputValue = 'a'. - - self assert: parser fail: '_'. - -! - -testChoiceOrder2 - | p1 p2 | - p1 := 'a' asParser token, 'b' asParser token. - p2 := 'b' asParser token / 'a' asParser token. - - parser := p1 / p2 compileWithConfiguration: configuration. - - self assert: parser parse: 'ab'. - self assert: result first inputValue = 'a'. - self assert: result second inputValue = 'b'. - - self assert: parser parse: 'a'. - self assert: result inputValue = 'a'. - - self assert: parser parse: 'b'. - self assert: result inputValue = 'b'. - - self assert: parser fail: 'c'. - -! - -testChoiceOrder3 - | p1 p2 a1 a2 | - a1 := 'a' asParser token name: 't1'; yourself. - a2 := 'a' asParser token name: 't2'; yourself. - - p1 := a1, 'b' asParser token. - p2 := a2. - - parser := p1 / p2 compileWithConfiguration: configuration. - - self assert: parser parse: 'ab'. - self assert: result first inputValue = 'a'. - self assert: result second inputValue = 'b'. - - self assert: parser parse: 'a'. - self assert: result inputValue = 'a'. - - self assert: parser fail: 'b'. - -! - -testChoiceOrder4 - | p1 p2 a1 a2 | - a1 := 'a' asParser token name: 't1'; yourself. - a2 := 'a' asParser token name: 't2'; yourself. - - p1 := a1, 'b' asParser token. - p2 := 'b' asParser token / a2. - - parser := p1 / p2 compileWithConfiguration: configuration. - - self assert: parser parse: 'ab'. - self assert: result first inputValue = 'a'. - self assert: result second inputValue = 'b'. - - self assert: parser parse: 'a'. - self assert: result inputValue = 'a'. - - self assert: parser parse: 'b'. - self assert: result inputValue = 'b'. - - self assert: parser fail: 'c'. - -! - -testCompileChoice - parser := ('foo' asParser / 'bar' asParser) compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: 'foo'. - self assert: parser parse: 'bar' to: 'bar'. - self assert: parser fail: '_'. - -! - -testCompileChoice2 - parser := ('foo' asParser token trim / 'bar' asParser token trim) compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: parser parse: 'bar'. - self assert: result inputValue = 'bar'. - self assert: parser fail: '_'. - -! - -testCompileLiteral - parser := 'foo' asParser token compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: parser fail: 'boo'. -! - -testCompileSequence - parser := ('foo' asParser token), ('bar' asParser token) - compileWithConfiguration: configuration. - - self assert: parser parse: 'foobar'. - self assert: result first inputValue = 'foo'. - self assert: result second inputValue = 'bar'. -! - -testCompileTokenComplex2 - | a b argumentsWith | - "based on the PPSmalltlakGrammar>>blockArgumentsWith" - a := $| asParser smalltalkToken - yourself. - b := $] asParser smalltalkToken - yourself. - argumentsWith := (a / b and ==> [:t | ]) wrapped - name: 'argumentsWith'; - yourself. - - parser := argumentsWith compileWithConfiguration: configuration. - self assert: parser parse: '|'. - - parser := argumentsWith compileWithConfiguration: configuration. - self assert: parser parse: ']'. -! - -testCompileTokenComplex3 - | choice1 choice2 a1 b1 a2 b2 tricky | - a1 := $| asParser token - yourself. - b1 := $] asParser token - yourself. - choice1 := (a1 / b1) wrapped - name: 'choice1'; - yourself. - - a2 := $| asParser token - yourself. - b2 := $] asParser token - yourself. - choice2 := (a2 / b2) wrapped - name: 'choice1'; - yourself. - - tricky := (a1 asParser, choice1) / (b2 asParser, choice2). - - parser := tricky compileWithConfiguration: configuration. - self assert: parser parse: '||'. - - parser := tricky compileWithConfiguration: configuration. - self assert: parser parse: '|]'. - - parser := tricky compileWithConfiguration: configuration. - self assert: parser parse: ']|'. - - parser := tricky compileWithConfiguration: configuration. - self assert: parser parse: ']]'. -! - -testCompileTrim - parser := 'foo' asParser token trim end compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: 'foo '. - self assert: result inputValue = 'foo'. - - - self assert: parser parse: ' foo'. - self assert: result inputValue = 'foo'. - - self assert: parser fail: 'boo'. -! - -testTokenName - | token | - token := 'foo' asParser token name: 'fooToken'; yourself. - parser := token plus - compileWithConfiguration: configuration. - - self assert: parser parse: 'foofoo'. - self assert: result first inputValue = 'foo'. - self assert: result second inputValue = 'foo'. - self assert: (parser class methodDictionary includesKey: #fooToken). -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCLTokenizingOptimizationTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCLTokenizingOptimizationTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,86 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +TestCase subclass:#PPCLTokenizingOptimizationTest + instanceVariableNames:'configuration parser result' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Tokenizing' +! + + +!PPCLTokenizingOptimizationTest methodsFor:'as yet unclassified'! + +assert: object type: class + self assert: (object isKindOf: class) +! + +configuration + configuration := PPCTokenizingConfiguration new. + configuration arguments generate: false. + ^ configuration +! + +optimize: aPPParser + ^ self configuration compile: aPPParser. +! + +testCompileToken + parser := 'foo' asParser token. + result := self optimize: parser. + + self assert: result type: PPCTokenizingParserNode. + self assert: result parser type: PPCTokenConsumeNode. + self assert: result parser child type: PPCTokenNode. + self assert: result parser child child literal = 'foo'. +! + +testCompileTrim + parser := 'foo' asParser token trim. + result := self optimize: parser. + + self assert: result type: PPCTokenizingParserNode. + self assert: result parser type: PPCTokenConsumeNode. + self assert: result parser child type: PPCTrimmingTokenNode. + self assert: result parser child whitespace type: PPCTokenStarSeparatorNode. +! + +testCompileTrimmingToken + parser := 'foo' asParser trimmingToken. + result := self optimize: parser. + + self assert: result type: PPCTokenizingParserNode. + self assert: result parser type: PPCTokenConsumeNode. + self assert: result parser child type: PPCTrimmingTokenNode. + self assert: result parser child whitespace type: PPCTokenStarSeparatorNode. + + self assert: result tokenizer children size = 2. + self assert: (result tokenizer children anySatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]) +! + +testCompileTrimmingToken2 + | token | + token := 'foo' asParser trimmingToken. + parser := token wrapped + name: 'fooToken'; + yourself. + result := self optimize: parser. + + self assert: result type: PPCTokenizingParserNode. + self assert: result parser type: PPCTokenConsumeNode. + self assert: result parser name = 'fooToken'. + self assert: result parser child name = 'token_fooToken'. + + self assert: result tokenizer children size = 2. + self assert: (result tokenizer children anySatisfy: [ :e | e isKindOf: PPCTrimmingTokenNode ]). + self assert: (result tokenizer children anySatisfy: [ :e | e name = 'token_fooToken']). +! ! + +!PPCLTokenizingOptimizationTest class methodsFor:'documentation'! + +version_HG + + ^ '$Changeset: $' +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCNodeFirstFollowNextTests.st --- a/compiler/tests/PPCNodeFirstFollowNextTests.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCNodeFirstFollowNextTests.st Thu May 21 14:12:22 2015 +0100 @@ -9,13 +9,22 @@ category:'PetitCompiler-Tests-Nodes' ! +PPCNodeFirstFollowNextTests class instanceVariableNames:'first' + +" + The following class instance variables are inherited by this class: + + TestCase - lastOutcomes + TestAsserter - + Object - +" +! + !PPCNodeFirstFollowNextTests methodsFor:'setup'! setUp configuration := PPCConfiguration default. configuration arguments generate: false. - "The First/Follow is not allowed after this phase:" - configuration arguments specialize: false. ! ! !PPCNodeFirstFollowNextTests methodsFor:'support'! @@ -336,7 +345,7 @@ first := self first: tree. self assert: first size: 2. - self assert: first anyMatchesType: PPCPredicateNode. + self assert: first anyMatchesType: PPCMessagePredicateNode. self assert: first anyMatchesType: PPCSentinelNode. ! @@ -344,7 +353,11 @@ configuration arguments specialize: true. tree := self treeFrom: #space asParser star. - self should: [ self first: tree ] raise: Exception. + first := self first: tree. + + self assert: first size: 2. + self assert: first anyMatchesType: PPCMessagePredicateNode. + self assert: first anyMatchesType: PPCSentinelNode. ! @@ -419,6 +432,20 @@ self assert: first anyOne name = 'foo'. ! ! +!PPCNodeFirstFollowNextTests methodsFor:'testing - first tokens'! + +testFirstTokenStar + | token | + token := '.' asParser token. + + tree := self treeFrom: token star. + + first := tree firstSetWithTokens. + self assert: first size = 2. + self assert: first anySatisfy: [ :e | e isTokenNode ]. + self assert: first anySatisfy: [ :e | e = PPCSentinelNode instance ]. +! ! + !PPCNodeFirstFollowNextTests methodsFor:'testing - follow'! testFollowSet1 diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCNodeTest.st --- a/compiler/tests/PPCNodeTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCNodeTest.st Thu May 21 14:12:22 2015 +0100 @@ -247,13 +247,24 @@ testConvertTrimmingToken3 | parser tree | + parser := 'foo' asParser trimmingToken star. + tree := self treeFrom: parser. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCTrimmingTokenNode. + self assert: tree child child type: PPCLiteralNode. + self assert: tree child child isMarkedForInline. +! + +testConvertTrimmingToken4 + | parser tree | + parser := $d asParser trimmingToken star. tree := self treeFrom: parser. self assert: tree type: PPCStarNode. - self assert: tree child type: PPCTrimmingTokenNode. - self assert: tree child child type: PPCCharacterNode. - self assert: tree child child isMarkedForInline. + self assert: tree child type: PPCTrimmingCharacterTokenNode. + self assert: tree child isMarkedForInline not. ! ! !PPCNodeTest methodsFor:'tests - epsilon'! diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCPrototype1OptimizingTest.st --- a/compiler/tests/PPCPrototype1OptimizingTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,360 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -TestCase subclass:#PPCPrototype1OptimizingTest - instanceVariableNames:'configuration' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-Core' -! - -!PPCPrototype1OptimizingTest methodsFor:'test support'! - -assert: object type: class - self assert: (object isKindOf: class) -! - -optimize: aPPParser - ^ aPPParser compileWithConfiguration: configuration. -! - -setUp - super setUp. - - configuration := PPCUniversalConfiguration new. - configuration arguments generate: false. - -" ^ configuration := PPCPluggableConfiguration on: - [ :_self | - _self toPPCIr. - _self createTokens. - _self specialize. - _self createRecognizingComponents. - _self inline. - _self merge. - ]" -! ! - -!PPCPrototype1OptimizingTest methodsFor:'tests'! - -testAnyPredicate - | tree | - tree := self optimize: #any asParser. - - self assert: tree type: PPCAnyNode. -! - -testCharSetPredicate - | tree | - tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo). - - self assert: tree type: PPCCharSetPredicateNode -! - -testChoiceInlining - | tree | - tree := self optimize: $a asParser / $b asParser. - - self assert: tree type: PPCChoiceNode. - self assert: tree children first type: PPCCharacterNode. - self assert: tree children first isMarkedForInline. - self assert: tree children second type: PPCCharacterNode. - self assert: tree children first isMarkedForInline. - -! - -testForwarding - | tree p1 p2 | - p2 := $a asParser. - p1 := p2 wrapped. - p1 name: 'p1'. - tree := self optimize: p1. - - self assert: tree type: PPCAbstractCharacterNode. - self assert: tree name = 'p1'. - - p2 name: 'p2'. - tree := self optimize: p1. - self assert: tree type: PPCForwardNode. - self assert: tree name = 'p1'. - self assert: tree child name = 'p2'. -! - -testInlineCharacter - | tree | - tree := self optimize: $a asParser plus. - - self assert: tree type: PPCPlusNode. - self assert: tree child type: PPCCharacterNode. - self assert: tree child isMarkedForInline. - self assert: tree child character = $a. -! - -testInlineCharacter2 - | tree | - tree := self optimize: $a asParser star. - - self assert: tree type: PPCStarNode. - self assert: tree child type: PPCCharacterNode. - self assert: tree child isMarkedForInline. - self assert: tree child character = $a. -! - -testInlineCharacter3 - | tree | - tree := self optimize: $a asParser, $b asParser. - - self assert: tree type: PPCSequenceNode. - self assert: tree children first type: PPCCharacterNode. - self assert: tree children first isMarkedForInline. - self assert: tree children first character = $a. - self assert: tree children second type: PPCCharacterNode. - self assert: tree children second isMarkedForInline. - self assert: tree children second character = $b. -! - -testInlineNil - | tree | - tree := self optimize: nil asParser star. - - self assert: tree type: PPCStarNode. - self assert: tree child type: PPCNilNode. - self assert: tree child isMarkedForInline. -! - -testInlineNotLiteral - | tree | - tree := self optimize: 'foo' asParser not star. - - self assert: tree type: PPCStarNode. - self assert: tree child type: PPCNotLiteralNode. - self assert: tree child literal = 'foo'. - self assert: tree child isMarkedForInline. -! - -testInlineNotPredicate - | tree | - tree := self optimize: (#letter asParser not, (PPPredicateObjectParser on: [ :e | e = $a or: [ e = $b ]] message: #foo) not). - - self assert: tree type: PPCSequenceNode. - self assert: tree children first type: PPCNotMessagePredicateNode. - self assert: tree children first isMarkedForInline. - self assert: tree children second type: PPCNotCharSetPredicateNode. - self assert: tree children second isMarkedForInline. - -! - -testInlinePluggable - | tree | - tree := self optimize: [:ctx | nil] asParser star. - - ((Smalltalk respondsTo:#isSmalltalkX) and:[ Smalltalk isSmalltalkX ]) - ifTrue:[ self skipIf: true description: 'not supported in St/X' ]. - - self assert: tree type: PPCStarNode. - self assert: tree child type: PPCPluggableNode. - self assert: tree child isMarkedForInline. - - "Modified: / 10-05-2015 / 07:30:58 / Jan Vrany " -! - -testInlinePredicate - | tree | - tree := self optimize: (#letter asParser, (PPPredicateObjectParser on: [ :e | e = $a or: [ e = $b ]] message: #foo)). - - self assert: tree type: PPCSequenceNode. - self assert: tree children first type: PPCMessagePredicateNode. - self assert: tree children first isMarkedForInline. - self assert: tree children second type: PPCCharSetPredicateNode. - self assert: tree children second isMarkedForInline. - -! - -testLetterPredicate - | tree | - tree := self optimize: #letter asParser. - - self assert: tree type: PPCMessagePredicateNode. - self assert: tree message = #isLetter. -! - -testNotAction - | tree | - tree := self optimize: (($f asParser, $o asParser) ==> #second) not. - - self assert: tree type: PPCNotNode. - self assert: tree child type: PPCRecognizingSequenceNode. -! - -testNotCharSetPredicate - | tree | - tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo) asParser not. - - self assert: tree type: PPCNotCharSetPredicateNode. -! - -testNotLiteral - | tree | - tree := self optimize: 'foo' asParser not. - - self assert: tree type: PPCNotLiteralNode. - self assert: tree literal = 'foo'. -! - -testNotMessagePredicate - | tree | - tree := self optimize: #letter asParser not. - - self assert: tree type: PPCNotMessagePredicateNode. -! - -testNotSequence - | tree | - tree := self optimize: ($f asParser, $o asParser) not. - - self assert: tree type: PPCNotNode. - self assert: tree child type: PPCRecognizingSequenceNode. -! - -testRecognizingSequence2 - | tree | - tree := self optimize: ($a asParser, $b asParser) token. - - self assert: tree type: PPCTokenNode. - self assert: tree child type: PPCRecognizingSequenceNode. - - tree := self optimize: ($a asParser, $b asParser) trimmingToken. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree child type: PPCRecognizingSequenceNode. -! - -testStarAny - | tree | - tree := self optimize: #any asParser star. - - self assert: tree type: PPCStarAnyNode. -! - -testStarCharSetPredicate - | tree | - tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo) star. - - self assert: tree type: PPCStarCharSetPredicateNode -! - -testStarMessagePredicate - | tree | - tree := self optimize: #letter asParser star. - - self assert: tree type: PPCStarMessagePredicateNode. -! - -testStarSeparator - | tree | - tree := self optimize: #space asParser star trimmingToken. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree child type: PPCTokenStarSeparatorNode. -! - -testStarSeparator2 - | tree | - tree := self optimize: (#space asParser star, 'whatever' asParser) trimmingToken. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree child type: PPCRecognizingSequenceNode. - self assert: tree child children first type: PPCTokenStarSeparatorNode. - self assert: tree child children first isMarkedForInline. -! - -testSymbolAction - | tree | - tree := self optimize: (#letter asParser) ==> #second. - - self assert: tree type: PPCSymbolActionNode. - - tree := self optimize: (#letter asParser) ==> [:e | e second ]. - self assert: tree type: PPCActionNode. -! - -testToken - | tree | - tree := self optimize: ((#letter asParser, #word asParser star) token). - - self assert: tree type: PPCTokenNode. - self assert: tree child type: PPCRecognizingSequenceNode. - self assert: tree child children size = 2. - self assert: tree child children first type: PPCMessagePredicateNode. - self assert: tree child children first isMarkedForInline. - self assert: tree child children second type: PPCTokenStarMessagePredicateNode. - self assert: tree child children second isMarkedForInline. - -! - -testTrimmingToken - | tree | - tree := self optimize: ((#letter asParser, #word asParser star) trimmingToken). - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree whitespace type: PPCTokenStarSeparatorNode. - self assert: tree whitespace isMarkedForInline. - - self assert: tree child type: PPCRecognizingSequenceNode. - self assert: tree child children size = 2. - self assert: tree child children first type: PPCMessagePredicateNode. - self assert: tree child children first isMarkedForInline. - self assert: tree child children second type: PPCTokenStarMessagePredicateNode. - self assert: tree child children first isMarkedForInline. -! - -testTrimmingToken2 - | parser tree | - parser := 'foo' asParser trimmingToken. - tree := self optimize: parser. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree child type: PPCLiteralNode. - self assert: tree child isMarkedForInline. - self assert: (tree whitespace allNodes allSatisfy: [ :n | n isKindOf: PPCNode ]). - - parser := ('foo' asParser, $b asParser) trimmingToken. - tree := self optimize: parser. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree child type: PPCRecognizingSequenceNode. - self assert: tree whitespace type: PPCTokenStarSeparatorNode. - self assert: tree whitespace isMarkedForInline. - - parser := $d asParser trimmingToken star. - tree := self optimize: parser. - - self assert: tree type: PPCStarNode. - self assert: tree child type: PPCTrimmingTokenNode. - self assert: tree child child type: PPCCharacterNode. - self assert: tree child child isMarkedForInline. -! - -testTrimmingToken3 - | parser tree | - parser := ('foo' asParser trimmingToken name: 'foo'), ('bar' asParser trimmingToken name: 'bar'). - tree := self optimize: parser. - - self assert: tree type: PPCSequenceNode. - self assert: tree children first type: PPCTrimmingTokenNode. - self assert: tree children second type: PPCTrimmingTokenNode. -! - -testTrimmingTokenNested - | parser tree foo| - foo := 'foo' asParser trimmingToken name: 'foo'. - parser := (foo not, 'bar' asParser) trimmingToken name: 'token'. - tree := self optimize: parser. - - self assert: tree type: PPCTrimmingTokenNode. - self assert: tree children second type: PPCRecognizingSequenceNode. - self assert: tree children second children first type: PPCNotLiteralNode. - self assert: tree children second children first isMarkedForInline. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCPrototype1Test.st --- a/compiler/tests/PPCPrototype1Test.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,579 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPAbstractParserTest subclass:#PPCPrototype1Test - instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 - arguments configuration' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-Core' -! - -!PPCPrototype1Test methodsFor:'context'! - -context - ^ context := PPCProfilingContext new -! ! - -!PPCPrototype1Test methodsFor:'test support'! - -assert: p parse: whatever - ^ result := super assert: p parse: whatever. -! - -parse: whatever - ^ result := super parse: whatever. -! - -tearDown - | parserClass | - - parserClass := (Smalltalk at: arguments name ifAbsent: [nil]). - parserClass notNil ifTrue:[ - parserClass removeFromSystem - ]. -! ! - -!PPCPrototype1Test methodsFor:'tests - compiling'! - -testCompileAnd - parser := #digit asParser and compileWithConfiguration: configuration. - - self assert: parser parse: '1' to: $1 end: 0. - self assert: parser fail: 'a'. - self assert: parser fail: ''. - - parser := ('foo' asParser, ($: asParser and)) compile. - self assert: parser parse: 'foo:' to: { 'foo'. $: } end: 3. -! - -testCompileAny - parser := #any asParser compile. - - self assert: parser parse: 'a' to: $a. - self assert: parser parse: '_' to: $_. - self assert: parser parse: ' -' to: Character cr. -! - -testCompileAnyStar - parser := #any asParser star compileWithConfiguration: configuration. - - - self assert: parser parse: 'aaa' to: { $a. $a . $a }. - self assert: context invocationCount = 1. - self assert: parser parse: '' to: { }. -! - -testCompileBlock - parser := (#letter asParser) plus ==> [ :res | res collect: [:each | each asUppercase ]]. - parser := parser compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: { $F . $O . $O}. - self assert: parser parse: 'bar' to: { $B . $A . $R}. - self assert: parser fail: ''. -! - -testCompileCharacter - parser := $a asParser compileWithConfiguration: configuration. - - self assert: parser parse: 'a' to: $a. - self assert: parser fail: 'b'. - - parser := $# asParser compileWithConfiguration: configuration. - self assert: parser parse: '#'. -! - -testCompileChoice - parser := (#digit asParser / #letter asParser) compileWithConfiguration: configuration. - - self assert: parser parse: '1' to: $1. - self assert: parser parse: 'a' to: $a. - self assert: parser fail: '_'. - -! - -testCompileChoice2 - parser := ('true' asParser / 'false' asParser) compileWithConfiguration: configuration. - - self assert: parser parse: 'true' to: 'true'. - self assert: parser parse: 'false' to: 'false'. - self assert: parser fail: 'trulse'. - -! - -testCompileLiteral - parser := 'foo' asParser compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: 'foo'. - self assert: parser parse: 'foobar' to: 'foo' end: 3. - self assert: parser fail: 'boo'. - - parser := '#[' asParser compileWithConfiguration: configuration. - self assert: parser parse: '#[1]' to: '#[' end: 2. -! - -testCompileLiteral2 - | quote | - quote := '''' asParser. - parser := (quote, $a asParser ) compileWithConfiguration: configuration. - self assert: parser parse: '''a' to: {'''' . $a}. -! - -testCompileNegate - parser := #letter asParser negate star, #letter asParser. - parser := parser compileWithConfiguration: configuration. - - self assert: parser parse: '...a' to: { { $. . $. . $. } . $a }. - self assert: parser parse: 'aaa' to: { {} . $a } end: 1. - self assert: parser fail: '...'. -! - -testCompileNil - parser := nil asParser compileWithConfiguration: configuration. - - self assert: parser parse: 'a' to: nil end: 0. - self assert: parser parse: '' to: nil end: 0. - - parser := nil asParser, 'foo' asParser. - self assert: parser parse: 'foo' to: { nil . 'foo' } -! - -testCompileNot - parser := #digit asParser not compileWithConfiguration: configuration. - - self assert: parser parse: 'a' to: nil end: 0. - self assert: parser fail: '1'. - self assert: parser parse: '' to: nil end: 0. - - parser := 'foo' asParser, $: asParser not. - parser := parser compileWithConfiguration: configuration. - self assert: parser parse: 'foo' to: { 'foo'. nil } end: 3. - - parser := 'foo' asParser, $: asParser not, 'bar' asParser. - parser := parser compileWithConfiguration: configuration. - self assert: parser parse: 'foobar' to: { 'foo'. nil . 'bar' } end: 6. -! - -testCompileNot2 - parser := ($a asParser, $b asParser) not compileWithConfiguration: configuration. - - self assert: parser parse: '' to: nil end: 0. - self assert: parser parse: 'a' to: nil end: 0. - self assert: parser parse: 'aa' to: nil end: 0. - self assert: parser fail: 'ab'. -! - -testCompileNot3 - parser := ('foo' asParser not, 'fee' asParser) compileWithConfiguration: configuration. - - self assert: parser parse: 'fee' to: #(nil 'fee'). - self assert: parser fail: 'foo'. -! - -testCompileNotLiteral - parser := 'foo' asParser not compileWithConfiguration: configuration. - self assert: parser class methodDictionary size = 1. - - self assert: parser parse: 'bar' to: nil end: 0. - - self assert: parser fail: 'foo'. - self assert: parser parse: '' to: nil end: 0. - - parser := '''' asParser not compile. - self assert: parser class methodDictionary size = 1. - - self assert: parser parse: 'a' to: nil end: 0. - self assert: parser fail: ''''. - self assert: parser parse: '' to: nil end: 0. - - - parser := ('foo' asParser, 'bar' asParser not) compile. - self assert: parser parse: 'foofoo' to: { 'foo'. nil } end: 3. - - parser := ('foo' asParser, 'foo' asParser not, #any asParser star) compile. - self assert: parser parse: 'foobar' to: { 'foo'. nil . #($b $a $r) } end: 6. - self assert: parser fail: 'foofoo'. -! - -testCompileOptional - parser := #digit asParser optional compileWithConfiguration: configuration. - - self assert: parser parse: '1' to: $1. - self assert: parser parse: 'a' to: nil end: 0. - - parser := (#digit asParser optional, #letter asParser) compile. - self assert: parser parse: '1a' to: { $1 . $a }. - self assert: parser parse: 'a' to: { nil . $a }. -! - -testCompilePlus - parser := #letter asParser plus compileWithConfiguration: configuration. - - self assert: parser parse: 'lorem' to: {$l. $o. $r. $e. $m} . - self assert: parser parse: 'a123' to: {$a} end: 1. - self assert: parser parse: 'ab123' to: {$a . $b} end: 2. - - self assert: parser fail: ''. - self assert: parser fail: '123'. -! - -testCompilePredicate - parser := #digit asParser compileWithConfiguration: configuration. - - self assert: parser parse: '1' to: $1. - self assert: parser parse: '0' to: $0. - self assert: parser fail: 'a'. -! - -testCompilePredicate2 - parser := #space asParser compileWithConfiguration: configuration. - - self assert: parser parse: ' ' to: Character space. - self assert: parser fail: 'a'. -! - -testCompileSequence - parser := (#digit asParser, #letter asParser) compileWithConfiguration: configuration. - - self assert: parser parse: '1a' to: {$1 .$a}. - - -! - -testCompileSequence2 - parser := (#digit asParser, #space asParser, #letter asParser) compileWithConfiguration: configuration. - - self assert: parser parse: '9 c' to: {$9 . Character space. $c }. - self assert: parser fail: '9c'. - -! - -testCompileSequence3 - parser := (#any asParser, #any asParser, #any asParser) compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: #($f $o $o). - self assert: parser fail: 'fo'. - -! - -testCompileStar - parser := #letter asParser star compileWithConfiguration: configuration. - - self assert: parser parse: 'lorem' to: {$l. $o. $r. $e. $m} . - self assert: parser parse: '' to: {}. - self assert: parser parse: '123' to: {} end: 0. - self assert: parser parse: 'ab123' to: {$a . $b} end: 2. -! - -testCompileStarLiteral - parser := 'foo' asParser star compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: #('foo' ) . - self assert: parser parse: 'foofoo' to: #('foo' 'foo') . - self assert: parser parse: 'foofoofoo' to: #('foo' 'foo' 'foo') . - self assert: parser parse: '' to: #(). - self assert: parser parse: 'bar' to: #() end: 0. -! - -testCompileStarPredicate - parser := #letter asParser star compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: #($f $o $o ) . - self assert: parser parse: '' to: #(). - self assert: parser parse: '123' to: #() end: 0. -! - -testCompileSymbolBlock - parser := (#letter asParser) plus ==> #second. - parser := parser compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: $o. - self assert: parser parse: 'bar' to: $a. - self assert: parser fail: ''. - self should: [ parser parse: 'f' ] raise: Error. -! - -testCompileTrim - parser := $a asParser trim compileWithConfiguration: configuration. - - self assert: parser fail: ''. - self assert: parser parse: 'a' to: $a. - self assert: parser parse: ' a' to: $a. - self assert: parser parse: 'a ' to: $a. - self assert: parser parse: ' a ' to: $a. -! - -testCompileTrimmingToken - | token1 token2 | - token1 := (#letter asParser) plus trimmingToken. - token2 := (#letter asParser) plus trimmingToken. - - parser := (token1, token2) compileWithConfiguration: configuration. - - self assert: parser parse: 'foo bar'. - self assert: parser parse: ' foo bar '. -! - -testCompileTrimmingToken2 - | token1 token2 | - token1 := (#letter asParser) plus trimmingToken. - token2 := (#letter asParser) plus trimmingToken / 'foo' asParser trimmingToken. - - parser := (token1, token2) compileWithConfiguration: configuration. - - self assert: parser parse: 'foo bar'. - self assert: parser parse: ' foo bar '. -! - -testCompileTrimmingToken3 - | token1 token2 | - token1 := ($a asParser, $b asParser) trimmingToken name: 'token1'. - token2 := (token1 not, $c asParser) trimmingToken name: 'token2'. - - parser := (token1 / token2) compileWithConfiguration: configuration. - - self assert: (parser class methodDictionary includesKey: #'token1'). - self assert: (parser class methodDictionary includesKey: #'token1_fast'). - - self assert: parser parse: 'ab'. - self assert: (result isKindOf: PPToken). - self assert: result inputValue = 'ab'. - - self assert: parser parse: 'c'. - self assert: (result isKindOf: PPToken). - self assert: result inputValue = 'c'. - -! ! - -!PPCPrototype1Test methodsFor:'tests - extra'! - -testCompileSmalltalkToken - parser := (#letter asParser, ((#letter asParser / #digit asParser) star)) smalltalkToken compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: parser parse: 'a'. - self assert: result inputValue = 'a'. - self assert: parser parse: 'f123a'. - self assert: result inputValue = 'f123a'. - - self assert: parser fail: ''. - self assert: parser fail: '12'. - - self assert: parser parse: ' "comment" foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: ' "comment" bar "another comment" '. - self assert: result inputValue = 'bar'. - self assert: parser parse: ' - "b" - "b" - foo - "and yet, another comment" - - "one more to make sure :)" - '. - self assert: result inputValue = 'foo'. -! - -testCycle - | p1 block | - - p1 := PPDelegateParser new. - block := ${ asParser, p1, $} asParser / nil asParser. - p1 setParser: block. - - parser := block compileWithConfiguration: configuration. - self assert: parser parse: '{}' to: { ${. nil . $} }. - self assert: parser parse: '{{}}' to: { ${. { ${ . nil . $} } . $} }. - -! - -testSmalltalkToken - parser := (#letter asParser, (#digit asParser / #letter asParser) star) smalltalkToken compileWithConfiguration: configuration. - - self assert: parser class methodDictionary size = 5. - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: context invocationCount = 8. - self assert: context rememberCount = 0. - self assert: context lwRememberCount = 1. - self assert: context lwRestoreCount = 0. -! - -testSmalltalkToken2 - id := (#letter asParser, (#digit asParser / #letter asParser) star) - name: 'identifier'; - yourself. - - parser := (id wrapped, $: asParser) smalltalkToken - name: 'kw'; - yourself. - - parser := parser compileWithConfiguration: configuration. - - self assert: parser parse: 'foo:'. - self assert: result inputValue = 'foo:'. -! - -testToken - parser := (#letter asParser, (#digit asParser / #letter asParser) star) flatten compileWithConfiguration: configuration. - - self assert: parser parse: 'foo' to: 'foo'. - self assert: parser parse: 'a' to: 'a'. - self assert: parser parse: 'f123a' to: 'f123a'. - self assert: parser fail: ''. -! - -testToken2 - parser := (#letter asParser, (#digit asParser / #letter asParser) star) token compileWithConfiguration: configuration. - - self assert: parser class methodDictionary size = 4. - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: context invocationCount = 6. - self assert: context rememberCount = 0. - self assert: context lwRememberCount = 1. - self assert: context lwRestoreCount = 0. -! - -testTrimmingToken - parser := (#letter asParser, (#digit asParser / #letter asParser) star) trimmingToken compileWithConfiguration: configuration. - - self assert: parser class methodDictionary size = 4. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: context invocationCount = 6. - self assert: context rememberCount = 0. - self assert: context lwRememberCount = 1. - self assert: context lwRestoreCount = 0. - - self assert: parser parse: ' foo '. - self assert: result inputValue = 'foo'. - - - - self assert: parser fail: '123'. - - self assert: context invocationCount = 1. - self assert: context rememberCount = 0. - self assert: context lwRememberCount = 0. - self assert: context lwRestoreCount = 0. - - - self assert: parser fail: ''. -! - -testTrimmingToken2 - - parser := 'foo' asParser trimmingToken, 'bar' asParser trimmingToken - compileWithConfiguration: configuration. - - self assert: parser parse: 'foobar'. - self assert: result first inputValue = 'foo'. - self assert: result second inputValue = 'bar'. - self assert: context invocationCount = 3. - - self assert: parser parse: ' foobar'. - self assert: result first inputValue = 'foo'. - self assert: result second inputValue = 'bar'. - self assert: context invocationCount = 3. - self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). - - self assert: parser fail: 'bar'. - self assert: context invocationCount = 1. - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). - -! - -testTrimmingToken3 - - parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken) - compileWithConfiguration: configuration. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - self assert: context invocationCount = 2. - - self assert: parser parse: ' bar'. - self assert: result inputValue = 'bar'. - self assert: context invocationCount = 2. - self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). - - self assert: parser fail: 'baz'. - self assert: context invocationCount = 2. - - self assert: parser fail: 'zaz'. - self assert: context invocationCount = 1. - self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). -! - -testTrimmingTokenNested - | identifier kw | - kw := 'false' asParser trimmingToken name: #kw. - identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. - - parser := identifier / kw. - parser := parser compileWithConfiguration: configuration. - self assert: parser class methodDictionary size = 5. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: 'false'. - self assert: result inputValue = 'false'. -! - -testTrimmingTokenNested2 - | identifier kw | - kw := 'false' asParser trimmingToken name: #kw. - identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. - - parser := identifier / kw. - parser := parser compileWithConfiguration: configuration. - self assert: parser class methodDictionary size = 5. - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: 'false'. - self assert: result inputValue = 'false'. -! - -testTrimmingTokenNested3 - | identifier kw | - kw := ('false' asParser, #word asParser not) trimmingToken name: #kw. - identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. - - parser := identifier / kw. - parser := parser compileWithConfiguration: configuration. - self assert: parser class methodDictionary size = 8. - self assert: (parser class methodDictionary values anySatisfy: [ :m | m selector = #kw ]). - self assert: (parser class methodDictionary values anySatisfy: [ :m | m selector = #kw_fast ]). - - self assert: parser parse: 'foo'. - self assert: result inputValue = 'foo'. - - self assert: parser parse: 'false'. - self assert: result inputValue = 'false'. - - "Modified: / 10-05-2015 / 07:33:19 / Jan Vrany " -! ! - -!PPCPrototype1Test methodsFor:'tests - ids'! - -setUp - arguments := PPCArguments default - profile: true; - debug: true; - yourself. - - configuration := PPCUniversalConfiguration new - arguments: arguments; - yourself. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCRecognizerComponentVisitorTest.st --- a/compiler/tests/PPCRecognizerComponentVisitorTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCRecognizerComponentVisitorTest.st Thu May 21 14:12:22 2015 +0100 @@ -182,6 +182,7 @@ | starNode | starNode := PPCStarMessagePredicateNode new message: #isLetter; + child: PPCSentinelNode instance; yourself. node := PPCTokenNode new @@ -197,6 +198,7 @@ | starNode | starNode := PPCStarMessagePredicateNode new message: #isSeparator; + child: PPCSentinelNode instance; yourself. node := PPCTokenNode new diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCSpecializingVisitorTest.st --- a/compiler/tests/PPCSpecializingVisitorTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCSpecializingVisitorTest.st Thu May 21 14:12:22 2015 +0100 @@ -138,6 +138,14 @@ self assert: result type: PPCNotCharSetPredicateNode. ! +testNotCharacter + node := self asNode: $: asParser not. + result := visitor visit: node. + + self assert: result type: PPCNotCharacterNode. + self assert: result character = $:. +! + testNotLiteral node := self asNode: 'foo' asParser not. result := visitor visit: node. diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCTokenGuardTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCTokenGuardTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,38 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +TestCase subclass:#PPCTokenGuardTest + instanceVariableNames:'guard compiler node tokenNode' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Guards' +! + +!PPCTokenGuardTest methodsFor:'as yet unclassified'! + +setUp + super setUp. + compiler := PPCMockCompiler new. +! + +testMakesSense + tokenNode := PPCTokenNode new + child: 'foo' asParser asCompilerTree. + node := PPCPlusNode new + child: tokenNode; + yourself. + guard := PPCTokenGuard on: node. + + self assert: guard makesSense. +! + +testMakesSense2 + node := PPCPlusNode new + child: ('foo' asParser asCompilerTree); + yourself. + guard := PPCTokenGuard on: node. + + self assert: guard makesSense not. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCTokenizingCodeGeneratorTest.st --- a/compiler/tests/PPCTokenizingCodeGeneratorTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/PPCTokenizingCodeGeneratorTest.st Thu May 21 14:12:22 2015 +0100 @@ -194,6 +194,7 @@ yourself. wsNode := PPCTokenStarSeparatorNode new name: 'consumeWhitespace'; + child: PPCSentinelNode instance; yourself. node := PPCTokenizingParserNode new diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCTokenizingTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCTokenizingTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,389 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +PPAbstractParserTest subclass:#PPCTokenizingTest + instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 + arguments configuration' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Tokenizing' +! + +!PPCTokenizingTest methodsFor:'as yet unclassified'! + +assert: p parse: whatever + ^ result := super assert: p parse: whatever. +! + +assert: p parse: whatever end: end + ^ result := super assert: p parse: whatever end: end +! + +cleanClass + | parserClass | + parserClass := (Smalltalk at: arguments name ifAbsent: [nil]). + parserClass notNil ifTrue:[ + self flag: 'uncomment'. +" parserClass removeFromSystem" + ]. +! + +context + ^ context := PPCProfilingContext new +! + +parse: whatever + ^ result := super parse: whatever. +! + +setUp + arguments := PPCArguments default + profile: true; + yourself. + + configuration := PPCTokenizingConfiguration new + arguments: arguments; + yourself. + + self cleanClass. +! + +tearDown + self cleanClass +! + +testChoiceOrder + parser := ( + 'a' asParser token, 'b' asParser token / + 'a' asParser token) + compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'b'. + + self assert: parser parse: 'a'. + self assert: result inputValue = 'a'. + + self assert: parser fail: '_'. + +! + +testChoiceOrder2 + | p1 p2 | + p1 := 'a' asParser token, 'b' asParser token. + p2 := 'b' asParser token / 'a' asParser token. + + parser := p1 / p2 compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'b'. + + self assert: parser parse: 'a'. + self assert: result inputValue = 'a'. + + self assert: parser parse: 'b'. + self assert: result inputValue = 'b'. + + self assert: parser fail: 'c'. + +! + +testChoiceOrder3 + | p1 p2 a1 a2 | + a1 := 'a' asParser token name: 't1'; yourself. + a2 := 'a' asParser token name: 't2'; yourself. + + p1 := a1, 'b' asParser token. + p2 := a2. + + parser := p1 / p2 compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'b'. + + self assert: parser parse: 'a'. + self assert: result inputValue = 'a'. + + self assert: parser fail: 'b'. + +! + +testChoiceOrder4 + | p1 p2 a1 a2 | + a1 := 'a' asParser token name: 't1'; yourself. + a2 := 'a' asParser token name: 't2'; yourself. + + p1 := a1, 'b' asParser token. + p2 := 'b' asParser token / a2. + + parser := p1 / p2 compileWithConfiguration: configuration. + + self assert: parser parse: 'ab'. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'b'. + + self assert: parser parse: 'a'. + self assert: result inputValue = 'a'. + + self assert: parser parse: 'b'. + self assert: result inputValue = 'b'. + + self assert: parser fail: 'c'. + +! + +testCompileAnd + parser := (('foo' asParser token and) / ('bar' asParser token and)), 'bar' asParser token + compileWithConfiguration: configuration. + + self assert: parser parse: 'bar'. + self assert: result second inputValue = 'bar'. +! + +testCompileChoice + parser := ('foo' asParser / 'bar' asParser) token compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: parser parse: 'bar'. + self assert: result inputValue = 'bar'. + self assert: parser fail: '_'. + +! + +testCompileChoice2 + parser := ('foo' asParser token trim / 'bar' asParser token trim) compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: parser parse: 'bar'. + self assert: result inputValue = 'bar'. + self assert: parser fail: '_'. + +! + +testCompileComplex1 + parser := ('foo' asParser token, 'bar' asParser token) / + ('foo' asParser token, 'baz' asParser token) compileWithConfiguration: configuration. + + self assert: parser parse: 'foobar'. + self assert: result second inputValue = 'bar'. + + self assert: parser parse: 'foobaz'. + self assert: result second inputValue = 'baz'. + + self assert: parser fail: 'foobaq'. + +! + +testCompileComplex2 + parser := ('foo' asParser token, 'bar' asParser token) star, 'foo' asParser token + compileWithConfiguration: configuration. + + self assert: parser parse: 'foobarfoobarfoo'. + self assert: parser parse: 'foo'. + + self assert: parser fail: 'bar'. + +! + +testCompileComplex3 + parser := ('foo' asParser token, 'bar' asParser token) star, 'foo' asParser token / + ('foo' asParser token, 'baz' asParser token) + compileWithConfiguration: configuration. + + self assert: parser parse: 'foobarfoobarfoo'. + self assert: parser parse: 'foo'. + + self assert: parser fail: 'bar'. + +! + +testCompileLiteral + parser := 'foo' asParser token compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: parser fail: 'boo'. +! + +testCompileSequence + parser := ('foo' asParser token), ('bar' asParser token) + compileWithConfiguration: configuration. + + self assert: parser parse: 'foobar'. + self assert: result first inputValue = 'foo'. + self assert: result second inputValue = 'bar'. +! + +testCompileStar + parser := 'foo' asParser token star compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result first inputValue = 'foo'. + + self assert: parser parse: 'boo' end: 0. + self assert: result isEmpty. +! + +testCompileStar2 + parser := ('foo' asParser token, 'bar' asParser token) star compileWithConfiguration: configuration. + + self assert: parser parse: 'foobar'. + self assert: context tokenReads size = 3. + + self assert: parser parse: 'bar' end: 0. + self assert: result isEmpty. + self assert: context tokenReads size = 1. + +! + +testCompileTokenComplex2 + | a b argumentsWith | + "based on the PPSmalltlakGrammar>>blockArgumentsWith" + a := $| asParser smalltalkToken + yourself. + b := $] asParser smalltalkToken + yourself. + argumentsWith := (a / b and ==> [:t | ]) wrapped + name: 'argumentsWith'; + yourself. + + parser := argumentsWith compileWithConfiguration: configuration. + self assert: parser parse: '|'. + + parser := argumentsWith compileWithConfiguration: configuration. + self assert: parser parse: ']'. +! + +testCompileTokenComplex3 + | choice1 choice2 a1 b1 a2 b2 tricky | + a1 := $| asParser token + yourself. + b1 := $] asParser token + yourself. + choice1 := (a1 / b1) wrapped + name: 'choice1'; + yourself. + + a2 := $| asParser token + yourself. + b2 := $] asParser token + yourself. + choice2 := (a2 / b2) wrapped + name: 'choice1'; + yourself. + + tricky := (a1 asParser, choice1) / (b2 asParser, choice2). + + parser := tricky compileWithConfiguration: configuration. + self assert: parser parse: '||'. + + parser := tricky compileWithConfiguration: configuration. + self assert: parser parse: '|]'. + + parser := tricky compileWithConfiguration: configuration. + self assert: parser parse: ']|'. + + parser := tricky compileWithConfiguration: configuration. + self assert: parser parse: ']]'. +! + +testCompileTrim + parser := 'foo' asParser token trim end compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: 'foo '. + self assert: result inputValue = 'foo'. + + + self assert: parser parse: ' foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: ' foo '. + self assert: result inputValue = 'foo'. + + self assert: parser fail: 'boo'. +! + +testTokenCharacter + | token | + token := $a asParser token. + parser := token plus + compileWithConfiguration: configuration. + + self assert: parser parse: 'a'. + self assert: result first inputValue = 'a'. + self assert: context invocations size = 5. +! + +testTokenCharacter2 + | token | + token := $a asParser token. + parser := token plus + compileWithConfiguration: configuration. + + self assert: parser parse: 'aaa'. + self assert: result first inputValue = 'a'. + self assert: result second inputValue = 'a'. + self assert: result third inputValue = 'a'. + self assert: context invocations size = 7. +! + +testTokenName + | token | + token := 'foo' asParser token name: 'fooToken'; yourself. + parser := token plus + compileWithConfiguration: configuration. + + self assert: parser parse: 'foofoo'. + self assert: result first inputValue = 'foo'. + self assert: result second inputValue = 'foo'. + self assert: (parser class methodDictionary includesKey: #fooToken). +! + +testWhitespace + | token ws trimmingToken | + configuration arguments inline: false. + + token := 'foo' asParser token. + ws := #blank asParser star name: 'consumeWhitespace'; yourself. + trimmingToken := ((ws, token, ws) ==> #second) + propertyAt: 'trimmingToken' put: true; + yourself. + + parser := trimmingToken plus + compileWithConfiguration: configuration. + + self assert: parser parse: ' foo '. + self assert: result first inputValue = 'foo'. + + self assert: (context invocations select: [:e | e = #consumeWhitespace ]) size = 2. +! + +testWhitespace2 + | token ws trimmingToken | + configuration arguments inline: false. + + token := 'foo' asParser token. + ws := #blank asParser star name: 'consumeWhitespace'; yourself. + trimmingToken := ((ws, token, ws) ==> #second) + propertyAt: 'trimmingToken' put: true; + yourself. + + parser := trimmingToken plus + compileWithConfiguration: configuration. + + self assert: parser parse: ' foo foo '. + self assert: result first inputValue = 'foo'. + self assert: result second inputValue = 'foo'. + + self assert: (context invocations select: [:e | e = #consumeWhitespace ]) size = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCUniversalOptimizationTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCUniversalOptimizationTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,360 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +TestCase subclass:#PPCUniversalOptimizationTest + instanceVariableNames:'configuration' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Universal' +! + +!PPCUniversalOptimizationTest methodsFor:'test support'! + +assert: object type: class + self assert: (object isKindOf: class) +! + +optimize: aPPParser + ^ aPPParser compileWithConfiguration: configuration. +! + +setUp + super setUp. + + configuration := PPCUniversalConfiguration new. + configuration arguments generate: false. + +" ^ configuration := PPCPluggableConfiguration on: + [ :_self | + _self toPPCIr. + _self createTokens. + _self specialize. + _self createRecognizingComponents. + _self inline. + _self merge. + ]" +! ! + +!PPCUniversalOptimizationTest methodsFor:'tests'! + +testAnyPredicate + | tree | + tree := self optimize: #any asParser. + + self assert: tree type: PPCAnyNode. +! + +testCharSetPredicate + | tree | + tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo). + + self assert: tree type: PPCCharSetPredicateNode +! + +testChoiceInlining + | tree | + tree := self optimize: $a asParser / $b asParser. + + self assert: tree type: PPCChoiceNode. + self assert: tree children first type: PPCCharacterNode. + self assert: tree children first isMarkedForInline. + self assert: tree children second type: PPCCharacterNode. + self assert: tree children first isMarkedForInline. + +! + +testForwarding + | tree p1 p2 | + p2 := $a asParser. + p1 := p2 wrapped. + p1 name: 'p1'. + tree := self optimize: p1. + + self assert: tree type: PPCCharacterNode. + self assert: tree name = 'p1'. + + p2 name: 'p2'. + tree := self optimize: p1. + self assert: tree type: PPCForwardNode. + self assert: tree name = 'p1'. + self assert: tree child name = 'p2'. +! + +testInlineCharacter + | tree | + tree := self optimize: $a asParser plus. + + self assert: tree type: PPCPlusNode. + self assert: tree child type: PPCCharacterNode. + self assert: tree child isMarkedForInline. + self assert: tree child character = $a. +! + +testInlineCharacter2 + | tree | + tree := self optimize: $a asParser star. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCCharacterNode. + self assert: tree child isMarkedForInline. + self assert: tree child character = $a. +! + +testInlineCharacter3 + | tree | + tree := self optimize: $a asParser, $b asParser. + + self assert: tree type: PPCSequenceNode. + self assert: tree children first type: PPCCharacterNode. + self assert: tree children first isMarkedForInline. + self assert: tree children first character = $a. + self assert: tree children second type: PPCCharacterNode. + self assert: tree children second isMarkedForInline. + self assert: tree children second character = $b. +! + +testInlineNil + | tree | + tree := self optimize: nil asParser star. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCNilNode. + self assert: tree child isMarkedForInline. +! + +testInlineNotLiteral + | tree | + tree := self optimize: 'foo' asParser not star. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCNotLiteralNode. + self assert: tree child literal = 'foo'. + self assert: tree child isMarkedForInline. +! + +testInlineNotPredicate + | tree | + tree := self optimize: (#letter asParser not, (PPPredicateObjectParser on: [ :e | e = $a or: [ e = $b ]] message: #foo) not). + + self assert: tree type: PPCSequenceNode. + self assert: tree children first type: PPCNotMessagePredicateNode. + self assert: tree children first isMarkedForInline. + self assert: tree children second type: PPCNotCharSetPredicateNode. + self assert: tree children second isMarkedForInline. + +! + +testInlinePluggable + | tree | + tree := self optimize: [:ctx | nil] asParser star. + + ((Smalltalk respondsTo:#isSmalltalkX) and:[ Smalltalk isSmalltalkX ]) + ifTrue:[ self skipIf: true description: 'not supported in St/X' ]. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCPluggableNode. + self assert: tree child isMarkedForInline. + + "Modified: / 10-05-2015 / 07:30:58 / Jan Vrany " +! + +testInlinePredicate + | tree | + tree := self optimize: (#letter asParser, (PPPredicateObjectParser on: [ :e | e = $a or: [ e = $b ]] message: #foo)). + + self assert: tree type: PPCSequenceNode. + self assert: tree children first type: PPCMessagePredicateNode. + self assert: tree children first isMarkedForInline. + self assert: tree children second type: PPCCharSetPredicateNode. + self assert: tree children second isMarkedForInline. + +! + +testLetterPredicate + | tree | + tree := self optimize: #letter asParser. + + self assert: tree type: PPCMessagePredicateNode. + self assert: tree message = #isLetter. +! + +testNotAction + | tree | + tree := self optimize: (($f asParser, $o asParser) ==> #second) not. + + self assert: tree type: PPCNotNode. + self assert: tree child type: PPCRecognizingSequenceNode. +! + +testNotCharSetPredicate + | tree | + tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo) asParser not. + + self assert: tree type: PPCNotCharSetPredicateNode. +! + +testNotLiteral + | tree | + tree := self optimize: 'foo' asParser not. + + self assert: tree type: PPCNotLiteralNode. + self assert: tree literal = 'foo'. +! + +testNotMessagePredicate + | tree | + tree := self optimize: #letter asParser not. + + self assert: tree type: PPCNotMessagePredicateNode. +! + +testNotSequence + | tree | + tree := self optimize: ($f asParser, $o asParser) not. + + self assert: tree type: PPCNotNode. + self assert: tree child type: PPCRecognizingSequenceNode. +! + +testRecognizingSequence2 + | tree | + tree := self optimize: ($a asParser, $b asParser) token. + + self assert: tree type: PPCTokenNode. + self assert: tree child type: PPCRecognizingSequenceNode. + + tree := self optimize: ($a asParser, $b asParser) trimmingToken. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree child type: PPCRecognizingSequenceNode. +! + +testStarAny + | tree | + tree := self optimize: #any asParser star. + + self assert: tree type: PPCStarAnyNode. +! + +testStarCharSetPredicate + | tree | + tree := self optimize: (PPPredicateObjectParser on: [:each | each = $b or: [each = $c] ] message: #foo) star. + + self assert: tree type: PPCStarCharSetPredicateNode +! + +testStarMessagePredicate + | tree | + tree := self optimize: #letter asParser star. + + self assert: tree type: PPCStarMessagePredicateNode. +! + +testStarSeparator + | tree | + tree := self optimize: #space asParser star trimmingToken. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree child type: PPCTokenStarSeparatorNode. +! + +testStarSeparator2 + | tree | + tree := self optimize: (#space asParser star, 'whatever' asParser) trimmingToken. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree child type: PPCRecognizingSequenceNode. + self assert: tree child children first type: PPCTokenStarSeparatorNode. + self assert: tree child children first isMarkedForInline. +! + +testSymbolAction + | tree | + tree := self optimize: (#letter asParser) ==> #second. + + self assert: tree type: PPCSymbolActionNode. + + tree := self optimize: (#letter asParser) ==> [:e | e second ]. + self assert: tree type: PPCActionNode. +! + +testToken + | tree | + tree := self optimize: ((#letter asParser, #word asParser star) token). + + self assert: tree type: PPCTokenNode. + self assert: tree child type: PPCRecognizingSequenceNode. + self assert: tree child children size = 2. + self assert: tree child children first type: PPCMessagePredicateNode. + self assert: tree child children first isMarkedForInline. + self assert: tree child children second type: PPCTokenStarMessagePredicateNode. + self assert: tree child children second isMarkedForInline. + +! + +testTrimmingToken + | tree | + tree := self optimize: ((#letter asParser, #word asParser star) trimmingToken). + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree whitespace type: PPCTokenStarSeparatorNode. + self assert: tree whitespace isMarkedForInline. + + self assert: tree child type: PPCRecognizingSequenceNode. + self assert: tree child children size = 2. + self assert: tree child children first type: PPCMessagePredicateNode. + self assert: tree child children first isMarkedForInline. + self assert: tree child children second type: PPCTokenStarMessagePredicateNode. + self assert: tree child children first isMarkedForInline. +! + +testTrimmingToken2 + | parser tree | + parser := 'foo' asParser trimmingToken. + tree := self optimize: parser. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree child type: PPCLiteralNode. + self assert: tree child isMarkedForInline. + self assert: (tree whitespace allNodes allSatisfy: [ :n | n isKindOf: PPCNode ]). + + parser := ('foo' asParser, $b asParser) trimmingToken. + tree := self optimize: parser. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree child type: PPCRecognizingSequenceNode. + self assert: tree whitespace type: PPCTokenStarSeparatorNode. + self assert: tree whitespace isMarkedForInline. + + parser := $d asParser trimmingToken star. + tree := self optimize: parser. + + self assert: tree type: PPCStarNode. + self assert: tree child type: PPCTrimmingTokenNode. + self assert: tree child child type: PPCCharacterNode. + self assert: tree child child isMarkedForInline. +! + +testTrimmingToken3 + | parser tree | + parser := ('foo' asParser trimmingToken name: 'foo'), ('bar' asParser trimmingToken name: 'bar'). + tree := self optimize: parser. + + self assert: tree type: PPCSequenceNode. + self assert: tree children first type: PPCTrimmingTokenNode. + self assert: tree children second type: PPCTrimmingTokenNode. +! + +testTrimmingTokenNested + | parser tree foo| + foo := 'foo' asParser trimmingToken name: 'foo'. + parser := (foo not, 'bar' asParser) trimmingToken name: 'token'. + tree := self optimize: parser. + + self assert: tree type: PPCTrimmingTokenNode. + self assert: tree children second type: PPCRecognizingSequenceNode. + self assert: tree children second children first type: PPCNotLiteralNode. + self assert: tree children second children first isMarkedForInline. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCUniversalTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/PPCUniversalTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,579 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests' }" + +"{ NameSpace: Smalltalk }" + +PPAbstractParserTest subclass:#PPCUniversalTest + instanceVariableNames:'parser result context node compiler id node2 id2 id1 node1 node3 + arguments configuration' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Tests-Core-Universal' +! + +!PPCUniversalTest methodsFor:'context'! + +context + ^ context := PPCProfilingContext new +! ! + +!PPCUniversalTest methodsFor:'test support'! + +assert: p parse: whatever + ^ result := super assert: p parse: whatever. +! + +parse: whatever + ^ result := super parse: whatever. +! + +tearDown + | parserClass | + + parserClass := (Smalltalk at: arguments name ifAbsent: [nil]). + parserClass notNil ifTrue:[ + parserClass removeFromSystem + ]. +! ! + +!PPCUniversalTest methodsFor:'tests - compiling'! + +testCompileAnd + parser := #digit asParser and compileWithConfiguration: configuration. + + self assert: parser parse: '1' to: $1 end: 0. + self assert: parser fail: 'a'. + self assert: parser fail: ''. + + parser := ('foo' asParser, ($: asParser and)) compile. + self assert: parser parse: 'foo:' to: { 'foo'. $: } end: 3. +! + +testCompileAny + parser := #any asParser compile. + + self assert: parser parse: 'a' to: $a. + self assert: parser parse: '_' to: $_. + self assert: parser parse: ' +' to: Character cr. +! + +testCompileAnyStar + parser := #any asParser star compileWithConfiguration: configuration. + + + self assert: parser parse: 'aaa' to: { $a. $a . $a }. + self assert: context invocationCount = 1. + self assert: parser parse: '' to: { }. +! + +testCompileBlock + parser := (#letter asParser) plus ==> [ :res | res collect: [:each | each asUppercase ]]. + parser := parser compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: { $F . $O . $O}. + self assert: parser parse: 'bar' to: { $B . $A . $R}. + self assert: parser fail: ''. +! + +testCompileCharacter + parser := $a asParser compileWithConfiguration: configuration. + + self assert: parser parse: 'a' to: $a. + self assert: parser fail: 'b'. + + parser := $# asParser compileWithConfiguration: configuration. + self assert: parser parse: '#'. +! + +testCompileChoice + parser := (#digit asParser / #letter asParser) compileWithConfiguration: configuration. + + self assert: parser parse: '1' to: $1. + self assert: parser parse: 'a' to: $a. + self assert: parser fail: '_'. + +! + +testCompileChoice2 + parser := ('true' asParser / 'false' asParser) compileWithConfiguration: configuration. + + self assert: parser parse: 'true' to: 'true'. + self assert: parser parse: 'false' to: 'false'. + self assert: parser fail: 'trulse'. + +! + +testCompileLiteral + parser := 'foo' asParser compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: 'foo'. + self assert: parser parse: 'foobar' to: 'foo' end: 3. + self assert: parser fail: 'boo'. + + parser := '#[' asParser compileWithConfiguration: configuration. + self assert: parser parse: '#[1]' to: '#[' end: 2. +! + +testCompileLiteral2 + | quote | + quote := '''' asParser. + parser := (quote, $a asParser ) compileWithConfiguration: configuration. + self assert: parser parse: '''a' to: {'''' . $a}. +! + +testCompileNegate + parser := #letter asParser negate star, #letter asParser. + parser := parser compileWithConfiguration: configuration. + + self assert: parser parse: '...a' to: { { $. . $. . $. } . $a }. + self assert: parser parse: 'aaa' to: { {} . $a } end: 1. + self assert: parser fail: '...'. +! + +testCompileNil + parser := nil asParser compileWithConfiguration: configuration. + + self assert: parser parse: 'a' to: nil end: 0. + self assert: parser parse: '' to: nil end: 0. + + parser := nil asParser, 'foo' asParser. + self assert: parser parse: 'foo' to: { nil . 'foo' } +! + +testCompileNot + parser := #digit asParser not compileWithConfiguration: configuration. + + self assert: parser parse: 'a' to: nil end: 0. + self assert: parser fail: '1'. + self assert: parser parse: '' to: nil end: 0. + + parser := 'foo' asParser, $: asParser not. + parser := parser compileWithConfiguration: configuration. + self assert: parser parse: 'foo' to: { 'foo'. nil } end: 3. + + parser := 'foo' asParser, $: asParser not, 'bar' asParser. + parser := parser compileWithConfiguration: configuration. + self assert: parser parse: 'foobar' to: { 'foo'. nil . 'bar' } end: 6. +! + +testCompileNot2 + parser := ($a asParser, $b asParser) not compileWithConfiguration: configuration. + + self assert: parser parse: '' to: nil end: 0. + self assert: parser parse: 'a' to: nil end: 0. + self assert: parser parse: 'aa' to: nil end: 0. + self assert: parser fail: 'ab'. +! + +testCompileNot3 + parser := ('foo' asParser not, 'fee' asParser) compileWithConfiguration: configuration. + + self assert: parser parse: 'fee' to: #(nil 'fee'). + self assert: parser fail: 'foo'. +! + +testCompileNotLiteral + parser := 'foo' asParser not compileWithConfiguration: configuration. + self assert: parser class methodDictionary size = 1. + + self assert: parser parse: 'bar' to: nil end: 0. + + self assert: parser fail: 'foo'. + self assert: parser parse: '' to: nil end: 0. + + parser := '''' asParser not compile. + self assert: parser class methodDictionary size = 1. + + self assert: parser parse: 'a' to: nil end: 0. + self assert: parser fail: ''''. + self assert: parser parse: '' to: nil end: 0. + + + parser := ('foo' asParser, 'bar' asParser not) compile. + self assert: parser parse: 'foofoo' to: { 'foo'. nil } end: 3. + + parser := ('foo' asParser, 'foo' asParser not, #any asParser star) compile. + self assert: parser parse: 'foobar' to: { 'foo'. nil . #($b $a $r) } end: 6. + self assert: parser fail: 'foofoo'. +! + +testCompileOptional + parser := #digit asParser optional compileWithConfiguration: configuration. + + self assert: parser parse: '1' to: $1. + self assert: parser parse: 'a' to: nil end: 0. + + parser := (#digit asParser optional, #letter asParser) compile. + self assert: parser parse: '1a' to: { $1 . $a }. + self assert: parser parse: 'a' to: { nil . $a }. +! + +testCompilePlus + parser := #letter asParser plus compileWithConfiguration: configuration. + + self assert: parser parse: 'lorem' to: {$l. $o. $r. $e. $m} . + self assert: parser parse: 'a123' to: {$a} end: 1. + self assert: parser parse: 'ab123' to: {$a . $b} end: 2. + + self assert: parser fail: ''. + self assert: parser fail: '123'. +! + +testCompilePredicate + parser := #digit asParser compileWithConfiguration: configuration. + + self assert: parser parse: '1' to: $1. + self assert: parser parse: '0' to: $0. + self assert: parser fail: 'a'. +! + +testCompilePredicate2 + parser := #space asParser compileWithConfiguration: configuration. + + self assert: parser parse: ' ' to: Character space. + self assert: parser fail: 'a'. +! + +testCompileSequence + parser := (#digit asParser, #letter asParser) compileWithConfiguration: configuration. + + self assert: parser parse: '1a' to: {$1 .$a}. + + +! + +testCompileSequence2 + parser := (#digit asParser, #space asParser, #letter asParser) compileWithConfiguration: configuration. + + self assert: parser parse: '9 c' to: {$9 . Character space. $c }. + self assert: parser fail: '9c'. + +! + +testCompileSequence3 + parser := (#any asParser, #any asParser, #any asParser) compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: #($f $o $o). + self assert: parser fail: 'fo'. + +! + +testCompileStar + parser := #letter asParser star compileWithConfiguration: configuration. + + self assert: parser parse: 'lorem' to: {$l. $o. $r. $e. $m} . + self assert: parser parse: '' to: {}. + self assert: parser parse: '123' to: {} end: 0. + self assert: parser parse: 'ab123' to: {$a . $b} end: 2. +! + +testCompileStarLiteral + parser := 'foo' asParser star compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: #('foo' ) . + self assert: parser parse: 'foofoo' to: #('foo' 'foo') . + self assert: parser parse: 'foofoofoo' to: #('foo' 'foo' 'foo') . + self assert: parser parse: '' to: #(). + self assert: parser parse: 'bar' to: #() end: 0. +! + +testCompileStarPredicate + parser := #letter asParser star compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: #($f $o $o ) . + self assert: parser parse: '' to: #(). + self assert: parser parse: '123' to: #() end: 0. +! + +testCompileSymbolBlock + parser := (#letter asParser) plus ==> #second. + parser := parser compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: $o. + self assert: parser parse: 'bar' to: $a. + self assert: parser fail: ''. + self should: [ parser parse: 'f' ] raise: Error. +! + +testCompileTrim + parser := $a asParser trim compileWithConfiguration: configuration. + + self assert: parser fail: ''. + self assert: parser parse: 'a' to: $a. + self assert: parser parse: ' a' to: $a. + self assert: parser parse: 'a ' to: $a. + self assert: parser parse: ' a ' to: $a. +! + +testCompileTrimmingToken + | token1 token2 | + token1 := (#letter asParser) plus trimmingToken. + token2 := (#letter asParser) plus trimmingToken. + + parser := (token1, token2) compileWithConfiguration: configuration. + + self assert: parser parse: 'foo bar'. + self assert: parser parse: ' foo bar '. +! + +testCompileTrimmingToken2 + | token1 token2 | + token1 := (#letter asParser) plus trimmingToken. + token2 := (#letter asParser) plus trimmingToken / 'foo' asParser trimmingToken. + + parser := (token1, token2) compileWithConfiguration: configuration. + + self assert: parser parse: 'foo bar'. + self assert: parser parse: ' foo bar '. +! + +testCompileTrimmingToken3 + | token1 token2 | + token1 := ($a asParser, $b asParser) trimmingToken name: 'token1'. + token2 := (token1 not, $c asParser) trimmingToken name: 'token2'. + + parser := (token1 / token2) compileWithConfiguration: configuration. + + self assert: (parser class methodDictionary includesKey: #'token1'). + self assert: (parser class methodDictionary includesKey: #'token1_fast'). + + self assert: parser parse: 'ab'. + self assert: (result isKindOf: PPToken). + self assert: result inputValue = 'ab'. + + self assert: parser parse: 'c'. + self assert: (result isKindOf: PPToken). + self assert: result inputValue = 'c'. + +! ! + +!PPCUniversalTest methodsFor:'tests - extra'! + +testCompileSmalltalkToken + parser := (#letter asParser, ((#letter asParser / #digit asParser) star)) smalltalkToken compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: parser parse: 'a'. + self assert: result inputValue = 'a'. + self assert: parser parse: 'f123a'. + self assert: result inputValue = 'f123a'. + + self assert: parser fail: ''. + self assert: parser fail: '12'. + + self assert: parser parse: ' "comment" foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: ' "comment" bar "another comment" '. + self assert: result inputValue = 'bar'. + self assert: parser parse: ' + "b" + "b" + foo + "and yet, another comment" + + "one more to make sure :)" + '. + self assert: result inputValue = 'foo'. +! + +testCycle + | p1 block | + + p1 := PPDelegateParser new. + block := ${ asParser, p1, $} asParser / nil asParser. + p1 setParser: block. + + parser := block compileWithConfiguration: configuration. + self assert: parser parse: '{}' to: { ${. nil . $} }. + self assert: parser parse: '{{}}' to: { ${. { ${ . nil . $} } . $} }. + +! + +testSmalltalkToken + parser := (#letter asParser, (#digit asParser / #letter asParser) star) smalltalkToken compileWithConfiguration: configuration. + + self assert: parser class methodDictionary size = 5. + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: context invocationCount = 8. + self assert: context rememberCount = 0. + self assert: context lwRememberCount = 0. + self assert: context lwRestoreCount = 0. +! + +testSmalltalkToken2 + id := (#letter asParser, (#digit asParser / #letter asParser) star) + name: 'identifier'; + yourself. + + parser := (id wrapped, $: asParser) smalltalkToken + name: 'kw'; + yourself. + + parser := parser compileWithConfiguration: configuration. + + self assert: parser parse: 'foo:'. + self assert: result inputValue = 'foo:'. +! + +testToken + parser := (#letter asParser, (#digit asParser / #letter asParser) star) flatten compileWithConfiguration: configuration. + + self assert: parser parse: 'foo' to: 'foo'. + self assert: parser parse: 'a' to: 'a'. + self assert: parser parse: 'f123a' to: 'f123a'. + self assert: parser fail: ''. +! + +testToken2 + parser := (#letter asParser, (#digit asParser / #letter asParser) star) token compileWithConfiguration: configuration. + + self assert: parser class methodDictionary size = 4. + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: context invocationCount = 6. + self assert: context rememberCount = 0. + self assert: context lwRememberCount = 0. + self assert: context lwRestoreCount = 0. +! + +testTrimmingToken + parser := (#letter asParser, (#digit asParser / #letter asParser) star) trimmingToken compileWithConfiguration: configuration. + + self assert: parser class methodDictionary size = 4. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: context invocationCount = 6. + self assert: context rememberCount = 0. + self assert: context lwRememberCount = 0. + self assert: context lwRestoreCount = 0. + + self assert: parser parse: ' foo '. + self assert: result inputValue = 'foo'. + + + + self assert: parser fail: '123'. + + self assert: context invocationCount = 1. + self assert: context rememberCount = 0. + self assert: context lwRememberCount = 0. + self assert: context lwRestoreCount = 0. + + + self assert: parser fail: ''. +! + +testTrimmingToken2 + + parser := 'foo' asParser trimmingToken, 'bar' asParser trimmingToken + compileWithConfiguration: configuration. + + self assert: parser parse: 'foobar'. + self assert: result first inputValue = 'foo'. + self assert: result second inputValue = 'bar'. + self assert: context invocationCount = 3. + + self assert: parser parse: ' foobar'. + self assert: result first inputValue = 'foo'. + self assert: result second inputValue = 'bar'. + self assert: context invocationCount = 3. + self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). + + self assert: parser fail: 'bar'. + self assert: context invocationCount = 2. +" self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). " + +! + +testTrimmingToken3 + + parser := ('foo' asParser trimmingToken / 'bar' asParser trimmingToken) + compileWithConfiguration: configuration. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + self assert: context invocationCount = 2. + + self assert: parser parse: ' bar'. + self assert: result inputValue = 'bar'. + self assert: context invocationCount = 2. + self assert: (context invocations anySatisfy: [ :e | e beginsWith: 'token' ]). + + self assert: parser fail: 'baz'. + self assert: context invocationCount = 2. + + self assert: parser fail: 'zaz'. + self assert: context invocationCount = 1. + self assert: (context invocations noneSatisfy: [ :e | e beginsWith: 'token' ]). +! + +testTrimmingTokenNested + | identifier kw | + kw := 'false' asParser trimmingToken name: #kw. + identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. + + parser := identifier / kw. + parser := parser compileWithConfiguration: configuration. + self assert: parser class methodDictionary size = 5. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: 'false'. + self assert: result inputValue = 'false'. +! + +testTrimmingTokenNested2 + | identifier kw | + kw := 'false' asParser trimmingToken name: #kw. + identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. + + parser := identifier / kw. + parser := parser compileWithConfiguration: configuration. + self assert: parser class methodDictionary size = 5. + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: 'false'. + self assert: result inputValue = 'false'. +! + +testTrimmingTokenNested3 + | identifier kw | + kw := ('false' asParser, #word asParser not) trimmingToken name: #kw. + identifier := (kw not, (#letter asParser, #word asParser star)) trimmingToken name: #identifier. + + parser := identifier / kw. + parser := parser compileWithConfiguration: configuration. + self assert: parser class methodDictionary size = 8. + self assert: (parser class methodDictionary values anySatisfy: [ :m | m selector = #kw ]). + self assert: (parser class methodDictionary values anySatisfy: [ :m | m selector = #kw_fast ]). + + self assert: parser parse: 'foo'. + self assert: result inputValue = 'foo'. + + self assert: parser parse: 'false'. + self assert: result inputValue = 'false'. + + "Modified: / 10-05-2015 / 07:33:19 / Jan Vrany " +! ! + +!PPCUniversalTest methodsFor:'tests - ids'! + +setUp + arguments := PPCArguments default + profile: true; + debug: true; + yourself. + + configuration := PPCUniversalConfiguration new + arguments: arguments; + yourself. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCompiledExpressionGrammarResource.st --- a/compiler/tests/PPCompiledExpressionGrammarResource.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,26 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -TestResource subclass:#PPCompiledExpressionGrammarResource - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPCompiledExpressionGrammarResource methodsFor:'as yet unclassified'! - -setUp - | time configuration | - configuration := PPCConfiguration universal. - configuration arguments name: #PPCompiledExpressionGrammar. - - - time := Time millisecondsToRun: [ - PPExpressionGrammar new compileWithConfiguration: configuration. - ]. - Transcript show: 'Expression grammar compiled in: ', time asString, 'ms'; cr. - -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPCompiledExpressionGrammarTest.st --- a/compiler/tests/PPCompiledExpressionGrammarTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,123 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParserTest subclass:#PPCompiledExpressionGrammarTest - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPCompiledExpressionGrammarTest class methodsFor:'as yet unclassified'! - -resources - ^ (OrderedCollection with: PPCompiledExpressionGrammarResource) - addAll: super resources; - yourself -! ! - -!PPCompiledExpressionGrammarTest methodsFor:'as yet unclassified'! - -compilerArguments - ^ PPCArguments default - profile: true; - ll: true; - yourself -! - -context - ^ PPCContext new -! - -parserClass - ^ Smalltalk at: #PPCompiledExpressionGrammar -! - -parserInstanceFor: aSymbol - ^ (Smalltalk at: #PPCompiledExpressionGrammar) new startSymbol: aSymbol -! - -testAdd - result := self parse: '1+2' rule: #add. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. -! - -testMul - result := self parse: '1 * 2' rule: #mul. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '*'. - self assert: result third = 2. -! - -testNumber - result := self parse: '1' rule: #number. - self assert: result = 1. -! - -testParens - result := self parse: '(1)' rule: #parens. - self assert: result size = 3. - self assert: result first inputValue = '('. - self assert: result second = 1. - self assert: result third inputValue = ')'. - -! - -testPrim - result := self parse: '1' rule: #prim. - self assert: result = 1. -! - -testPrim2 - result := self parse: '(1)' rule: #prim. - self assert: result size = 3. - self assert: result second = 1. -! - -testProd - result := self parse: '1' rule: #prod. - self assert: result = 1. -! - -testTerm - result := self parse: '1' rule: #term. - self assert: result = 1. - -! - -testTerm11 - result := self parse: '1 + 2' rule: #term. - self assert: result size = 3. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. - -! - -testTerm12 - result := self parse: '1 + 2 * 3' rule: #term. - self assert: result size = 3. - self assert: result second inputValue = '+'. - self assert: result first = 1. - self assert: result third isArray. - self assert: result third first = 2. - self assert: result third second inputValue = '*'. - self assert: result third third = 3. -! - -testTerm13 - result := self parse: '1 * 2 + 3' rule: #term. - self assert: result size = 3. - self assert: result first isArray. - self assert: result first first = 1. - self assert: result first second inputValue = '*'. - self assert: result first third = 2. - self assert: result second inputValue = '+'. - self assert: result third = 3. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPExpressionGrammar.st --- a/compiler/tests/PPExpressionGrammar.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,45 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParser subclass:#PPExpressionGrammar - instanceVariableNames:'add prod term mul prim parens number' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPExpressionGrammar methodsFor:'as yet unclassified'! - -add - ^ prod , $+ asParser trimmingToken , term -! - -mul - ^ prim , $* asParser trimmingToken , prod -! - -number - ^ #digit asParser plus trimmingToken ==> [ :token | token inputValue asNumber ] -! - -parens - ^ ($( asParser token trim), term , ($) asParser token trim) -! - -prim - ^ parens / number -! - -prod - ^ mul / prim -! - -start - ^ term end -! - -term - ^ add / prod -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPExpressionGrammarTest.st --- a/compiler/tests/PPExpressionGrammarTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,100 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParserTest subclass:#PPExpressionGrammarTest - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPExpressionGrammarTest methodsFor:'as yet unclassified'! - -parserClass - ^ PPExpressionGrammar -! - -testAdd - result := self parse: '1+2' rule: #add. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. -! - -testMul - result := self parse: '1 * 2' rule: #mul. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '*'. - self assert: result third = 2. -! - -testNumber - result := self parse: '1' rule: #number. - self assert: result = 1. -! - -testParens - result := self parse: '(1)' rule: #parens. - self assert: result size = 3. - self assert: result first inputValue = '('. - self assert: result second = 1. - self assert: result third inputValue = ')'. - -! - -testPrim - result := self parse: '1' rule: #prim. - self assert: result = 1. -! - -testPrim2 - result := self parse: '(1)' rule: #prim. - self assert: result size = 3. - self assert: result second = 1. -! - -testProd - result := self parse: '1' rule: #prod. - self assert: result = 1. -! - -testTerm - result := self parse: '1' rule: #term. - self assert: result = 1. - -! - -testTerm11 - result := self parse: '1 + 2' rule: #term. - self assert: result size = 3. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. - -! - -testTerm12 - result := self parse: '1 + 2 * 3' rule: #term. - self assert: result size = 3. - self assert: result second inputValue = '+'. - self assert: result first = 1. - self assert: result third isArray. - self assert: result third first = 2. - self assert: result third second inputValue = '*'. - self assert: result third third = 3. -! - -testTerm13 - result := self parse: '1 * 2 + 3' rule: #term. - self assert: result size = 3. - self assert: result first isArray. - self assert: result first first = 1. - self assert: result first second inputValue = '*'. - self assert: result first third = 2. - self assert: result second inputValue = '+'. - self assert: result third = 3. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPLL1ExpressionGrammar.st --- a/compiler/tests/PPLL1ExpressionGrammar.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,74 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParser subclass:#PPLL1ExpressionGrammar - instanceVariableNames:'add prod term mul prim parens number mulPrime addPrime termPrime' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPLL1ExpressionGrammar methodsFor:'as yet unclassified'! - -add - ^ prod, addPrime optional - map: [ :_prod :_addPrime | - _addPrime isNil - ifTrue: [ _prod ] - ifFalse: [ Array with: _prod withAll: _addPrime ] - - ] -! - -addPrime - ^ $+ asParser trimmingToken , term -! - -mul - ^ prim, mulPrime optional - map: [ :_prim :_mulPrime | - _mulPrime isNil - ifTrue: [ _prim ] - ifFalse: [ Array with: _prim withAll: _mulPrime ] - - ] -! - -mulPrime - ^ $* asParser trimmingToken, prod -! - -number - ^ #digit asParser plus trimmingToken ==> [ :token | token inputValue asNumber ] -! - -parens - ^ ($( asParser token trim), term , ($) asParser token trim) -! - -prim - ^ parens / number -! - -prod - ^ mul -! - -start - ^ term end -! - -term - ^ prod, termPrime optional - map: [ :_prod :_termPrime | - _termPrime isNil - ifTrue: [ _prod ] - ifFalse: [ Array with: _prod withAll: _termPrime ] - ] -! - -termPrime - ^ $+ asParser trimmingToken, term -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPLL1ExpressionGrammarTest.st --- a/compiler/tests/PPLL1ExpressionGrammarTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,100 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParserTest subclass:#PPLL1ExpressionGrammarTest - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPLL1ExpressionGrammarTest methodsFor:'as yet unclassified'! - -parserClass - ^ PPLL1ExpressionGrammar -! - -testAdd - result := self parse: '1+2' rule: #add. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. -! - -testMul - result := self parse: '1 * 2' rule: #mul. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '*'. - self assert: result third = 2. -! - -testNumber - result := self parse: '1' rule: #number. - self assert: result = 1. -! - -testParens - result := self parse: '(1)' rule: #parens. - self assert: result size = 3. - self assert: result first inputValue = '('. - self assert: result second = 1. - self assert: result third inputValue = ')'. - -! - -testPrim - result := self parse: '1' rule: #prim. - self assert: result = 1. -! - -testPrim2 - result := self parse: '(1)' rule: #prim. - self assert: result size = 3. - self assert: result second = 1. -! - -testProd - result := self parse: '1' rule: #prod. - self assert: result = 1. -! - -testTerm - result := self parse: '1' rule: #term. - self assert: result = 1. - -! - -testTerm11 - result := self parse: '1 + 2' rule: #term. - self assert: result size = 3. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. - -! - -testTerm12 - result := self parse: '1 + 2 * 3' rule: #term. - self assert: result size = 3. - self assert: result second inputValue = '+'. - self assert: result first = 1. - self assert: result third isArray. - self assert: result third first = 2. - self assert: result third second inputValue = '*'. - self assert: result third third = 3. -! - -testTerm13 - result := self parse: '1 * 2 + 3' rule: #term. - self assert: result size = 3. - self assert: result first isArray. - self assert: result first first = 1. - self assert: result first second inputValue = '*'. - self assert: result first third = 2. - self assert: result second inputValue = '+'. - self assert: result third = 3. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPTokenizedExpressionGrammarResource.st --- a/compiler/tests/PPTokenizedExpressionGrammarResource.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,26 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -TestResource subclass:#PPTokenizedExpressionGrammarResource - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPTokenizedExpressionGrammarResource methodsFor:'as yet unclassified'! - -setUp - | time configuration | - configuration := PPCLL1Configuration new. - configuration arguments name:#PPTokenizedExpressionGrammar. - - - time := Time millisecondsToRun: [ - PPExpressionGrammar new compileWithConfiguration: configuration. - ]. - Transcript crShow: 'Expression grammar tokenized in: ', time asString, 'ms'. - -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPTokenizedExpressionGrammarTest.st --- a/compiler/tests/PPTokenizedExpressionGrammarTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,130 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParserTest subclass:#PPTokenizedExpressionGrammarTest - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - - -!PPTokenizedExpressionGrammarTest class methodsFor:'as yet unclassified'! - -resources - ^ (OrderedCollection with: PPTokenizedExpressionGrammarResource) - addAll: super resources; - yourself -! ! - -!PPTokenizedExpressionGrammarTest methodsFor:'as yet unclassified'! - -compilerArguments - ^ PPCArguments default - profile: true; - yourself -! - -context - ^ PPCContext new -! - -parserClass - ^ Smalltalk at: #PPTokenizedExpressionGrammar -! - -parserInstanceFor: aSymbol - ^ (Smalltalk at: #PPTokenizedExpressionGrammar) new startSymbol: aSymbol -! - -testAdd - result := self parse: '1+2' rule: #add. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. -! - -testMul - result := self parse: '1 * 2' rule: #mul. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '*'. - self assert: result third = 2. -! - -testNumber - result := self parse: '1' rule: #number. - self assert: result = 1. -! - -testParens - result := self parse: '(1)' rule: #parens. - self assert: result size = 3. - self assert: result first inputValue = '('. - self assert: result second = 1. - self assert: result third inputValue = ')'. - -! - -testPrim - result := self parse: '1' rule: #prim. - self assert: result = 1. -! - -testPrim2 - result := self parse: '(1)' rule: #prim. - self assert: result size = 3. - self assert: result second = 1. -! - -testProd - result := self parse: '1' rule: #prod. - self assert: result = 1. -! - -testTerm - result := self parse: '1' rule: #term. - self assert: result = 1. - -! - -testTerm11 - result := self parse: '1 + 2' rule: #term. - self assert: result size = 3. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. - -! - -testTerm12 - result := self parse: '1 + 2 * 3' rule: #term. - self assert: result size = 3. - self assert: result second inputValue = '+'. - self assert: result first = 1. - self assert: result third isArray. - self assert: result third first = 2. - self assert: result third second inputValue = '*'. - self assert: result third third = 3. -! - -testTerm13 - result := self parse: '1 * 2 + 3' rule: #term. - self assert: result size = 3. - self assert: result first isArray. - self assert: result first first = 1. - self assert: result first second inputValue = '*'. - self assert: result first third = 2. - self assert: result second inputValue = '+'. - self assert: result third = 3. -! ! - -!PPTokenizedExpressionGrammarTest class methodsFor:'documentation'! - -version_HG - - ^ '$Changeset: $' -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPTokenizedLL1ExpressionGrammarResource.st --- a/compiler/tests/PPTokenizedLL1ExpressionGrammarResource.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,26 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -TestResource subclass:#PPTokenizedLL1ExpressionGrammarResource - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPTokenizedLL1ExpressionGrammarResource methodsFor:'as yet unclassified'! - -setUp - | time configuration | - configuration := PPCLL1Configuration new. - configuration arguments name:#PPTokenizedLL1ExpressionGrammar. - - - time := Time millisecondsToRun: [ - PPLL1ExpressionGrammar new compileWithConfiguration: configuration. - ]. - Transcript crShow: 'Expression grammar compiled in: ', time asString, 'ms'. - -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/PPTokenizedLL1ExpressionGrammarTest.st --- a/compiler/tests/PPTokenizedLL1ExpressionGrammarTest.st Tue May 12 01:24:03 2015 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,123 +0,0 @@ -"{ Package: 'stx:goodies/petitparser/compiler/tests' }" - -"{ NameSpace: Smalltalk }" - -PPCompositeParserTest subclass:#PPTokenizedLL1ExpressionGrammarTest - instanceVariableNames:'' - classVariableNames:'' - poolDictionaries:'' - category:'PetitCompiler-Tests-ExpressionGrammar' -! - -!PPTokenizedLL1ExpressionGrammarTest class methodsFor:'as yet unclassified'! - -resources - ^ (OrderedCollection with: PPTokenizedLL1ExpressionGrammarResource) - addAll: super resources; - yourself -! ! - -!PPTokenizedLL1ExpressionGrammarTest methodsFor:'as yet unclassified'! - -compilerArguments - ^ PPCArguments default - profile: true; - ll: true; - yourself -! - -context - ^ PPCContext new -! - -parserClass - ^ Smalltalk at: #PPTokenizedLL1ExpressionGrammar -! - -parserInstanceFor: aSymbol - ^ (Smalltalk at: #PPCompiledExpressionGrammar) new startSymbol: aSymbol -! - -testAdd - result := self parse: '1+2' rule: #add. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. -! - -testMul - result := self parse: '1 * 2' rule: #mul. - self assert: result isArray. - self assert: result first = 1. - self assert: result second inputValue = '*'. - self assert: result third = 2. -! - -testNumber - result := self parse: '1' rule: #number. - self assert: result = 1. -! - -testParens - result := self parse: '(1)' rule: #parens. - self assert: result size = 3. - self assert: result first inputValue = '('. - self assert: result second = 1. - self assert: result third inputValue = ')'. - -! - -testPrim - result := self parse: '1' rule: #prim. - self assert: result = 1. -! - -testPrim2 - result := self parse: '(1)' rule: #prim. - self assert: result size = 3. - self assert: result second = 1. -! - -testProd - result := self parse: '1' rule: #prod. - self assert: result = 1. -! - -testTerm - result := self parse: '1' rule: #term. - self assert: result = 1. - -! - -testTerm11 - result := self parse: '1 + 2' rule: #term. - self assert: result size = 3. - self assert: result first = 1. - self assert: result second inputValue = '+'. - self assert: result third = 2. - -! - -testTerm12 - result := self parse: '1 + 2 * 3' rule: #term. - self assert: result size = 3. - self assert: result second inputValue = '+'. - self assert: result first = 1. - self assert: result third isArray. - self assert: result third first = 2. - self assert: result third second inputValue = '*'. - self assert: result third third = 3. -! - -testTerm13 - result := self parse: '1 * 2 + 3' rule: #term. - self assert: result size = 3. - self assert: result first isArray. - self assert: result first first = 1. - self assert: result first second inputValue = '*'. - self assert: result first third = 2. - self assert: result second inputValue = '+'. - self assert: result third = 3. -! ! - diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/abbrev.stc --- a/compiler/tests/abbrev.stc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/abbrev.stc Thu May 21 14:12:22 2015 +0100 @@ -6,32 +6,23 @@ PPCContextMementoTest PPCContextMementoTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Context' 1 PPCContextTest PPCContextTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Context' 1 PPCCopyVisitorTest PPCCopyVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 -PPCGuardTest PPCGuardTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 +PPCGuardTest PPCGuardTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Guards' 1 PPCInliningVisitorTest PPCInliningVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 -PPCLL1OptimizingTest PPCLL1OptimizingTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 -PPCLL1Test PPCLL1Test stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 PPCLL1VisitorTest PPCLL1VisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 +PPCLTokenizingOptimizationTest PPCLTokenizingOptimizationTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Tokenizing' 1 PPCMergingVisitorTest PPCMergingVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 PPCMockCompiler PPCMockCompiler stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 0 -PPCNodeFirstFollowNextTests PPCNodeFirstFollowNextTests stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Nodes' 1 +PPCNodeFirstFollowNextTests PPCNodeFirstFollowNextTests stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Nodes' 2 PPCNodeTest PPCNodeTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Nodes' 1 PPCOptimizeChoicesTest PPCOptimizeChoicesTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 -PPCPrototype1OptimizingTest PPCPrototype1OptimizingTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 -PPCPrototype1Test PPCPrototype1Test stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core' 1 PPCRecognizerComponentDetectorTest PPCRecognizerComponentDetectorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 PPCRecognizerComponentVisitorTest PPCRecognizerComponentVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 PPCSpecializingVisitorTest PPCSpecializingVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 PPCTokenDetectorTest PPCTokenDetectorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 +PPCTokenGuardTest PPCTokenGuardTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Guards' 1 PPCTokenizingCodeGeneratorTest PPCTokenizingCodeGeneratorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 +PPCTokenizingTest PPCTokenizingTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Tokenizing' 1 PPCTokenizingVisitorTest PPCTokenizingVisitorTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Visitors' 1 -PPCompiledExpressionGrammarResource PPCompiledExpressionGrammarResource stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPCompiledExpressionGrammarTest PPCompiledExpressionGrammarTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPExpressionGrammar PPExpressionGrammar stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 0 -PPExpressionGrammarTest PPExpressionGrammarTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPLL1ExpressionGrammar PPLL1ExpressionGrammar stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 0 -PPLL1ExpressionGrammarTest PPLL1ExpressionGrammarTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPTokenizedExpressionGrammarResource PPTokenizedExpressionGrammarResource stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPTokenizedExpressionGrammarTest PPTokenizedExpressionGrammarTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPTokenizedLL1ExpressionGrammarResource PPTokenizedLL1ExpressionGrammarResource stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 -PPTokenizedLL1ExpressionGrammarTest PPTokenizedLL1ExpressionGrammarTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-ExpressionGrammar' 1 +PPCUniversalOptimizationTest PPCUniversalOptimizationTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Universal' 1 +PPCUniversalTest PPCUniversalTest stx:goodies/petitparser/compiler/tests 'PetitCompiler-Tests-Core-Universal' 1 stx_goodies_petitparser_compiler_tests stx_goodies_petitparser_compiler_tests stx:goodies/petitparser/compiler/tests '* Projects & Packages *' 3 diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/bc.mak --- a/compiler/tests/bc.mak Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/bc.mak Thu May 21 14:12:22 2015 +0100 @@ -82,32 +82,23 @@ $(OUTDIR)PPCCopyVisitorTest.$(O) PPCCopyVisitorTest.$(H): PPCCopyVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCGuardTest.$(O) PPCGuardTest.$(H): PPCGuardTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCInliningVisitorTest.$(O) PPCInliningVisitorTest.$(H): PPCInliningVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1OptimizingTest.$(O) PPCLL1OptimizingTest.$(H): PPCLL1OptimizingTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCLL1Test.$(O) PPCLL1Test.$(H): PPCLL1Test.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCLL1VisitorTest.$(O) PPCLL1VisitorTest.$(H): PPCLL1VisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCLTokenizingOptimizationTest.$(O) PPCLTokenizingOptimizationTest.$(H): PPCLTokenizingOptimizationTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCMergingVisitorTest.$(O) PPCMergingVisitorTest.$(H): PPCMergingVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCMockCompiler.$(O) PPCMockCompiler.$(H): PPCMockCompiler.st $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeFirstFollowNextTests.$(O) PPCNodeFirstFollowNextTests.$(H): PPCNodeFirstFollowNextTests.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCNodeTest.$(O) PPCNodeTest.$(H): PPCNodeTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCOptimizeChoicesTest.$(O) PPCOptimizeChoicesTest.$(H): PPCOptimizeChoicesTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCPrototype1OptimizingTest.$(O) PPCPrototype1OptimizingTest.$(H): PPCPrototype1OptimizingTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCPrototype1Test.$(O) PPCPrototype1Test.$(H): PPCPrototype1Test.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizerComponentDetectorTest.$(O) PPCRecognizerComponentDetectorTest.$(H): PPCRecognizerComponentDetectorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCRecognizerComponentVisitorTest.$(O) PPCRecognizerComponentVisitorTest.$(H): PPCRecognizerComponentVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCSpecializingVisitorTest.$(O) PPCSpecializingVisitorTest.$(H): PPCSpecializingVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenDetectorTest.$(O) PPCTokenDetectorTest.$(H): PPCTokenDetectorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenGuardTest.$(O) PPCTokenGuardTest.$(H): PPCTokenGuardTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingCodeGeneratorTest.$(O) PPCTokenizingCodeGeneratorTest.$(H): PPCTokenizingCodeGeneratorTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCTokenizingTest.$(O) PPCTokenizingTest.$(H): PPCTokenizingTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)PPCTokenizingVisitorTest.$(O) PPCTokenizingVisitorTest.$(H): PPCTokenizingVisitorTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCompiledExpressionGrammarResource.$(O) PPCompiledExpressionGrammarResource.$(H): PPCompiledExpressionGrammarResource.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestResource.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPCompiledExpressionGrammarTest.$(O) PPCompiledExpressionGrammarTest.$(H): PPCompiledExpressionGrammarTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPCompositeParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPExpressionGrammar.$(O) PPExpressionGrammar.$(H): PPExpressionGrammar.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPExpressionGrammarTest.$(O) PPExpressionGrammarTest.$(H): PPExpressionGrammarTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPCompositeParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPLL1ExpressionGrammar.$(O) PPLL1ExpressionGrammar.$(H): PPLL1ExpressionGrammar.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPLL1ExpressionGrammarTest.$(O) PPLL1ExpressionGrammarTest.$(H): PPLL1ExpressionGrammarTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPCompositeParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedExpressionGrammarResource.$(O) PPTokenizedExpressionGrammarResource.$(H): PPTokenizedExpressionGrammarResource.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestResource.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedExpressionGrammarTest.$(O) PPTokenizedExpressionGrammarTest.$(H): PPTokenizedExpressionGrammarTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPCompositeParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedLL1ExpressionGrammarResource.$(O) PPTokenizedLL1ExpressionGrammarResource.$(H): PPTokenizedLL1ExpressionGrammarResource.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestResource.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) -$(OUTDIR)PPTokenizedLL1ExpressionGrammarTest.$(O) PPTokenizedLL1ExpressionGrammarTest.$(H): PPTokenizedLL1ExpressionGrammarTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPCompositeParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalOptimizationTest.$(O) PPCUniversalOptimizationTest.$(H): PPCUniversalOptimizationTest.st $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPCUniversalTest.$(O) PPCUniversalTest.$(H): PPCUniversalTest.st $(INCLUDE_TOP)\stx\goodies\petitparser\tests\PPAbstractParserTest.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestAsserter.$(H) $(INCLUDE_TOP)\stx\goodies\sunit\TestCase.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests.$(O) stx_goodies_petitparser_compiler_tests.$(H): stx_goodies_petitparser_compiler_tests.st $(INCLUDE_TOP)\stx\libbasic\LibraryDefinition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\ProjectDefinition.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/Make.proto --- a/compiler/tests/extras/Make.proto Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/Make.proto Thu May 21 14:12:22 2015 +0100 @@ -34,7 +34,7 @@ # add the path(es) here:, # ********** OPTIONAL: MODIFY the next lines *** # LOCALINCLUDES=-Ifoo -Ibar -LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler/tests -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/petitparser/tests -I$(INCLUDE_TOP)/stx/goodies/sunit -I$(INCLUDE_TOP)/stx/libbasic -I$(INCLUDE_TOP)/stx/libbasic2 +LOCALINCLUDES= -I$(INCLUDE_TOP)/stx/goodies/petitparser -I$(INCLUDE_TOP)/stx/goodies/petitparser/compiler -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/java -I$(INCLUDE_TOP)/stx/goodies/petitparser/parsers/smalltalk -I$(INCLUDE_TOP)/stx/goodies/petitparser/tests -I$(INCLUDE_TOP)/stx/goodies/sunit -I$(INCLUDE_TOP)/stx/libbasic -I$(INCLUDE_TOP)/stx/libbasic2 # if you need any additional defines for embedded C code, @@ -129,6 +129,8 @@ # BEGINMAKEDEPEND --- do not remove this line; make depend needs it +$(OUTDIR)PPExpressionGrammar.$(O) PPExpressionGrammar.$(H): PPExpressionGrammar.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) +$(OUTDIR)PPLL1ExpressionGrammar.$(O) PPLL1ExpressionGrammar.$(H): PPLL1ExpressionGrammar.st $(INCLUDE_TOP)/stx/goodies/petitparser/PPCompositeParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPDelegateParser.$(H) $(INCLUDE_TOP)/stx/goodies/petitparser/PPParser.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests_extras.$(O) stx_goodies_petitparser_compiler_tests_extras.$(H): stx_goodies_petitparser_compiler_tests_extras.st $(INCLUDE_TOP)/stx/libbasic/LibraryDefinition.$(H) $(INCLUDE_TOP)/stx/libbasic/Object.$(H) $(INCLUDE_TOP)/stx/libbasic/ProjectDefinition.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/Make.spec --- a/compiler/tests/extras/Make.spec Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/Make.spec Thu May 21 14:12:22 2015 +0100 @@ -51,12 +51,16 @@ STCWARNINGS=-warnNonStandard COMMON_CLASSES= \ + PPExpressionGrammar \ + PPLL1ExpressionGrammar \ stx_goodies_petitparser_compiler_tests_extras \ COMMON_OBJS= \ + $(OUTDIR_SLASH)PPExpressionGrammar.$(O) \ + $(OUTDIR_SLASH)PPLL1ExpressionGrammar.$(O) \ $(OUTDIR_SLASH)stx_goodies_petitparser_compiler_tests_extras.$(O) \ diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPCExpressionsVerificationTest.st --- a/compiler/tests/extras/PPCExpressionsVerificationTest.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/PPCExpressionsVerificationTest.st Thu May 21 14:12:22 2015 +0100 @@ -58,7 +58,7 @@ normalParser := self grammar. compiledParser := self compiledGrammar. - fileResources expressionSourcesBig do: [ :source | + fileResources expressionSourcesMedium do: [ :source | expected := normalParser parse: source. expected isPetitFailure ifFalse: [ actual := (compiledParser parse: source withContext: self context). diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPCResources.st --- a/compiler/tests/extras/PPCResources.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/PPCResources.st Thu May 21 14:12:22 2015 +0100 @@ -177,6 +177,6 @@ ! smalltalkSourcesSmall - ^ (self smalltalkInDirectory: '../smalltalk-src/') copyFrom: 1 to: 5000. + ^ (self smalltalkInDirectory: '../smalltalk-src/') copyFrom: 1 to: 1000. ! ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPCSmalltalkTests.st --- a/compiler/tests/extras/PPCSmalltalkTests.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/PPCSmalltalkTests.st Thu May 21 14:12:22 2015 +0100 @@ -16,7 +16,7 @@ profile: true; yourself. - configuration := PPCLL1Configuration new + configuration := PPCTokenizingConfiguration new arguments: arguments; yourself. ! diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPCompiledExpressionGrammarResource.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPCompiledExpressionGrammarResource.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,26 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +TestResource subclass:#PPCompiledExpressionGrammarResource + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPCompiledExpressionGrammarResource methodsFor:'as yet unclassified'! + +setUp + | time configuration | + configuration := PPCConfiguration universal. + configuration arguments name: #PPCompiledExpressionGrammar. + + + time := Time millisecondsToRun: [ + PPExpressionGrammar new compileWithConfiguration: configuration. + ]. + Transcript show: 'Expression grammar compiled in: ', time asString, 'ms'; cr. + +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPCompiledExpressionGrammarTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPCompiledExpressionGrammarTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,124 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParserTest subclass:#PPCompiledExpressionGrammarTest + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPCompiledExpressionGrammarTest class methodsFor:'as yet unclassified'! + +resources + ^ (OrderedCollection with: PPCompiledExpressionGrammarResource) + addAll: super resources; + yourself +! ! + +!PPCompiledExpressionGrammarTest methodsFor:'as yet unclassified'! + +compilerArguments + ^ PPCArguments default + profile: true; + ll: true; + yourself +! + +context + + ^ PPCContext new +! + +parserClass + ^ Smalltalk at: #PPCompiledExpressionGrammar +! + +parserInstanceFor: aSymbol + ^ (Smalltalk at: #PPCompiledExpressionGrammar) new startSymbol: aSymbol +! + +testAdd + result := self parse: '1+2' rule: #add. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. +! + +testMul + result := self parse: '1 * 2' rule: #mul. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '*'. + self assert: result third = 2. +! + +testNumber + result := self parse: '1' rule: #number. + self assert: result = 1. +! + +testParens + result := self parse: '(1)' rule: #parens. + self assert: result size = 3. + self assert: result first inputValue = '('. + self assert: result second = 1. + self assert: result third inputValue = ')'. + +! + +testPrim + result := self parse: '1' rule: #prim. + self assert: result = 1. +! + +testPrim2 + result := self parse: '(1)' rule: #prim. + self assert: result size = 3. + self assert: result second = 1. +! + +testProd + result := self parse: '1' rule: #prod. + self assert: result = 1. +! + +testTerm + result := self parse: '1' rule: #term. + self assert: result = 1. + +! + +testTerm11 + result := self parse: '1 + 2' rule: #term. + self assert: result size = 3. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. + +! + +testTerm12 + result := self parse: '1 + 2 * 3' rule: #term. + self assert: result size = 3. + self assert: result second inputValue = '+'. + self assert: result first = 1. + self assert: result third isArray. + self assert: result third first = 2. + self assert: result third second inputValue = '*'. + self assert: result third third = 3. +! + +testTerm13 + result := self parse: '1 * 2 + 3' rule: #term. + self assert: result size = 3. + self assert: result first isArray. + self assert: result first first = 1. + self assert: result first second inputValue = '*'. + self assert: result first third = 2. + self assert: result second inputValue = '+'. + self assert: result third = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPExpressionGrammar.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPExpressionGrammar.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,45 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParser subclass:#PPExpressionGrammar + instanceVariableNames:'add prod term mul prim parens number' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPExpressionGrammar methodsFor:'as yet unclassified'! + +add + ^ prod , $+ asParser trimmingToken , term +! + +mul + ^ prim , $* asParser trimmingToken , prod +! + +number + ^ #digit asParser plus trimmingToken ==> [ :token | token inputValue asNumber ] +! + +parens + ^ ($( asParser token trim), term , ($) asParser token trim) +! + +prim + ^ parens / number +! + +prod + ^ mul / prim +! + +start + ^ term end +! + +term + ^ add / prod +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPExpressionGrammarTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPExpressionGrammarTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,100 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParserTest subclass:#PPExpressionGrammarTest + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPExpressionGrammarTest methodsFor:'as yet unclassified'! + +parserClass + ^ PPExpressionGrammar +! + +testAdd + result := self parse: '1+2' rule: #add. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. +! + +testMul + result := self parse: '1 * 2' rule: #mul. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '*'. + self assert: result third = 2. +! + +testNumber + result := self parse: '1' rule: #number. + self assert: result = 1. +! + +testParens + result := self parse: '(1)' rule: #parens. + self assert: result size = 3. + self assert: result first inputValue = '('. + self assert: result second = 1. + self assert: result third inputValue = ')'. + +! + +testPrim + result := self parse: '1' rule: #prim. + self assert: result = 1. +! + +testPrim2 + result := self parse: '(1)' rule: #prim. + self assert: result size = 3. + self assert: result second = 1. +! + +testProd + result := self parse: '1' rule: #prod. + self assert: result = 1. +! + +testTerm + result := self parse: '1' rule: #term. + self assert: result = 1. + +! + +testTerm11 + result := self parse: '1 + 2' rule: #term. + self assert: result size = 3. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. + +! + +testTerm12 + result := self parse: '1 + 2 * 3' rule: #term. + self assert: result size = 3. + self assert: result second inputValue = '+'. + self assert: result first = 1. + self assert: result third isArray. + self assert: result third first = 2. + self assert: result third second inputValue = '*'. + self assert: result third third = 3. +! + +testTerm13 + result := self parse: '1 * 2 + 3' rule: #term. + self assert: result size = 3. + self assert: result first isArray. + self assert: result first first = 1. + self assert: result first second inputValue = '*'. + self assert: result first third = 2. + self assert: result second inputValue = '+'. + self assert: result third = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPLL1ExpressionGrammar.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPLL1ExpressionGrammar.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,82 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParser subclass:#PPLL1ExpressionGrammar + instanceVariableNames:'add prod term mul prim parens number mulPrime addPrime termPrime' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + + +!PPLL1ExpressionGrammar methodsFor:'as yet unclassified'! + +add + ^ prod, addPrime optional + map: [ :_prod :_addPrime | + _addPrime isNil + ifTrue: [ _prod ] + ifFalse: [ Array with: _prod withAll: _addPrime ] + + ] +! + +addPrime + ^ $+ asParser trimmingToken , term +! + +mul + ^ prim, mulPrime optional + map: [ :_prim :_mulPrime | + _mulPrime isNil + ifTrue: [ _prim ] + ifFalse: [ Array with: _prim withAll: _mulPrime ] + + ] +! + +mulPrime + ^ $* asParser trimmingToken, prod +! + +number + ^ #digit asParser plus trimmingToken ==> [ :token | token inputValue asNumber ] +! + +parens + ^ ($( asParser token trim), term , ($) asParser token trim) +! + +prim + ^ parens / number +! + +prod + ^ mul +! + +start + ^ term end +! + +term + ^ prod, termPrime optional + map: [ :_prod :_termPrime | + _termPrime isNil + ifTrue: [ _prod ] + ifFalse: [ Array with: _prod withAll: _termPrime ] + ] +! + +termPrime + ^ $+ asParser trimmingToken, term +! ! + +!PPLL1ExpressionGrammar class methodsFor:'documentation'! + +version_HG + + ^ '$Changeset: $' +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPLL1ExpressionGrammarTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPLL1ExpressionGrammarTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,100 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParserTest subclass:#PPLL1ExpressionGrammarTest + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPLL1ExpressionGrammarTest methodsFor:'as yet unclassified'! + +parserClass + ^ PPLL1ExpressionGrammar +! + +testAdd + result := self parse: '1+2' rule: #add. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. +! + +testMul + result := self parse: '1 * 2' rule: #mul. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '*'. + self assert: result third = 2. +! + +testNumber + result := self parse: '1' rule: #number. + self assert: result = 1. +! + +testParens + result := self parse: '(1)' rule: #parens. + self assert: result size = 3. + self assert: result first inputValue = '('. + self assert: result second = 1. + self assert: result third inputValue = ')'. + +! + +testPrim + result := self parse: '1' rule: #prim. + self assert: result = 1. +! + +testPrim2 + result := self parse: '(1)' rule: #prim. + self assert: result size = 3. + self assert: result second = 1. +! + +testProd + result := self parse: '1' rule: #prod. + self assert: result = 1. +! + +testTerm + result := self parse: '1' rule: #term. + self assert: result = 1. + +! + +testTerm11 + result := self parse: '1 + 2' rule: #term. + self assert: result size = 3. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. + +! + +testTerm12 + result := self parse: '1 + 2 * 3' rule: #term. + self assert: result size = 3. + self assert: result second inputValue = '+'. + self assert: result first = 1. + self assert: result third isArray. + self assert: result third first = 2. + self assert: result third second inputValue = '*'. + self assert: result third third = 3. +! + +testTerm13 + result := self parse: '1 * 2 + 3' rule: #term. + self assert: result size = 3. + self assert: result first isArray. + self assert: result first first = 1. + self assert: result first second inputValue = '*'. + self assert: result first third = 2. + self assert: result second inputValue = '+'. + self assert: result third = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedExpressionGrammarResource.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPTokenizedExpressionGrammarResource.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,26 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +TestResource subclass:#PPTokenizedExpressionGrammarResource + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPTokenizedExpressionGrammarResource methodsFor:'as yet unclassified'! + +setUp + | time configuration | + configuration := PPCTokenizingConfiguration new. + configuration arguments name:#PPTokenizedExpressionGrammar. + + + time := Time millisecondsToRun: [ + PPExpressionGrammar new compileWithConfiguration: configuration. + ]. + Transcript crShow: 'Expression grammar tokenized in: ', time asString, 'ms'. + +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedExpressionGrammarTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPTokenizedExpressionGrammarTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,122 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParserTest subclass:#PPTokenizedExpressionGrammarTest + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPTokenizedExpressionGrammarTest class methodsFor:'as yet unclassified'! + +resources + ^ (OrderedCollection with: PPTokenizedExpressionGrammarResource) + addAll: super resources; + yourself +! ! + +!PPTokenizedExpressionGrammarTest methodsFor:'as yet unclassified'! + +compilerArguments + ^ PPCArguments default + profile: true; + yourself +! + +context + ^ PPCContext new +! + +parserClass + ^ Smalltalk at: #PPTokenizedExpressionGrammar +! + +parserInstanceFor: aSymbol + ^ (Smalltalk at: #PPTokenizedExpressionGrammar) new startSymbol: aSymbol +! + +testAdd + result := self parse: '1+2' rule: #add. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. +! + +testMul + result := self parse: '1 * 2' rule: #mul. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '*'. + self assert: result third = 2. +! + +testNumber + result := self parse: '1' rule: #number. + self assert: result = 1. +! + +testParens + result := self parse: '(1)' rule: #parens. + self assert: result size = 3. + self assert: result first inputValue = '('. + self assert: result second = 1. + self assert: result third inputValue = ')'. + +! + +testPrim + result := self parse: '1' rule: #prim. + self assert: result = 1. +! + +testPrim2 + result := self parse: '(1)' rule: #prim. + self assert: result size = 3. + self assert: result second = 1. +! + +testProd + result := self parse: '1' rule: #prod. + self assert: result = 1. +! + +testTerm + result := self parse: '1' rule: #term. + self assert: result = 1. + +! + +testTerm11 + result := self parse: '1 + 2' rule: #term. + self assert: result size = 3. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. + +! + +testTerm12 + result := self parse: '1 + 2 * 3' rule: #term. + self assert: result size = 3. + self assert: result second inputValue = '+'. + self assert: result first = 1. + self assert: result third isArray. + self assert: result third first = 2. + self assert: result third second inputValue = '*'. + self assert: result third third = 3. +! + +testTerm13 + result := self parse: '1 * 2 + 3' rule: #term. + self assert: result size = 3. + self assert: result first isArray. + self assert: result first first = 1. + self assert: result first second inputValue = '*'. + self assert: result first third = 2. + self assert: result second inputValue = '+'. + self assert: result third = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedLL1ExpressionGrammarResource.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPTokenizedLL1ExpressionGrammarResource.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,26 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +TestResource subclass:#PPTokenizedLL1ExpressionGrammarResource + instanceVariableNames:'' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPTokenizedLL1ExpressionGrammarResource methodsFor:'as yet unclassified'! + +setUp + | time configuration | + configuration := PPCTokenizingConfiguration new. + configuration arguments name:#PPTokenizedLL1ExpressionGrammar. + + + time := Time millisecondsToRun: [ + PPLL1ExpressionGrammar new compileWithConfiguration: configuration. + ]. + Transcript crShow: 'LL1 Expression grammar tokenized in: ', time asString, 'ms'. + +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedLL1ExpressionGrammarTest.st --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/compiler/tests/extras/PPTokenizedLL1ExpressionGrammarTest.st Thu May 21 14:12:22 2015 +0100 @@ -0,0 +1,140 @@ +"{ Package: 'stx:goodies/petitparser/compiler/tests/extras' }" + +"{ NameSpace: Smalltalk }" + +PPCompositeParserTest subclass:#PPTokenizedLL1ExpressionGrammarTest + instanceVariableNames:'context' + classVariableNames:'' + poolDictionaries:'' + category:'PetitCompiler-Extras-Tests-Expressions' +! + +!PPTokenizedLL1ExpressionGrammarTest class methodsFor:'as yet unclassified'! + +resources + ^ (OrderedCollection with: PPTokenizedLL1ExpressionGrammarResource) + addAll: super resources; + yourself +! ! + +!PPTokenizedLL1ExpressionGrammarTest methodsFor:'as yet unclassified'! + +compilerArguments + ^ PPCArguments default + profile: true; + ll: true; + yourself +! + +context + ^ context := PPCProfilingContext new +! + +parserClass + ^ Smalltalk at: #PPTokenizedLL1ExpressionGrammar +! + +parserInstanceFor: aSymbol + ^ (Smalltalk at: #PPTokenizedLL1ExpressionGrammar) new startSymbol: aSymbol +! + +testAdd + result := self parse: '1+2' rule: #term. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. +! + +testMul + result := self parse: '1 * 2' rule: #mul. + self assert: result isArray. + self assert: result first = 1. + self assert: result second inputValue = '*'. + self assert: result third = 2. +! + +testNumber + result := self parse: '1' rule: #number. + self assert: result = 1. +! + +testParens + result := self parse: '(1)' rule: #parens. + self assert: result size = 3. + self assert: result first inputValue = '('. + self assert: result second = 1. + self assert: result third inputValue = ')'. + +! + +testPrim + result := self parse: '1' rule: #prim. + self assert: result = 1. +! + +testPrim2 + result := self parse: '(1)' rule: #prim. + self assert: result size = 3. + self assert: result second = 1. +! + +testProd + result := self parse: '1' rule: #prod. + self assert: result = 1. +! + +testTerm01 + result := self parse: '1' rule: #term. + self assert: result = 1. + + self assert: context lwRememberCount = 2. + self assert: context lwRestoreCount = 0. +! + +testTerm02 + result := self parse: '1+2' rule: #term. + self assert: result size = 3. + + self assert: context lwRestoreCount = 0. +! + +testTerm03 + result := self parse: '1*2+3*4' rule: #term. + self assert: result size = 3. + + + self assert: context lwRestoreCount = 0. +! + +testTerm11 + result := self parse: '1 + 2' rule: #term. + self assert: result size = 3. + self assert: result first = 1. + self assert: result second inputValue = '+'. + self assert: result third = 2. + +! + +testTerm12 + result := self parse: '1 + 2 * 3' rule: #term. + self assert: result size = 3. + self assert: result second inputValue = '+'. + self assert: result first = 1. + self assert: result third isArray. + self assert: result third first = 2. + self assert: result third second inputValue = '*'. + self assert: result third third = 3. +! + +testTerm13 + result := self parse: '1 * 2 + 3' rule: #term. + self assert: result size = 3. + self assert: result first isArray. + self assert: result first first = 1. + self assert: result first second inputValue = '*'. + self assert: result first third = 2. + self assert: result second inputValue = '+'. + self assert: result third = 3. +! ! + diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedSmalltalkGrammarResource.st --- a/compiler/tests/extras/PPTokenizedSmalltalkGrammarResource.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/PPTokenizedSmalltalkGrammarResource.st Thu May 21 14:12:22 2015 +0100 @@ -14,7 +14,7 @@ setUp | time configuration | - configuration := PPCConfiguration LL1. + configuration := PPCConfiguration tokenizing. configuration arguments name:#PPTokenizedSmalltalkGrammar. time := Time millisecondsToRun: [ diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/PPTokenizedSmalltalkGrammarTests.st --- a/compiler/tests/extras/PPTokenizedSmalltalkGrammarTests.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/PPTokenizedSmalltalkGrammarTests.st Thu May 21 14:12:22 2015 +0100 @@ -159,6 +159,13 @@ rule: #method ! +testMethod4 + self + parse: 'foo: bar + foo:= bar' + rule: #method +! + testSequence1 self parse: '| a | 1 . 2' diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/abbrev.stc --- a/compiler/tests/extras/abbrev.stc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/abbrev.stc Thu May 21 14:12:22 2015 +0100 @@ -6,10 +6,20 @@ PPCResources PPCResources stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Support' 1 PPCSmalltalkTests PPCSmalltalkTests stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 PPCSmalltalkVerificationTest PPCSmalltalkVerificationTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 +PPCompiledExpressionGrammarResource PPCompiledExpressionGrammarResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPCompiledExpressionGrammarTest PPCompiledExpressionGrammarTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 PPCompiledJavaResource PPCompiledJavaResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Java' 1 PPCompiledJavaSyntaxTest PPCompiledJavaSyntaxTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Java' 1 PPCompiledSmalltalkGrammarResource PPCompiledSmalltalkGrammarResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 PPCompiledSmalltalkGrammarTests PPCompiledSmalltalkGrammarTests stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 +PPExpressionGrammar PPExpressionGrammar stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 0 +PPExpressionGrammarTest PPExpressionGrammarTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPLL1ExpressionGrammar PPLL1ExpressionGrammar stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 0 +PPLL1ExpressionGrammarTest PPLL1ExpressionGrammarTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPTokenizedExpressionGrammarResource PPTokenizedExpressionGrammarResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPTokenizedExpressionGrammarTest PPTokenizedExpressionGrammarTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPTokenizedLL1ExpressionGrammarResource PPTokenizedLL1ExpressionGrammarResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 +PPTokenizedLL1ExpressionGrammarTest PPTokenizedLL1ExpressionGrammarTest stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Expressions' 1 PPTokenizedSmalltalkGrammarResource PPTokenizedSmalltalkGrammarResource stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 PPTokenizedSmalltalkGrammarTests PPTokenizedSmalltalkGrammarTests stx:goodies/petitparser/compiler/tests/extras 'PetitCompiler-Extras-Tests-Smalltalk' 1 stx_goodies_petitparser_compiler_tests_extras stx_goodies_petitparser_compiler_tests_extras stx:goodies/petitparser/compiler/tests/extras '* Projects & Packages *' 3 diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/bc.mak --- a/compiler/tests/extras/bc.mak Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/bc.mak Thu May 21 14:12:22 2015 +0100 @@ -35,7 +35,7 @@ -LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler\tests -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\petitparser\tests -I$(INCLUDE_TOP)\stx\goodies\sunit -I$(INCLUDE_TOP)\stx\libbasic -I$(INCLUDE_TOP)\stx\libbasic2 +LOCALINCLUDES= -I$(INCLUDE_TOP)\stx\goodies\petitparser -I$(INCLUDE_TOP)\stx\goodies\petitparser\compiler -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\java -I$(INCLUDE_TOP)\stx\goodies\petitparser\parsers\smalltalk -I$(INCLUDE_TOP)\stx\goodies\petitparser\tests -I$(INCLUDE_TOP)\stx\goodies\sunit -I$(INCLUDE_TOP)\stx\libbasic -I$(INCLUDE_TOP)\stx\libbasic2 LOCALDEFINES= STCLOCALOPT=-package=$(PACKAGE) -I. $(LOCALINCLUDES) -headerDir=. $(STCLOCALOPTIMIZATIONS) $(STCWARNINGS) $(LOCALDEFINES) -varPrefix=$(LIBNAME) @@ -76,6 +76,8 @@ # BEGINMAKEDEPEND --- do not remove this line; make depend needs it +$(OUTDIR)PPExpressionGrammar.$(O) PPExpressionGrammar.$(H): PPExpressionGrammar.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) +$(OUTDIR)PPLL1ExpressionGrammar.$(O) PPLL1ExpressionGrammar.$(H): PPLL1ExpressionGrammar.st $(INCLUDE_TOP)\stx\goodies\petitparser\PPCompositeParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPDelegateParser.$(H) $(INCLUDE_TOP)\stx\goodies\petitparser\PPParser.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(STCHDR) $(OUTDIR)stx_goodies_petitparser_compiler_tests_extras.$(O) stx_goodies_petitparser_compiler_tests_extras.$(H): stx_goodies_petitparser_compiler_tests_extras.st $(INCLUDE_TOP)\stx\libbasic\LibraryDefinition.$(H) $(INCLUDE_TOP)\stx\libbasic\Object.$(H) $(INCLUDE_TOP)\stx\libbasic\ProjectDefinition.$(H) $(STCHDR) # ENDMAKEDEPEND --- do not remove this line diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/libInit.cc --- a/compiler/tests/extras/libInit.cc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/libInit.cc Thu May 21 14:12:22 2015 +0100 @@ -27,6 +27,8 @@ void _libstx_goodies_petitparser_compiler_tests_extras_Init(pass, __pRT__, snd) OBJ snd; struct __vmData__ *__pRT__; { __BEGIN_PACKAGE2__("libstx_goodies_petitparser_compiler_tests_extras", _libstx_goodies_petitparser_compiler_tests_extras_Init, "stx:goodies/petitparser/compiler/tests/extras"); +_PPExpressionGrammar_Init(pass,__pRT__,snd); +_PPLL1ExpressionGrammar_Init(pass,__pRT__,snd); _stx_137goodies_137petitparser_137compiler_137tests_137extras_Init(pass,__pRT__,snd); diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st --- a/compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/extras/stx_goodies_petitparser_compiler_tests_extras.st Thu May 21 14:12:22 2015 +0100 @@ -57,6 +57,7 @@ by searching along the inheritance chain of all of my classes." ^ #( + #'stx:goodies/petitparser' "PPCompositeParser - superclass of PPExpressionGrammar" #'stx:goodies/petitparser/parsers/java' "PPJavaLexiconTest - superclass of PPCompiledJavaSyntaxTest" #'stx:goodies/petitparser/tests' "PPAbstractParserTest - superclass of PPCCompiledJavaVerificationTest" #'stx:goodies/sunit' "TestAsserter - superclass of PPCCompiledJavaVerificationTest" @@ -75,9 +76,7 @@ by searching all classes (and their packages) which are referenced by my classes." ^ #( - #'stx:goodies/petitparser' "PPFailure - referenced by PPCResources>>workingJavaInDirectory:" #'stx:goodies/petitparser/compiler' "PPCArguments - referenced by PPCSmalltalkTests>>setUp" - #'stx:goodies/petitparser/compiler/tests' "PPCompiledExpressionGrammarResource - referenced by PPCompiledExpressionsVerificationTest class>>resources" #'stx:goodies/petitparser/parsers/smalltalk' "PPSmalltalkGrammar - referenced by PPCSmalltalkVerificationTest>>smalltalkGrammar" #'stx:libbasic2' "Random - referenced by PPCResources>>expressionOfSize:stream:" ) @@ -108,10 +107,20 @@ (PPCResources autoload) (PPCSmalltalkTests autoload) (PPCSmalltalkVerificationTest autoload) + (PPCompiledExpressionGrammarResource autoload) + (PPCompiledExpressionGrammarTest autoload) (PPCompiledJavaResource autoload) (PPCompiledJavaSyntaxTest autoload) (PPCompiledSmalltalkGrammarResource autoload) (PPCompiledSmalltalkGrammarTests autoload) + PPExpressionGrammar + (PPExpressionGrammarTest autoload) + PPLL1ExpressionGrammar + (PPLL1ExpressionGrammarTest autoload) + (PPTokenizedExpressionGrammarResource autoload) + (PPTokenizedExpressionGrammarTest autoload) + (PPTokenizedLL1ExpressionGrammarResource autoload) + (PPTokenizedLL1ExpressionGrammarTest autoload) (PPTokenizedSmalltalkGrammarResource autoload) (PPTokenizedSmalltalkGrammarTests autoload) #'stx_goodies_petitparser_compiler_tests_extras' diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/libInit.cc --- a/compiler/tests/libInit.cc Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/libInit.cc Thu May 21 14:12:22 2015 +0100 @@ -34,32 +34,23 @@ _PPCCopyVisitorTest_Init(pass,__pRT__,snd); _PPCGuardTest_Init(pass,__pRT__,snd); _PPCInliningVisitorTest_Init(pass,__pRT__,snd); -_PPCLL1OptimizingTest_Init(pass,__pRT__,snd); -_PPCLL1Test_Init(pass,__pRT__,snd); _PPCLL1VisitorTest_Init(pass,__pRT__,snd); +_PPCLTokenizingOptimizationTest_Init(pass,__pRT__,snd); _PPCMergingVisitorTest_Init(pass,__pRT__,snd); _PPCMockCompiler_Init(pass,__pRT__,snd); _PPCNodeFirstFollowNextTests_Init(pass,__pRT__,snd); _PPCNodeTest_Init(pass,__pRT__,snd); _PPCOptimizeChoicesTest_Init(pass,__pRT__,snd); -_PPCPrototype1OptimizingTest_Init(pass,__pRT__,snd); -_PPCPrototype1Test_Init(pass,__pRT__,snd); _PPCRecognizerComponentDetectorTest_Init(pass,__pRT__,snd); _PPCRecognizerComponentVisitorTest_Init(pass,__pRT__,snd); _PPCSpecializingVisitorTest_Init(pass,__pRT__,snd); _PPCTokenDetectorTest_Init(pass,__pRT__,snd); +_PPCTokenGuardTest_Init(pass,__pRT__,snd); _PPCTokenizingCodeGeneratorTest_Init(pass,__pRT__,snd); +_PPCTokenizingTest_Init(pass,__pRT__,snd); _PPCTokenizingVisitorTest_Init(pass,__pRT__,snd); -_PPCompiledExpressionGrammarResource_Init(pass,__pRT__,snd); -_PPCompiledExpressionGrammarTest_Init(pass,__pRT__,snd); -_PPExpressionGrammar_Init(pass,__pRT__,snd); -_PPExpressionGrammarTest_Init(pass,__pRT__,snd); -_PPLL1ExpressionGrammar_Init(pass,__pRT__,snd); -_PPLL1ExpressionGrammarTest_Init(pass,__pRT__,snd); -_PPTokenizedExpressionGrammarResource_Init(pass,__pRT__,snd); -_PPTokenizedExpressionGrammarTest_Init(pass,__pRT__,snd); -_PPTokenizedLL1ExpressionGrammarResource_Init(pass,__pRT__,snd); -_PPTokenizedLL1ExpressionGrammarTest_Init(pass,__pRT__,snd); +_PPCUniversalOptimizationTest_Init(pass,__pRT__,snd); +_PPCUniversalTest_Init(pass,__pRT__,snd); _stx_137goodies_137petitparser_137compiler_137tests_Init(pass,__pRT__,snd); diff -r 4751c407bb40 -r f6d77fee9811 compiler/tests/stx_goodies_petitparser_compiler_tests.st --- a/compiler/tests/stx_goodies_petitparser_compiler_tests.st Tue May 12 01:24:03 2015 +0100 +++ b/compiler/tests/stx_goodies_petitparser_compiler_tests.st Thu May 21 14:12:22 2015 +0100 @@ -56,7 +56,6 @@ by searching along the inheritance chain of all of my classes." ^ #( - #'stx:goodies/petitparser' "PPCompositeParser - superclass of PPExpressionGrammar" #'stx:goodies/petitparser/tests' "PPAbstractParserTest - superclass of PPCCodeGeneratorTest" #'stx:goodies/sunit' "TestAsserter - superclass of PPCCodeGeneratorTest" #'stx:libbasic' "LibraryDefinition - superclass of stx_goodies_petitparser_compiler_tests" @@ -74,7 +73,8 @@ by searching all classes (and their packages) which are referenced by my classes." ^ #( - #'stx:goodies/petitparser/compiler' "PPCAbstractCharacterNode - referenced by PPCPrototype1OptimizingTest>>testForwarding" + #'stx:goodies/petitparser' "PPCharSetPredicate - referenced by PPCCodeGeneratorTest>>testCharSetPredicateNode" + #'stx:goodies/petitparser/compiler' "PPCAbstractLiteralNode - referenced by PPCNodeFirstFollowNextTests>>testFirst1" #'stx:goodies/petitparser/parsers/java' "PPJavaWhitespaceParser - referenced by PPCMergingVisitorTest>>javaWsNode" ) ! @@ -114,32 +114,23 @@ PPCCopyVisitorTest PPCGuardTest PPCInliningVisitorTest - PPCLL1OptimizingTest - PPCLL1Test PPCLL1VisitorTest + PPCLTokenizingOptimizationTest PPCMergingVisitorTest PPCMockCompiler PPCNodeFirstFollowNextTests PPCNodeTest PPCOptimizeChoicesTest - PPCPrototype1OptimizingTest - PPCPrototype1Test PPCRecognizerComponentDetectorTest PPCRecognizerComponentVisitorTest PPCSpecializingVisitorTest PPCTokenDetectorTest + PPCTokenGuardTest PPCTokenizingCodeGeneratorTest + PPCTokenizingTest PPCTokenizingVisitorTest - PPCompiledExpressionGrammarResource - PPCompiledExpressionGrammarTest - PPExpressionGrammar - PPExpressionGrammarTest - PPLL1ExpressionGrammar - PPLL1ExpressionGrammarTest - PPTokenizedExpressionGrammarResource - PPTokenizedExpressionGrammarTest - PPTokenizedLL1ExpressionGrammarResource - PPTokenizedLL1ExpressionGrammarTest + PPCUniversalOptimizationTest + PPCUniversalTest #'stx_goodies_petitparser_compiler_tests' ) !