An event for syntax highlighting is lost when reopening file or pasting code into Workspace

[Patrik Svestka]

A syntax of a source code is not highlighted when opened via Open File... or just copy & paste into Workspace.

When typing the source code gets properly highlighted.

[Jan Vrany]

The problem is that CodeView2::EditView does not react on #textChanged notification. Attached fix does that, i.e., it forces source code reprocessing whenever #textChanges.

However, this is not desirable. The problem is that - bad enough - the calls to #textChanged does not carry the information of which part of text actually changed. Therefore the clients have no chance than just reprocess whole text which is OK for short souces such as smalltalk methods but takes unacceptable amount of time for large sources such as Java source. I'm really reluctant to commit attached "patch" because of this.

Another problem is that the notification is emitted multiple times for a single keystroke.

The proper fix would be to fix all senders of #textChanged and pass down an information which part actually changed (by means of offsets or line/col pairs).

[Jan Vrany]

Better fix, ready for testing

[Jan Vrany]

I found methods when the modification notification was missing. Please try second patch - libtool_fix_1_of_1_rev_daeb49f45c7a_Issue_99__Send_notification_to_services_also_from__basicWithoutRedraw___.patch​ and let me know. If OK, I'll commit.

[vranyj1]

Hmm, seems to be a bit more complicated (sorry for czech, copy-pasted from private communication):

(07:30:40 PM) Patrik Svestka: vzít jakýkoliv zdroják
(07:30:47 PM) Patrik Svestka: z textového editoru
(07:30:53 PM) Patrik Svestka: a dát paste
(07:30:56 PM) Patrik Svestka: to nefungovalo
(07:31:23 PM) Patrik Svestka: což stále nefunguje
(07:31:27 PM) Patrik Svestka: co teď zafungovalo
(07:31:40 PM) Patrik Svestka: je neoznačený text z StX do nového tabu StX se obarvil
(07:31:58 PM) Patrik Svestka: a teď to zase nefunguje
(07:32:00 PM) Patrik Svestka: hmm
(07:32:17 PM) Patrik Svestka: a když dám menší část tak to funguje
(07:32:29 PM) Patrik Svestka: nahrávání nefunguje v žádném případě
(07:32:44 PM) Patrik Svestka: zkoušel jsem to natahovat z *.txt nebo *.st souboru
(07:32:48 PM) Patrik Svestka: ani jedno nefunguje
(07:33:24 PM) Patrik Svestka: zajímavé
(07:33:27 PM) Patrik Svestka: tohle se obarví
(07:33:28 PM) Patrik Svestka: [:exit|

model ifNil: [model := ValueHolder? with:'Enter a value'].

box := DialogBox? new.
box addTextLabel:'Enter some string, please:'.
box addInputFieldOn:model.
box addAbortButton; addOkButton.
box open.

(07:33:48 PM) Patrik Svestka: ale když dám
(07:33:49 PM) Patrik Svestka: |box field model warning condition|

min_range := -50.
max_range := 50.

(07:33:54 PM) Patrik Svestka: tak se neobarv9
(07:33:55 PM) Patrik Svestka: í
(07:44:29 PM) Patrik Svestka: hmm zvláštní

[Jan Vrany]

I have added some tests and the patch seem to work fine. Could you please be more specific
and provide more detailed information of what works and what not?

[Patrik Svestka]

The patches did not help much. I have applied both of them but the behaviour remained the same:

1) I have saved following code to a text file (check_for_integer_smalltalk_way.txt):

       | v |
       v := '-adb34s'.
       (v first = $-) | (v first = $+) 
         ifTrue: [
            v allButFirst notNil and: [v allButFirst allSatisfy: [:c|c isDigit]]
         ]  
         ifFalse: [
           v notNil and: [v allSatisfy: [:c|c isDigit]]
         ]

2) When in Workspace I performed File -> Open file and loaded this file the code remained unhighlighted.

3) The same situation remains when I copy (ctrl+c) and paste the code (ctrl+v) into new Workspace tab.

[Jan Vrany]

I see what's going on here. I tried copy-paste the above snippet to a new workspace and actually does highlight it but since it's not a valid method nor expression it fails parse it and thus the code after first parse error is not highlighted.

If you look carefully, you should see that first vertical bar (just before x at line 1)
is red, not black. Could you confirm?

I do agree however that this is not very nice and we can do better. There are two things we can do:

1. Improve the parser and add another starting rule, basically a statement list that would allow for local declarations before each statement. This way the above would be syntactically valid as well ss (for example):

| a b |
(a + b) printStringRadix: 8.

| x |
x := OrderedCollection new.
x add: 1; add:2. 
x max.

2. Improve the syntax highlighter so that upon syntax error it would resort in lexical highlighting augmented with simple token-level regexp matcher to highlight the rest. Lexical highlighting is less accurate but faster and still may do OK job for things like literals, selectors, comments and so on.

See:

• ​TokenPatternMatcher
• ​JavaLexicalHighlighter

Both things should be implemented. The latter would also give a better experience when editing methods in browser, debugger and so on. But it requires some effort...

[Patrik Svestka]

I see what's going on here. I tried copy-paste the above snippet to a new workspace and actually does highlight it but since it's not a valid method nor expression it fails parse it and thus the code after first parse error is not highlighted.

I see so the first error stops the parsing.

If you look carefully, you should see that first vertical bar (just before v at line 1)
is red, not black. Could you confirm?

Yes I do confirm that. The first "pipe" is colored red.

ad 1) Yes, that would make sense.

ad 2) Good idea. It would improve the highlighting process in case of workspace.

[Patrik Svestka]

The syntax highlighting does not work even for simple example without \| \|

An example:

Dictionary new
  at: 'Win32OperatingSystem::RegistryEntry(HKEY_USERS\S-1-5-21-119559289-1840127793-336618761-855951\Software\Altap\Altap Salamander 3.08\User Menu\1)'
  put: (Dictionary new
          at: 'Arguments'
         put:'-O -A "\\rm2ms001157\e$$\GR_Prisma_Interface"');
  yourself.