Starting StX with saved image kills the whole environment

[Patrik Svestka]

I have exited and saved the StX and now it starts directly into MiniDebugger?:

C:\prg_sdk\smalltalkx-jv-branch-6.2.5_x86_64-win32\bin>stx.bat --image st.img
IMG [info]: executable and/or shared libraries changed address(es).
IMG [info]: updating cached function pointers.
Font [warning]: encoding for "a segoe UI-medium-roman-9.0 (ms-default)-Font" sho
uld be ms-default; is ms-easteurope
Font [warning]: encoding for "a Segoe UI-medium-roman-9.0 (ms-default)-Font" sho
uld be ms-default; is ms-easteurope
Font [warning]: encoding for "a Segoe UI-bold-roman-9.0 (ms-default)-Font" shoul
d be ms-default; is ms-easteurope
Font [warning]: encoding for "a courier-medium-roman-12 (iso10646-1)-Font" shoul
d be iso10646-1; is ms-easteurope
WinWorkstation? [info]: CreateWindow? S failed: 1158 (0x486) [4796]
WinWorkstation? [error]: CreateWindow? failed: 1158 (0x486) [8992]
NoHandlerError?: primitive failed: a WinWorkstation?(local)>>setEventMask:in:
process: id=0 name=scheduler
context: WinWorkstation? >> setEventMask:in: [24]

......: CheckBox?(SimpleView?) >> recreate [20]
......: CheckBox?(SimpleView?) >> reinitialize [25]
......: [] in DeviceWorkstation?>>reinitializeFor: >> value [75]
......: Signal >> handle:do: [14]
......: [] in DeviceWorkstation?>>reinitializeFor: >> value [71]
......: SignalSet? >> handle:do: [14]
......: SignalSet? >> catch: [11]
......: [] in DeviceWorkstation?>>reinitializeFor: >> value: [70]
......: WeakArray? >> do: [8]
......: WinWorkstation?(DeviceWorkstation?) >> reinitializeFor: [66]
......: WinWorkstation?(Object) >> perform:with: [31]
......: MessageNode? >> evaluateIn:forCascade: [32]
......: MessageNode? >> evaluateIn: [2]
......: StatementNode? >> evaluateExpressionIn: [2]
......: StatementNode? >> evaluateAllIn: [7]
......: StatementNode? >> evaluateIn: [2]
......: StatementNode?(ParseNode?) >> evaluate [2]
......: BlockNode? >> value [7]
......: [] in BlockNode?>>doesNotUnderstand: >> value [11]
......: [ intermediate recursive contexts skipped ]
......: [] in BlockNode?>>doesNotUnderstand: >> value [11]
......: Block >> on:do: [11]
......: Block(Object) >> perform:withArguments: [135]
......: BlockNode? >> doesNotUnderstand: [31]
......: BlockNode?(NONE) >> on:do: [31]
......: BlockNode?(Object) >> perform:with:with: [31]
......: MessageNode? >> evaluateIn:forCascade: [36]
......: MessageNode? >> evaluateIn: [2]
......: StatementNode? >> evaluateExpressionIn: [2]
......: StatementNode? >> evaluateAllIn: [7]
......: StatementNode? >> evaluateIn: [2]
......: StatementNode?(ParseNode?) >> evaluate [2]
......: BlockNode? >> value [7]
......: True >> ifTrue: [8]
......: True(Object) >> perform:with: [31]
......: MessageNode? >> evaluateIn:forCascade: [32]
......: MessageNode? >> evaluateIn: [2]
......: StatementNode? >> evaluateExpressionIn: [2]
......: StatementNode? >> evaluateAllIn: [7]
......: StatementNode? >> evaluateIn: [2]
......: StatementNode?(ParseNode?) >> evaluate [2]
......: [] in Parser>>evaluate:in:receiver:notifying:logged:ifFail:compile:chec

kForEndOfInput: >> value [138]

......: QuerySignal? >> answer:do: [10]
......: [] in Parser>>evaluate:in:receiver:notifying:logged:ifFail:compile:chec

kForEndOfInput: >> value [120]

......: UndefinedVariableError? class(GenericException? class) >> handle:do: [14]

......: ByteCodeCompiler?(Parser) >> evaluate:in:receiver:notifying:logged:ifFai

l:compile:checkForEndOfInput: [113]

......: ByteCodeCompiler?(Parser) >> evaluate:receiver:notifying:compile: [3]
......: EncodedStream?(PeekableStream?) >> fileInNextChunkNotifying:passChunk:sil

ent: [37]

......: EncodedStream?(PeekableStream?) >> fileInNextChunkNotifying:passChunk: [1

4]

......: [] in EncodedStream?>>basicFileInNotifying:passChunk: >> value [146]
......: SignalSet? >> handle:do: [14]
......: EncodedStream? >> basicFileInNotifying:passChunk: [100]
......: EncodedStream?(PeekableStream?) >> fileInNotifying:passChunk: [6]
......: [] in PeekableStream?>>fileIn >> value [14]
......: QuerySignal?(Signal) >> handle:do: [14]
......: EncodedStream?(PeekableStream?) >> fileIn [11]
......: SmalltalkChunkFileSourceReader? >> fileInStream: [2]
......: SmalltalkChunkFileSourceReader? class(AbstractSourceFileReader? class) >>
fileInStream: [3]
......: [ intermediate recursive contexts skipped ]
......: SmalltalkChunkFileSourceReader? class(AbstractSourceFileReader? class) >>
fileInStream: [3]
......: SmalltalkLanguage?(ProgrammingLanguage?) >> fileInStream: [3]
......: [ intermediate recursive contexts skipped ]
......: SmalltalkLanguage?(ProgrammingLanguage?) >> fileInStream: [3]
......: [] in Smalltalk class>>fileInStream:lazy:silent:logged:addPath: >> valu

e [39]

......: SignalSet? >> answer:do: [10]
......: [] in Smalltalk class>>fileInStream:lazy:silent:logged:addPath: >> valu

e [36]

......: Block >> ensure: [12]
......: Smalltalk class >> fileInStream:lazy:silent:logged:addPath: [41]
......: Smalltalk class >> fileIn:lazy:silent:logged: [60]
......: Smalltalk class >> fileIn: [10]
......: [] in Smalltalk class>>restart >> value [129]
......: [] in ClassDescription?>>withoutUpdatingChangesDo: >> value [9]
......: [ intermediate recursive contexts skipped ]
......: [] in ClassDescription?>>withoutUpdatingChangesDo: >> value [9]
......: SignalSet? >> answer:do: [10]
......: Class class(ClassDescription?) >> withoutUpdatingChangesDo: [7]
......: [] in Smalltalk class>>restart >> value [126]
......: Block >> on:do: [11]
......: Smalltalk class >> restart [132]
......: UndefinedObject? >> nil [0]
......: nil

Type "c" to proceed, "?" for help
MiniDebugger?>

[Jan Vrany]

The problem is that it cannot recreate a window:

WinWorkstation [info]: CreateWindow S failed: 1158 (0x486) [4796]
WinWorkstation [error]: CreateWindow failed: 1158 (0x486) [8992]

The error code is 1158 - ​ERROR_NO_MORE_USER_HANDLES:

The current process has used all of its system allowance of handles for Window Manager objects.

What's the cause is not yet clear...

[Jan Vrany]

I cannot reproduce it.

Could you try on a freshly booted machine? If it happens again and you see " CreateWindow? S failed: " messages, could you check event log for an event saying:

A desktp heap allocation failed

[Patrik Svestka]

Thank you for the additional information.

Based on that information I have performed addition testing and it seems that StX depletes all User Objects available for one application (which is by default set to 10 000 - I have raised the limit to 18 000 but not tested with that limit yet) - more at User Objects: ​https://msdn.microsoft.com/en-us/library/windows/desktop/ms725486%28v=vs.85%29.aspx

I suspect some kind of memory leak to be resposible for depleting the User Objects. I have also check other applications and how many User Objects they do use. From the observation I can say the maximum the other applications (IDE Rubymine, outlook, virtual desktop (multiple screens like on linux), SQL server studio... are using maximum around 600 User Objects so using more than 1000 is probably unreasonable (I will try to monitor the behavior for longer period of time).

The simple tests are compiled into this picture (uploaded the same picture also as attachment so we do not lose it)

[Patrik Svestka]

You were right at 10000 (the usual Win 7 x64 limit) the system kills the application by not giving it more handles thus the error you found.

[Jan Vrany]

That's a good analysis - so surely this is St/X problem (I suspected that, but I learned not to trust anything and anybody) A wild guess is that it tries to recreate dead
windows or - more likely - bitmaps.

Still, I cannot reproduce it so it's hard to debug. Can you reproduce it? I mean, if you start from scratch (-I) then do your evil working and than save the image, could you produce an image that exhibit this? That would help a lot.

[Patrik Svestka]

Steps to follow to have similar situation (I have uploaded a picture so you can verify you have similar situation):

1. Start stx.bat -I
2. Click on GuiPainter? (on the launcher window)
3. Create a simple GUI Stub (don't change even the name of the application - just leave everything at default)
4. click on show raster (raster zeigen) and am raster ausrichten (align with raster)
5. Place OK and Cancel button from Widget galery (Buttons tab)
6. From tab Menus place Combo List
7. From tab Text tab place text editor
8. From Buttons tab place 3 more Buttons (objects)
9. From Groups select Framed Box
10. Inside the Framed Box place Slider (Misc tab), Entry field, Label (both Text tab)
11. Now save the image (file/save image from launcher)
12. Now exit StX without saving the image
13. After reload the StX uses 4845 USER Objects

P.S. Trust no one in this situation, only what you find and can measure, is only natural .))

[Jan Vrany]

Very good, that should help. The raster thing is indeed fishy and I feel is first symptom.
The raster is implemented as on-the-fly generated bitmap which is then set as canvas background. So perhaps upon snapshot restart, something gets terribly wrong. Will see.

I'll have a look and let you know, but certainly not today. Meanwhile, you may want to file out your code or (learn how to) commit.

[Patrik Svestka]

I'm trying to get to know StX and how to fix simple things first (that will take some time but I'm stubborn ;) ). And yes learn to file out and commit on the list too.

[Jan Vrany]

As for the raster issue: the problem is likely that the raster pixmap is created from scratch by drawing onto a Windowx bitmap. When an image is saved, the Form has not backing
data so upon snapshot restart, the form is recreated but with random data (what used to be at place where Windows allocated them).
So, we need to save the data by #getBits before image is saved. That should fix it.

As for exhausting user object limit, I tried stepsdescribed above but after reloading
the image a number of user objects us pretty much the same, some 300 of them. Tried on build 2109.

[Patrik Svestka]

What I have written above was on build 2107. I've downloaded now build 2109 ( btw. I'm getting new warning there when starting
INIT [warn] (16-04-28 20:22:07): version mismatch in libstx_libjavascript: HTMLD
ocGeneratorForJavaScript (2d674011) vs. HTMLDocGenerator (1aeb8012) )

Maybe try moving the elements around? I have noticed that the number of objects goes up (even on build 2109) when I move around the objects then save and reload then I got increase in number of objects.

[Jan Vrany]

Attached script shows increase in user object consumption after an image restart. A first step to debug it.

[Jan Vrany]

Maybe fix ( 1)

[Jan Vrany]

Could you load the SimpleView-recreate.st​ right after starting a fresh system (i.e., no restart from a snapshot) and check if it helps on your system?

I tried and it seems to help however - as you have seen - I have trouble to reproduce the problem. Also, I haven't tested it extensively, there might be some side-effects of this patch. Thanks a lot!

[Patrik Svestka]

I have managed to measure the usage using the script above (slight adjustments for my environment was needed). However the measurements seem to produce different results than the process explorer from sysinternals .

The question is what is really counted (handlers, GDI, etc.?) within smalltalk. I have checked the MSDN ​https://msdn.microsoft.com/en-us/library/ms683192.aspx and it looks like GR_USEROBJECTS should return the USER OBJECTS.

The following example - process explorer shows 1096 USER Objects but script within StX shows:

...
WINWORKSTATION: pW=123pB=185pGc=285pbmpdc=185

Will save a snapshot and exit in 3 seconds.
WINWORKSTATION: pW=169pB=192pGc=259pbmpdc=192
IMG [info]: executable and/or shared libraries changed address(es).
IMG [info]: updating cached function pointers.
Processor [info]: timeslicer started
Smalltalk [info]: startup process 1 active.
Smalltalk [info]: execute imageStartBlocks...
Restarted from snapshot.
WINWORKSTATION: pW=389pB=197pGc=179pbmpdc=197
WINWORKSTATION: pW=419pB=338pGc=327pbmpdc=338

[Patrik Svestka]

From my tests it appears that this patch fixes the issue with the depletion of the USER objects. I've managed to use around 900 USER Objects but then it gets collected and even after reload all the objects are getting collected to around 200-300 objects which is reasonable.

The one thing I have noticed is that the GDI object do not get collected too. The number of the GDI objects only raises slowly but I have managed to get it around 725 and it did not get any lower (only saving and reloading did lower the number of GDI Objects) - I probably could have continued I'll monitor it for now.

Only minor thing remains - So, we need to save the data by #getBits before image is saved. That should fix it. ;)

Great work!

[Jan Vrany]

OK, I pushed the fix (ff1fb34dbf49/stx.libview). I've also added an smalltalk interface to
GetGuiResources() for more accurate debugging as you suggested (ac94a91f7626/stx.libview). Usage:

WinWorkstation printGuiResourceCounts

Will print counts on stdout.

The above fixes will be included from build 2125 onwards. I'm still not 100% sure there won't be any (unwanted) side effects - let me know if you experience some.

---

The one thing I have noticed is that the GDI object do not get collected too. The number of the GDI objects only raises slowly but I have managed to get it around 725 and it did not get any lower (only saving and reloading did lower the number of GDI Objects) - I probably could have continued I'll monitor it for now.

I don't understand - do you mean that number of GDI object rises but never drops (even if number of user object drops)? Within one session or after snapshot restart?

Only minor thing remains - So, we need to save the data by #getBits before image is saved. That should fix it. ;)

Well, seems to be bit more tricky, I created a separate issue for this - see #88. I'll deal with this a little later.

[Patrik Svestka]

The WinWorkstation printGuiResourceCounts ;(alias GetGuiResources()) now shows correct values for the Windows environment (GDI & USER Objects). (note: Hmm no _PEAK constants in MinGW/64, there is always something missing.)

The above fixes will be included from build 2125 onwards. I'm still not 100% sure there won't be any (unwanted) side effects - let me know if you experience some.

I have not yet experienced any side-effects of this change. I'll report them when I encounter them.

The one thing I have noticed is that the GDI object do not get collected too. The number of the GDI objects only raises slowly but I have managed to get it around 725 and it did not get any lower (only saving and reloading did lower the number of GDI Objects) - I probably could have continued I'll monitor it for now.

I don't understand - do you mean that number of GDI object rises but never drops (even if number of user object drops)? Within one session or after snapshot restart?

That was meant for one session only (performing actions with GUI implicated steady rise of GDI Objects). When the snapshot was restarted the GDI dropped to normal levels. Now the issue seems to be solved for the build '6.2.5.2127'.

[Jan Vrany]

Since it seems the problem has been fixed, I'm closing this issue. If any problems arose, please reopen.