How to crash hard StX using a code paste

[patrik.svestka@…]

How to crash StX hard:

1) Smalltalk loadPackage: 'stx:libview/tests'. (do it in workspace)
2) start SystemBrowser
3) view Package
4) click on tests
5) click on ImageTests
6) click tests - regressions
7) select whole text (ctrl +a)
8) paste test:

test_issue_126_font_size_24
    "
    https://swing.fit.cvut.cz/projects/stx-jv/ticket/126
    "
       | testString topView textView image allColors |
   [ 
        testString := Unicode16String new.
        testString := 'コーヒーアイスクリームケーキビールすしかき'.

        topView := StandardSystemView new.
        topView extent:300@200.

        textView := EditTextView new.
        textView origin:0.0 @ 0.0 corner:1.0 @ 1.0.
        topView addSubView:textView.
        "/ courier sizes 8, 12, 24, 48, 96, 288
        textView font:(Font family:'courier' size:12).

        1 to: 1000 do: [ :each | textView paste:testString ].
    
        textView unselect.
        topView open. 

        "/ wait for the topView to appear
        Delay waitForSeconds:1.

        image := Image fromView: topView grab: false.

        "/ x starts from 20 to avoid cursor (its color)
        allColors :=  Set new.
          1 to: 20 do: [:y| 
                     20 to: 40 do: [:x | allColors add: (image colorAtX:x y:y)].
                 ]. 
    ] ensure:[ 
        topView close.
    ].  

    self assert: (allColors size > 1)

    "Created: / 22-03-2018 / 14:07:31 / svestkap"
   

9) Right click on the workspace, while the text is still selected
10) Smalltalk/X crashes hard

[jan vrany]

I can reproduce it. It looks you've earned your tea.

The crash (segmentation violation) looks to happen in Win32OperatingSystem >> #primGetFileAttributes:. Sigh.

[jan vrany]

This is going to be many-fold. The minimal testcase (so far) is:

smalltalk -I --quick -P "OperatingSystem primGetFileAttributes: (String streamContents:[:s|s next:2000 put: $x]) asUnicode16String"

So it seems that really long Unicode16String passed to primGetFileAttributes: cause the crash.

Why opening a context menu ends up in primGetFileAttributes: is another story.

[jan vrany]

Patrik,

could you please try attached patch​ and let me know if it helps?

Event if it does, please DO NOT close the ticket. There's more to be investigated!

[patrik.svestka@…]

Hi Jan, the patch is working for the issue reported. I'm leaving the issue opened as you wished.

[jan vrany]

Fixed in:

• cd0581d5639b/stx.libbasic - Issue #197: fixed buffer overflow in _makeWchar()

The crash was caused by buffer overflow in _makeWchar() in Win32OperatingSystem when passed string argument was Unicode16String. It (wrongly) used passed buffer size as size of the buffer in characters while caller passed it as a size in bytes (a result of sizeof()).

This naturally caused a buffer overflow when size of (unicode) string was greater (MAXPATHLEN / 2), in turn causing the segmentation violation.