CharacterEncoderImplementations__ISO10646_to_JavaText.st
author convert-repo
Thu, 05 Jul 2018 03:33:35 +0000
changeset 23197 823c765c176d
parent 22476 b30058f26971
permissions -rw-r--r--
update tags
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
     1
"{ Encoding: utf8 }"
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
     2
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     3
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     4
 COPYRIGHT (c) 2006 by eXept Software AG
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     5
              All Rights Reserved
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     6
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     7
 This software is furnished under a license and may be used
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     8
 only in accordance with the terms of that license and with the
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
     9
 inclusion of the above copyright notice.   This software may not
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    10
 be provided or otherwise made available to, or used by, any
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    11
 other person.  No title to or ownership of the software is
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    12
 hereby transferred.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    13
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    14
"{ Package: 'stx:libbasic' }"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    15
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    16
"{ NameSpace: CharacterEncoderImplementations }"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    17
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    18
VariableBytesEncoder subclass:#ISO10646_to_JavaText
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    19
	instanceVariableNames:''
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    20
	classVariableNames:''
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    21
	poolDictionaries:''
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    22
	category:'Collections-Text-Encodings'
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    23
!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    24
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    25
!ISO10646_to_JavaText class methodsFor:'documentation'!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    26
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    27
copyright
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    28
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    29
 COPYRIGHT (c) 2006 by eXept Software AG
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    30
              All Rights Reserved
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    31
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    32
 This software is furnished under a license and may be used
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    33
 only in accordance with the terms of that license and with the
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    34
 inclusion of the above copyright notice.   This software may not
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    35
 be provided or otherwise made available to, or used by, any
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    36
 other person.  No title to or ownership of the software is
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    37
 hereby transferred.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    38
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    39
!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    40
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    41
documentation
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    42
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    43
    Translates \uXXXX-escapes in the text
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    44
"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    45
! !
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    46
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    47
!ISO10646_to_JavaText methodsFor:'encoding & decoding'!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    48
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    49
decodeString:aStringOrByteCollection
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    50
    "given a string in JavaText encoding (i.e. with \uXXXX escaped characters),
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    51
     return a new string containing the same characters, in 16bit (or more) encoding.
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    52
     Returns either a normal String, a Unicode16String or a Unicode32String instance.
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    53
     Only useful, when reading Java property and resource files.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    54
     This only handles up-to 30bit characters."
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    55
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    56
    |nBits ch 
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    57
     in out codePoint|
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    58
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    59
    nBits := 8.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    60
    in := aStringOrByteCollection readStream.
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    61
    out := CharacterWriteStream on:(String new:10).
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    62
    [in atEnd] whileFalse:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    63
        ch := in next.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    64
        ch == $\ ifTrue:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    65
            in peekOrNil == $u ifTrue:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    66
                in next.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    67
                codePoint := 0.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    68
                4 timesRepeat:[
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    69
                    codePoint := (codePoint * 16) + in next digitValue.
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    70
                ].
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    71
                out nextPut:(Character codePoint:codePoint).
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    72
            ] ifFalse:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    73
                out nextPut:ch
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    74
            ]
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    75
        ] ifFalse:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    76
            out nextPut:ch
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    77
        ].
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    78
    ].
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    79
    ^ out contents
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    80
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    81
    "
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    82
     CharacterEncoderImplementations::ISO10646_to_JavaText
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    83
        decodeString:'AB\u1234CD' 
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    84
    "
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    85
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    86
    "Modified: / 23-10-2006 / 13:23:18 / cg"
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
    87
    "Modified: / 17-01-2018 / 18:34:52 / stefan"
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    88
!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    89
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    90
encodeString:aUnicodeString
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    91
    "return the JavaText representation of aUnicodeString.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    92
     The resulting string is only useful to be stored on some external file,
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    93
     not for being used inside ST/X."
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    94
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    95
    |ch in out codePoint|
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    96
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    97
    in := aUnicodeString readStream.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    98
    out := WriteStream on:(String new:10).
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
    99
    [in atEnd] whileFalse:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   100
        ch := in next.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   101
        codePoint := ch codePoint.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   102
        (codePoint between:16r20 and:16r7F) ifTrue:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   103
            out nextPut:ch.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   104
        ] ifFalse:[
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   105
            out nextPutAll:'\u'.
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   106
            out nextPutAll:((codePoint printStringRadix:16) leftPaddedTo:4 with:$0).
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   107
        ].
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   108
    ].
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   109
    ^ out contents
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   110
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   111
    "
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   112
     CharacterEncoderImplementations::ISO10646_to_JavaText
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   113
        encodeString:'hello '  
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   114
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   115
     CharacterEncoderImplementations::ISO10646_to_JavaText
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   116
        decodeString:(CharacterEncoderImplementations::ISO10646_to_JavaText encodeString:'hello ') 
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   117
    "
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   118
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   119
    "Modified: / 23-10-2006 / 13:25:03 / cg"
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   120
! !
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   121
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   122
!ISO10646_to_JavaText methodsFor:'queries'!
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   123
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   124
characterSize:aCharacter
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   125
    (aCharacter codePoint between:16r20 and:16r7F) ifTrue:[
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   126
        ^ 1.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   127
    ].
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   128
    ^ 6    "\u1234"
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   129
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   130
    "Created: / 17-01-2018 / 17:58:59 / stefan"
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   131
! !
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   132
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   133
!ISO10646_to_JavaText methodsFor:'stream support'!
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   134
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   135
readNextCharacterFrom:aStream
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   136
    |char codePoint|
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   137
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   138
    char := aStream next.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   139
    (char ~~ $\ and:[aStream peek ~~ $u]) ifTrue:[
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   140
        ^ char.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   141
    ].
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   142
    aStream next.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   143
    codePoint := 0.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   144
    4 timesRepeat:[
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   145
        codePoint := (codePoint * 16) + aStream next digitValue.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   146
    ].
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   147
    ^ Character codePoint:codePoint.
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   148
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   149
    "Created: / 17-01-2018 / 18:33:22 / stefan"
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   150
! !
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   151
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   152
!ISO10646_to_JavaText class methodsFor:'documentation'!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   153
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   154
version
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   155
    ^ '$Header$'
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   156
!
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   157
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   158
version_CVS
22476
b30058f26971 #REFACTORING by stefan
Stefan Vogel <sv@exept.de>
parents: 15944
diff changeset
   159
    ^ '$Header$'
15944
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   160
! !
8299575b9314 initial checkin
Claus Gittinger <cg@exept.de>
parents:
diff changeset
   161